1/26
Looks like no tags are added yet.
Using the Saga pattern to maintain data consistency
Sagas are mechanisms to maintain data consistency in a microservice architecture without having to use distributed transactions.
You define a saga for each system command that needs to update data in multiple services.
A saga is a sequence of local transactions. Each local transaction updates data within a single service using the familiar ACID transaction frameworks and libraries.
The system operation initiates the first step of the saga. The completion of a local transaction triggers the execution of the next local transaction.
Sagas vs ACIS
Sagas differ from ACID transactions in a couple of important ways. As I describe in detail in section 4.3, they lack the isolation property of ACID transactions.
Also, because each local transaction commits its changes, a saga must be rolled back using compensating transactions.
An example SAGA: the createOrder() saga
services that participate in a saga communicate using asynchronous messaging. A service publishes a message when a local transaction completes. This message then triggers the next step in the saga. Not only does using messaging ensure the saga participants are loosely coupled, it also guarantees that a saga completes. That’s because if the recipient of a message is temporarily unavailable, the message broker buffers the message until it can be delivered.

Sagas use compensating transactions to roll back changes
Unlike ACIS transaction, sagas can’t be automatically rolled back, because each step commits its changes to the local database. This means, for example, that if the authorization of the credit card fails in the fourth step of the Create Order Saga, the FTGO application must explicitly undo the changes made by the first three steps. You must write what are known as compensating transactions.
What happens if a step in a Saga fails?
If step Tn+1 fails, all previous steps T1…Tn must be undone.
Each step Ti has a compensating transaction Ci that reverses it.
Undo is done in reverse order: T1… Tn, Cn…C1.
Example: If credit card authorization fails in step 4, the first 3 steps must be rolled back with compensating transactions.
It’s important to note that not all steps need compensating transactions. Read-only steps, such as verify ConsumerDetails(), don’t need compensating transactions. Nor do steps such as authorizeCreditCard() that are followed by steps that always succeed.
(145-147)

Coordinating Sagas
A saga’s implementation consists of logic that coordinates the steps of the saga. When a saga is initiated by system command, the coordination logic must select and tell the first saga participant to execute a local transaction. Once that transaction completes, the saga’s sequencing coordination selects and invokes the next saga participant. This process continues until the saga has executed all the steps. If any local transaction fails, the saga must execute the compensating transactions in reverse order.
There are a couple of different ways to structure a saga’s coordination logic:
Choreography—Distribute the decision making and sequencing among the saga participants. They primarily communicate by exchanging events.
Orchestration—Centralize a saga’s coordination logic in a saga orchestrator class. A saga orchestrator sends command messages to saga participants telling them which operations to perform.
Coordinating Sagas - Choreography-based sagas
One way you can implement a saga is by using choreography. When using choreography, there’s no central coordinator telling the saga participants what to do. Instead, the saga participants subscribe to each other’s events and respond accordingly.

Implementation of Create order SAGA using Choreography
Figure 4.4 shows the design of the choreography-based version of the Create Order Saga. The participants communicate by exchanging events. Each participant, starting with the Order Service, updates its database and publishes an event that triggers the next participant.
The Create Order Saga must also handle the scenario where a saga participant rejects the Order and publishes some kind of failure event. For example, the authorization of the consumer’s credit card might fail. The saga must execute the compensating transactions to undo what’s already been done. Figure 4.5 shows the flow of events when the AccountingService can’t authorize the consumer’s credit card.

communication-related issues that you must consider when implementing choreography-based sagas.
The first issue is ensuring that a saga participant updates its database and publishes an event as part of a database transaction. Each step of a choreography-based saga updates the database and publishes an event. For example, in the Create Order Saga, Kitchen Service receives a Consumer Verified event, creates a Ticket, and publishes a Ticket Created event. It’s essential that the database update and the publishing of the event happen atomically. Consequently, to communicate reliably, the saga participants must use transactional messaging, described in chapter 3.
The second issue you need to consider is ensuring that a saga participant must be able to map each event that it receives to its own data. For example, when OrderService receives a Credit Card Authorized event, it must be able to look up the corresponding Order. The solution is for a saga participant to publish events containing a correlation id, which is data that enables other participants to perform the mapping.
Benefits & Drawbacks of Choreography-based sagas

Orchestration-based sagas
Orchestration is another way to implement sagas. When using orchestration, you define an orchestrator class whose sole responsibility is to tell the saga participants what to do.
The saga orchestrator communicates with the participants using command/async reply-style interaction.
To execute a saga step, it sends a command message to a participant telling it what operation to perform. After the saga participant has performed the operation, it sends a reply message to the orchestrator. The orchestrator then processes the message and determines which saga step to perform next.

Implementing the the createOrder (as an example) Saga using Orchestration
Note that in final step, the saga orchestrator sends a command message to OrderService, even though it’s a component of Order Service. In principle, the Create Order Saga could approve the Order by updating it directly. But in order to be consistent, the saga treats Order Service as just another participant.
Diagrams such as figure 4.6 each depict one scenario for a saga, but a saga is likely to have numerous scenarios. For example, the Create Order Saga has four scenarios. In addition to the happy path, the saga can fail due to a failure in either Consumer Service, Kitchen Service, or Accounting Service. It’s useful, therefore, to model a saga as a state machine, because it describes all possible scenarios.

Modeling saga orchestration as state machines
A state machine consists of a set of states and a set of transitions between states that are triggered by events. Each transition can have an action, which for a saga is the invocation of a saga participant.
The transitions between states are triggered by the completion of a local transaction performed by a saga participant.
The current state and the specific outcome of the local transaction determine the state transition and what action, if any, to perform.
(check img) → The state machine’s initial action is to send the VerifyConsumer command to Consumer Service. The response from Consumer Service triggers the next state transition. If the consumer was successfully verified, the saga creates the Ticket and transitions to the Creating Ticket state. But if the consumer verification failed, the saga rejects the Order and transitions to the Rejecting Order state. The state machine undergoes numerous other state transitions, driven by the responses from saga participants, until it reaches a final state of either Order Approved or Order Rejected.
(153-154)

Saga orchestration and transactional messaging
Each step of an orchestration-based saga consists of a service updating a database and publishing a message.
For example, Order Service persists an Order and a Create Order Saga orchestrator and sends a message to the first saga participant. A saga participant, such as Kitchen Service, handles a command message by updating its database and sending a reply message. Order Service processes the participant’s reply message by updating the state of the saga orchestrator and sending a command message to the next saga participant. As described in chapter 3, a service must use transactional messaging in order to atomically update the database and publish messages. Later on in section 4.4, I’ll describe the implementation of the Create Order Saga orchestrator in more detail, including how it uses transaction messaging.
I recommend using orchestration for all but the simplest sagas.

Handling the lack of isolation when using sagas
The challenge with using sagas is that they lack the isolation property of ACID transactions. That’s because the updates made by each of a saga’s local transactions are immediately visible to other sagas once that transaction commits.
This behavior can cause two problems. First, other sagas can change the data accessed by the saga while it’s executing. And other sagas can read its data before the saga has completed its updates, and consequently can be exposed to inconsistent
data. You can, in fact, consider a saga to be ACD without I (Isolation):
Atomicity—The saga implementation ensures that all transactions are executed
or all changes are undone.
Consistency—Referential integrity within a service is handled by local databases.
Referential integrity across services is handled by the services.
Durability—Handled by local databases.
Overview of anomalies
This lack of isolation potentially causes what the database literature calls anomalies. An anomaly is when a transaction reads or writes data in a way that it wouldn’t if transactions were executed one at time.
Lost updates—One saga overwrites without reading changes made by another saga.
Dirty reads—A transaction or a saga reads the updates made by a saga that has
not yet completed those updates.
Fuzzy/nonrepeatable reads—Two different steps of a saga read the same data and get different results because another saga has made updates.
All three anomalies can occur, but the first two are the most common and the most challenging. Let’s take a look at those two types of anomaly, starting with lost updates.
Anomalies - Lost updates

Anomalies - Dirt updates

Countermeasures for handling the lack of isolation
The structure of a saga
The countermeasures paper mentioned in the last section defines a useful model for the structure of a saga.
shown in figure 4.8, a saga consists of three types of transactions:
Compensatable transactions—Transactions that can potentially be rolled back using a compensating transaction.
Pivot transaction—The go/no-go point in a saga. If the pivot transaction commits, the saga will run until completion. A pivot transaction can be a transaction that’s neither compensatable nor retriable. Alternatively, it can be the last compensatable transaction or the first retriable transaction.
Retriable transactions—Transactions that follow the pivot transaction and are guaranteed to succeed.

Countermeasure: Semantic Lock
When using the semantic lock countermeasure, a saga’s compensatable transaction sets a flag in any record that it creates or updates.
The flag indicates that the record isn’t committed and could potentially change. The flag can either be a lock that prevents other transactions from accessing the record or a warning that indicates that other transactions should treat that record with suspicion.
It’s cleared by either a retriable transaction—saga is completing successfully—or by a compensating transaction: the saga is rolling back.

Countermeasure: Commutative updates
One straightforward countermeasure is to design the update operations to be commutative. Operations are commutative if they can be executed in any order. An Account’s debit() and credit() operations are commutative (if you ignore overdraft checks). This countermeasure is useful because it eliminates lost updates.
Consider, for example, a scenario where a saga needs to be rolled back after a compensatable transaction has debited (or credited) an account. The compensating transaction can simply credit (or debit) the account to undo the update. There’s no possibility of overwriting updates made by other sagas.
Countermeasure: pessimistic view
Another way to deal with the lack of isolation is the pessimistic view countermeasure. It reorders the steps of a saga to minimize business risk due to a dirty read.

Countermeasure: Read value
The reread value countermeasure prevents lost updates. A saga that uses this countermeasure rereads a record before updating it, verifies that it’s unchanged, and then updates the record. If the record has changed, the saga aborts and possibly restarts.

Countermeasure: Version file
The version file countermeasure is so named because it records the operations that are performed on a record so that it can reorder them. It’s a way to turn noncommutative operations into commutative operations.

Countermeasure: By value
The final countermeasure is the by value countermeasure. It’s a strategy for selecting concurrency mechanisms based on business risk.
An application that uses this countermeasure uses the properties of each request to decide between using sagas and distributed transactions. It executes low-risk requests using sagas, perhaps applying the countermeasures described in the preceding section. But it executes high-risk requests involving,

Creating sagas - OrderService example
This would need the following steps:
creating a new order
publish the createOrder event
save the new created order in the database
create the saga state
publish the saga state to consumers
(complete example: 163-175)
