Library · Summary & review

Event-Driven Architecture in Golang

By Michael Stack. Moving a Go monolith to events without losing a message, and what is worth keeping from the book.

FR EN
Event-Driven Architecture in Golang book cover, Michael Stack, 1st edition

Event-Driven Architecture in Golang

Event-Driven Architecture in Golang: Building complex systems with asynchronicity and eventual consistency

6.4 /10

« A build told step by step, honest about its shortcuts, solid when no message may be lost, dated on tooling. »

  • AuthorMichael Stack
  • OriginalPackt, 2022 · 1st edition, 380 pages
  • LanguageEnglish only
  • This page~29 min read
Book rating across 5 dimensionsIdeas5/10Practical7/10Readability6/10Aged well6/10Examples8/10

Move a Go monolith to event-based exchanges, step by step, without losing or duplicating messages.

Why this book

Two services that call each other directly end up glued together. They must be deployed together, and one's outage stops the other. Michael Stack, the author, calls the result a distributed monolith: a monolith with needless complexity added on top (preface).

His remedy is EDA (event-driven architecture): services no longer call each other, they publish events. An event is a past fact nobody undoes, such as "order created". The book puts it this way: "The event itself is in the past and it is an immutable fact." (p. 8).

The book follows a single thread, MallBots, a shopping mall application. Kiosks let shoppers pick items from several stores, and robots go and fetch them (p. 12). You start from a modular monolith: one program to deploy, cut into modules that each own a business area. Chapter after chapter, the direct calls between modules become messages. They go through NATS JetStream, a broker (a server that receives messages and redistributes them), with PostgreSQL for the data.

This note assumes the basics are known: storing events, separating reads from writes. On the blog, I already wrote articles on the PostgreSQL event store (idea 1), on choreographed sagas (idea 7) and on idempotence: a job replayed twice gives the same result as once. So I keep only what the book adds. The last two chapters deploy the result on AWS with Terraform, then monitor it. They are off topic and I leave them out.

The ideas that stick

The nine ideas fall into three groups:

1Three meanings of the word EDA, three lifespans for an event

On a team, "we do event-driven" can cover three different practices. The book separates them from the start: "Three different uses or patterns exist that can be called EDA individually or altogether" (p. 4).

  • notification: the event carries almost nothing, sometimes only an identifier. The book shows PaymentReceived with just the PaymentID. Whoever wants the detail must call the payments service back (p. 5).
  • event-carried state transfer (ECST): the same event also carries the customer, the order and the amount. Nobody has to call back anymore. The book calls it "an asynchronous cousin to representational state transfer (REST)" (p. 5). REST means the usual HTTP calls, where you fetch data on demand and wait for the answer: that is synchronous. ECST pushes the data to everyone interested without waiting for an answer: that is asynchronous.
  • event sourcing: you no longer store an object's state but the sequence of its changes. They go into an event store, an append-only table. The current state is recomputed by replaying the events (p. 7).

A second grid measures an event's lifespan and how far it travels (p. 143). An aggregate here is a group of business objects treated as one, an order with its lines for example.

  • domain event: the shortest, it never leaves the application and lives in memory for the time of the current processing. Since it is neither stored nor sent, it has no version (no format number to track) and it is synchronous. It tells the rest of the code that an aggregate changed (pp. 91, 143).
  • event sourcing event: the longest, it stays stored as long as needed. It never crosses the boundary of the service, the module that owns the data. It is versioned: you never edit an event already written, you create a new version. It is synchronous too. It is the aggregate's change log (pp. 91, 116, 143).
  • integration event: its lifespan and its number of consumers are unknown, it is versioned and asynchronous. It informs the other components (p. 143).

The book links the two lists in one sentence: "Both notification and event-carried state transfer events are integration events" (p. 143). Notification and ECST are therefore two ways to fill an integration event. Asynchrony starts there: "Integration events are strictly asynchronous" (p. 87).

Two lines then separate EDA from its neighbors. First, event sourcing is not event broadcasting between modules. It keeps the history of one module and stays "an implementation detail and not an architectural choice that has application-wide ramifications" (p. 101). Second, CQRS, which separates writes from reads, is not part of it: "CQRS is not, in my opinion, an event-driven pattern." (p. 39).

2A modular monolith first, asynchrony later

With the vocabulary set, the starting point remains to choose. The book opens on a chain of synchronous calls. Picture seven people passing a message down a line: each waits for the next one's reply, and a single absentee blocks everyone. When a customer checks out a basket, the Baskets module calls the Ordering module, which manages orders (p. 86). Creating the order then calls four other modules: Customers, Payments, Depot, where the robots bring the items, and Notifications. Depot in turn queries the Stores module, twice (p. 86). In total, "A total of seven modules are involved in the process of checking out a basket." (p. 86).

The author owns the exaggeration: an extreme example, for demonstration. He adds that "call chains such as these can develop in real applications" (p. 86). His answer (ch. 2) is not to cut the program into microservices, independent services that are heavy to run (p. 41). His recommendation fits in one sentence: "A modular monolith is the recommended architecture to start with for any project of reasonable complexity." (p. 41). The program stays a single block, and the call chain is replaced step by step with events. It is the same building, but the phone chain gives way to a notice board where everyone reads what concerns them.

Each module is a bounded context, a business area with its own vocabulary (p. 76). Folders are named baskets, stores, depot, ordering, not controllers or models. Each module has its own internal directory, which the Go compiler protects from imports by another module (pp. 76-77). Modules call each other over gRPC, a remote function call protocol (p. 84). In a single program, a plain function call would do. Yet the book wants each module to expose only a fixed interface to the others. That contract will survive if the module ever becomes a separate service (pp. 41-42).

Before adding any message, the book sets out a choice grid. Two reasons push one context to call another: it needs data, or it needs an action (p. 84). The action is the simpler one: you send the request to the target module's entry point (its API), the most common route (p. 85). For data, there are three options:

  • share the database: everyone writes in the same address book. Avoid it, especially if several places change it (p. 84).
  • push: the owner sends the data to a list of known subscribers, like a mailing list. The author sees "This is a maintenance nightmare." (p. 84), because the list grows with the components.
  • pull: the requester fetches the data when it needs it, like opening the directory at the moment of calling.

The verdict surprises in an EDA book: "Given the options, pulling data is the better choice in most cases." (p. 84). Pull costs more calls. Caching the data lightens those calls, but the cache is hard to invalidate (p. 84). There remains the risk that a called service goes down. The caller prepares for it: it retries, and it adds a circuit breaker, a guard that stops calling a failing service for a few moments (p. 84). Same principle as the breaker in an electrical panel, which cuts before everything burns out.

The book's first step is not asynchronous at all. It replaces the side effects of a handler, the function that processes a request, with in-memory domain events. The example: CreateOrder saved the order, then called notifications. With every new rule, it would have needed renaming to CreateOrderAndNotifyAndReward… (p. 88). The author admits it at the end of the chapter: "this didn’t change much or add any new asynchronous communication methods" (p. 97). This first step serves as a foundation: the book then uses it as the starting point to make the whole application reactive (p. 97). One useful detail: these events stay inside the context, so they can embed the whole aggregate (p. 91). Only the events that leave the module must be cut down to the strict minimum (p. 91).

3The silent bug: Save empties the events before they are published

The domain events of idea 2 stay in memory. Later (ch. 5), the book moves aggregates to event sourcing, and a bug appears with no error at all. Picture a mail counter. The clerk files the letter, then empties the outgoing tray. The mail carrier comes later and finds the tray empty. The repository (AggregateRepository) is the object that loads and saves an aggregate. It relies on a store, the adapter that talks to PostgreSQL. Its Save ends with CommitEvents(). That call updates the aggregate's version (the number of its last event). It also empties the list of pending events (pp. 117, 131).

Now the handler for the "add a product" command (in CQRS terms, a command is a request for change) then called Publish(product.GetEvents()...). The list is already empty, so nothing goes out: the carrier leaves empty-handed. The catalog read model, the read table fed by these events, is never updated. Nothing crashes, and the expected log lines never appear (p. 131).

// command handler, from the book (p. 131)
if err = h.products.Save(ctx, product); err != nil { // Save empties the events
    return err
}
if err = h.domainPublisher.Publish(ctx, product.GetEvents()...); err != nil { // empty list
    return err
}

The book compares three remedies:

  • publish before saving. It works if "Everyone remembers that Publish() must always precede Save()" (p. 131). Save must also never fail because of a badly applied event: events would then be published without being recorded.
  • inject the publisher into the repository. The repository then depends on the publisher. To do without it, you give it a nil that it must test before each use (p. 132).
  • wrap the store in a middleware, a function that surrounds an object to add behavior without changing its interface. This is the chosen solution (p. 132).
// from the book (pp. 133-134), shortened
type AggregateStoreMiddleware func(AggregateStore) AggregateStore

func AggregateStoreWithMiddleware(store AggregateStore, mws ...AggregateStoreMiddleware) AggregateStore {
    s := store
    for i := len(mws) - 1; i >= 0; i-- { s = mws[i](s) } // store, A, B, C gives A(B(C(store)))
    return s
}

type EventPublisher struct { AggregateStore; publisher ddd.EventPublisher }

func (p EventPublisher) Save(ctx context.Context, a EventSourcedAggregate) error {
    if err := p.AggregateStore.Save(ctx, a); err != nil { return err } // save first
    return p.publisher.Publish(ctx, a.Events()...)                     // publish next
}

The middleware wraps the store, which the repository calls in its own Save. It therefore publishes during store.Save, before the repository calls CommitEvents(): the list is not empty yet. It is a checkpoint that photocopies the letter before the tray is emptied. The guarantee fits in one sentence: "Domain events will always be published after the events have been successfully persisted into the database." (p. 134). Command handlers no longer need the publisher.

Two usage rules follow. The order of the chain matters little. Only a middleware that catches panics must come first in the list. It is then the outermost (A in A(B(C(store)))) and covers the whole chain, like a safety net (p. 133). Second rule: the old Publish calls left in the handlers are a trap. Normally they do nothing, since the list is empty, but "when things go wrong, they may still publish events we would not want to be published" (p. 135). The book does not say in which case.

The book states a limit: all of this stays synchronous, inside a single context (p. 134). A crash between the save and a message to another service remains possible. Idea 6 deals with that.

4Never serialize the aggregate in a snapshot, serialize a versioned copy

A second cost of event sourcing remains: rereading all of an aggregate's events on each load. The book distinguishes two kinds of aggregates. Short ones, like an order or a basket, see few events. Long ones, like a store or a customer, pile up many (p. 135). For the long ones, you take snapshots, photos of the state at a given version.

The book gives the definition and a ban: "A snapshot is a serialization of the aggregate and the version of the aggregate it came from" (p. 136). Yet you must not serialize the aggregate itself, because that would "limit the flexibility to change the structure of the aggregate in the future" (p. 136). Renaming a field of the aggregate would indeed make the snapshots already recorded unreadable. You serialize a versioned representation instead, ProductV1. It is a numbered form: an old V1 form stays readable even when the business moves to a V2.

The aggregate implements two methods, ToSnapshot() and ApplySnapshot(). The second one reads back every known version:

// Product's ApplySnapshot, from the book (p. 137)
switch ss := snapshot.(type) {
case *ProductV1:
    p.StoreID, p.Name, p.Description, p.SKU, p.Price = ss.StoreID, ss.Name, ss.Description, ss.SKU, ss.Price
// made-up example based on the text: ProductV2 adds Weight int
case *ProductV2:
    /* same fields as V1, plus p.Weight = ss.Weight */
default:
    return errors.ErrInternal.Msgf("%T received the unexpected snapshot %T", p, snapshot)
}

The text gives the evolution rule. We add Weight, so we create ProductV2 with all the fields of ProductV1 plus the weight. ToSnapshot() now returns the V2, and ApplySnapshot() handles both (p. 137). The V1 branch of ApplySnapshot does not change. As long as no event sets a product's weight, Weight keeps 0, the default value of an integer in Go. You still have to keep the V1 reading code. Indeed, "An older snapshot may continue to exist in the database" as long as the strategy does not take a new one (p. 137).

The book lists three frequency strategies, to pick according to the business (p. 137):

  • every N events, for example every 50.
  • every period, once a day or once an hour.
  • at a pivot event, for example when a store changes its brand.

Its own code takes a snapshot every three events, and it says so plainly: "Every three events is not a good strategy outside of demonstration purposes." (p. 138).

The snapshots table has (stream_id, stream_name) as its key, without the version. A stream here is the sequence of an aggregate's events, identified by an id and a name (p. 120). An UPDATE therefore overwrites the old photo, and you can change the key to keep a history (p. 138). On load, the middleware looks for a snapshot and applies it. The store then reads only events with a higher version. It "skips all events that have a version equal to or lower than what the snapshot has for its version" (p. 139). It works like a bookmark: you resume reading where the photo was taken.

The author warns against overuse: "A snapshot is a type of cache, and like other caches, it has various downsides, such as the duplication and invalidation of data." (p. 139). Later, he makes it a performance optimization that "should only be used when absolutely necessary" (p. 139). The book gives no performance measurement and no event count from which the photo pays off.

5One transaction per request, carried by an injection container

In ideas 3 and 4, the code writes several things in a row (events, version, snapshot). It assumes the whole thing succeeds or fails as one block. Between services, that assumption falls apart. The book illustrates it with payments: the Payments module marks an invoice paid and notifies the Ordering module. If saving the invoice fails after the call, Ordering has already moved on (pp. 230-231). With an event in place of the call, it is worse: you no longer know who consumed it or what it triggered, so you don't know what to fix (p. 231).

The book names the problem: the dual write. It shows up wherever a state change must be written "split in two or more places" (p. 232). Here, those two places are the database and the broker. It is like paying for the bread at one till and the milk at another, with the second till failing. A transaction is a set of database writes that succeed or fail together: one till for the whole basket. The chapter's thesis: "Even more important than having a single write is that we need to have a single transactional boundary around what we will be writing." (p. 232). The book also rules out a two-step mini-saga (a saga is a sequence of steps that can be rolled back, see idea 7), which would be "like using a sledgehammer to swat a fly" (p. 232).

Creating a transaction is easy. The challenge is "ensuring the same transaction is used for every database interaction for the entire life of the request" (p. 233). The book rejects three leads:

  • a global variable: a nightmare to test and to evolve (p. 233).
  • a tx parameter added to every method: the application would be coupled to the database, which the author finds "completely out of the question" (p. 233).
  • a transaction slipped into the context and read by each repository. You would have to change them all, and third-party code could not read it (p. 233).

The chosen lead creates the repositories on demand, with the transaction injected. Dependency injection means giving an object what it needs instead of letting it create it. The book adds a small di package, a container: a registry of object factories. It knows two lifetimes (pp. 233-236):

  • singleton: a single instance for the whole application, like the coffee machine on the floor. The logger, for example.
  • scoped: one instance per scope, meaning per request. The transaction and the repositories that use it are of this kind. It is the customer's tray: each request gets its own.

The book flags a trap. Take NewShoppingListRepository(..., mono.DB().Begin()), the repository for the Depot module's shopping lists, wired to the monolith's database. It does give one transaction per scope, but only that repository sees it (p. 238). Each aisle would get its own tray. You need a single scoped "tx" that all repositories receive. A DB interface replaces *sql.DB to accept either a database or a transaction (p. 239).

Each entry point then opens a scope, then closes the transaction. For gRPC, the book wraps the original server in a serverTx. On each request, it opens a scope, builds a fresh server that receives the scope's transaction, passes the request to it, then commits or rolls back:

// serverTx, from the book (pp. 241-242), shortened
func (s serverTx) CreateShoppingList(ctx context.Context, req *depotpb.CreateShoppingListRequest) (resp *depotpb.CreateShoppingListResponse, err error) {
    ctx = s.c.Scoped(ctx)                      // new scope in a new context
    defer func(tx *sql.Tx) {
        err = s.closeTx(tx, err)               // commit or rollback, named returns to replace err
    }(di.Get(ctx, "tx").(*sql.Tx))
    next := server{app: di.Get(ctx, "app").(application.App)} // fresh application, same tx
    return next.CreateShoppingList(ctx, req)
}

func (s serverTx) closeTx(tx *sql.Tx, err error) error {
    if p := recover(); p != nil { _ = tx.Rollback(); panic(p) } // we don't swallow the panic
    if err != nil { _ = tx.Rollback(); return err }
    return tx.Commit()
}

Domain events, for their part, run inside an already open scope. They find its container in the request's Go context. They don't create another one, otherwise they would get their own transaction (pp. 243-244). Message and command handlers open a scope like serverTx (p. 245).

The book describes the gain without overselling it: "Every query and insert will be made within a single transaction, effectively turning multiple writes into one." (p. 245). It adds right away that the dual write is not solved (p. 245). The database is consistent, the message is still to handle. The price is admitted too. The injection package adds complexity. The container returns untyped values, and the code asserts their type (.(*sql.Tx), a "type assertion") without the compiler checking it: if the promise is wrong, the error only shows at run time. Finally, Get panics if the key does not exist (pp. 234-235).

6The outbox writes the message inside the transaction, the inbox absorbs its duplicates

The transaction of idea 5 surrounds the database. Messages still have to get inside it. The outbox is a table in the same database. Instead of publishing, you write the message there, in the request's transaction. A separate process publishes it afterwards. The inbox is the table of messages already received, used to recognize a duplicate. The outbox is an office's outgoing tray: the clerk drops the letter in as they record the order, and the mail carrier empties it later. If the order is cancelled, the letter goes with it. The inbox is the receptionist's logbook: they tick off the number of each parcel and refuse a second one with the same number.

On the sending side, the book leaves the handlers alone. It puts a middleware on the raw message stream. This stream is the channel that carries bytes, unrelated to the stream of idea 4. The middleware writes to depot.outbox instead of publishing (pp. 251-252). The typed streams (events, commands, replies) are rebuilt on top of it, inside the request's scope (p. 252). The table has a published_at column that stays empty while nothing has left, and a partial index that covers only those rows: WHERE published_at IS NULL (pp. 250-251).

Without a sending process, "everything quickly comes to a halt" (p. 252), since no message leaves the local database anymore. The processor polls the table, meaning it rereads it at a fixed interval. It takes a batch of unpublished messages, publishes them one by one, then fills in published_at. In the demo, it reads 50 messages at most and waits half a second between two reads (p. 254). It doesn't wait if it just published. It runs as a goroutine in each module, with no transaction: a transaction lasts for one request, and the processor runs continuously (pp. 253-254). The book cites a more efficient option: reading PostgreSQL's transaction log, the WAL (p. 252).

The processor has a dual write of its own: it publishes, then it records. If it crashes between the two, the message goes out twice. The book draws the conclusion: these crashes sometimes publish several times, but "We already have deduplication in place thanks to our implementation of the inbox" (p. 253). Exactly-once processing, one execution per message, is something that "can be achieved by adding deduplication to at-least-once delivery" (p. 149). At-least-once means the broker retries until it gets an acknowledgment, so duplicates are possible.

On the receiving side, deduplication becomes atomic. The depot.inbox table has the message identifier as its key. The middleware tries to insert each received message there. On conflict, the message is acknowledged without being processed. Otherwise it is processed, and its identifier is recorded in the same transaction as the rest of the changes (p. 246).

The inbox works on raw messages, because the subscription side of typed streams cannot live in a request's scope (p. 247). The book does not explain it, but the repository gives the reason: the subscription is created once at startup, before any request. So the book writes a NewEventMessageHandler that converts each raw message back into a Go event (pp. 247-248).

The schemas are minimal, and the book knows it. The inbox table could carry the aggregate version, the publication date or the message order (p. 246). The outbox table could get ordering or partitioning columns (p. 251). The book places no lock on the rows the processor reads, and never mentions SKIP LOCKED. That is a SQL option that lets a reader skip rows already locked by another one. Without it, two instances of the same module could read the same rows and publish the same messages twice. By the book's logic, only the consumers' inbox would absorb these duplicates.

Here, crash by crash, is what remains and who catches it.

The crashWhat remainsWho catches it
Before the commitNothing, neither in the database nor on the brokerNobody: there is nothing to catch up
After the commit, before publishingThe outbox row, published_at emptyThe processor publishes it on its next pass
After publishing, before published_atA message already gone, which will go againThe consumer's inbox ignores the duplicate
Two instances read the same rowsThe same message published twiceThe inbox, by the book's logic (it never mentions SKIP LOCKED)

7An orchestrated saga is a database row that moves one step forward on each reply

Messages now flow without gaps. What remains is an operation that crosses several modules and must be able to roll back. Here, creating an order: authorize the customer, create the shopping list at the depot, confirm the payment, start the shopping, approve the order. If the payment fails, the list must be cancelled and the order rejected.

A saga is a sequence of steps, each with an action and a compensation, the inverse action that undoes the step if a later one fails. Picture a trip booked with several suppliers: if the car fails, you cancel the hotel, then the flight, in reverse order. You don't go back in time, you do the opposite action. The book compares it to two-phase commit (2PC). In it, a coordinator keeps transactions open, and so resources blocked, until its final order (pp. 200-201). The saga avoids this blocking, but it loses isolation: "Sagas drop support for the isolation guarantee, making them ACD transactions." (p. 201). An ACID transaction offers four guarantees: atomic, consistent, isolated, durable. A saga keeps A, C and D but not the I: the effects of a step are visible to concurrent operations during execution, like the hotel booking a customer sees before the flight is confirmed.

There are two ways to drive it. In choreography, each module listens for the event that announces its turn. The book counts six of them for order creation, from OrderCreated to OrderApproved (pp. 202-203). The trap shows up at compensation. If the payment fails, Payments publishes UnauthorizedPayment. Depot cancels its list, Ordering rejects the order, and Customers doesn't listen to the event: either it has nothing to do there, or someone forgot to subscribe it, and nothing tells you which. The author doesn't hide the ambiguity: "Choreographed compensations can be tricky this way." (p. 203).

In orchestration, an orchestrator sends commands, messages that ask a module to act, and receives success or failure replies. Order creation gives ten messages, five commands and five replies (pp. 204-205). A failure reply triggers the compensation. For a refused payment, it takes five messages. First the failure reply, then the cancellation of the list and its reply, finally the rejection of the order and its reply (p. 205). The book's rule: as soon as there is more than one step and more than one compensation, an orchestrator beats a choreography (p. 205).

The two ways, side by side.

ChoreographyOrchestration
Who decides what comes nextEach module, on the event that announces its turnAn orchestrator that sends commands
Where the sequence livesSpread across the modules' subscriptionsIn the orchestrator and the database row
The compensationEach module reacts to the failure eventA failure reply triggers the cancellation commands
The trapA module forgotten in the subscriptions, and nothing flags itThe book assumes every compensation succeeds
When to pick itThe book no longer recommends it once there is more than one step and more than one compensation (p. 205)As soon as there is more than one step and more than one compensation (p. 205)

The cosec module (Create-Order-Saga-Execution-Coordinator) implements it (p. 211). It starts on the OrderCreated integration event, and the event's identifier becomes the saga identifier (p. 225). The trick is that the orchestrator keeps nothing in memory: "The orchestrator does not keep the sagas running or in memory." (p. 225). All the state lives in a PostgreSQL row:

-- from the book (p. 221)
CREATE TABLE cosec.sagas (
  id text NOT NULL, name text NOT NULL, data bytea NOT NULL,
  step int NOT NULL, done bool NOT NULL, compensating bool NOT NULL,
  updated_at timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (id, name)
);

The cursor is step plus compensating, and done marks the end. After each interaction, the orchestrator writes the saga's data (the data column) to the database, for example the identifier of the shopping list just created. A reply arrives, it loads the saga by the correlated identifier, applies the reply, moves on, saves. It is a clerk who forgets everything between two calls: they reread the case's tracking card, act, then update it. A command carries in its headers the address where replies should go, and these headers are copied onto the reply. That is how the saga is found again (p. 208). The benefit: the saga doesn't get in the way of a stopped service, and "so long as it eventually comes back up, the saga will also eventually get back to executing the steps it needs to" (p. 228).

Steps are declared with a builder. Step 0 has no action, because the order already exists, pending, when the saga starts. It has only the compensation that rejects it, and since compensations play in reverse, it runs last on a rollback:

// the six steps, from the book (pp. 223-224)
saga.AddStep().Compensation(saga.rejectOrder)                   // 0. -RejectOrder
saga.AddStep().Action(saga.authorizeCustomer)                   // 1. AuthorizeCustomer
saga.AddStep().Action(saga.createShoppingList).                 // 2. CreateShoppingList
    OnActionReply(depotpb.CreatedShoppingListReply, saga.onCreatedShoppingListReply).
    Compensation(saga.cancelShoppingList)                       //    -CancelShoppingList
saga.AddStep().Action(saga.confirmPayment)                      // 3. ConfirmPayment
saga.AddStep().Action(saga.initiateShopping)                    // 4. InitiateShopping
saga.AddStep().Action(saga.approveOrder)                        // 5. ApproveOrder

A step without compensation is skipped on rollback (p. 211). So let's follow a payment refused at step 3. The orchestrator switches to compensating and walks back. It sends CancelShoppingList for step 2, skips step 1, then sends RejectOrder for step 0. This matches the five messages seen above (p. 205). The reply of step 2 also serves: onCreatedShoppingListReply copies the list identifier into the saga's data, for the InitiateShopping command (p. 224).

The gaps are real. The book is content to state that the orchestrator "expects each compensating action to complete successfully" (p. 205). It says nothing more. Nowhere does it mention a compensation that fails, a maximum saga duration or a stuck saga. No workflow engine is cited either. Temporal, for example, is software that runs sequences of durable steps and resumes them after a crash. Finally, the book shows the orchestrator through type diagrams. They give the signatures, not the function bodies, so nothing about an unexpected reply.

My reading

Outside the book, read in the example code (ch. 8). The figures only give the signatures (pp. 209-211), so I read the repository, internal/sec/orchestrator.go. It confirms the gap: if a cancellation fails, it returns an internal error and never retries. It also silently drops a reply without a saga header, because it cannot tell which saga the reply belongs to. I found a flaw too: if the first action fails, Start does not return the error, and the caller thinks all is well (ch. 8, 9 and 12). The last example in the repository adds the scoped transaction, the outbox and the inbox of ideas 5 and 6, but the gap in cancellations remains.

8A read model can feed on three modules, joined when the order arrives

Each module receives events from the others. What does it do with these messages? The book's first answer is a local copy. The Baskets module keeps a table of stores and products, fed by events from Stores. The book calls it a "cache repository" to remind us that the data doesn't change owner (p. 176). It plays the role of an anti-corruption layer, an airlock between two models: if Store or Product change shape, only the code that receives the event moves (p. 176).

Each consumer sizes its own copy. The Depot module doesn't need a product's price, but it keeps the store's location (p. 180). A gRPC call serves as a fallback when the row is missing, but it cannot tell that a copy is stale: "This will not help us to determine if our cache is stale" (p. 177). The insert also ignores the uniqueness violation, because the incoming event and the fallback may insert the same row at the same time (p. 178).

A warning closes the part: "when we transfer state with events, we do not transfer domain responsibilities along with it" (p. 181). If the Customers module owns authentication, it is still the one you call for that.

The Search module pushes the principle further. It answers five searches: by customer, by store and product, by date range, by total range, by status (p. 192). The book refuses to graft it onto the Ordering module, which manages orders. Managing an order and searching among orders share data, not all their logic. Adding search to an existing module would risk causing bugs in it or slowing it down (p. 192). The book also justifies it with a team argument: standing up an event consumer doesn't disturb the other teams (p. 192).

The read model is a single table, search.orders. It holds the customer name, the order lines and the status. Two array columns, product_ids and store_ids, serve to find the orders placed for a given product or store (p. 193). Two choices are worth keeping. The filtering columns live in the repository implementation, not in the model, so you can change database (p. 193). The identifiers and the status stay text, so that incoming types can change without breaking the table (p. 194).

The build happens in two stages. Search listens to three modules: Customers (the customers), Stores (the stores and products) and Ordering (the orders) (p. 189). It stores the customer, store and product events as they arrive. At OrderCreated, they are joined with what is already in the database to build the rich row. Statuses then update the row (pp. 194-195). It is a secretary who receives notes from three departments: the customer card, the catalogue, the order. She files them, and when the order arrives she writes a complete summary sheet. Searching becomes a glance, instead of questioning three departments. The author owns the eventual consistency: "Our read model is going to be eventually consistent, and, under normal conditions, no one may ever notice." (p. 195).

One question stays open: what happens to the join if OrderCreated arrives before the customer's event? The book describes the setup without covering this case (pp. 194-195).

9A Pact contract is verified on both sides, even for messages

The book sets out four levels of tests: unit, integration, contract and end-to-end (pp. 260-261). Unit and integration tests look like standard Go. The contract is the new part. A contract test checks that an API provider honors what a consumer expects, without running the two together. The book calls it fast: "Despite being rather high on the testing pyramid, these contract tests are expected to run just as fast as unit tests" (p. 262).

The tool is Pact, and the book picks the consumer-driven contract (p. 278). The mechanism comes in two halves (p. 279):

  • the consumer calls a fake provider with its real requests. The tool records a contract file.
  • the provider replays those requests against the real provider, whose dependencies are replaced by doubles, fake versions written for the test. It compares the real responses to what was expected.

One half alone proves nothing. Picture a restaurant writing on an order slip what it really uses from the delivery, say five kilos of red tomatoes, without insisting on a precise variety. The supplier checks that it can deliver exactly that. Without the slip, the supplier checks in a vacuum, and without the check, the slip is only a wish. The expectations cover only what the consumer uses: a subset of routes and fields (p. 279). In the book's REST example, a web interface calls the baskets API. The contract announces adding a negative quantity and expects a 400 code. For the response, it checks only the message field, with a matcher: a rule that accepts any value of the same type rather than an exact value. The real response contains other fields (p. 284).

The provider builds the starting state with StateHandlers. This table maps the name of a state ("a basket exists") to a function that sets up that state in the doubles. The book warns: "Using provider states will require communication and collaboration between teams." (p. 286). A Pact Broker, a server that keeps the contracts and verification results, can share them in CI (pp. 280-282).

The chapter's real novelty is Pact for asynchronous messages, not just REST. There is no request, hence no fake provider. The consumer declares the message it expects (ExpectsToReceive, WithJSONContent) and a ConsumedBy that passes it to its real handlers, with doubles behind them (pp. 287-289). On the producer side, the verifier reads each message described in the contract ("a StoreCreated message"), and the book writes a handler for each description. That handler sets up just enough of the module to produce the event, for example by calling CreateStore(). A fake publisher keeps the last message produced, which the verifier compares to the contract (pp. 290-292). A limit is admitted: "Other formats could be used but the Pact tools support JSON the best and the matchers only work with JSON." (p. 290). The message contract is therefore in JSON, while the real transport uses protobuf, the compact binary format of the book's messages.

The two cases side by side.

RESTMessages
What the consumer declaresIts real requests, played against a fake providerThe expected message (ExpectsToReceive, WithJSONContent), passed to its real handlers
What the provider doesReplays the requests against the real providerOne handler per description sets up just enough of the module to produce the event
What is comparedThe real response to what was expectedThe last message kept by a fake publisher to what was expected
FormatJSONJSON, while the real transport uses protobuf (p. 290)

The book saves end-to-end tests for the end. They are written in Gherkin, a language of readable scenarios ("Given… When… Then…"), run by godog, the official Cucumber library for Go, which ties each sentence to a Go function (pp. 294-295). The author advises restraint: you start with the critical business flows (p. 295). The chapter never names the saga or the outbox: it tests neither.

My honest take

Two questions mattered to me. Does this book teach me something my blog doesn't already say? Yes, on four precise topics. Are its mechanisms reliable as they stand? Half: the substance is good, the safeguards are missing in places. I read it as a refactoring build told by someone who prettifies nothing, and that is its charm.

Its strength: it shows the complete path. A working monolith becomes, step by step, an asynchronous system that still works, and each step is a folder in the public repository. The orchestrated saga keeps its state in the database. The inbox and the outbox live in a transaction per request. Snapshots are versioned, and contract tests cover the messages. The author even admits his shortcuts (p. 84).

Two reservations. Idempotence fits on one page (pp. 149-150), with no idempotency key (an identifier attached to the request, to recognize a replay) and no partial errors: if that's your topic, read my articles instead. And the saga assumes the rollbacks succeed (p. 205), with no maximum duration and no retry: you'll add yourself what makes it workable at night.

My advice: read pages 131 to 139 of chapter 5 first (publication and snapshots). Follow with chapters 8 and 9 (the saga, the transaction, the outbox). Finish with chapter 10 from page 277, for the contract tests. Skim chapter 3, which is long. Chapter 11 (AWS and Terraform) is another book for a web developer. Chapter 12 is worth your time for observability (pp. 325-345). If event sourcing is new to you, go through my notes Exploring CQRS and Event Sourcing and Understanding Eventsourcing first.

Odilon

Still valid in 2026?

The patterns last: transaction per request, inbox and outbox, saga with persisted state, versioned snapshots, Pact contracts. The public repository compiles too: go build ./... passes in Docker with Go 1.26 on the folders of chapters 5, 8, 9 and 12. Seven points to watch:

Who is it for?

Read it if

  • you already know CQRS and event sourcing in Go, and you want to see what comes next: the saga, the outbox, the inbox.
  • you are looking for a complete example of refactoring a modular monolith toward asynchronous exchanges.
  • you want to write contract tests for messages, not only for REST.
  • you are comfortable with intermediate Go and have a local Docker to follow the repository.

Skip it if

  • you are discovering event sourcing: start with a foundations book.
  • idempotence and partial errors are your main topic.
  • you expect to deploy on AWS: chapter 11 stands apart and is outside this note.

Going further

Note based on: chapters 1 to 10 and chapter 12 read in full, chapter 11 only skimmed. The "My reading" boxes and the checks in the repository are mine, not the book's.

Comments (0)

Browse the whole library

More book notes coming: one book at a time, the marrow only.