Why architecture matters here
The architectural argument for event sourcing starts where CRUD ends. An UPDATE destroys information: the fact that the order was placed, then modified, then partially refunded collapses into whatever columns look like now. Whole classes of requirements — audit ('who changed what when'), temporal queries ('what did the account look like on March 3rd'), retroactive fixes ('replay with the corrected fee logic'), debugging ('what sequence of operations produced this corrupt state') — become either impossible or bolt-on shadows (audit tables that drift from truth). When the events are the store, those capabilities are queries, not projects.
The single-writer actor model matters just as much as the log. Concurrency bugs in entity logic — two requests reading balance 100 and both withdrawing 80 — are eliminated structurally: sharding guarantees one actor instance per entity cluster-wide, the mailbox serializes commands, and state lives in memory between them. No optimistic-lock retry loops, no SELECT FOR UPDATE contention; command handling is a pure function call at memory speed, with the journal append as the only IO. This is the shape that lets a modest cluster run tens of millions of live entities with per-command latencies dominated by one database write.
The costs are real and architectural, not incidental. Events are immutable contracts: the event you persist today will be replayed by code five years from now, so serialization and schema evolution (adapters upcasting old versions) are core engineering, not plumbing. Reads move elsewhere: folding events answers 'this entity's state', not 'all orders over ₹10k this month' — projections must build every query shape you need, and they lag the write side by design. Teams that adopt the pattern for entities with genuine lifecycle and audit needs — accounts, orders, devices, games — and keep reference data in ordinary tables, get the payoff; teams that event-source their country-code lookup learn the ceremony has a price.
The architecture: every piece explained
Top row: the write path. A command arrives at the sharded entity (a typed message: Withdraw(amount, replyTo)). The EventSourcedBehavior's command handler sees current in-memory state and the command, and returns an Effect: persist one or more events, reply, or reject — validation, invariants, and decisions all happen here, and only here. Persisted events go to the journal plugin: an append of (persistenceId, sequenceNr, payload, metadata) with per-entity sequence numbers providing optimistic-integrity (a duplicate writer — the split-brain scenario — fails on sequence collision rather than corrupting history). Only after the journal confirms does the event handler apply the event to state and any thenReply side effects run — the actor never lies about durability.
Middle row: staying fast and current. The event handler is the fold: State × Event → State, total and side-effect-free, because it runs both live and during replay. Snapshots persist the folded state every N events (or on size triggers); recovery — on node crash, rebalance, or passivation wake-up — loads the latest snapshot and replays only subsequent events, turning what could be a million-event replay into snapshot-plus-fifty. Serialization is where longevity lives: events serialize via schema-evolvable formats (protobuf, Avro, Jackson-CBOR with explicit versioning), and event adapters upcast stored v1 payloads to the v3 shapes current handlers expect — the mechanism that lets five-year-old history replay through today's code.
Bottom rows: the read side and the topology. Projections consume the journal's event stream by tag or slice (eventsByTag / eventsBySlices), tracking offsets durably, and build read models — SQL tables shaped for queries, Elasticsearch indexes, or outbound Kafka topics for other services. Delivery is at-least-once with offset commits, so projection handlers are written idempotently (upserts keyed by entity + sequenceNr); grouped and exactly-once flavors exist for JDBC targets. Sharded entities complete the picture: Cluster Sharding places each persistenceId on exactly one node, passivates idle entities, and recovers them on demand — the single-writer guarantee that makes the whole model sound. The ops strip: journal latency and error monitoring (it is your system of record), replay-duration percentiles (your real recovery SLO), and an event schema registry with review rules, because an event, once written, is an API you shipped to your own future.
End-to-end flow
Follow money through the system. Withdraw(80) for account A-9931 arrives at the API, which sends it into the cluster; sharding routes to the entity's home node, wakes the passivated actor, and recovery runs: latest snapshot (state at seq 4,180) plus 23 events replayed — 40ms, then the actor is live with balance 100. The command handler validates (sufficient funds, account open, daily limit) and emits Withdrawn(80, txnId); the Cassandra journal appends it at seq 4,204 in ~3ms; the event handler folds balance to 20; the reply confirms. A second withdraw racing the first simply queues in the mailbox and is rejected by the same validation against the updated state — the concurrency bug that plagues read-modify-write systems cannot be expressed here.
Downstream, the projection tier is already working. The account-events projection (running as sharded daemon processes across the cluster) receives Withdrawn by slice, upserts the account-balances read table and inserts into the transactions listing table (idempotent on (persistenceId, seqNr)), and commits its offset. A second projection publishes an integration event to Kafka for the fraud service. The mobile app's balance screen reads the projection — perhaps 200ms behind the write, which the UX absorbs by showing the command's synchronous reply immediately.
Now the interesting days. A node dies: sharding reallocates its entities; each wakes on first command via snapshot+replay; the journal was always the truth, so nothing was lost — the incident is a latency blip. An event schema evolves: Withdrawn v2 adds a channel field; the adapter upcasts v1 events with channel=UNKNOWN during replay; old history flows through new logic untouched. A projection bug (wrong fee categorization) is found: the fix is deployed and the projection's offset is rewound — the read table rebuilds itself from history over an hour, while the write side never blinked. Finally the auditor asks for account A-9931's complete history with reasons: it is a journal query, delivered in minutes, because the system never knew how to forget.