State as a fold over the log

Diagram

The Projection lifecycle

The projection transitions from a historical reconstruction to a live stream through two distinct phases. First, it performs a replay() from the Earliest position to catch up with history, a process that ends when the stream reaches the high watermark as it was when the service started (Main.kt:98-104). Once the replay is complete, the service enters the follow() phase, which does not end, allowing the view to stay current (Main.kt:108-128). The seamless transition between these two phases is achieved via RecordBatch.nextOffset, which ensures the follow starts exactly where the replay stopped, preventing gaps or double-counting (Main.kt:36-39).

The Projection state and Progress

The projection is designed such that the service stores nothing locally; its state is purely a functional fold over the log (Main.kt:28-30). To avoid silent corruption, the service deliberately does not persist its position; persisting a position without the corresponding state would result in a service that resumes at a late offset with an empty view (Main.kt:40-42). The current status of the service is exposed via ProjectionStats, which includes:

(Main.kt:176-188)

Rebuilt view and the replay guarantee

When a service restarts, it must rebuild its entire state from the log. The rebuilt mode in the testing suite asserts that a restarted projection must return at least as many events as it had before the restart (check.py:193-207). A failure is triggered if the applied count is less than the previous state or if the fromReplay count does not match the expected history, ensuring that the view is not left incomplete (check.py:209-211).

Key files

Behaviour that surprises

• Non-persistence of position: The Projection class does not persist its position because doing so without the state would lead to silent corruption (Main.kt:40-42).
• The replayComplete signal: The transition from history to live data is signaled by the replayComplete flag, which is set only after the replay flow completes (Main.kt:105).
• At-least-once semantics: Because the position is moved only after the consumer handles a batch, a crash between handling and saving will cause the batch to be replayed upon restart (README.md:35-37).