The build lifecycle

What this module is responsible for

The dev/projection module implements a read model where state is a pure function of the event log. It is responsible for consuming a stream of events from a BooblikSubscriber, building an in-memory view, and providing an HTTP query surface.

Diagram

The replay and follow transition

To ensure the query surface transitions from "building" to "current" without losing or duplicating data, the projection uses a two-phase approach. First, it performs a replay() to consume history. The RecordBatch.nextOffset property is critical here; it reports where each partition stopped during the replay, allowing the subsequent follow() phase to start exactly at that offset Main.kt:137-139. This mechanism ensures that the view catches up to the high watermark without gaps or redundant processing.

The statelessness of Projection

A core design principle is that the projection stores nothing on disk. While a consumer might persist its position, the projection's state is purely in-memory. As noted in the documentation, persisting a position without the corresponding state would lead to silent corruption README.md:141-144. Instead, upon any restart, the service must rebuild its entire state by replaying the log from the beginning to ensure the view is consistent with the truth of the log.

Rebuild verification

The integrity of the rebuild process is verified by ensuring that a restarted service recovers at least the same amount of data it held previously. The check-projection.sh script performs this by capturing the applied count before a restart and asserting that the rebuilt view contains at least that many events check-projection.sh:42-47. This prevents a scenario where a service resumes from a late offset with an empty view, which would result in a "successful" but incorrect state.

At-least-once semantics in checkpointing

The system guarantees at-least-once delivery through the mechanics of checkpointing. The offset is only saved after the collector has successfully handled a batch of records SubscriptionTest.kt:143-172. If an error occurs during the apply phase, the checkpoint is not updated, meaning the next attempt will replay the same batch, ensuring no event is lost at the cost of potential duplicates.

Key files

Behaviour that surprises

• Silent Corruption Risk: If a service were to persist its position without its state, it would result in a silent corruption rather than an optimization README.md:41-42.
• Replay Completion: The replayComplete flag is only set once the replay flow from the subscriber actually completes Main.kt:105.
• Non-monotonicity in follow: While follow ensures the view stays current, the follow phase does not end, unlike replay which terminates once the high watermark is reached Main.kt:35.