dev/projection

Diagram

Projection

The core in-memory state machine that derives UserView from the event log. The Projection class manages a ConcurrentHashMap of users and uses AtomicLong to track applied and skipped events View.kt:39-43. It processes Event objects by updating user-specific statistics like event counts and action frequencies View.kt:55-67.

More: Projection

The build lifecycle

The transition from the replay stage to the follow stage to ensure a gapless view. The build function first executes a replay from StartPosition.Earliest to catch up with history Main.kt:98. Once the replay completes, the progress.replayComplete flag is set to true Main.kt:105, and the system transitions to a follow stage for each partition Main.kt:112-130. This ensures that the follow starts exactly at the nextOffset reported by the replay Main.kt:117-119.

More: The build lifecycle

Progress

Monitoring metrics including replay/follow counts, rebuild counts, and partition positions. The Progress class tracks the number of events from replay and follow stages using AtomicLong Main.kt:146-147, and maintains a map of current partition positions Main.kt:149.

Recovery after a broker failure

The mechanism for rebuilding the entire state from the beginning of the log when the connection is lost. If an exception occurs during the build loop, the catch block increments the rebuild count and resets the live and replayComplete flags Main.kt:135-140. The entire process then restarts, re-reading the log from the beginning to ensure state consistency Main.kt:87-143.

Key files

Behaviour that should surprise

• The Projection class does not persist its state to disk; instead, it relies on the fact that it can rebuild the entire state by replaying the log from the beginning View.kt:25-32.
• If a record cannot be decoded, the apply function increments the skipped counter but continues processing the log rather than stopping View.kt:49-53.
• The follow stage uses a separate CoroutineScope for each partition to allow concurrent consumption of partition streams Main.kt:111-130.