Atomic offset persistence

Diagram

FileOffsetStore

The mechanism for persisting consumer positions using atomic file moves to prevent corruption during crashes. As described in FileOffsetStore.kt:23-25, a position saved halfway is considered worse than no position at all, as a truncated offset like "1" parsed from "12" would cause silent replays. To prevent this, the save method writes to a temporary file and uses StandardCopyOption.ATOMIC_MOVE to ensure the offset is updated completely or not at all FileOffsetStore.kt:55-57.

The write-ahead-log and SegmentMode

The system provides two distinct write paths for log segments, which can be selected via the SegmentMode enum LogSegment.kt:25:

The MAPPED mode is the default and is chosen based on performance measurements LogSegment.kt:19-20. While MAPPED can be faster, it can suffer from performance degradation when the writer outruns the OS writeback, a phenomenon explored in SustainedWriteProbe.kt:18-19.

Recovery after a crash

When a LogSegment is opened, it must undergo a recovery process to rebuild the SparseOffsetIndex by walking the record headers LogSegment.kt:253-254. The recovery mechanism reads headers in chunks using a RECOVERY_BUFFER to avoid excessive syscalls LogSegment.kt:327-330. Because recovery relies on the length prefix, a record whose declared length extends beyond the file boundary is discarded, and the segment is truncated to the last intact boundary LogSegment.kt:302-304.

Corruption detection and checksums

To address the risk where a torn write inside a record body is not detectable by length prefixes alone, the system implements checksums (M-60) LogSegment.kt:258-259. During recovery, the verify function checks the record's body against its stored CRC LogSegment.kt:405. If a mismatch is detected, the recovery process stops to prevent the system from treating garbage as valid data LogSegment.kt:364-366.

CorruptionTest

The CorruptionTest class validates that the system correctly identifies and reacts to data corruption CorruptionTest.kt:23. Key test cases include:

• Bit-flipping: Ensuring a single bit flip in a record body is caught CorruptionTest.kt:50-51.
• Torn bodies: Verifying that even if a length prefix survives, a modified body is detected CorruptionTest.kt:108-109.
• Recovery boundaries: Ensuring that once a bad record is encountered, the system stops and does not allow subsequent records to be incorrectly processed CorruptionTest.kt:64-65.

Key files

Behaviour that surprising

• The LogSegment.open function performs a recover operation that skips record bodies, only reading headers to rebuild the index LogSegment.kt:319.
• In FileOffsetStore, the save operation is designed to be "all or nothing" by using a temporary file and an atomic move to prevent partial writes of the offset FileOffsetStore.kt:55-57.
• The SustainedWriteProbe is designed to intentionally saturate the OS writeback to observe the performance degradation of the MAPPED mode SustainedWriteProbe.kt:32-33.