FetchDecodeBenchmark and the JVM reader cost

What this module is responsible for

This module provides a performance measurement suite to resolve a specific architectural conflict: whether using a shared, multiplatform decoder for the FETCH response imposes a performance penalty on JVM clients compared to a specialized, JVM-only reader.

Diagram

The Fetch decoder dilemma

The architectural conflict arises from the need for checksum verification. Because the broker streams segment bytes to the socket without looking at them (zero-copy), the client is the only party that can verify the data. This creates a tension between two approaches: a ResponseReader optimized for the JVM and a ResponseDecoder designed for multiplatform compatibility. As noted in FetchDecodeBenchmark.kt:19-31, the decision to merge these paths was deferred until measurements could prove whether the shared decoder's abstraction would cost the JVM reader performance.

ResponseReader vs ResponseDecoder

The benchmark compares two distinct paths:

• reader(): Uses ResponseReader.fetch over ByteBuffer, which is a JVM-only implementation (FetchDecodeBenchmark.kt:74).
• decoder(): Uses ResponseDecoder.fetch over ByteArray and index arithmetic, which is shared with Kotlin/Native (FetchDecodeBenchmark.kt:24-25).

The core of the comparison is whether the CRC32C intrinsic on the JVM is efficient enough to offset the overhead of the shared decoder's abstraction.

RecordSize and throughput scaling

The recordSize parameter is used to isolate the cost of bookkeeping versus the cost of checksumming. The benchmark tests three specific sizes:

CorruptRecordException and offset mapping

When a checksum mismatch is detected, the protocol-level CorruptRecordException contains an offset relative to the start of the fetch. The ResponseReader is responsible for translating this into a client-level exception. Specifically, it catches the protocol exception and re-throws a client-side CorruptRecordException where the offset is adjusted to be relative to the fetch, ensuring the caller receives a consistent API (ResponseReader.kt:93-99).

Truncated response handling

The decoder must distinguish between different types of incomplete data to prevent false alarms. The fetch logic handles two specific edge cases:

1. Truncated Payload: If the size of a record is greater than the remaining bytes in the buffer, it returns a FetchResponse with truncated = true and the truncatedRecordBytes set to the expected size (Responses.kt:129-137).
2. Truncated Header: If the response ends while reading a record header (fewer bytes left than Protocol.RECORD_HEADER_BYTES), the response is marked as truncated, but the truncatedRecordBytes remains zero because the size could not be determined (Responses.kt:152).

Key files

Behaviour that surprises

• ResponseDecoder.fetch uses a check(promised == reader.remaining) to ensure the frame length matches the promised payload, which serves as a consistency check for the log state (Responses.kt:114).
• The ResponseReader.fetch method performs a translation of exceptions rather than a direct propagation to maintain the library's public API contract (ResponseReader.kt:94-99).