FetchMode

What this module is responsible for

The FetchMode determines the data transfer strategy used by the broker to move log data from the storage layer to the network socket during a FetchRequest. It is a critical performance knob that dictates whether data passes through the JVM heap or stays within the OS page cache.

Diagram

FetchMode

The available strategies are defined in BooblikServer.kt:37-43:

ZERO_COPY mechanics

When FetchMode.ZERO_COPY is selected, the Session utilizes the Connection.transferFrom method to perform a zero-copy transfer Session.kt:221-223. This approach leverages the operating system's ability to move data from the file system cache directly to the network stack, avoiding the overhead of copying bytes into the application's memory space.

HEAP staging and buffer reuse

In HEAP mode, the Session manages a reusable staging buffer to minimize allocation pressure Session.kt:289-294. The lifecycle of this buffer is as follows:

1. Allocation: A buffer is allocated with an initial size of 64 KiB Session.kt:297.
2. Growth: If a request requires more space, the staging function allocates a new ByteBuffer of the required size Session.kt:290.
3. Reuse: The same buffer is cleared and reused for subsequent requests to avoid frequent GC cycles.

Fetch slice lifecycle and retention

To prevent a long-running FetchRequest from blocking log retention, the Session manages data access through a "slice" mechanism Session.kt:206.

• Opening: A slice is opened only after the maxWaitMillis period has passed and the highWatermark is checked Session.kt:188.
• Retention: The slice is held open only during the actual I/O operation (the header and the body) Session.kt:212.
• Closing: The slice is automatically closed via a use block, ensuring that the log segment is not held open longer than necessary, which would otherwise prevent the retention policy from deleting old segments Session.kt:212.

Fetch response integrity and truncation

The server ensures that responses respect the maxBytes constraint and segment boundaries:

• Truncation: If a record is cut off by the maxBytes limit, the Session marks the response as truncated ServerTest.kt:117.
• Segment Boundaries: A fetch operation will stop at a segment boundary; if a request spans multiple segments, the client must perform subsequent fetches to retrieve data from the next segment ServerTest.kt:137.
• Integrity: The maxBytes constraint is applied to the total bytes returned, and the truncated flag is used to signal to the client that the full record was not sent ServerTest.kt:117.

Key files

Behaviour that surprises

• Silent Requests: When AckPolicy.NONE is used in a ProduceRequest, the server does not send any response frame at all, even if the write was successful Session.kt:169.
• High Watermark vs. Log End: A FetchRequest at the current highWatermark is considered a valid "caught-up" state and returns an empty response rather than an error Session.kt:197.
• Slice Timing: The available check for minBytes is performed by opening and immediately closing a slice to avoid holding a segment lock during the awaitRecords wait period Session.kt:276-277.