Connection and Topic Management

Diagram

BooblikConnection and the Pipelined Request Queue

The BooblikConnection implements a pipelined architecture where many requests can be in flight simultaneously. To achieve this, it uses a ConcurrentLinkedQueue of Pending objects to track requests that have been sent but not yet answered BooblikConnection.kt:60. Because the broker is guaranteed to answer requests in the order they were received, the client uses a FIFO queue to match incoming responses back to their original callers via a correlationId BooblikConnection.kt:34-37. To prevent interleaved writes on the socket, a single writer coroutine drains the outbound channel, ensuring that the wire order matches the request order BooblikConnection.kt:74-83.

Session and the Protocol Loop

The Session class runs the primary protocol loop, which continuously reads frames from a connection and dispatches them for handling Session.kt:50. The lifecycle begins with readFrame, which validates the length prefix to prevent large allocation attacks Session.kt:68-70. Once a frame is read, handle uses a when expression to decode the request; if RequestDecoder.decode returns a DecodeResult.Failed, the session immediately responds with an error to the client Session.kt:81-89.

MetadataRequest and Topic Discovery

MetadataRequest is used to discover the state of topics and partitions. When a request is processed, the broker checks if the requested topics exist in its registry Session.kt:122. If a named topic is not found, the broker does not simply omit it from the response; instead, it fails the entire request with an UNKNOWN_TOPIC_OR_PARTITION error to prevent clients from waiting indefinitely for a topic that does not exist Session.kt:127-130.

ConnectionClosedException and Failure Propagation

When a network error or a fatal protocol error occurs, the fail(cause: Throwable) function is invoked to clean up the connection BooblikConnection.kt:161-163. This function performs several critical tasks:

• It marks the connection as failed.
• It closes the outbound channel.
• It iterates through all pending requests in the queue and completes them exceptionally with a ConnectionClosedException BooblikConnection.kt:166-167.

The Producer-Connection Ownership Model

To maintain the integrity of the pipelined protocol, a Producer must own its Connection. This is because a Producer manages its own pending records and is the only writer to its specific socket README.md:49-53. If a user attempts to use the same Connection directly while a Producer is active, the responses may be mismatched because the broker's responses are matched strictly by order README.md:54-56.

Key files

Behaviour that surprising

• A MetadataRequest that names a non-existent topic will result in an error for the entire request rather than just omitting the missing topic Session.kt:127-130.
• If a Connection fails, all currently waiting Pending requests are immediately failed with a ConnectionClosedException BooblikConnection.kt:167.
• The Session will continue to keep a connection open even if a DecodeResult.Failed occurs, as long as the framing remains intact Session.kt:78-79.