Metrics reporting and retention

Diagram

Metrics snapshotting and rate reporting

The Metrics class provides a snapshot of the current state of the Broker, which is used to calculate performance. Instead of printing cumulative counters—which would require the operator to perform manual differentiation during an incident—the reportMetrics function transforms these raw values into rates (e.g., records per second) by comparing the current snapshot against a previous one over a specific time interval Main.kt:119-123. This interval is configurable via metricsIntervalMillis in the BooblikConfig BooblikConfig.kt:47.

The applyRetention timer

The broker itself is designed to be stateless regarding time, allowing tests to advance time manually by calling broker.applyRetention directly Main.kt:131-133. The actual temporal logic is encapsulated in a dedicated background coroutine launched in Main.kt:86 Main.kt:86. This loop uses delay(config.retentionCheckMillis) to determine when to trigger the retention logic Main.kt:141, ensuring that the "when" is decoupled from the "how" of the broker's internal logic Main.kt:136-144.

Retention by size and age

Retention is governed by two optional parameters defined in BooblikConfig BooblikConfig.kt:42-43:

When applyRetention is called, the broker removes segments that exceed these thresholds Main.kt:142.

Shutdown sequence and data integrity

To prevent data loss, the application follows a strict, layered shutdown sequence within the Runtime shutdown hook Main.kt:91-103. The order is critical:

1. The BooblikServer is closed first Main.kt:94.
2. Background coroutines (metrics and retention) are cancelled via background.cancel() to stop any pending tasks Main.kt:95.
3. The broker is closed last Main.kt:98. This ensures that the broker's writers are still active while the server is shutting down, allowing any accepted batches to reach the disk before the underlying log is closed Main.kt:97-98.

Key files

Behaviour that surprises

• Decoupled Time: The Broker does not have its own clock; it relies on an external caller (like the applyRetention loop in Main.kt) to tell it when to perform maintenance Main.kt:131-133.
• Shutdown Order: The broker is closed after the background coroutines are cancelled to ensure that any data currently being processed by the server can be flushed to the log before the log itself is closed Main.kt:95-98.
• Rate-based Metrics: reportMetrics specifically avoids printing cumulative counters to prevent the need for manual differentiation during incident response Main.kt:110-111.