Block refcounting (atomic refs + condemned flag) lets live queries
survive concurrent compaction and retention. Levelled compactor merges
2h->8h->32h blocks via raw chunk passthrough. Retention drops blocks
older than the configured window. Background goroutine drives
flush/compact/retain cycles; exported RunCompaction/ApplyRetention allow
deterministic test control via injectable clock.
Soak test: 10k series x 48h simulated at 15s intervals (115M samples).
Validates flat memory (158 MiB peak), bounded disk (13 blocks), and zero
query errors during compaction.
All existing tests refactored to table-driven with uniform assertions.
Label matchers (equal, not-equal, regexp, not-regexp) in labels/,
postings intersection/union/difference in internal/postings/,
LabelValues and AllPostings on index, block, and head readers.
ingot.go wired to real DB backed by head + block readers. Querier
snapshots overlapping blocks, Select resolves matchers to postings,
merged iterator chains blocks then head with block-wins dedup.
Oracle tests compare every query against a naive []sample reference
across head-only, block-only, head+block merge seam, multiple blocks,
all four matcher types, and time range filtering.
Flush sealed head chunks to ULID-named block directories on disk. Each
block contains CRC'd chunk segment files (mmap'd for reads), a binary
index (symbol table, series, postings with TOC), and a meta.json written
last as the immutability gate. WAL is truncated after block fsync,
preserving the crash-safety ording invariant.
Head connects chunkenc and WAL: label-to-ref resolution with striped
concurrent maps, active Gorilla chunk per series sealing at 120 samples,
and an Appender that buffers samples, writes WAL on commit, then applies
to head. OOO rejection checks both committed and batch state. WAL replay
on Open rebuilds the full in-memory state. Exported
ChunkAppender/ChunkIterator interfaces from chunkenc.
david