In running Prometheus instances, compressing the records was shown to
reduce disk usage by half while incurring a negligible CPU cost.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
Make WAL live tailer return EOF when the there is a half-written record at the end of the file.
Previously, this would cause an infinite loop as we ignored EOFs when filling the buffer. We now differentiate between EOFs that read >0 bytes, and EOFs that didn't.
Add some more unit tests for tailing a corrupt WAL, and unify interfaces Reader and LiveReader for the purposes of testing.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
Chris Marchbanks
Frederic Branczyk
Tom Wilkie