Ideally haring should be compiled with the same options as haproxy so
that ring headers have the same size (e.g. with/without locks, with/
without lock debugging). But when enabling DEBUG_STRICT, BUG_ON() is
enabled and breaks the build by making references to complain() and
ha_backtrace_to_stderr().
Let's just disable DEBUG_STRICT before opening include files. This is
sufficient to address the problem.
This may be backorted to older versions that include haring.
The ring's offset currently contains a perpetually growing custor which
is the number of bytes written from the start. It's used by readers to
know where to (re)start reading from. It was made absolute because both
the head and the tail can change during writes and we needed a fixed
position to know where the reader was attached. But this is complicated,
error-prone, and limits the ability to reduce the lock's coverage. In
fact what is needed is to know where the reader is currently waiting, if
at all. And this location is exactly where it stored its count, so the
absolute position in the buffer (the seek offset from the first storage
byte) does represent exactly this, as it doesn't move (we don't realign
the buffer), and is stable regardless of how head/tail changes with writes.
This patch modifies this so that the application code now uses this
representation instead. The most noticeable change is the initialization,
where we've kept ~0 as a marker to go to the end, and it's now set to
the tail offset instead of trying to resolve the current write offset
against the current ring's position.
The offset was also used at the end of the consuming loop, to detect
if a new write had happened between the lock being released and taken
again, so as to wake the consumer(s) up again. For this we used to
take a copy of the ring->ofs before unlocking and comparing with the
new value read in the next lock. Since it's not possible to write past
the current reader's location, there's no risk of complete rollover, so
it's sufficient to check if the tail has changed.
Note that the change also has an impact on the haring consumer which
needs to adapt as well. But that's good in fact because it will rely
on one less variable, and will use offsets relative to the buffer's
head, and the change remains backward-compatible.
In case a file-backed ring was not properly synced before being dumped,
the output can look bogus due to the head pointer not being perfectly
up to date. In this case, passing "-r" will make haring automatically
skip entries not starting with a zero, and resynchronize with the rest
of the messages.
This should be backported to 2.6.
Some traces may contain LF characters which are quite cumbersome to
deal with using the common tools. Given that the utility still has
access to the raw traces and knows where the delimiters are, let's
offer the possibility to remap LF characters to a different sequence.
Here we're using CR VT which will have the same visual appearance but
will remain on the same line for grep etc. This behavior is enabled by
the -l option. It's not enabled by default because it's 50% slower due
to content processing.
With the ability to back a memory ring into an mmapped file, it makes
sense to be able to dump these files. That's what this utility does.
The entire ring is dumped to stdout. It's well suited to large dumps,
it converts roughly 6 GB of logs per second.
The utility is really meant for developers at the moment. It might
evolve into a more general tool but at the moment it's still possible
that it might need to be run under gdb to process certain crash dumps.
Also at the moment it must not be used on a ring being actively written
to or it will dump garbage.
The code is made so that we can envision later to attach to a live
ring and dump live contents, but this requires that the utility is
built with the exact same options (threads etc), and that the file
is opened read-write. For now these parts have been commented out,
waiting for a reasonably balanced and non-intrusive solution to be
found (e.g. signals must be intercepted so that the tool cannot
leave the ring with a watcher present).
If it is detected that the memory layout of the ring struct differs,
a warning is emitted. At the end, if an error occurs, a warning is
printed as well (this does happen when the process is not cleanly
stopped, but it indicates the end was reached).
Willy Tarreau