Add PCRE_CONFIG and PCRE2_CONFIG variables to allow the user to
configure path of pcre-config or pcre2-config instead of using the one
in his path.
This is particulary useful when cross-compiling.
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Released version 1.9-dev3 with the following main changes :
- BUG/MINOR: h1: don't consider the status for each header
- MINOR: h1: report in the h1m struct if the HTTP version is 1.1 or above
- MINOR: h1: parse the Connection header field
- DOC: Fix typos in lua documentation
- MINOR: h1: Add H1_MF_XFER_LEN flag
- MINOR: http: add http_hdr_del() to remove a header from a list
- MINOR: h1: add headers to the list after controls, not before
- MEDIUM: h1: better handle transfer-encoding vs content-length
- MEDIUM: h1: deduplicate the content-length header
- BUG/MEDIUM: patterns: fix possible double free when reloading a pattern list
- BUG/MEDIUM: h1: Really skip all updates when incomplete messages are parsed
- CLEANUP/CONTRIB: hpack: remove some h1 build warnings
- BUG/MINOR: tools: fix set_net_port() / set_host_port() on IPv4
- BUG/MINOR: cli: make sure the "getsock" command is only called on connections
- MINOR: stktable: provide an unchecked version of stktable_data_ptr()
- MINOR: stream-int: make si_appctx() never fail
- BUILD: ssl_sock: remove build warnings on potential null-derefs
- BUILD: stats: remove build warnings on potential null-derefs
- BUILD: stream: address null-deref build warnings at -Wextra
- BUILD: http: address a couple of null-deref warnings at -Wextra
- BUILD: log: silent build warnings due to unchecked __objt_{server,applet}
- BUILD: dns: fix null-deref build warning at -Wextra
- BUILD: checks: silence a null-deref build warning at -Wextra
- BUILD: connection: silence a couple of null-deref build warnings at -Wextra
- BUILD: backend: fix 3 build warnings related to null-deref at -Wextra
- BUILD: sockpair: silence a build warning at -Wextra
- BUILD: build with -Wextra and sort out certain warnings
- BUG/CRITICAL: hpack: fix improper sign check on the header index value
- BUG/MEDIUM: http: Don't parse chunked body if there is no input data
- DOC: Update configuration doc about the maximum number of stick counters.
- BUG/MEDIUM: process_stream: Don't use si_cs_io_cb() in process_stream().
- MINOR: h2/stream_interface: Reintroduce te wake() method.
- BUG/MEDIUM: h2: Wake the task instead of calling h2_recv()/h2_process().
- BUG/MEDIUM: process_stream(): Don't wake the task if no new data was received.
- MEDIUM: lua: Add stick table support for Lua.
This ads support for accessing stick tables from Lua. The supported
operations are reading general table info, lookup by string/IP key, and
dumping the table.
Similar to "show table", a data filter is available during dump, and as
an improvement over "show table" it's possible to use up to 4 filter
expressions instead of just one (with implicit AND clause binding the
expressions). Dumping with/without filters can take a long time for
large tables, and should be used sparingly.
At the eand of process_stream(), we wake the task if there's something in
the input buffer, after attempting a recv. However this is wrong, and we should
only do so if we received new data. Just check the CF_READ_PARTIAL flag.
This is 1.9-specific and should not be backported.
Instead of using si_cs_io_cb() in process_stream() use si_cs_send/si_cs_recv
instead, as si_cs_io_cb() may lead to process_stream being woken up when it
shouldn't be, and thus timeout would never get triggered.
With recent modifications on the buffers API, when a buffer is released (calling
b_free), we replace it by BUF_NULL where the area pointer is NULL. So many
operations, like b_peek, must be avoided on a released or not allocated
buffer. These changes were mainly made in the commit c9fa048 ("MAJOR: buffer:
finalize buffer detachment").
Since this commit, HAProxy can crash during the body parsing of chunked HTTP
messages because there is no check on the channel's buffer in HTTP analyzers
(http_request_forward_body and http_response_forward_body) nor in H1 functions
reponsible to parse chunked content (h1_skip_chunk_crlf & co). If a stream is
woken up after all input data were forwarded, its input channel's buffer is
released (so set to BUF_NULL). In this case, if we resume the parsing of a
chunk, HAProxy crashes.
To fix this issue, we just skip the parsing of chunks if there is no input data
for the corresponding channel. This is only done if the message state is
strickly lower to HTTP_MSG_ENDING.
Tim Düsterhus found using afl-fuzz that some parts of the HPACK decoder
use incorrect bounds checking which do not catch negative values after
a type cast. The first culprit is hpack_valid_idx() which takes a signed
int and is fed with an unsigned one, but a few others are affected as
well due to being designed to work with an uint16_t as in the table
header, thus not being able to detect the high offset bits, though they
are not exposed if hpack_valid_idx() is fixed.
The impact is that the HPACK decoder can be crashed by an out-of-bounds
read. The only work-around without this patch is to disable H2 in the
configuration.
CVE-2018-14645 was assigned to this bug.
This patch addresses all of these issues at once. It must be backported
to 1.8.
We're not far from being able to build with -Wextra -Werror. The
following warnings had to be disabled to enable a clean build at
-Wextra on x86_64 using gcc 4.7, 5.5, 6.4 and 7.3 :
sign-compare, unused-parameter, old-style-declaration,
ignored-qualifiers, clobbered, missing-field-initializers,
implicit-fallthrough
The following extra warnings could be added without side effects :
type-limits, shift-negative-value, shift-overflow=2 duplicated-cond,
null-dereference
As a result, -Wextra was enabled by default, hoping it will help catch
issues over the long term. If new undesired warnings pop up, it's easy
to disable them using the nowarn call.
An invalid null-deref warning is emitted because cmsg is not checked,
though it definitely is valid given the test performed 10 lines above,
but the compiler cannot necessarily guess this. Adding a null test to
the problematic condition is enough to get rid of it and cheap enough.
Like for the other checks, the type is being tested just before calling
objt_{server,dns_srvrq}() so let's use the unguarded version instead to
silence the warning.
When building with -Wnull-dereferences, gcc sees some cases where a
pointer is dereferenced after a check may set it to null. While all of
these are already guarded by either a preliminary test or the code's
construction (eg: listeners code being called only on listeners), it
cannot be blamed for not "seeing" this, so better use the unguarded
calls everywhere this happens, particularly after checks. This is a
step towards building with -Wextra.
Callers of si_appctx() always use the result without checking it because
they know by construction that it's valid. This results in unchecked null
pointer warnings at -Wextra, so let's remove this test and make it clear
that it's up to the caller to check validity first.
stktable_data_ptr() currently performs null pointer checks but most
callers don't check the result since they know by construction that
it cannot be null. This causes valid warnings when building with
-Wextra which are worth addressing since it will result in better
code. Let's provide an unguarded version of this function for use
where the check is known to be useless and untested.
Theorically nothing would prevent a front applet form connecting to a stats
socket, and if a "getsock" command was issued, it would cause a crash. Right
now nothing in the code does this so in its current form there is no impact.
It may or may not be backported to 1.8.
These two functions were apparently written on the same model as their
parents when added by commit 11bcb6c4f ("[MEDIUM] IPv6 support for syslog")
except that they perform an assignment instead of a return, and as a
result fall through the next case where the assigned value may possibly
be partially overwritten. At least under Linux the port offset is the
same in both sockaddr_in and sockaddr_in6 so the value is written twice
without side effects.
This needs to be backported as far as 1.5.
In h1_headers_to_hdr_list, when an incomplete message is parsed, all updates
must be skipped until the end of the message is found. Then the parsing is
restarted from the beginning. But not all updates were skipped, leading to
invalid rewritting or segfault.
No backport is needed.
A null pointer assignment was missing after free() in function
pat_ref_reload() which can lead to segfault.
This bug was introduced in commit b5997f7 ("MAJOR: threads/map: Make
acls/maps thread safe").
Must be backported to 1.8.
Just like we used to do in proto_http, we now check that each and every
occurrence of the content-length header field and each of its values are
exactly identical, and we normalize the header to return the last value
of the first header with spaces trimmed.
The transfer-encoding header processing was a bit lenient in this part
because it was made to read messages already validated by haproxy. We
absolutely need to reinstate the strict processing defined in RFC7230
as is currently being done in proto_http.c. That is, transfer-encoding
presence alone is enough to cancel content-length, and must be
terminated by the "chunked" token, except in the response where we
can fall back to the close mode if it's not last.
For this we now use a specific parsing function which updates the
flags and we introduce a new flag H1_MF_XFER_ENC indicating that the
transfer-encoding header is present.
Last, if such a header is found, we delete all content-length header
fields found in the message.
This flag is usefull to handle cases where there is no body, regardless of CL or
TE headers (for instance, responses to HEAD requests). It will not be set by the
parser itself.
The new function h1_parse_connection_header() is called when facing a
connection header in the generic parser, and it will set up to 3 bits
in h1m->flags indicating if at least one "close", "keep-alive" or "upgrade"
tokens was seen.
This will be needed for the mux to know how to process the Connection
header, and will save it from having to re-parse the request line since
it's captured on the fly.
While it was possible to consider the status before parsing response
headers, it's wrong to do it for request headers and could lead to
random behaviours due to this status matching other fields instead.
Additionnally there is little to no value in doing this for each and
every new header field. It's much better to reset the content-length
at once in the callerwhen seeing such statuses (which currently is only
the H2 mux).
No backport is needed, this is purely 1.9.
Released version 1.9-dev2 with the following main changes :
- BUG/MINOR: buffers: Fix b_slow_realign when a buffer is realign without output
- BUG/MEDIUM: threads: fix the no-thread case after the change to the sync point
- BUG/MEDIUM: servers: check the queues once enabling a server
- BUG/MEDIUM: queue: prevent a backup server from draining the proxy's connections
- MEDIUM: mux: Remove const on the buffer in mux->snd_buf()
- CLEANUP: backend: Move mux install to call it at only one place
- MINOR: conn_stream: add an tx buffer to the conn_stream
- MINOR: conn_stream: add cs_send() as a default snd_buf() function
- MINOR: backend: Try to find the best mux for outgoing connections
- MEDIUM: backend: don't rely on mux_pt_ops in connect_server()
- MINOR: mux: Add info about the supported side in alpn_mux_list structure
- MINOR: mux: Unlink ALPN and multiplexers to rather speak of mux protocols
- MINOR: mux: Print the list of existing mux protocols during HA startup
- MEDIUM: checks: use the new rendez-vous point to spread check result
- MEDIUM: haproxy: don't use sync_poll_loop() anymore in the main loop
- MINOR: threads: remove the previous synchronization point
- MAJOR: server: make server state changes synchronous again
- CLEANUP: server: remove the update list and the update lock
- BUG/MINOR: threads: Remove the unexisting lock label "UPDATED_SERVERS_LOCK"
- BUG/MEDIUM: stream_int: Don't check CO_FL_SOCK_RD_SH flag to trigger cs receive
- MINOR: mux: Change get_mux_proto to get an ist as parameter
- MINOR: mux: Improve the message with the list of existing mux protocols
- MINOR: mux/frontend: Add 'proto' keyword to force the mux protocol
- MINOR: mux/server: Add 'proto' keyword to force the multiplexer's protocol
- MEDIUM: mux: Use the mux protocol specified on bind/server lines
- BUG/MEDIUM: connection/mux: take care of serverless proxies
- MINOR: queue: make sure the pendconn is released before logging
- MINOR: stream: rename {srv,prx}_queue_size to *_queue_pos
- MINOR: queue: store the queue index in the stream when enqueuing
- MINOR: queue: replace the linked list with a tree
- MEDIUM: add set-priority-class and set-priority-offset
- MEDIUM: queue: adjust position based on priority-class and priority-offset
- DOC: update the roadmap about priority queues
- BUG/MINOR: ssl: empty connections reported as errors.
- MINOR: connections: Make rcv_buf mandatory and nuke cs_recv().
- MINOR: connections: Move rxbuf from the conn_stream to the h2s.
- MINOR: connections: Get rid of txbuf.
- MINOR: tasks: Allow tasklet_wakeup() to wakeup a task.
- MINOR: connections/mux: Add the wait reason(s) to wait_list.
- MINOR: stream_interface: Don't use si_cs_send() as a task handler.
- MINOR: stream_interface: Give stream_interface its own wait_list.
- MINOR: mux_h2: Don't use h2_send() as a callback.
- MINOR: checks: Add event_srv_chk_io().
- BUG/MEDIUM: tasks: Don't insert in the global rqueue if nbthread == 1
- BUG/MEDIUM: sessions: Don't use t->state.
- BUG/MEDIUM: ssl: fix missing error loading a keytype cert from a bundle.
- BUG/MEDIUM: ssl: loading dh param from certifile causes unpredictable error.
- BUG/MINOR: map: fix map_regm with backref
- DOC: dns: explain set server ... fqdn requires resolver
- DOC: add documentation for prio_class and prio_offset sample fetches.
- DOC: ssl: Use consistent naming for TLS protocols
- DOC: update the layering design notes
- MINOR: tasks: Don't special-case when nbthreads == 1
- MINOR: fd cache: And the thread_mask with all_threads_mask.
- BUG/MEDIUM: lua: socket timeouts are not applied
- BUG/MINOR: lua: fix extra 500ms added to socket timeouts
- BUG/MEDIUM: server: update our local state before propagating changes
- BUG/MEDIUM: cli/threads: protect all "proxy" commands against concurrent updates
- DOC: server/threads: document which functions need to be called with/without locks
- BUG/MEDIUM: cli/threads: protect some server commands against concurrent operations
- BUG/MEDIUM: streams: Don't forget to remove the si from the wait list.
- BUG/MEDIUM: tasklets: Add the thread as active when waking a tasklet.
- BUG/MEDIUM: stream-int: Check if the conn_stream exist in si_cs_io_cb.
- BUG/MEDIUM: H2: Activate polling after successful h2_snd_buf().
- BUG/MEDIUM: stream_interface: Call the wake callback after sending.
- BUG/MAJOR: queue/threads: make pendconn_redistribute not lock the server
- BUG/MEDIUM: connection: don't forget to always delete the list's head
- BUG/MEDIUM: lb/threads: always properly lock LB algorithms on maintenance operations
- BUG/MEDIUM: check/threads: do not involve the rendez-vous point for status updates
- BUG/MINOR: chunks: do not store -1 into chunk_printf() in case of error
- BUG/MEDIUM: http: don't store exp_replace() result in the trash's length
- BUG/MEDIUM: http: don't store url_decode() result in the samples's length
- BUG/MEDIUM: dns: don't store dns_build_query() result in the trash's length
- BUG/MEDIUM: map: don't store exp_replace() result in the trash's length
- BUG/MEDIUM: connection: don't store recv() result into trash.data
- BUG/MEDIUM: cli/ssl: don't store base64dec() result in the trash's length
- MINOR: chunk: remove impossible tests on negative chunk->data
- MINOR: sample: remove impossible tests on negative smp->data.u.str.data
- DOC: Fix spelling error in configuration doc
- REGTEST/MINOR: Missing mandatory "ignore_unknown_macro".
- REGTEST/MINOR: Add a new class of regression testing files.
- BUG/MEDIUM: unix: provide a ->drain() function
- MINOR: connection: make conn_sock_drain() work for all socket families
- BUG/MINOR: lua: Bad HTTP client request duration.
- REGEST/MINOR: Add reg testing files.
- BUG/MEDIUM: mux_pt: dereference the connection with care in mux_pt_wake()
- REGTEST/MINOR: Add a reg testing file for b406b87 commit.
- BUG/MEDIUM: lua: reset lua transaction between http requests
- MINOR: add be_conn_free sample fetch
- MINOR: Add srv_conn_free sample fetch
- BUG/MEDIUM: hlua: Make sure we drain the output buffer when done.
- MINOR: checks: Call wake_srv_chk() when we can finally send data.
- BUG/MEDIUM: stream_interface: try to call si_cs_send() earlier.
- BUG/MAJOR: thread: lua: Wrong SSL context initialization.
- REGTEST/MINOR: Add a reg testing file for 3e60b11.
- BUG/MEDIUM: hlua: Don't call RESET_SAFE_LJMP if SET_SAFE_LJMP returns 0.
- REGTEST/MINOR: lua: Add reg testing files for 70d318c.
- BUG/MEDIUM: dns/server: fix incomatibility between SRV resolution and server state file
- BUG/MEDIUM: ECC cert should work with TLS < v1.2 and openssl >= 1.1.1
- MINOR: tools: make date2str_log() take some consts
- MINOR: thread: implement HA_ATOMIC_XADD()
- BUG/MINOR: stream: use atomic increments for the request counter
- BUG/MEDIUM: session: fix reporting of handshake processing time in the logs
- BUG/MEDIUM: h2: fix risk of memory leak on malformated wrapped frames
- BUG/MAJOR: buffer: fix incorrect check in __b_putblk()
- MINOR: log: move the log code to sess_build_logline() to add extra arguments
- MINOR: log: make the backend fall back to the frontend when there's no stream
- MINOR: log: make sess_build_logline() not dereference a NULL stream for txn
- MINOR: log: don't unconditionally pick log info from s->logs
- CLEANUP: log: make the low_level lf_{ip,port,text,text_len} functions take consts
- MINOR: log: keep a copy of the backend connection early in sess_build_logline()
- MINOR: log: do not dereference a null stream to access captures
- MINOR: log: be sure not to dereference a null stream for a target
- MINOR: log: don't check the stream-int's conn_retries if the stream is NULL
- MINOR: log: use NULL for the unique_id if there is no stream
- MINOR: log: keep a copy of s->flags early to avoid a dereference
- MINOR: log: use zero as the request counter if there is no stream
- MEDIUM: log: make sess_build_logline() support being called with no stream
- MINOR: log: provide a function to emit a log for a session
- MEDIUM: h2: produce some logs on early errors that prevent streams from being created
- BUG/MINOR: h1: fix buffer shift after realignment
- MINOR: connection: make the initialization more consistent
- MINOR: connection: add new function conn_get_proxy()
- MINOR: connection: add new function conn_is_back()
- MINOR: log: One const should be enough.
- BUG/MINOR: dns: check and link servers' resolvers right after config parsing
- BUG/MINOR: http/threads: atomically increment the error snapshot ID
- MINOR: snapshot: restart on the event ID and not the stream ID
- MINOR: snapshot: split the error snapshots into common and proto-specific parts
- MEDIUM: snapshot: start to reorder the HTTP snapshot output a little bit
- MEDIUM: snapshot: implement a show() callback and use it for HTTP
- MINOR: proxy: add a new generic proxy_capture_error()
- MINOR: http: make the HTTP error capture rely on the generic proxy code
- MINOR: http: remove the pointer to the error snapshot in http_capture_bad_message()
- REORG: cli: move the "show errors" handler from http to proxy
- BUG/MEDIUM: snapshot: take the proxy's lock while dumping errors
- MEDIUM: snapshots: dynamically allocate the snapshots
- MEDIUM: snapshot: merge the captured data after the descriptor
- MEDIUM: mworker: remove register/unregister signal functions
- MEDIUM: mworker: use the haproxy poll loop
- BUG/MINOR: mworker: no need to stop peers for each proxy
- MINOR: mworker: mworker_cleanlisteners() delete the listeners
- MEDIUM: mworker: block SIGCHLD until the master is ready
- MEDIUM: mworker: never block SIG{TERM,INT} during reload
- MEDIUM: startup: unify signal init between daemon and mworker mode
- MINOR: mworker: don't deinit the poller fd when in wait mode
- MEDIUM: mworker: master wait mode use its own initialization
- MEDIUM: mworker: replace the master pipe by socketpairs
- MINOR: mworker: keep and clean the listeners
- MEDIUM: threads: close the thread-waker pipe during deinit
- MEDIUM: mworker: call per_thread deinit in mworker_reload()
- REORG: http: move the HTTP semantics definitions to http.h/http.c
- REORG: http: move http_get_path() to http.c
- REORG: http: move error codes production and processing to http.c
- REORG: http: move the log encoding tables to log.c
- REORG: http: move some header value processing functions to http.c
- BUG/MAJOR: kqueue: Don't reset the changes number by accident.
- MEDIUM: protocol: use a custom AF_MAX to help protocol parser
- MEDIUM: protocol: sockpair protocol
- TESTS: add a python wrapper for sockpair@
- BUG/MINOR: server: Crash when setting FQDN via CLI.
- BUG/MINOR: h2: report asynchronous end of stream on closed connections
- BUILD: fix build without thread
- BUG/MEDIUM: tasks: Don't forget to decrement task_list_size in tasklet_free().
- MEDIUM: connections: Don't reset the polling flags in conn_fd_handler().
- MEDIUM: connections/mux: Add a recv and a send+recv wait list.
- MEDIUM: connections: Get rid of the recv() method.
- MINOR: h2: Let user of h2_recv() and h2_send() know xfer has been done.
- MEDIUM: h2: always subscribe to receive if allowed.
- MEDIUM: h2: Don't use a wake() method anymore.
- MEDIUM: stream_interface: Make recv() subscribe when more data is needed.
- MINOR: connections: Add a "handle" field to wait_list.
- MEDIUM: mux_h2: Revamp the send path when blocking.
- MEDIUM: stream_interfaces: Starts receiving from the upper layers.
- MINOR: checks: Give checks their own wait_list.
- MINOR: conn_streams: Remove wait_list from conn_streams.
- REORG: h1: create a new h1m_state
- MINOR: h1: add the restart offsets into struct h1m
- MINOR: h1: remove the unused states from h1m_state
- MINOR: h1: provide a distinct init() function for request and response
- MINOR: h1: add a message flag to indicate that a message carries a response
- MINOR: h2: make sure h1m->err_pos field is correct on chunk error
- MINOR: h1: properly pre-initialize err_pos to -2
- MINOR: mux_h2: replace the req,res h1 messages with a single h1 message
- MINOR: h2: pre-initialize h1m->err_pos to -1 on the output path
- MEDIUM: h1: consider err_pos before deciding to accept a header name or not
- MEDIUM: h1: make the parser support a pointer to a start line
- MEDIUM: h1: let the caller pass the initial parser's state
- MINOR: h1: make the message parser support a null <hdr> argument
- MEDIUM: h1: support partial message parsing
- MEDIUM: h1: remove the useless H1_MSG_BODY state
- MINOR: h2: store the HTTP status into the H2S, not the H1M
- MINOR: h1: remove the HTTP status from the H1M struct
- MEDIUM: h1: implement the request parser as well
- MINOR: h1: add H1_MF_TOLOWER to decide when to turn header names to lower case
- MINOR: connection: pass the proxy when creating a connection
- BUG/MEDIUM: h2: Don't forget to empty the wait lists on destroy.
- BUG/MEDIUM: h2: Don't forget to set recv_wait_list to NULL in h2_detach.
- BUG/MAJOR: h2: reset the parser's state on mux buffer full
The h2 parser has this specificity that if it cannot send the headers
frame resulting from the headers it just parsed, it needs to drop it
and parse it again later. Since commit 8852850 ("MEDIUM: h1: let the
caller pass the initial parser's state"), when this happens the parser
remains in the data state and the headers are not parsed again next
time, resulting in a parse error. Let's reset the parser on exit there.
No backport is needed.
If we're detaching the conn_stream, and it was subscribed to be waken up
when more data was available to receive, unsubscribe it.
No backport is needed.
Empty both send_list and fctl_list when destroying the h2 context, so that
if we're freeing the stream after, it doesn't try to remove itself from the
now-deleted list.
No backport is needed.
Till now it was very difficult for a mux to know what proxy it was
working for. Let's pass the proxy when the mux is instanciated at
init() time. It's not yet used but the H1 mux will definitely need
it, just like the H2 mux when dealing with backend connections.
The h1 parser used to systematically turn header field names to lower
case because it was designed for H2. Let's add a flag which is off by
default to condition this behaviour so that when using it from an H1
parser it will not affect the message.
The original H1 request parsing code was reintroduced into the generic
H1 parser so that it can be used regardless of the direction. If the
parser is interrupted and restarts, it makes use of the H1_MF_RESP
flag to decide whether to re-parse a request or a response. While
parsing the request, the method is decoded and set into the start line
structure.
The HTTP status is not relevant to the H1 message but to the H2 stream
itself. It used to be placed there by pure convenience but better move
it before it's too hard to remove.
This state was only a delimiter between headers and body but it now
causes more harm than good because it requires someone to change it.
Since the H1 parser knows if we're in DATA or CHUNK_SIZE, simply let
it set the right next state so that h1m->state constantly matches
what is expected afterwards.
While it was not needed in the H2 mux which was reading full H1 messages
from the channel, it is mandatory for the H1 mux reading contents from
outside to be able to restart on a message. The problem is that the
headers are indexed on the fly, and it's not fun to have to store
everything between calls.
The solution here is to complete the first pass doing a partial restart,
and only once the end of message was found, to start over it again at
once, filling entries. This way there is a bounded number of passes on
the contents and no need to store an intermediary result anymore. Later
this principle could even be used to decide to completely drop an output
buffer to save memory.
