Since last commit, we now have a pointer to the applet in the
applet context. So we don't need the si->release function pointer
anymore, it can be extracted from applet->applet.release. At many
places, the ->release function was still tested for real connections
while it is only limited to applets, so most of them were simply
removed. For the remaining valid uses, a new inline function
si_applet_release() was added to simplify the check and the call.
In preparation for a later move of all the applet context outside of the
stream interface, we'll need to have access to the applet itself from the
context. Let's have a pointer to it inside the context.
si_prepare_embedded() was used both to attach an applet and to detach
anything from a stream interface. Split it into si_prepare_none() to
detach and si_prepare_applet() to attach an applet.
si->conn->target is now assigned from within these two functions instead
of their respective callers.
Since this is the applet context, call it ->appctx to avoid the confusion
with the pointer to the applet. Many places were changed but it's only a
renaming.
The object type was added to "struct appctx". The purpose will be
to identify an appctx when the applet context is detached from the
stream interface. For now, it's still attached, so this patch only
adds the new type and does not replace its use.
In preparation of making the applet context dynamically allocatable,
we create a "struct appctx". Nothing else was changed, it's the same
struct as the one which was in the stream interface.
A long time ago when peers were introduced, there was no applet nor
applet context. Applet contexts were introduced but the peers still
did not make use of them and the "ptr" pointer remains present in
every stream interface in addition to the other contexts.
Simply move this pointer to its own location in the context.
Note that this pointer is still a void* because its type and contents
varies depending on the peers session state. Probably that this could
be cleaned up in the future given that all other contexts already store
much more than a single pointer.
Most of the times, the caller of objt_<type>(ptr) will know that <ptr>
is valid and of the correct type (eg: in an "if" condition). Let's provide
an unsafe variant that does not perform the check again for these usages.
The new functions are called "__objt_<type>".
This is to be more consistent with the other functions. The only
reason why these functions used to return a value was to let the
caller adjust polling by itself, but now their only callers were
the si_shutr()/si_shutw() inline functions. Now these functions
do not depend anymore on the connection.
These connection variant of these functions now call
conn_data_stop_recv()/conn_data_stop_send() before returning order
not to require a return code anymore. The applet version does not
need this at all.
These functions induce a lot of ifs everywhere because they consider two
different cases, one which is where the connection exists and has a file
descriptor, and the other one which is the default case where at most an
applet has to be notified.
Let's have them in si_ops and automatically decide which one to use.
The connection shutdown sequence has been slightly simplified, and we
now clear the flags at the end.
Also we remove SHUTR_NOW after a shutw with nolinger, as it's cleaner
not to keep it.
Now that applets work like real connections, there is no reason for
them to evade the response analysers. The stats applet emits valid
HTTP responses, it can flow through the HTTP response analyser just
fine. This now allows http-response/rsprep/rspadd rules to be applied
on top of stats. Cookie insertion does nothing since applets are not
servers and thus do not have a cookie. We can imagine compression to be
applied later if the stats output is emitted in chunks and in HTTP/1.1.
A minor visible effect of this change is that there is no more "-1" in
the timers presented in the logs when viewing the stats, all timers are
real.
Instead of having applets bypass the whole connection process, we now
follow the common path through sess_prepare_conn_req(). It is this
function which detects an applet an sets the output state so SI_ST_EST
instead of initiating a connection to a server. It is made possible
because we now have s->target pointing to the applet.
We used to rely on the stream interface's target to detect an applet
from within the session while trying to process the connection request,
but this is incorrect, as this target is the one currently connected
and not the next one to process. This will make a difference when we
later support keep-alive. The only "official" value indicating where
we want to connect is the session's target, which can be :
- &applet : connect to this applet
- NULL : connect using the normal LB algos
- anything else : direct connection to some entity
Since we're interested in detecting the specific case of applets, it's
OK to make use of s->target then.
Also, applets are being isolated from connections, and as such there
will not be any ->connect method available when an applet is running,
so we can get rid of this test as well.
At the moment, stats require some preliminary storage just to store
some flags and codes that are parsed very early and used later. In
fact that doesn't make much sense and makes it very hard to allocate
the applet dynamically.
This patch changes this. Now stats_check_uri() only checks for the
validity of the request and the fact that it matches the stats uri.
It's handle_stats() which parses it. It makes more sense because
handle_stats() used to already perform some preliminary processing
such as verifying that POST contents are not missing, etc...
There is only one minor hiccup in doing so : the reqrep rules might
be processed in between. This has been addressed by moving
http_handle_stats() just after stats_check_uri() and setting s->target
at the same time. Now that s->target is totally operational, it's used
to mark the current request as being targetted at the stats, and this
information is used after the request processing to remove the HTTP
analysers and only let the applet handle the request.
Thus we guarantee that the storage for the applet is filled with the
relevant information and not overwritten when we switch to the applet.
There is a big trouble with the way POST is handled for the admin
stats page. The POST parameters are extracted from some http-request
rules, and if not round they return zero hoping for being called again
when more data passes. This results in the HTTP analyser being called
several times and all the rules prior to the stats being executed
multiple times as well. That includes rewrite rules.
So instead of doing this, we now move all the processing of the stats
into the stats applet.
That way we just set the stats applet in the HTTP analyser when a stats
request is detected, and the applet takes the time it needs to read the
arguments and respond. We could even imagine improving the applet to
support requests larger than a single buffer.
The code was almost only moved and minimally changed. Several new HTTP
states were added to the stats applet to emit headers, redirects and
to read POST. It was necessary to do this because the headers sent
depend on the parsing of the POST request. In the end it's beneficial
because we removed two stream_int_retnclose() calls.
In preparation for moving the POST processing to the applet, we first
add new states to the HTTP I/O handler. Till now st0 was only 0/1 for
start/end. We now replace it with an enum.
Currently a connection is required on the remote side to emit a proxy
protocol header line. Let's support NULL addresses to emit an UNKNOWN
tag as well.
These two fetch methods predate the samples and used to store the
destination address into the server-facing connection's address field
because we had no other place at this time.
This will become problematic with the current connection changes, so
let's fix this.
We make the peers code use applet->ptr instead of conn->xprt_ctx to
store the pointer to the current peer. That way it does not depend
on a connection anymore.
This field was used by dumpstats to retrieve a pointer to the current
session, which may already be found from ->owner. With this change,
the stats code doesn't need the connection at all anymore.
We're trying to move the applets out of the struct connection. So
let's remove the dependence on xprt_st and introduce si->applet.st2
to store the missing contextual data instead.
The free() function must free the "struct pat_idx_elt".
This bug was introduced by commit ed66c29 (REORG: acl/pattern: extract
pattern matching from the acl file and create pattern.c), no backport
is needed.
The commit 37e340c (BUG/MEDIUM: stick: completely remove the unused flag
from the store entries) was incomplete. We also need to ensure that only
the first store-response for a table is applied and that it may coexist
with a possible store-request that was already done on this table.
This patch with the previous one should be backported to 1.4.
struct eb_node is 36 bytes on a 64-bit machine. It's thus rounded
up to 40 bytes, and when forming a struct eb32_node, another 4 bytes
are added, rounded up to 48 bytes. We waste 8 bytes of space on 48
bytes because of alignments. It's basically the same with memory
blocks and immediate strings.
By packing the structure, eb32_node is down to 40 bytes. This saves
16 bytes per struct task and 20 bytes per struct stksess, used to
store each stick-table key.
The store[] array in the session holds a flag which probably aimed to
differenciate store entries learned from the request from those learned
from the response, and allowing responses to overwrite only the request
ones (eg: have a server set a response cookie which overwrites the request
one).
But this flag is set when a response data is stored, and is never cleared.
So in practice, haproxy always runs with this flag set, meaning that
responses prevent themselves from overriding the request data.
It is desirable anyway to keep the ability not to override data, because
the override is performed only based on the table and not on the key, so
that would mean that it would be impossible to retrieve two different
keys to store into a same table. For example, if a client sets a cookie
and a server another one, both need to be updated in the table in the
proper order. This is especially true when multiple keys may be tracked
on each side into the same table (eg: list of IP addresses in a header).
So the correct fix which also maintains the current behaviour consists in
simply removing this flag and never try to optimize for the overwrite case.
This fix also has the benefit of significantly reducing the session size,
by 64 bytes due to alignment issues caused by this flag!
The bug has been there forever (since 1.4-dev7), so a backport to 1.4
would be appropriate.
Now instead of seeing many send() calls from multiple "tcp-check send"
rules, we fill the output buffer and try to send all only when we're
not in a send state or when the output buffer is too small for sending
the next message.
This results in a lot less syscalls and avoids filling the network with
many small packets. It will also improve the behaviour of some bogus
servers which expect a complete request in the first packet.
In recent commit 5ecb77f (MEDIUM: checks: add send/expect tcp based check),
bitfields were mistakenly used at some places for the actions. Fortunately,
the only two actions right now are 1 and 2 so they don't share any bit in
common and the bug has no impact.
No backport is needed.
ACL parse errors are not easy to understand since recent commit 348971e
(MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()' into the
ACL keyword) :
[ALERT] 339/154717 (26437) : parsing [check-bug.cfg:10] : error detected while parsing a 'stats admin' rule : unknown ACL or sample keyword 'env(a,b,c)': invalid arg 2 in fetch method 'env' : end of arguments expected at position 2, but got ',b,c'..
This error is only relevant to sample fetch keywords, so the new form is
a bit easier to understand :
[ALERT] 339/160011 (26626) : parsing [check-bug.cfg:12] : error detected while parsing a 'stats admin' rule : invalid arg 2 in fetch method 'env' : end of arguments expected at position 2, but got ',b,c' in sample expression 'env(a,b,c),upper'.
No backport is needed.
William Lallemand reported a double free on the args parser used in fetches
and ACLs. The cause is that the arg expression is not fully initialized nor
deinitialized when killed and that one of the pointers was already freed once
in certain error conditions.
Simply set it to NULL after the first call to free().
The bug was apparently introduced in 1.5-dev9 with commit 2ac5718
(MEDIUM: add a new typed argument list parsing framework).
If a "tcp-check send" experiences an EAGAIN on a send() call, it will
nevertheless go to next rule, and will not try to send again if the next
rule is an expect.
Change this so that we always try to send whatever remains in the buffer
before doing anything else.
A config with just a "tcp-check expect string XXX" loops at 100% CPU
because the connect() wakes the function and there's nothing to send,
but it does not disable the polling.
Rearrange the polling setup to fix this. This was just caused by latest
commit, no backport is needed.
This is a generic health check which can be used to match a
banner or send a request and analyse a server response.
It works in a send/expect ways and many exchange can be done between
HAProxy and a server to decide the server status, making HAProxy able to
speak the server's protocol.
It can send arbitrary regular or binary strings and match content as a
regular or binary string or a regex.
Signed-off-by: Baptiste Assmann <bedis9@gmail.com>
We currently use such an hex parser in pat_parse_bin() to parse hex
string patterns. We'll need another generic one so let's move it to
standard.c and have pat_parse_bin() make use of it.
This patch permits to use the same struct pattern for two indentical maps.
This permits to preserve memory, and permits to update only one
"struct pattern" when the dynamic map update is supported.
Commit 348971e (MEDIUM: acl: use the fetch syntax 'fetch(args),conv(),conv()'
into the ACL keyword) introduced a regression in the ACL parser. The second
argument of an ACL keyword is now mistakenly confused with a converter.
This bug is post-dev19 and does not require any backport.
We happened to preform this call twice on some checks, once in the
recv event handler, and another one in the main function. Remove
the one from the event handler which does not make any more sense
there.
When pure TCP checks are used, we see a useless call to recvfrom()
in strace resulting from an inconditional poll on recv after the
connect() succeeds. Let's remove this one and properly report
connection success in the write events.
Error reporting in health checks is unreliable as the number of recent
patch shows. The main reason is that the code required to detect the
exact situation where the error occurred is not simple, and the errors
have to be handled closer to where they occur in order to be accurate
(rely on getsockopt(SO_ERROR) and errno).
To solve this, we introduce chk_report_conn_err(). It does its best to
consider a possible errno passed in argument, a possible timeout passed
as well, then it completes this with getsockopt() if needed, and takes
into account the current status of the connection. The result is that
by simply calling this function with errno when it's known, we can emit
accurate log messages from every location. We can now see a messages
like "Connection error during SSL handshake (No route to host)" which
were not previously possible.
The only case where errno is supposed to be valid is when the connection
has just got the CO_FL_ERROR flag and errno is not zero, because it will
have been set by the same function that has set the flag. For all other
situations, we need to check the socket using getsockopt(), but only do
it once, since it clears the pending error code. For this reason, we
assign the error code to errno in order not to lose it. The same call
is made at the entry of event_srv_chk_r(), event_srv_chk_w(), and
wake_srv_chk() so that we get a chance to collect errors reported by
the poller or by failed syscalls.
Note that this fix relies on the 4 previous patches, so backporters
must be very careful.
At some places, we report an error by just detecting FD_POLL_ERR.
The problem is that the caller never knows if it must use errno or
call getsockopt(SO_ERROR). And since this last one clears the
pending error from the queue, it cannot be used inconditionally.
An elegant solution consists in clearing errno prior to inspecting
FD_POLL_ERR. The caller then knows that if it gets CO_FL_ERROR and
errno == 0, it must call getsockopt().
Since commit 348971e (MEDIUM: acl: use the fetch syntax
'fetch(args),conv(),conv()' into the ACL keyword), ACLs wait on input
that may change. This is visible in the configuration below :
tcp-request inspect-delay 3s
tcp-request content accept if REQ_CONTENT
Nothing will pass before the end of the timer. This is because
historically, sample_process() was dedicated to stick tables where
it was absolutely necessary to wait for a stable sample. Now samples
are used by many other things and we can't afford this. So let's move
this check to the stick tables after the call to sample_process()
instead.
This is post-1.5-dev19 work, no backport is required.
When we get a hard error from a syscall indicating the socket is dead,
it makes sense to set the CO_FL_SOCK_WR_SH and CO_FL_SOCK_RD_SH flags
to indicate that the socket may not be used anymore. It will ease the
error processing in health checks where the state of socket is very
important. We'll also be able to avoid some setsockopt(nolinger) after
an error.
For now, the rest of the code is not impacted because CO_FL_ERROR is
always tested prior to these flags.
