httpclient_new() sets the hc->req.uri ist without duplicating its
memory, which is a problem since the string in the ist could be
inaccessible at some point. The API was made to use a ist which was
allocated dynamically, but httpclient_new() didn't do that, which result
in a crash when calling istfree().
This patch fixes the problem by doing an istdup()
Fix issue #1452.
The "show proc" output changed and it's time to update the example.
The output does not display the relative PID anymore since the nbproc
keyword disappeared, and it displays the number of failed reloads since
the last successful one.
When in wait mode, the mworker-prog postparser is launched, but
unfortunately the child structure doesn't contain all required
information to be able to launch the test.
This test is only required when doing a configuration parsing.
Must be backported as far as 2.0.
Since the wait mode is always used once we successfuly loaded the
configuration, every processes were marked as old workers.
To fix this, the PROC_O_LEAVING flag is set only on the processes which
have a number of reloads greater than the current processes.
The ReloadFailed prompt in the master CLI is shown only when
failedreloads > 0. It was previously using a check on the wait mode, but
we always use the wait mode now.
Implement a reload failure counter which counts the number of failure
since the last success. This counter is available in 'show proc' over
the master CLI.
Clarify the startup and reload messages:
On a successful configuration load, haproxy will emit "Loading success."
after successfuly forked the children.
When it didn't success to load the configuration it will emit "Loading failure!".
When trying to reload the master process, it will emit "Reloading
HAProxy".
Use the waitpid mode after successfully loading the configuration, this
way the memory will be freed in the master, and will preserve the memory.
This will be useful when doing a reload with a configuration which has
large maps or a lot of SSL certificates, avoiding an OOM because too
much memory was allocated in the master.
nbproc was removed, it's time to remove any reference to the relative
PID in the master-worker, since there can be only 1 current haproxy
process.
This patch cleans up the alerts and warnings emitted during the exit of
a process, as well as the "show proc" output.
cs_close() and cs_drain_and_close() are called to close a conn-stream.
cs_shutr() and cs_shutw() are called with the appropriate modes. But the
conn-stream is not released at this stage. However the flags are
reset. Thus, after a cs_close(), we loose shutdown flags. If cs_close() is
performed several times, it is a problem. And indeed, it is possible. On one
hand, the stream-interface may close the conn-stream. On the other end, the
stream always closes it when it is released.
It is a problem for the H1 multiplexer. Because the conn-stream flags are
reset, the shutr and shutw are performed twice. For a delayed shutdown, the
dirty mode is used instead of the normal one because the last call to
h1_shutw() overwrite H1C flags set by the first call. This leads to dirty
shutdowns while normal ones are required. At the end, it is possible to
truncate the messages.
This bug was revealed by the commit a85c522d4 ("BUG/MINOR: mux-h1: Save
shutdown mode if the shutdown is delayed").
This patch is related to the issue #1450. It must be backported as far as
2.0.
In HTTP, when a lua action is evaluated, a reply object can be used to send
a response to the client and interrupt the transaction. This reply object is
converted into HTX and is limited to the response channel buffer. Its size,
once converted, cannot exceed the buffer size. There is no streaming at this
stage. However, this limitation was not documented.
Note that, for now, there is no easy way to know if the reply will fit or
not int the response channel buffer. Thus the reply must be reasonably
small. Otherwise a 500-Internal-Error message is returned.
This patch is related to the issue #1447. It may be backported as far as
2.2.
TCP/HTTP allow actions stop rules evaluation of the current section
only. Only the http-response description was accurate on this
point. Thus, the documentation is now explicit on this point for all
other concerned rulesets.
This patch may be backported, to all supported versions for tcp-request
and http-request documentation, and as far as 2.2 for http-after-response
documentation.
This reverts commit 597909f4e6
http-after-response rules evaluation was changed to do the same that was
done for http-response, in the code. However, the opposite must be performed
instead. Only the rules of the current section must be stopped. Thus the
above commit is reverted and the http-response rules evaluation will be
fixed instead.
Note that only "allow" action is concerned. It is most probably an uncommon
action for an http-after-request rule.
This patch must be backported as far as 2.2 if the above commit was
backported.
A TCP/HTTP action can stop the rules evaluation. However, it should be
applied on the current section only. For instance, for http-requests rules,
an "allow" on a frontend must stop evaluation of rules defined in this
frontend. But the backend rules, if any, must still be evaluated.
For http-response rulesets, according the configuration manual, the same
must be true. Only "allow" action is concerned. However, since the
beginning, this action stops evaluation of all remaining rules, not only
those of the current section.
This patch may be backported to all supported versions. But it is not so
critical because the bug exists since a while. I doubt it will break any
existing configuration because the current behavior is
counterintuitive.
In ssl_fc_unique_id decription, threre is a reference to the wrong sample
fetch. ssl_bc_unique_id is used instead of ssl_fc_unique_id.
This patch should fix the issue #1449. It may be backported to all
supportted versions.
- add new metric: `haproxy_backend_agg_server_check_status`
it counts the number of servers matching a specific check status
this permits to exclude per server check status as the usage is often
to rely on the total. Indeed in large setup having thousands of
servers per backend the memory impact is not neglible to store the per
server metric.
- realign promex_str_metrics array
quite simple implementation - we could improve it later by adding an
internal state to the prometheus exporter, thus to avoid counting at
every dump.
this patch is an attempt to close github issue #1312. It may bebackported
to 2.4 if requested.
Signed-off-by: William Dauchy <wdauchy@gmail.com>
The httpclient uses channel_add_input() to notify the channel layer that
it must forward some data. This function was used with b_data(&req->buf)
which ask to send the size of a buffer (because of the HTX metadata
which fill the buffer completely).
This is wrong and will have the consequence of trying to send data that
doesn't exist, letting HAProxy looping at 100% CPU.
When using htx channel_add_input() must be used with the size of the htx
payload, and not the size of a buffer.
When sending the request payload it also need to sets the buffer size to
0, which is achieved with a htx_to_buf() when the htx payload is empty.
This patch fixes the receive part of the lua httpclient when no payload
was sent.
The lua task was not awoken once it jumped into
hlua_httpclient_rcv_yield(), which caused the lua client to freeze.
It works with a payload because the payload push is doing the wakeup.
A change in the state machine of the IO handler is also require to
achieve correctly the change from the REQ state to the RES state, it has
to detect if there is the right EOM flag in the request.
This one was missing. It should be easier to use now. It is obvious that
some functions are missing, and it looks like ist2str() and istpad() are
exactly the same.
`info_field_names` and `stat_field_names` no longer exist and have been
moved in stats.c
To avoid changing this comment, just mention the name of the new table
`info_fields` and `stat_fields`
Signed-off-by: William Dauchy <wdauchy@gmail.com>
When "max-age" or "s-maxage" receive their values in quotes, the pointer
to the integer to be parsed is advanced by one, but the error pointer
check doesn't consider this advanced offset, so it will not match a
parse error such as max-age="a" and will take the value zero instead.
This probably needs to be backported, though it's unsure it has any
effect in the real world.
This function claims to perform an strncat()-like operation but it does
not, it always copies the indicated number of bytes, regardless of the
presence of a NUL character (what is currently done by chunk_memcat()).
Let's remove it and explicitly replace it with chunk_memcat().
Some architectures like x86_64 and aarch64 support efficient unaligned
64-bit reads. On such architectures, we already know that each string
passed to field_start() has some margin at the end because it's parsed
using fgets2() which looks for the trailing LF using the same method.
Thus let's skip spaces by packs of 8. This increases the parsing speed
by 35%.
Modern compilers were producing producing less efficient code in the
field_start() loop, by not emitting two conditional jumps for a single
test. However by reordering the test we can merge the optimal case and
the default one and get back to good performance so let's simplify the
test. This improves the parsing speed by 5%.
Fix potential allocation failure of HTX start-line during H3 request
decoding. In this case, h3_decode_qcs returns -1 as error code.
This addresses in part github issue #1445.
The usage message was starting to have long lines, it's preferable that
it still fits well into a default 80-col display so that options are
easy to find. Also cut that into the 3 parts (input filter, modifier,
output format) for improved legibility.
Released version 2.5-dev13 with the following main changes :
- SCRIPTS: git-show-backports: re-enable file-based filtering
- MINOR: jwt: Make invalid static JWT algorithms an error in `jwt_verify` converter
- MINOR: mux-h2: add trace on extended connect usage
- BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
- MINOR: stream/mux: implement websocket stream flag
- MINOR: connection: implement function to update ALPN
- MINOR: connection: add alternative mux_ops param for conn_install_mux_be
- MEDIUM: server/backend: implement websocket protocol selection
- MINOR: server: add ws keyword
- BUG/MINOR: resolvers: fix sent messages were counted twice
- BUG/MINOR: resolvers: throw log message if trash not large enough for query
- MINOR: resolvers/dns: split dns and resolver counters in dns_counter struct
- MEDIUM: resolvers: rename dns extra counters to resolvers extra counters
- BUG/MINOR: jwt: Fix jwt_parse_alg incorrectly returning JWS_ALG_NONE
- DOC: add QUIC instruction in INSTALL
- CLEANUP: halog: Remove dead stores
- DEV: coccinelle: Add ha_free.cocci
- CLEANUP: Apply ha_free.cocci
- DEV: coccinelle: Add rule to use `istnext()` where possible
- CLEANUP: Apply ist.cocci
- REGTESTS: Use `feature cmd` for 2.5+ tests (2)
- DOC: internals: move some API definitions to an "api" subdirectory
- MINOR: quic: Allocate listener RX buffers
- CLEANUP: quic: Remove useless code
- MINOR: quic: Enhance the listener RX buffering part
- MINOR: quic: Remove a useless lock for CRYPTO frames
- MINOR: quic: Use QUIC_LOCK QUIC specific lock label.
- MINOR: backend: Get client dst address to set the server's one only if needful
- MINOR: compression: Warn for 'compression offload' in defaults sections
- MEDIUM: connection: rename fc_conn_err and bc_conn_err to fc_err and bc_err
- DOC: configuration: move the default log formats to their own section
- MINOR: ssl: make the ssl_fc_sni() sample-fetch function always available
- MEDIUM: log: add the client's SNI to the default HTTPS log format
- DOC: config: add an example of reasonably complete error-log-format
- DOC: config: move error-log-format before custom log format
All default formats were described before the custom one, except this
one. Better place them all together before the custom log format. This
only swaps and renumbers the sections.
During a troublehooting it came obvious that the SNI always ought to
be logged on httpslog, as it explains errors caused by selection of
the default certificate (or failure to do so in case of strict-sni).
This expectation was also confirmed on the mailing list.
Since the field may be empty it appeared important not to leave an
empty string in the current format, so it was decided to place the
field before a '/' preceding the SSL version and ciphers, so that
in the worst case a missing field leads to a field looking like
"/TLSv1.2/AES...", though usually a missing element still results
in a "-" in logs.
This will change the log format for users who already deployed the
2.5-dev versions (hence the medium level) but no released version
was using this format yet so there's no harm for stable deployments.
The reg-test was updated to check for "-" there since we don't send
SNI in reg-tests.
Link: https://www.mail-archive.com/haproxy@formilux.org/msg41410.html
Cc: William Lallemand <wlallemand@haproxy.org>
Its definition is enclosed inside an ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
which is defined since OpenSSL 0.9.8. Having it conditioned like this
prevents us from using it by default in a log format, which could cause
an error on an old or exotic library.
Let's just always define it and make the sample fetch fail to return
anything on such libs instead.
I'm always having a very hard time finding the log-format definition of
httplog, because it's not in the httplog description, and looking for
"httplog" doesn't yield the custom log formats section.
It would make more sense to write these log-formats into their respective
sections where they will be easier to find. That's what this commit does.
