Commit Graph

19 Commits

Author SHA1 Message Date
Christopher Faulet
0de0becf0b BUG/MINOR: mqtt: Support empty client ID in CONNECT message
As specified by the MQTT specification (MQTT-3.1.3-6), the client ID may be
empty. That means the length of the client ID string may be 0. However, The
MQTT parser does not support empty strings.

So, to fix the bug, the mqtt_read_string() function may now parse empty
string. 2 bytes must be found to decode the string length, but the length
may be 0 now. It is the caller responsibility to test the string emptiness
if necessary. In addition, in mqtt_parse_connect(), the client ID may be
empty now.

This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 16:29:44 +02:00
Christopher Faulet
ca925c9c28 BUG/MINOR: mqtt: Fix parser for string with more than 127 characters
Parsing of too long strings (> 127 characters) was buggy because of a wrong
cast on the length bytes. To fix the bug, we rely on mqtt_read_2byte_int()
function. This way, the string length is properly decoded.

This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 16:29:44 +02:00
Tim Duesterhus
d8be0018fe REGTESTS: Remove REQUIRE_VERSION=1.6 from all tests
HAProxy 1.6 is EOL, thus this always matches.
2021-06-11 19:21:28 +02:00
Willy Tarreau
e1465c1e46 REGTESTS: disable inter-thread idle connection sharing on sensitive tests
Some regtests involve multiple requests from multiple clients, which can
be dispatched as multiple requests to a server. It turns out that the
idle connection sharing works so well that very quickly few connections
are used, and regularly some of the remaining idle server connections
time out at the moment they were going to be reused, causing those random
"HTTP header incomplete" traces in the logs that make them fail often. In
the end this is only an artefact of the test environment.

And indeed, some tests like normalize-uri which perform a lot of reuse
fail very often, about 20-30% of the times in the CI, and 100% of the
time in local when running 1000 tests in a row. Others like ubase64,
sample_fetches or vary_* fail less often but still a lot in tests.

This patch addresses this by adding "tune.idle-pool.shared off" to all
tests which have at least twice as many requests as clients. It proves
very effective as no single error happens on normalize-uri anymore after
10000 tests. Also 100 full runs of all tests yield no error anymore.

One test is tricky, http_abortonclose, it used to fail ~10 times per
1000 runs and with this workaround still fails once every 1000 runs.
But the test is complex and there's a warning in it mentioning a
possible issue when run in parallel due to a port reuse.
2021-05-09 14:41:41 +02:00
Alex
51c8ad45ce MINOR: sample: converter: Add json_query converter
With the json_query can a JSON value be extacted from a header
or body of the request and saved to a variable.

This converter makes it possible to handle some JSON workload
to route requests to different backends.
2021-04-15 17:07:03 +02:00
Daniel Corbett
befef70e23 BUG/MINOR: sample: Rename SenderComID/TargetComID to SenderCompID/TargetCompID
The recently introduced Financial Information eXchange (FIX)
converters have some hard coded tags based on the specification that
were misspelled. Specifically, SenderComID and TargetComID should
be SenderCompID and TargetCompID according to the specification [1][2].

This patch updates all references, which includes the converters
themselves, the regression test, and the documentation.

[1] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag49.html
[2] https://fiximate.fixtrading.org/en/FIX.5.0SP2_EP264/tag56.html
2021-03-10 10:44:20 +01:00
Christopher Faulet
85a813676f REGTESTS: Fix required versions for several scripts
The following scripts require HAProxy 2.4 :

 * cache/caching_rules.vtc
 * cache/post_on_entry.vtc
 * cache/vary.vtc
 * checks/1be_40srv_odd_health_checks.vtc
 * checks/40be_2srv_odd_health_checks.vtc
 * checks/4be_1srv_health_checks.vtc
 * converter/fix.vtc
 * converter/mqtt.vtc
 * http-messaging/protocol_upgrade.vtc
 * http-messaging/websocket.vtc
 * http-set-timeout/set_timeout.vtc
 * log/log_uri.vtc

However it may change is features are backported.
2021-01-28 16:37:14 +01:00
William Dauchy
5417e898ff CLEANUP: sample: remove uneeded check in json validation
- check functions are never called with a NULL args list, it is always
  an array, so first check can be removed
- the expression parser guarantees that we can't have anything else,
  because we mentioned json converter takes a mandatory string argument.
  Thus test on `ARGT_STR` can be removed as well
- also add breaking line between enum and function declaration

In order to validate it, add a simple json test testing very simple
cases but can be improved in the future:

- default json converter without args
- json converter failing on error (utf8)
- json converter with error being removed (utf8s)

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-10 07:39:58 +01:00
William Dauchy
888b0ae8cf MINOR: converter: adding support for url_enc
add base support for url encode following RFC3986, supporting `query`
type only.

- add test checking url_enc/url_dec/url_enc
- update documentation
- leave the door open for future changes

this should resolve github issue #941

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2021-01-06 23:43:04 +01:00
William Dauchy
a2a46ee572 REGTESTS: converter: add url_dec test
while looking at `url_dec` implementation I realised there was not yet a
simple test to avoid future regressions.
This one is testing simple case, including the "+" behaviour depending
on the argument passed to `url_dec`

Signed-off-by: William Dauchy <wdauchy@gmail.com>
2020-11-17 11:53:25 +01:00
Christopher Faulet
32186472cb REGTEST: converter: Add a regtest for MQTT converters
This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used
to validate and extract information from a MQTT (Message Queuing Telemetry
Transport) message.
2020-11-05 19:27:08 +01:00
Christopher Faulet
7983b8687e REGTEST: converter: Add a regtest for fix converters
This new script tests fix_is_valid() and fix_tag_value() converters used to
validate and extract information from a FIX (Financial Information eXchange)
message.
2020-11-05 19:26:40 +01:00
Willy Tarreau
0f08c37f0d REGTEST: the iif converter test requires 2.3
This one was recently added in 2.3.
2020-09-29 10:52:32 +02:00
Tim Duesterhus
3943e4fc3e MINOR: sample: Add iif(<true>,<false>) converter
iif() takes a boolean as input and returns one of the two argument
strings depending on whether the boolean is true.

This converter most likely is most useful to return the proper scheme
depending on the value returned by the `ssl_fc` fetch, e.g. for use within
the `x-forwarded-proto` request header.

However it can also be useful for use within a template that is sent to
the client using `http-request return` with a `lf-file`. It allows the
administrator to implement a simple condition, without needing to prefill
variables within the regular configuration using `http-request
set-var(req.foo)`.
2020-09-11 16:59:27 +02:00
Tim Duesterhus
f38175cf6e MINOR: sample: Add secure_memcmp converter
secure_memcmp compares two binary strings in constant time. It's only
available when haproxy is compiled with USE_OPENSSL.
2020-06-09 22:04:13 +02:00
Patrick Gansterer
8e36651ed3 MINOR: sample: Add digest and hmac converters
Make the digest and HMAC function of OpenSSL accessible to the user via
converters. They can be used to sign and validate content.

Reviewed-by: Tim Duesterhus <tim@bastelstu.be>
2020-05-12 10:08:11 +02:00
Tim Duesterhus
cd3732456b MINOR: sample: Validate the number of bits for the sha2 converter
Instead of failing the conversion when an invalid number of bits is
given the sha2 converter now fails with an appropriate error message
during startup.

The sha2 converter was introduced in d437630237,
which is in 2.1 and higher.
2019-12-17 13:28:00 +01:00
Tim Duesterhus
4381d26edc BUG/MINOR: sample: Make the field converter compatible with -m found
Previously an expression like:

    path,field(2,/) -m found

always returned `true`.

Bug exists since the `field` converter exists. That is:
f399b0debf

The fix should be backported to 1.6+.
2019-10-21 15:49:42 +02:00
Tim Duesterhus
d437630237 MINOR: sample: Add sha2([<bits>]) converter
This adds a converter for the SHA-2 family, supporting SHA-224, SHA-256
SHA-384 and SHA-512.

The converter relies on the OpenSSL implementation, thus only being available
when HAProxy is compiled with USE_OPENSSL.

See GitHub issue #123. The hypothetical `ssl_?_sha256` fetch can then be
simulated using `ssl_?_der,sha2(256)`:

  http-response set-header Server-Cert-FP %[ssl_f_der,sha2(256),hex]
2019-06-17 13:36:42 +02:00