RepoMirrors
/
haproxy
mirror of http://git.haproxy.org/git/haproxy.git/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
haproxy/reg-tests/converter/mqtt.vtc

238 lines
10 KiB
Plaintext
Raw Normal View History

REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
varnishtest "mqtt converters Test"
REGTESTS: Fix required versions for several scripts The following scripts require HAProxy 2.4 : * cache/caching_rules.vtc * cache/post_on_entry.vtc * cache/vary.vtc * checks/1be_40srv_odd_health_checks.vtc * checks/40be_2srv_odd_health_checks.vtc * checks/4be_1srv_health_checks.vtc * converter/fix.vtc * converter/mqtt.vtc * http-messaging/protocol_upgrade.vtc * http-messaging/websocket.vtc * http-set-timeout/set_timeout.vtc * log/log_uri.vtc However it may change is features are backported.
2020-12-15 16:13:39 +00:00
#REQUIRE_VERSION=2.4
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
feature ignore_unknown_macro
server s1 {
BUG/MINOR: mqtt: Fix parser for string with more than 127 characters Parsing of too long strings (> 127 characters) was buggy because of a wrong cast on the length bytes. To fix the bug, we rely on mqtt_read_2byte_int() function. This way, the string length is properly decoded. This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 13:26:00 +00:00
# MQTT 3.1.1 CONNECT packet (id: test_subaaaaaa... [len = 200])
recv 215
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
sendhex "20020000"
close
BUG/MINOR: mqtt: Support empty client ID in CONNECT message As specified by the MQTT specification (MQTT-3.1.3-6), the client ID may be empty. That means the length of the client ID string may be 0. However, The MQTT parser does not support empty strings. So, to fix the bug, the mqtt_read_string() function may now parse empty string. 2 bytes must be found to decode the string length, but the length may be 0 now. It is the caller responsibility to test the string emptiness if necessary. In addition, in mqtt_parse_connect(), the client ID may be empty now. This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 13:37:59 +00:00
# MQTT 3.1.1 CONNECT packet (id: <empty> - username: test - passwd: passwd)
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
accept
BUG/MINOR: mqtt: Support empty client ID in CONNECT message As specified by the MQTT specification (MQTT-3.1.3-6), the client ID may be empty. That means the length of the client ID string may be 0. However, The MQTT parser does not support empty strings. So, to fix the bug, the mqtt_read_string() function may now parse empty string. 2 bytes must be found to decode the string length, but the length may be 0 now. It is the caller responsibility to test the string emptiness if necessary. In addition, in mqtt_parse_connect(), the client ID may be empty now. This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 13:37:59 +00:00
recv 28
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
sendhex "20020000"
close
# MQTT 3.1.1 CONNECT packet (id: test_sub - username: test - passwd: passwd - will_topic: willtopic - will_payload: willpayload)
accept
recv 60
sendhex "20020000"
close
# MQTT 5.0 CONNECT packet (id: test_sub)
accept
recv 26
sendhex "200600000322000a"
# MQTT 5.0 CONNECT packet (id: test_sub - username: test - passwd: passwd)
accept
recv 40
sendhex "200600000322000a"
# MQTT 5.0 complex CONNECT/CONNACK packet
accept
recv 128
sendhex "20250000221100000078217fff24012501270000ffff22000a2600016100016226000163000164"
close
# Invalid MQTT 3.1.1 CONNACK packet with invalid flags (!= 0x00)
accept
recv 22
sendhex "21020000"
expect_close
MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value converters for MQTTv3.1 In MQTTv3.1, protocol name is "MQIsdp" and protocol level is 3. The mqtt converters(mqtt_is_valid and mqtt_field_value) did not work for clients on mqttv3.1 because the mqtt_parse_connect() marked the CONNECT message invalid if either the protocol name is not "MQTT" or the protocol version is other than v3.1.1 or v5.0. To fix it, we have added the mqttv3.1 protocol name and version as part of the checks. This patch fixes the mqtt converters to support mqttv3.1 clients as well (issue #1600). It must be backported to 2.4.
2022-03-21 14:34:00 +00:00
# MQTT 3.1 CONNECT packet (id: test_sub - username: test - passwd: passwd)
accept
recv 38
sendhex "20020000"
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
} -start
server s2 {
# MQTT 5.0 complex CONNECT packet
recv 128
sendhex "20250000221100000078217fff24012501270000ffff22000a2600016100016226000163000164"
} -start
haproxy h1 -conf {
defaults
mode tcp
REGTESTS: extend the default I/O timeouts and make them overridable With the CI occasionally slowing down, we're starting to see again some spurious failures despite the long 1-second timeouts. This reports false positives that are disturbing and doesn't provide as much value as this could. However at this delay it already becomes a pain for developers to wait for the tests to complete. This commit adds support for the new environment variable HAPROXY_TEST_TIMEOUT that will allow anyone to modify the connect, client and server timeouts. It was set to 5 seconds by default, which should be plenty for quite some time in the CI. All relevant values that were 200ms or above were replaced by this one. A few larger values were left as they are special. One test for the set-timeout action that used to rely on a fixed 1-sec value was extended to a fixed 5-sec, as the timeout is normally not reached, but it needs to be known to compare the old and new values.
2021-11-18 16:46:22 +00:00
timeout connect "${HAPROXY_TEST_TIMEOUT-5s}"
timeout client "${HAPROXY_TEST_TIMEOUT-5s}"
timeout server "${HAPROXY_TEST_TIMEOUT-5s}"
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
frontend fe1
bind "fd@${fe1}"
tcp-request inspect-delay 1s
tcp-request content reject unless { req.payload(0,0),mqtt_is_valid }
default_backend be1
frontend fe2
bind "fd@${fe2}"
tcp-request inspect-delay 1s
tcp-request content reject unless { req.payload(0,0),mqtt_is_valid }
tcp-request content set-var(req.flags) req.payload(0,0),mqtt_field_value(connect,flags)
tcp-request content set-var(req.protoname) req.payload(0,0),mqtt_field_value(connect,protocol_name)
tcp-request content set-var(req.protovsn) req.payload(0,0),mqtt_field_value(connect,protocol_version)
tcp-request content set-var(req.clientid) req.payload(0,0),mqtt_field_value(connect,client_identifier)
tcp-request content set-var(req.willtopic) req.payload(0,0),mqtt_field_value(connect,will_topic)
tcp-request content set-var(req.willbody) req.payload(0,0),mqtt_field_value(connect,will_payload)
tcp-request content set-var(req.user) req.payload(0,0),mqtt_field_value(connect,username)
tcp-request content set-var(req.pass) req.payload(0,0),mqtt_field_value(connect,password)
tcp-request content set-var(req.maxpktsz) req.payload(0,0),mqtt_field_value(connect,39)
tcp-request content set-var(req.reqpbinfo) req.payload(0,0),mqtt_field_value(connect,23)
tcp-request content set-var(req.ctype) req.payload(0,0),mqtt_field_value(connect,3)
tcp-request content set-var(req.willrsptopic) req.payload(0,0),mqtt_field_value(connect,8)
MEDIUM: vars: make the var() sample fetch function really return type ANY A long-standing issue was reported in issue #1215. In short, var() was initially internally declared as returning a string because it was not possible by then to return "any type". As such, users regularly get trapped thinking that when they're storing an integer there, then the integer matching method automatically applies. Except that this is not possible since this is related to the config parser and is decided at boot time where the variable's type is not known yet. As such, what is done is that the output being declared as type string, the string match will automatically apply, and any value will first be converted to a string. This results in several issues like: http-request set-var(txn.foo) int(-1) http-request deny if { var(txn.foo) lt 0 } not working. This is because the string match on the second line will in fact compare the string representation of the variable against strings "lt" and "0", none of which matches. The doc says that the matching method is mandatory, though that's not the case in the code due to that default string type being permissive. There's not even a warning when no explicit match is placed, because this happens very deep in the expression evaluator and making a special case just for "var" can reveal very complicated. The set-var() converter already mandates a matching method, as the following will be rejected: ... if { int(12),set-var(txn.truc) 12 } while this one will work: ... if { int(12),set-var(txn.truc) -m int 12 } As such, this patch this modifies var() to match the doc, returning the type "any", and mandating the matching method, implying that this bogus config which does not work: http-request set-var(txn.foo) int(-1) http-request deny if { var(txn.foo) lt 0 } will need to be written like this: http-request set-var(txn.foo) int(-1) http-request deny if { var(txn.foo) -m int lt 0 } This *will* break some configs (and even 3 of our regtests relied on this), but except those which already match string exclusively, all other ones are already broken and silently fail (and one of the 3 regtests, the one on FIX, was bogus regarding this). In order to fix existing configs, one can simply append "-m str" after a "var()" in an ACL or "if" expression: http-request deny unless { var(txn.jwt_alg) "ES" } must become: http-request deny unless { var(txn.jwt_alg) -m str "ES" } Most commonly, patterns such as "le", "lt", "ge", "gt", "eq", "ne" in front of a number indicate that the intent was to match an integer, and in this case "-m int" would be desired: tcp-response content reject if ! { var(res.size) gt 3800 } ought to become: tcp-response content reject if ! { var(res.size) -m int gt 3800 } This must not be backported, but if a solution is found to at least detect this exact condition in the generic expression parser and emit a warning, this could probably help spot configuration bugs. Link: https://www.mail-archive.com/haproxy@formilux.org/msg41341.html Cc: Christopher Faulet <cfaulet@haproxy.com> Cc: Tim Dsterhus <tim@bastelstu.be>
2021-11-02 16:08:15 +00:00
tcp-request content reject if ! { var(req.protoname) -m str "MQTT" } || ! { var(req.protovsn) -m str "5" }
tcp-request content reject if ! { var(req.flags) -m str "238" } || ! { var(req.clientid) -m str "test_sub" }
tcp-request content reject if ! { var(req.user) -m str "test" } || ! { var(req.pass) -m str "passwd" }
tcp-request content reject if ! { var(req.willtopic) -m str "willtopic" } || ! { var(req.willbody) -m str "willpayload" }
tcp-request content reject if ! { var(req.maxpktsz) -m str "20" } || ! { var(req.reqpbinfo) -m str "1" }
tcp-request content reject if ! { var(req.ctype) -m str "text/plain" } || ! { var(req.willrsptopic) -m str "willrsptopic" }
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
default_backend be2
backend be1
server s1 ${s1_addr}:${s1_port}
tcp-response inspect-delay 1s
tcp-response content reject unless { res.payload(0,0),mqtt_is_valid }
backend be2
server s2 ${s2_addr}:${s2_port}
tcp-response inspect-delay 1s
tcp-response content reject unless { res.payload(0,0),mqtt_is_valid }
tcp-response content set-var(res.flags) res.payload(0,0),mqtt_field_value(connack,flags)
tcp-response content set-var(res.protovsn) res.payload(0,0),mqtt_field_value(connack,protocol_version)
tcp-response content set-var(res.rcode) res.payload(0,0),mqtt_field_value(connack,reason_code)
tcp-response content set-var(res.sessexpint) res.payload(0,0),mqtt_field_value(connack,17)
tcp-response content set-var(res.recvmax) res.payload(0,0),mqtt_field_value(connack,33)
tcp-response content set-var(res.maxqos) res.payload(0,0),mqtt_field_value(connack,36)
tcp-response content set-var(res.retainavail) res.payload(0,0),mqtt_field_value(connack,37)
tcp-response content set-var(res.maxpktsz) res.payload(0,0),mqtt_field_value(connack,39)
tcp-response content set-var(res.topicaliasmax) res.payload(0,0),mqtt_field_value(connack,34)
MEDIUM: vars: make the var() sample fetch function really return type ANY A long-standing issue was reported in issue #1215. In short, var() was initially internally declared as returning a string because it was not possible by then to return "any type". As such, users regularly get trapped thinking that when they're storing an integer there, then the integer matching method automatically applies. Except that this is not possible since this is related to the config parser and is decided at boot time where the variable's type is not known yet. As such, what is done is that the output being declared as type string, the string match will automatically apply, and any value will first be converted to a string. This results in several issues like: http-request set-var(txn.foo) int(-1) http-request deny if { var(txn.foo) lt 0 } not working. This is because the string match on the second line will in fact compare the string representation of the variable against strings "lt" and "0", none of which matches. The doc says that the matching method is mandatory, though that's not the case in the code due to that default string type being permissive. There's not even a warning when no explicit match is placed, because this happens very deep in the expression evaluator and making a special case just for "var" can reveal very complicated. The set-var() converter already mandates a matching method, as the following will be rejected: ... if { int(12),set-var(txn.truc) 12 } while this one will work: ... if { int(12),set-var(txn.truc) -m int 12 } As such, this patch this modifies var() to match the doc, returning the type "any", and mandating the matching method, implying that this bogus config which does not work: http-request set-var(txn.foo) int(-1) http-request deny if { var(txn.foo) lt 0 } will need to be written like this: http-request set-var(txn.foo) int(-1) http-request deny if { var(txn.foo) -m int lt 0 } This *will* break some configs (and even 3 of our regtests relied on this), but except those which already match string exclusively, all other ones are already broken and silently fail (and one of the 3 regtests, the one on FIX, was bogus regarding this). In order to fix existing configs, one can simply append "-m str" after a "var()" in an ACL or "if" expression: http-request deny unless { var(txn.jwt_alg) "ES" } must become: http-request deny unless { var(txn.jwt_alg) -m str "ES" } Most commonly, patterns such as "le", "lt", "ge", "gt", "eq", "ne" in front of a number indicate that the intent was to match an integer, and in this case "-m int" would be desired: tcp-response content reject if ! { var(res.size) gt 3800 } ought to become: tcp-response content reject if ! { var(res.size) -m int gt 3800 } This must not be backported, but if a solution is found to at least detect this exact condition in the generic expression parser and emit a warning, this could probably help spot configuration bugs. Link: https://www.mail-archive.com/haproxy@formilux.org/msg41341.html Cc: Christopher Faulet <cfaulet@haproxy.com> Cc: Tim Dsterhus <tim@bastelstu.be>
2021-11-02 16:08:15 +00:00
tcp-response content reject if ! { var(res.protovsn) -m str "5" } || ! { var(res.flags) -m str "0" }
tcp-response content reject if ! { var(res.rcode) -m str "0" } || ! { var(res.sessexpint) -m str "120" }
tcp-response content reject if ! { var(res.recvmax) -m str "32767" } || ! { var(res.maxqos) -m str "1" }
tcp-response content reject if ! { var(res.retainavail) -m str "1" } || ! { var(res.maxpktsz) -m str "65535" }
tcp-response content reject if ! { var(res.topicaliasmax) -m str "10" }
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
} -start
client c1_311_1 -connect ${h1_fe1_sock} {
# Valid MQTT 3.1.1 CONNECT packet (id: test_sub)
BUG/MINOR: mqtt: Fix parser for string with more than 127 characters Parsing of too long strings (> 127 characters) was buggy because of a wrong cast on the length bytes. To fix the bug, we rely on mqtt_read_2byte_int() function. This way, the string length is properly decoded. This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 13:26:00 +00:00
sendhex "10d40100044d5154540402003c00c8746573745f737562616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161"
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
recv 4
expect_close
} -run
client c1_311_2 -connect ${h1_fe1_sock} {
BUG/MINOR: mqtt: Support empty client ID in CONNECT message As specified by the MQTT specification (MQTT-3.1.3-6), the client ID may be empty. That means the length of the client ID string may be 0. However, The MQTT parser does not support empty strings. So, to fix the bug, the mqtt_read_string() function may now parse empty string. 2 bytes must be found to decode the string length, but the length may be 0 now. It is the caller responsibility to test the string emptiness if necessary. In addition, in mqtt_parse_connect(), the client ID may be empty now. This patch should partely fix the issue #1310. It must be backported to 2.4.
2021-06-28 13:37:59 +00:00
# Valid MQTT 3.1.1 CONNECT packet (id: <empty> - username: test - passwd: passwd)
sendhex "101a00044d51545404c2003c00000004746573740006706173737764"
REGTEST: converter: Add a regtest for MQTT converters This new script tests mqtt_is_valid() and mqtt_get_field_value() converters used to validate and extract information from a MQTT (Message Queuing Telemetry Transport) message.
2020-10-30 16:46:00 +00:00
recv 4
expect_close
} -run
client c1_311_3 -connect ${h1_fe1_sock} {
# Valid MQTT 3.1.1 CONNECT packet (id: test_sub - username: test - passwd: passwd - will_topic: willtopic - will_payload: willpayload)
sendhex "103a00044d51545404ee003c0008746573745f737562000977696c6c746f706963000b77696c6c7061796c6f61640004746573740006706173737764"
recv 4
expect_close
} -run
client c1_50_1 -connect ${h1_fe1_sock} {
# Valid MQTT 5.0 CONNECT packet (id: test_sub)
sendhex "101800044d5154540502003c032100140008746573745f737562"
recv 8
expect_close
} -run
client c1_50_2 -connect ${h1_fe1_sock} {
# Valid MQTT 5.0 CONNECT packet (id: test_sub - username: test - passwd: passwd)
sendhex "102600044d51545405c2003c032100140008746573745f7375620004746573740006706173737764"
recv 8
expect_close
} -run
client c1_50_3 -connect ${h1_fe1_sock} {
# Valid MQTT 5.0 complex CONNECT/CONNACK packet
sendhex "107e00044d51545405ee003c182700000014170126000161000162260001630001642100140008746573745f7375622a03000a746578742f706c61696e08000c77696c6c727370746f7069632600016500016626000167000168000977696c6c746f706963000b77696c6c7061796c6f61640004746573740006706173737764"
recv 39
expect_close
} -run
client c2_311_1 -connect ${h1_fe1_sock} {
# Invalid MQTT 3.1.1 PINREQ
sendhex "d000"
expect_close
} -run
client c2_311_2 -connect ${h1_fe1_sock} {
# Invalid MQTT 3.1.1 CONNECT packet with invalid flags (!= 0x00)
sendhex "111400044d5154540402003c0008746573745f737562"
expect_close
} -run
client c2_311_3 -connect ${h1_fe1_sock} {
# Invalid MQTT 3.1.1 CONNACK packet with invalid flags (!= 0x00)
sendhex "101400044d5154540402003c0008746573745f737562"
expect_close
} -run
client c2_311_4 -connect ${h1_fe1_sock} {
# Invalid MQTT 3.1.1 CONNECT with too long remaing_length ( > 4 bytes)
sendhex "10ffffffff1400044d5154540402003c0008746573745f737562"
expect_close
} -run
client c2_311_4 -connect ${h1_fe1_sock} {
# Invalid MQTT 3.1.1 CONNECT with not matching ( 0x13 != 0x14)
sendhex "101300044d5154540402003c000874657374a5f737562"
expect_close
} -run
client c2_311_4 -connect ${h1_fe1_sock} {
# Invalid MQTT 3.1.1 CONNECT with not matching ( 0x18 != 0x14)
sendhex "101800044d5154540402003c000874657374a5f737562ffffffff"
expect_close
} -run
client c2_50_1 -connect ${h1_fe2_sock} {
# complex MQTT 5.0 CONNECT/CONNACK packet
# - CONNECT :
# client-id : test_sub
# username : test
# password : passwd
# will-topic : willtopic
# will-payload: willpayload
# connect props:
# maximum-packet-size : 20
# request-problem-information: 1
# user-property : name=a value=b
# user-property : name=c value=d
# will props:
# content-type : text/plain
# response-topic: willrsptopic
# user-property : name=e value=f
# user-property : name=g value=h
# - CONNACK :
# flags : 0x00
# reason-code: 0x00
# connack props:
# session-Expiry-interval: 120
# receive-maximum : 32767
# maximum-qos : 1
# retain-available : 1
# maximum-packet-size : 65535
# topic-alias-maximum : 10
# user-property : name=a value=b
# user-property : name=c value=d
sendhex "107e00044d51545405ee003c182700000014170126000161000162260001630001642100140008746573745f7375622a03000a746578742f706c61696e08000c77696c6c727370746f7069632600016500016626000167000168000977696c6c746f706963000b77696c6c7061796c6f61640004746573740006706173737764"
recv 39
expect_close
} -run
MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value converters for MQTTv3.1 In MQTTv3.1, protocol name is "MQIsdp" and protocol level is 3. The mqtt converters(mqtt_is_valid and mqtt_field_value) did not work for clients on mqttv3.1 because the mqtt_parse_connect() marked the CONNECT message invalid if either the protocol name is not "MQTT" or the protocol version is other than v3.1.1 or v5.0. To fix it, we have added the mqttv3.1 protocol name and version as part of the checks. This patch fixes the mqtt converters to support mqttv3.1 clients as well (issue #1600). It must be backported to 2.4.
2022-03-21 14:34:00 +00:00
client c3_31_1 -connect ${h1_fe1_sock} {
# Valid MQTT 3.1 CONNECT packet (id: test_sub - username: test - passwd: passwd)
sendhex "102400064d514973647003c200000008746573745f7375620004746573740006706173737764"
recv 4
} -run