haproxy/reg-tests/http-messaging/http_request_buffer.vtc

136 lines
3.7 KiB
Plaintext
Raw Normal View History

varnishtest "A test for http-request-buffer option"
feature ignore_unknown_macro
# This test checks HTTP request buffering feature.
# We run one server s1 which can serve only one client (no -repeat argument here).
# c1 client uses a malformed request which is not transferred to s1 server
# thanks to "http-buffer-request". If this was the case, c2 client
# could not connect to s1 server and this would lead to make this test fail.
barrier b1 cond 2 -cyclic
server s1 {
rxreq
expect req.bodylen == 257
txresp
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
accept
rxreq
expect req.bodylen == 2
txresp
} -start
syslog S -level info {
recv
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 fe1/<NOSRV> .* 408 .* - - cD-- .* .* \"GET /this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url HTTP/1\\.1\""
barrier b1 sync
recv
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/srv1 [0-9]*/[0-9]*/[0-9]*/[0-9]*/[0-9]* 200 .* - - ---- .* .* \"GET / HTTP/1\\.1\""
barrier b1 sync
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
recv
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe2 be1/srv1 [0-9]*/[0-9]*/[0-9]*/[0-9]*/[0-9]* 200 .* - - ---- .* .* \"POST /1 HTTP/1\\.1\""
barrier b1 sync
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
recv
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe2 be1/<NOSRV> [0-9]*/-1/-1/-1/[0-9]* -1 .* - - CR-- .* .* \"POST /2 HTTP/1\\.1\""
} -start
haproxy h1 -conf {
defaults
mode http
timeout client 100
timeout server "${HAPROXY_TEST_TIMEOUT-5s}"
timeout connect "${HAPROXY_TEST_TIMEOUT-5s}"
backend be1
server srv1 ${s1_addr}:${s1_port}
frontend fe1
option httplog
option http-buffer-request
log ${S_addr}:${S_port} local0 debug err
bind "fd@${fe1}"
use_backend be1
frontend fe2
timeout client 10s
option httplog
option http-buffer-request
log ${S_addr}:${S_port} local0 debug err
bind "fd@${fe2}"
use_backend be1
} -start
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
# 1 byte of the payload is missing.
# ==> The request must timed out with a 408 response
client c1 -connect ${h1_fe1_sock} {
send "GET"
send " "
send "/this-is-a-long-url"
send "-this-is-a-long-url"
send "-this-is-a-long-url"
send "-this-is-a-long-url"
send "-this-is-a-long-url"
send "-this-is-a-long-url"
send "-this-is-a-long-url"
send " HTT"
send "P/1.1"
send "\r"
send "\n"
send "Content-Length: 209\r\n\r\n"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
send "abcdefghijklmnopqrstuvwxyz"
rxresp
expect resp.status == 408
} -run
# Wait matching on log message
barrier b1 sync
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
# Payload is fully sent
# ==> Request must be sent to the server. A 200 must be received
client c2 -connect ${h1_fe1_sock} {
txreq -bodylen 257
rxresp
expect resp.status == 200
} -run
# Wait matching on log message
barrier b1 sync
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
# Payload is fully sent in 2 steps (with a small delay, smaller than the client
# timeout) and split on a chunk size.
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
# ==> Request must be sent to the server. A 200 must be received
client c3 -connect ${h1_fe2_sock} {
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
send "POST /1 HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n1\r\n1\r\n1"
delay 0.01
send "\r\n1\r\n0\r\n\r\n"
rxresp
expect resp.status == 200
} -run
# Wait matching on log message
barrier b1 sync
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
# Last CRLF of the request payload is missing but payload is sent in 2 steps
# (with a small delay, smaller than the client timeout) and split on a chunk
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
# size. The client aborts before sending the last CRLF.
# ==> Request must be handled as an error with 'CR--' termination state.
client c4 -connect ${h1_fe2_sock} {
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer When a message is parsed and copied into the channel buffer, in h1_process_demux(), more space is requested if some pending data remain after the parsing while the channel buffer is not empty. To do so, CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the channel buffer to continue. In the stream-interface, when this flag is set, the SI is considered as blocked on the RX path. It is only unblocked when some data are sent. However, it is not accurrate because the parsing may be stopped because there is not enough data to continue. For instance in the middle of a chunk size. In this case, some data may have been already copied but the parser is blocked because it must receive more data to continue. If the calling SI is blocked on RX at this stage when the stream is waiting for the payload (because http-buffer-request is set for instance), the stream remains stuck infinitely. To fix the bug, we must request more space to the app layer only when it is not possible to copied more data. Actually, this happens when data remain in the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or when we are unable to copy headers or trailers into a non-empty buffer. The first condition is quite easy to handle. The second one requires an API refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been updated. Now it is possible to know when we need more space in the buffer to copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag (H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux(). This patch is part of a series related to the issue #1362. It should be backported as far as 2.0, probably with some adaptations. So be careful during backports.
2021-09-20 05:47:27 +00:00
send "POST /2 HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n1\r\n1\r\n1"
delay 0.01
send "\r\n1\r\n0\r\n"
} -run
syslog S -wait