2018-12-21 10:56:26 +00:00
|
|
|
varnishtest "A test for http-request-buffer option"
|
|
|
|
feature ignore_unknown_macro
|
|
|
|
|
|
|
|
|
|
|
|
# This test checks HTTP request buffering feature.
|
|
|
|
# We run one server s1 which can serve only one client (no -repeat argument here).
|
|
|
|
# c1 client uses a malformed request which is not transferred to s1 server
|
|
|
|
# thanks to "http-buffer-request". If this was the case, c2 client
|
|
|
|
# could not connect to s1 server and this would lead to make this test fail.
|
|
|
|
|
|
|
|
server s1 {
|
|
|
|
rxreq
|
|
|
|
expect req.bodylen == 257
|
|
|
|
txresp
|
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
When a message is parsed and copied into the channel buffer, in
h1_process_demux(), more space is requested if some pending data remain
after the parsing while the channel buffer is not empty. To do so,
CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the
channel buffer to continue. In the stream-interface, when this flag is set,
the SI is considered as blocked on the RX path. It is only unblocked when
some data are sent.
However, it is not accurrate because the parsing may be stopped because
there is not enough data to continue. For instance in the middle of a chunk
size. In this case, some data may have been already copied but the parser is
blocked because it must receive more data to continue. If the calling SI is
blocked on RX at this stage when the stream is waiting for the payload
(because http-buffer-request is set for instance), the stream remains stuck
infinitely.
To fix the bug, we must request more space to the app layer only when it is
not possible to copied more data. Actually, this happens when data remain in
the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or
when we are unable to copy headers or trailers into a non-empty buffer.
The first condition is quite easy to handle. The second one requires an API
refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been
updated. Now it is possible to know when we need more space in the buffer to
copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag
(H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux().
This patch is part of a series related to the issue #1362. It should be
backported as far as 2.0, probably with some adaptations. So be careful
during backports.
2021-09-20 05:47:27 +00:00
|
|
|
|
|
|
|
accept
|
|
|
|
|
|
|
|
rxreq
|
|
|
|
expect req.bodylen == 2
|
|
|
|
txresp
|
2018-12-21 10:56:26 +00:00
|
|
|
} -start
|
|
|
|
|
|
|
|
syslog S -level info {
|
|
|
|
recv
|
|
|
|
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 fe1/<NOSRV> .* 408 .* - - cD-- .* .* \"GET /this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url HTTP/1\\.1\""
|
|
|
|
recv
|
|
|
|
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/srv1 [0-9]*/[0-9]*/[0-9]*/[0-9]*/[0-9]* 200 .* - - ---- .* .* \"GET / HTTP/1\\.1\""
|
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
When a message is parsed and copied into the channel buffer, in
h1_process_demux(), more space is requested if some pending data remain
after the parsing while the channel buffer is not empty. To do so,
CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the
channel buffer to continue. In the stream-interface, when this flag is set,
the SI is considered as blocked on the RX path. It is only unblocked when
some data are sent.
However, it is not accurrate because the parsing may be stopped because
there is not enough data to continue. For instance in the middle of a chunk
size. In this case, some data may have been already copied but the parser is
blocked because it must receive more data to continue. If the calling SI is
blocked on RX at this stage when the stream is waiting for the payload
(because http-buffer-request is set for instance), the stream remains stuck
infinitely.
To fix the bug, we must request more space to the app layer only when it is
not possible to copied more data. Actually, this happens when data remain in
the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or
when we are unable to copy headers or trailers into a non-empty buffer.
The first condition is quite easy to handle. The second one requires an API
refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been
updated. Now it is possible to know when we need more space in the buffer to
copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag
(H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux().
This patch is part of a series related to the issue #1362. It should be
backported as far as 2.0, probably with some adaptations. So be careful
during backports.
2021-09-20 05:47:27 +00:00
|
|
|
recv
|
|
|
|
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/srv1 [0-9]*/[0-9]*/[0-9]*/[0-9]*/[0-9]* 200 .* - - ---- .* .* \"POST /1 HTTP/1\\.1\""
|
|
|
|
recv
|
|
|
|
expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/<NOSRV> [0-9]*/-1/-1/-1/[0-9]* -1 .* - - CR-- .* .* \"POST /2 HTTP/1\\.1\""
|
2018-12-21 10:56:26 +00:00
|
|
|
} -start
|
|
|
|
|
|
|
|
haproxy h1 -conf {
|
|
|
|
defaults
|
|
|
|
mode http
|
|
|
|
timeout client 100ms
|
|
|
|
timeout server 1s
|
|
|
|
timeout connect 1s
|
|
|
|
|
|
|
|
backend be1
|
|
|
|
server srv1 ${s1_addr}:${s1_port}
|
|
|
|
|
|
|
|
frontend fe1
|
|
|
|
option httplog
|
|
|
|
option http-buffer-request
|
|
|
|
log ${S_addr}:${S_port} local0 debug err
|
|
|
|
bind "fd@${fe1}"
|
|
|
|
use_backend be1
|
|
|
|
} -start
|
|
|
|
|
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
When a message is parsed and copied into the channel buffer, in
h1_process_demux(), more space is requested if some pending data remain
after the parsing while the channel buffer is not empty. To do so,
CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the
channel buffer to continue. In the stream-interface, when this flag is set,
the SI is considered as blocked on the RX path. It is only unblocked when
some data are sent.
However, it is not accurrate because the parsing may be stopped because
there is not enough data to continue. For instance in the middle of a chunk
size. In this case, some data may have been already copied but the parser is
blocked because it must receive more data to continue. If the calling SI is
blocked on RX at this stage when the stream is waiting for the payload
(because http-buffer-request is set for instance), the stream remains stuck
infinitely.
To fix the bug, we must request more space to the app layer only when it is
not possible to copied more data. Actually, this happens when data remain in
the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or
when we are unable to copy headers or trailers into a non-empty buffer.
The first condition is quite easy to handle. The second one requires an API
refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been
updated. Now it is possible to know when we need more space in the buffer to
copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag
(H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux().
This patch is part of a series related to the issue #1362. It should be
backported as far as 2.0, probably with some adaptations. So be careful
during backports.
2021-09-20 05:47:27 +00:00
|
|
|
# 1 byte of the payload is missing.
|
|
|
|
# ==> The request must timed out with a 408 response
|
2018-12-21 10:56:26 +00:00
|
|
|
client c1 -connect ${h1_fe1_sock} {
|
|
|
|
send "GET"
|
|
|
|
send " "
|
|
|
|
send "/this-is-a-long-url"
|
|
|
|
send "-this-is-a-long-url"
|
|
|
|
send "-this-is-a-long-url"
|
|
|
|
send "-this-is-a-long-url"
|
|
|
|
send "-this-is-a-long-url"
|
|
|
|
send "-this-is-a-long-url"
|
|
|
|
send "-this-is-a-long-url"
|
|
|
|
send " HTT"
|
|
|
|
send "P/1.1"
|
|
|
|
send "\r"
|
|
|
|
send "\n"
|
|
|
|
send "Content-Length: 209\r\n\r\n"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
send "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
rxresp
|
|
|
|
expect resp.status == 408
|
|
|
|
} -run
|
|
|
|
|
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
When a message is parsed and copied into the channel buffer, in
h1_process_demux(), more space is requested if some pending data remain
after the parsing while the channel buffer is not empty. To do so,
CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the
channel buffer to continue. In the stream-interface, when this flag is set,
the SI is considered as blocked on the RX path. It is only unblocked when
some data are sent.
However, it is not accurrate because the parsing may be stopped because
there is not enough data to continue. For instance in the middle of a chunk
size. In this case, some data may have been already copied but the parser is
blocked because it must receive more data to continue. If the calling SI is
blocked on RX at this stage when the stream is waiting for the payload
(because http-buffer-request is set for instance), the stream remains stuck
infinitely.
To fix the bug, we must request more space to the app layer only when it is
not possible to copied more data. Actually, this happens when data remain in
the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or
when we are unable to copy headers or trailers into a non-empty buffer.
The first condition is quite easy to handle. The second one requires an API
refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been
updated. Now it is possible to know when we need more space in the buffer to
copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag
(H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux().
This patch is part of a series related to the issue #1362. It should be
backported as far as 2.0, probably with some adaptations. So be careful
during backports.
2021-09-20 05:47:27 +00:00
|
|
|
# Payload is fully sent
|
|
|
|
# ==> Request must be sent to the server. A 200 must be received
|
2018-12-21 10:56:26 +00:00
|
|
|
client c2 -connect ${h1_fe1_sock} {
|
|
|
|
txreq -bodylen 257
|
|
|
|
rxresp
|
|
|
|
expect resp.status == 200
|
|
|
|
} -run
|
|
|
|
|
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
When a message is parsed and copied into the channel buffer, in
h1_process_demux(), more space is requested if some pending data remain
after the parsing while the channel buffer is not empty. To do so,
CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the
channel buffer to continue. In the stream-interface, when this flag is set,
the SI is considered as blocked on the RX path. It is only unblocked when
some data are sent.
However, it is not accurrate because the parsing may be stopped because
there is not enough data to continue. For instance in the middle of a chunk
size. In this case, some data may have been already copied but the parser is
blocked because it must receive more data to continue. If the calling SI is
blocked on RX at this stage when the stream is waiting for the payload
(because http-buffer-request is set for instance), the stream remains stuck
infinitely.
To fix the bug, we must request more space to the app layer only when it is
not possible to copied more data. Actually, this happens when data remain in
the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or
when we are unable to copy headers or trailers into a non-empty buffer.
The first condition is quite easy to handle. The second one requires an API
refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been
updated. Now it is possible to know when we need more space in the buffer to
copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag
(H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux().
This patch is part of a series related to the issue #1362. It should be
backported as far as 2.0, probably with some adaptations. So be careful
during backports.
2021-09-20 05:47:27 +00:00
|
|
|
# Payload is fully sent in 2 steps (with a small delay, smaller than the client
|
|
|
|
# timeout) and splitted on a chunk size.
|
|
|
|
# ==> Request must be sent to the server. A 200 must be received
|
|
|
|
client c3 -connect ${h1_fe1_sock} {
|
|
|
|
send "POST /1 HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n1\r\n1\r\n1"
|
|
|
|
delay 0.01
|
|
|
|
send "\r\n1\r\n0\r\n\r\n"
|
|
|
|
rxresp
|
|
|
|
expect resp.status == 200
|
|
|
|
} -run
|
2018-12-21 10:56:26 +00:00
|
|
|
|
BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel buffer
When a message is parsed and copied into the channel buffer, in
h1_process_demux(), more space is requested if some pending data remain
after the parsing while the channel buffer is not empty. To do so,
CS_FL_WANT_ROOM flag is set. It means the H1 parser needs more space in the
channel buffer to continue. In the stream-interface, when this flag is set,
the SI is considered as blocked on the RX path. It is only unblocked when
some data are sent.
However, it is not accurrate because the parsing may be stopped because
there is not enough data to continue. For instance in the middle of a chunk
size. In this case, some data may have been already copied but the parser is
blocked because it must receive more data to continue. If the calling SI is
blocked on RX at this stage when the stream is waiting for the payload
(because http-buffer-request is set for instance), the stream remains stuck
infinitely.
To fix the bug, we must request more space to the app layer only when it is
not possible to copied more data. Actually, this happens when data remain in
the input buffer while the H1 parser is in states MSG_DATA or MSG_TUNNEL, or
when we are unable to copy headers or trailers into a non-empty buffer.
The first condition is quite easy to handle. The second one requires an API
refactoring. h1_parse_msg_hdrs() and h1_parse_msg_tlrs() fnuctions have been
updated. Now it is possible to know when we need more space in the buffer to
copy headers or trailers (-2 is returned). In the H1 mux, a new H1S flag
(H1S_F_RX_CONGESTED) is used to track this state inside h1_process_demux().
This patch is part of a series related to the issue #1362. It should be
backported as far as 2.0, probably with some adaptations. So be careful
during backports.
2021-09-20 05:47:27 +00:00
|
|
|
# Last CRLF of the request payload is missing but payload is sent in 2 steps
|
|
|
|
# (with a small delay, smaller than the client timeout) and splitted on a chunk
|
|
|
|
# size. The client aborts before sending the last CRLF.
|
|
|
|
# ==> Request must be handled as an error with 'CR--' termination state.
|
|
|
|
client c4 -connect ${h1_fe1_sock} {
|
|
|
|
send "POST /2 HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n1\r\n1\r\n1"
|
|
|
|
delay 0.01
|
|
|
|
send "\r\n1\r\n0\r\n"
|
|
|
|
} -run
|
|
|
|
|
|
|
|
syslog S -wait
|