RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-28 14:34:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

rgw: fix chunked-encoding for chunks >1MiB

Browse Source 
For HTTP responses sent with chunked-encoding, and greater than 1MiB in
size, the chunk-size field was being printed wrong.

Specifically, the chunk-size field was being sent with a mangled or
missing trailer of '\r\n'.

This bug manifested as HTTP clients being unable to read the response:
Chrome generates ERR_INCOMPLETE_CHUNKED_ENCODING
Python/boto generates httplib.LineTooLong: got more than 65536 bytes when reading chunk size

The wrong variable was being used to determine the size of the buffer
used for the chunk-size field.

Fix it by using the correct variable, and rename the variables to
clearly reflect their purpose.

Prior to PR#23940, this would only have been seen in some Swift
operations. PR#23940 changed some S3 operations to also use chunked
encoding to get responses sent faster, and made the bug easier to
detect. It was initially reported for a ListBucket call with a high
max-keys argument.

Backport: luminous, mimic
Reference: https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
Reference: https://github.com/ceph/ceph/pull/23940
Fixes: http://tracker.ceph.com/issues/35990
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
This commit is contained in:
Robin H. Johnson 2018-09-14 14:23:49 -07:00
parent 07a3e8d7a6
commit 3b864482d6
No known key found for this signature in database
GPG Key ID: 19395F23C58826C4
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
src/rgw/rgw_client_io_filters.h
View File

@ -286,11 +286,15 @@ public:
return DecoratedRestfulClient<T>::send_body(buf, len);
} else {
static constexpr char HEADER_END[] = "\r\n";
char sizebuf[32];
const auto slen = snprintf(sizebuf, sizeof(buf), "%" PRIx64 "\r\n", len);
/* https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1 */
// TODO: we have no support for sending chunked-encoding
// extensions/trailing headers.
char chunk_size[32];
const auto chunk_size_len = snprintf(chunk_size, sizeof(chunk_size),
"%" PRIx64 "\r\n", len);
size_t sent = 0;
sent += DecoratedRestfulClient<T>::send_body(sizebuf, slen);
sent += DecoratedRestfulClient<T>::send_body(chunk_size, chunk_size_len);
sent += DecoratedRestfulClient<T>::send_body(buf, len);
sent += DecoratedRestfulClient<T>::send_body(HEADER_END,
sizeof(HEADER_END) - 1);