This can be run by hand with:
./mvnw clean package
./mvnw test -Dtest=PutObjects
The following properties are sourced from the environment:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
RGW_HTTP_ENDPOINT_URL
RGW_HTTPS_ENDPOINT_URL
Then adds:
qa/workunits/rgw: add test driver script for maven suite
Launch it fromn cls.yaml, as with test_librgw_file.h.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
These changes address checksum header identification and signing
algorithm selection, including checksum trailer verification
for signed- and unsigned-payload cases.
These changes address all the actual S3 request failures I have
so far been able to reproduce, with and without content checksums
and/or new trailing checksum headers, and with and without
SSL.
Fixes: https://tracker.ceph.com/issues/63153
Specifically, it fixes the request failures that motivated the
initial tracker filing. It extracts but does not validate new client
content checksums if present. Validation and management of new
S3 content-checksum headers will follow in a subsequent change.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
squashed commits:
* wip chunk meta parsing--seem to have first AWSv4ComplMulti::ChunkMeta::create_next sort of parsing
* use constexpr sarlen(...) for static array lengths throughout rgw_auth_s3.cc
* link AWSv4CompleMulti::ChunkMeta to its enclosing completer
* capture original content-length header before AWSv4ComplMulti overwrites it
* mostly extract the trailer
* fix misordered content-length, experiment w/exbuf
* save leftover bytes between calls to AWSv4ComplMulti::recv_chunk()
* propagate data_offset_in_stream from AWSv4ComplMulti::recv_chunk()
* clean up trailer section extract
* trailer section cleanup and introduce extract_helper
* unrolled checksum extract--fixup
* fix sv_trailer end pos, and cleanup
* add proplist interface to rgw::auth::Completer and AWSv4ComplMulti
* spliterate trailers
* check completer props
* redefine prop_map to point into already-allocated trailer_vec
* hax: thread a counter onto AWSv4ComplMulti recv_body() and recv_chunk path
* fix apparent bug where due to reads less than chunk_size induce a final, zero-length read that was skipped before forcing recognition of the last chunk in the stream
* check only for a trailing checksum named in x-amz-trailer
* don't try to match signatures when no signature provided (because streaming unsigned)
* oops, fix content_length decl
* fix recognition of next chunk envelope in unsigned aws-chunk case
* clean up AWSv4CompMulti flags and correctly detect aws unsigned chunked
* rework checksum-trailer extraction and introduce AWSv4ComplMulti::calc_v4_trailing_signature
* thread const struct req_state* into AWSv4ComplMulti
* large cleanup of trailer parsing, no regression
* fix trailer signature calculation--checks
* correctly generate final chunk hmac
* typo in comment
* verify trailing signature when expected (using expected final chunk signature)
* move trailer_vec back onto recv_body()'s stack
* remove strange completer comment
* remove last_frag (now points into parsing_buf)
* remove implied dependency on content_length
* move trailer recognition to AWSv4ComplMulti::complete()
* remove now-unused is_last_chunk() predicate
* remove unused ChunkMeta::completer
* responses to review comments
* when trailer is sig expected, fail (only) if none present or if it does not match calculated
* remove stale parse_content_length(...) decl
* remove now-unused AWSv4ComplMulti::content_length
* fix extract_helper end search position as in mut_extract_helper
* change "\n" reserve term in get_canon_amz_hdrs() part of the sum (review)
and initialize length to 0
* remove debugging code
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
s/nfs-ganesha/.nfs/ in doc/cephadm/services/nfs, in accordance with
information given to me by John Mulligan over Slack.
Signed-off-by: Zac Dover <zac.dover@proton.me>
user info as 'uid' in dashboard. 'full_user_id' includes
tenant and user id, and also namespace wherever it is non
empty.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
cephadm/rgw: make rgw perf counters cache and it's size configurable
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
This commit adds support for subvolume snap scheduling and translations for snap schedule repeat schedules and retention policies
Fixes: https://tracker.ceph.com/issues/64372
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
This is v2 of the rbd/nvmeof test: It deploys 1 gateway and 1 initiator.
Then does basic verification on nvme commands and runs fio.
This commit creates:
1. qa/tasks/nvmeof.py: adds a new 'Nvmeof' task which deploys
the gateway and shares config with the initiator hosts.
Sharing config was previously done by 'nvmeof_gateway_cfg' task
in qa/tasks/cephadm.py (that task is removed in this commit).
2. qa/workunits/rbd/nvmeof_basic_tests.sh:
Runs nvme commands (discovery, connect, connect-all, disconnect-all,
and list-subsys) and does basic verification of the output.
3. qa/workunits/rbd/nvmeof_fio_test.sh:
Runs fio command. Also runs iostat in parallel if IOSTAT_INTERVAL
variable is set. This variable configures the delay between each iostat
print.
nvmeof-cli upgrade from v0.0.6 to v0.0.7 introduced major changes
to all nvmeof commands. This commit changes v0.0.6 commands to
v0.0.7 in qa/workunits/rbd/nvmeof_initiator.sh
Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
before this change, we link against libcap without finding it. this
works fine as long as libcap-devel or libcap-dev is installed in the
system. but if it is not, the source would fail to build due to missing
`sys/capability.h`. this is not a great developer experience.
in this change, a `Findcap.cmake` is added to find the capability
library. which would fail the build at the configure phase.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
boost has some different predefined build variants. they are quite
like CMake's CMAKE_BUILD_TYPE. in which, "debug" enables some
features related features. so it would be nice if we can have it
enabled for the Debug build, if the boost is built from source.
see also
https://www.boost.org/build/doc/html/bbv2/overview/builtins/features.html
before this change, we always build the "release" variant. in this
change, "debug" variant is built if Ceph's is built with
CMAKE_BUILD_TYPE=Debug. please note, this change does not change
the way how boost is built when packaging Ceph, as our debian/rpm
receipts do not define CMAKE_BUILD_TYPE and respect the distros'
settings, in that case, the "release" variant is still built.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Remove "background", "getdata", and "putdata" from the list of LUA
context options. Passing these options throws the following error:
"ERROR: invalid script context: background. must be one of: preRequest,
postRequest".
Fixes: https://tracker.ceph.com/issues/64327
Signed-off-by: Zac Dover <zac.dover@proton.me>
work around a deadlock under RGWDataChangesLog::add_entry() with
multi-object delete by making the push() call synchronous
this is a partial revert of d38a69b6e1
Fixes: https://tracker.ceph.com/issues/63373
Signed-off-by: Casey Bodley <cbodley@redhat.com>
In order to enable users who might like to use `radosgw-admin object
manifest ...` to get a complete list of raw objects that comprise the
RGW object, we'll include the head object when it includes not
data. It will be indicated with an "index" of -1.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Adds a new sub-command to radosgw-admin to list the manifest, which
are the rados objects that contain the RGW object's data.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
