because multisite replicates multipart objects as a single part, we lose
information about the part sizes from the original manifest that is
necessary to correctly decrypt across those part boundaries
on replication, parse the part lengths out of the source object's
manifest, and store them in a separate RGW_ATTR_CRYPT_PARTS for use on
decryption
Fixes: https://tracker.ceph.com/issues/46062
Signed-off-by: Casey Bodley <cbodley@redhat.com>
users now call a static read_manifest_parts() function, and pass the
resulting vector into the BlockDecrypt constructor
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Update the doc to match the reality in the code. I don't know where
the recommendation to have min_size = k+2 came from, but for awhile
now we've defaulted to K+1. See PR #8008.
Signed-off-by: Dan van der Ster <dan.vanderster@clyso.com>
* refs/pull/52754/head:
test: explicitly link to ceph-common for some libcephfs tests
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
current idle timeout is 30s, so, making the test sleep for 30s may not
be enough. setting sleep time to be longer, and skippign the test so it
won't take too long.
Fixes: https://tracker.ceph.com/issues/62264
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
exporter: fix instance_id labeld value for rgw in exporter
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Instead of saying "[...] support is not supported"
change the first "support" to "operation".
Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
* refs/pull/52675/head:
test/TestOSDMap: don't use the deprecated std::random_shuffle method
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
So stdin-killer and other utilities are installed in the bin directory.
vstart_runner.py now relies on their presence.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Now that the teuthology tools can be run in vstart_runner, there's no
reason to override this method.
Importantly, this enables the use of the new stdin-killer tool [1].
[1] https://github.com/ceph/teuthology/pull/1846
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
With [1], these tools are now installed in the teuthology virtualenv.
Update the path in the command arguments so these tools can be run via
sudo.
[1] https://github.com/ceph/teuthology/pull/1846
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
There's no technical reason to disallow this. The original intent was to
avoid deadlocks but this possibility is already present when interacting
with a teuthology RemoteProcess. Avoiding it only for local processes
does not make sense.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This relies on the new stdin-killer [1] teuthology helper that allows
interacting with the command's stdin.
[1] https://github.com/ceph/teuthology/pull/1846
Fixes: 8bb77ed9e1
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This change causes the program to exit gracefully when stdin is closed
rather than with a Python exception.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Prior to this set of commits, the MDS would write the ESubtreeMap to the
journal, trim everything up to that segment, then finally force the
trimming of that last segment (`MDLog::trim(0)`). This is awkward in the
new code which preserves a major segment boundary at the beginning of
the journal during trimming. Instead of writing a special case for this
situation, it seems more natural to just use a new "lid" or "cap" event
to mark the beginning of the journal when no subtree map can yet be
written but we need sequence numbers to tie in other MDS tables.
Like ESegment, ELid doesn't actually contain any state. It's just a
marker for the beginning the log after rank deactivation or rank
creation. It can appear in the middle of the log if the shutdown
sequence is interrupted while writing the event but the MDS will skip it
during replay in that case.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
When the MDS journal is wiped, EResetJournal is a major segment boundary
as it necessarily begins the journal.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
When the ESubtreeMap is very large (~5k+ subtrees), the MDS will
end up logging only a few events (as bad as 1) per segment as the
subtree map dominates the segment size.
This test simply creates an artificially large subtree and confirms that
other file system activity completes in a timely manner. This is now
taking advantage of the minor segments which allows for a normal set of
events per log segment (and fewer subtree maps). The test fails on the
current main HEAD.
Historical note: when I first observed this abberant behavior, the
vstart cluster was actually using mds_debug_subtrees = True (the default
for every vstart cluster). This caused the MDS to write out the subtree
map (for debugging reasons) with every event. When testing the MDS with
large subtrees (distributed ephemeral pinning), this caused the MDS to
slow to a trickle of operations per second. Despite this unintentional
misconfiguration, the problem still exists but the number of auth
subtrees must be large for a particlar rank to replicate the behavior.
On main HEAD, the creation of 10k files (workload stage) takes ~110
seconds. On this branch, it takes ~30 seconds.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This commit adds a new ESegment event type which can delineate
LogSegments. This event can be used as an alternative to the heavy
weight ESubtreeMap which can be very expensive to generate when the MDS
has a large subtree map.
Fixes: https://tracker.ceph.com/issues/58154
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
The major problem here is that the MDLog::_start_entry method puts the
current event sequence number in the EMetaBlob of the event (if
present). Because of this, no other event can be submitted as this would
invalidate the event sequence. Instead, fixup the event sequence during
submission and simplify related logic that uses it during EMetaBlob
construction.
Secondarily, for the purposes of this commit series, _start_entry
introduced recursive locks when generating the ESubtreeMap within
MDLog::_segment_upkeep. So, this commit is a necessary cleanup.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
