C_AioRead::finish needs to add in each chunk of a partial read
request to the 'partial' map in the AioCompletion's state
(in destriper, of type StripedReadResult). That map is global
and must be protected from simultaneous access. Use the
AioCompletion lock; could create a separate lock if contention is an
issue.
Fixes: #3567
Signed-off-by: Dan Mick <dan.mick@inktank.com>
During a test_librbd_fsx run including flatten, ImageCtx->parent
was being dereferenced while null. Between the time the parent
overlap is calculated and the time the guard+write completes
with ENOENT and submits the copyup+write, the parent image
could have changed (by resize) or been made irrelevant (by
child flatten) such that the parent overlap is now incorrect.
Handle "no parent" by just sending the copyup+write; the copyup
part will be a no-op. Move to WRITE_FLAT state in this case
because there's no more child to deal with.
Handle "overlap changed" by recalculating overlap before
reading parent data; if none is left, don't read, but rather
just clear m_object_image_extents, in which case the copyup
will again be a no-op because it will be of zero length.
However we still have a parent, so stay in WRITE_COPYUP state
and come back through as usual.
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Fixes: #3524
Server::_rename_prepare() adds remote inode's parent instead of
projected parent to the journal. So during journal replay, the
journal entry for the rename operation will wrongly revert the
remote inode's projected rename. This issue can be reproduced by:
touch file1
ln file1 file2
rm file1
mv file2 file3
After journal replay, file1 reappears and directory's fragstat
gets corrupted.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Creating bloom filter for incomplete dir that was added by log
replay will confuse subsequent dir lookup and can create null
dentry for existing file. The erroneous null dentry confuses the
fragstat accounting and causes undeletable empty directory.
The fix is check if the dir is complete before creating the bloom
filter. For the MDCache::trim_non_auth{,_subtree} cases, just do
not call CDir::add_to_bloom because bloom filter is useless for
replica.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
These asserts are valid for a uniform cluster, but they won't hold
for a replica running a version without the info.last_epoch_started
patch.
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Server::_rename_prepare() adds remote inode's parent instead of
projected parent to the journal. So during journal replay, the
journal entry for the rename operation will wrongly revert the
remote inode's projected rename. This issue can be reproduced by:
touch file1
ln file1 file2
rm file1
mv file2 file3
After journal replay, file1 reappears and directory's fragstat
gets corrupted.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Creating bloom filter for incomplete dir that was added by log
replay will confuse subsequent dir lookup and can create null
dentry for existing file. The erroneous null dentry confuses the
fragstat accounting and causes undeletable empty directory.
The fix is check if the dir is complete before creating the bloom
filter. For the MDCache::trim_non_auth{,_subtree} cases, just do
not call CDir::add_to_bloom because bloom filter is useless for
replica.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Avoid anything on stdout that will generate cron emails for people.
Reported-by: Stefan Priebe <s.priebe@profihost.ag>
Signed-off-by: Sage Weil <sage@inktank.com>
If a subdirectory is specified to ceph_mount, the
root inode does not have an ino of CEPH_INO_ROOT, so
cwd will fail to ever find root and eventially hits
an assertion in in->get_first_parent(). This fix uses
the inode stored in the root member instead, ensuring
that we stop wherever the mount is rooted.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
After replay, we don't know if the dentry removal has already been
committed. Use a sloppy removal so that we succeed even if we are
repeating the operation.
Conveniently, the previous implementation (pre v0.55) silently ignored
tmap op codes it did not understand, which means this new RMSLOPPY will
be interpreted the same as an actual RMSLOPPY. That means an v0.55
mds can run against an older osd (say, argonaut) without problems.
Signed-off-by: Sage Weil <sage@inktank.com>
This reverts 29fae494d0 and fixes the
alternate implmentation added by 8e91d00b52.
librbd relies the ENOENT return value.
Reported-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Rename applied_seq to max_applied_seq, since it is a bound; there may be
seq's < max_applied_seq that are not applied. This aligns the naming with
max_applying_seq.
Signed-off-by: Sage Weil <sage@inktank.com>
We can have a large number of operations in the op_wq waiting to be applied
to the fs. Currently, when we want to commit, we want for them *all* to
apply. This can take a very long time (the default queue length is 500
operations!).
Instead, mark an Op as started ("applying") when the thread pool actually
starts to apply it. At that point, only wait for applying ops to complete.
We let any threads with an op seq < max_applying_seq begin as well so that
we have a proper ordering/barrier. When those flush, applied_seq will ==
max_applying_seq, and that becomes the committing_seq value.
Note that 'applied_seq' is still maintain, but serves no real purpose
except to populate our asserts with sanity checks. max_applying_seq serves
the purpose applied_seq used to.
This removes once unnecessary source of latency associated with fs
commits.
Signed-off-by: Sage Weil <sage@inktank.com>
If we apply or commit a RepModify from a prevous perring interval, we need
to free it.
This fixes 'slow request' messages when in fact clients requests are not
delayed, and plugs the related memory leak.
Signed-off-by: Sage Weil <sage@inktank.com>
CID 745933 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "data_size" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
CID 745935 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "expiration" is not initialized in this constructor nor in any functions that it calls.
At (2): Non-static class member "min_len" is not initialized in this constructor nor in any functions that it calls.
At (4): Non-static class member "max_len" is not initialized in this constructor nor in any functions that it calls.
At (6): Non-static class member "ret" is not initialized in this constructor nor in any functions that it calls.
At (8): Non-static class member "len" is not initialized in this constructor nor in any functions that it calls.
At (10): Non-static class member "ofs" is not initialized in this constructor nor in any functions that it calls.
At (12): Non-static class member "supplied_md5_b64" is not initialized in this constructor nor in any functions that it calls.
At (14): Non-static class member "supplied_etag" is not initialized in this constructor nor in any functions that it calls.
CID 745934 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
At (16): Non-static class member "data_pending" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
CID 745936 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "m_off" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
Instead of a special-case exit, just skip the con replacement. Continue
on to mark the old con down.
CID 745920 (#1 of 1): Missing unlock (LOCK)
At (8): Returning without unlocking "this->heartbeat_lock._m".
Signed-off-by: Sage Weil <sage@inktank.com>
CInode::freeze_inode() is used in the case of cross authority rename.
Server::handle_slave_rename_prep() calls it to wait for all other
operations on source inode to complete. This happens after all locks
for the rename operation are acquired. But to acquire locks, we need
auth pin locks' parent objects first. So there is an ABBA deadlock
if someone auth pins the source inode after locks for rename are
acquired and before Server::handle_slave_rename_prep() is called.
The fix is freeze and auth pin the source inode at the same time.
This patch introduces CInode::freeze_auth_pin(), it waits for all
other MDRequests to release auth pins, then change the inode to
FROZENAUTHPIN state, this state prevents other MDRequests from
getting new auth pins.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Use rdlock_try() instead can_read() when path_traverse encounters
a NULL dentry. This can partly avoid infinitely waiting for the
dentry to become readable when the dentry is replica.
Strictly speaking, use rdlock_try() is still enough because auth
MDS may drop the REQRDLOCK message in some cases.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
discover_ino() has a parameter want_xlocked. The parameter indicates
if remote discover handler can proceed when xlocked dentry is
encountered. open_remote_ino() uses discover_ino() to find non-auth
inode, but always set 'want_xlocked' to false. This may cause
dead lock in some corner cases. For example:
we rename a inode's primary dentry to one of its remote dentry and
send slave request to one witness MDS. but before the slave request
reaches the witness MDS, the inode is trimmed from the witness MDS'
cache. Then when the slave request arrives, open_remote_ino() will
be called during traversing the destpath. open_remote_ino() calls
discover_ino() with 'want_xlocled=false' to find the inode.
discover_ino() sends MDiscover message to the inode's authority MDS.
The handler of MDiscover message finds the inode's primary dentry
is xlocked and it sleeps.
The fix is add a parameter 'want_xlocked' to open_remote_ino() and
make open_remote_ino() pass the parameter to discover_ino().
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
During export, it's possible to get cache expire messages in
DISCOVERING, FREEZING and PREPPING state.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
discover_ino() may return -ENOENT if it races with other FS activities.
so use C_MDC_RetryOpenRemoteIno instead of C_MDC_OpenRemoteIno as
onfinish callback.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
The clients may already send caps release message to the exporting
MDS, so the importing MDS waits for the release message forever.
consider revoking caps as issued can avoid this issue.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Locker::acquire_locks() skip auth pinning replica object if we only
request a rdlock and the lock is read-lockable. To get all locks,
we may call Locker::acquire_locks() several times, locks in replca
objects may become not read-lockable between calls. So it is
possible we need auth pin new objects after already take some locks.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Forwarding client request that was from MDS will trigger assertion
in MDS::forward_message_mds(). MDS only send client requests for
stray migration/reintegration, so it's safe to drop them.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
For an inode just changed authority, if the new auth MDS want to
change a lock in the inode from 'sync' to 'lock' state before caps
are exported. The lock in replica can be in 'sync->lock' state
because client caps prevent it from transitting to 'lock' state.
So we should call eval() after clearing client caps.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
So eval_gather() will not skip calling scatter_writebehind(),
otherwise the replica lock may be in flushing state forever.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
