If we apply or commit a RepModify from a prevous perring interval, we need
to free it.
This fixes 'slow request' messages when in fact clients requests are not
delayed, and plugs the related memory leak.
Signed-off-by: Sage Weil <sage@inktank.com>
CID 745933 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "data_size" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
CID 745935 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "expiration" is not initialized in this constructor nor in any functions that it calls.
At (2): Non-static class member "min_len" is not initialized in this constructor nor in any functions that it calls.
At (4): Non-static class member "max_len" is not initialized in this constructor nor in any functions that it calls.
At (6): Non-static class member "ret" is not initialized in this constructor nor in any functions that it calls.
At (8): Non-static class member "len" is not initialized in this constructor nor in any functions that it calls.
At (10): Non-static class member "ofs" is not initialized in this constructor nor in any functions that it calls.
At (12): Non-static class member "supplied_md5_b64" is not initialized in this constructor nor in any functions that it calls.
At (14): Non-static class member "supplied_etag" is not initialized in this constructor nor in any functions that it calls.
CID 745934 (#1 of 1): Uninitialized pointer field (UNINIT_CTOR)
At (16): Non-static class member "data_pending" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
CID 745936 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "m_off" is not initialized in this constructor nor in any functions that it calls.
Signed-off-by: Sage Weil <sage@inktank.com>
Instead of a special-case exit, just skip the con replacement. Continue
on to mark the old con down.
CID 745920 (#1 of 1): Missing unlock (LOCK)
At (8): Returning without unlocking "this->heartbeat_lock._m".
Signed-off-by: Sage Weil <sage@inktank.com>
CInode::freeze_inode() is used in the case of cross authority rename.
Server::handle_slave_rename_prep() calls it to wait for all other
operations on source inode to complete. This happens after all locks
for the rename operation are acquired. But to acquire locks, we need
auth pin locks' parent objects first. So there is an ABBA deadlock
if someone auth pins the source inode after locks for rename are
acquired and before Server::handle_slave_rename_prep() is called.
The fix is freeze and auth pin the source inode at the same time.
This patch introduces CInode::freeze_auth_pin(), it waits for all
other MDRequests to release auth pins, then change the inode to
FROZENAUTHPIN state, this state prevents other MDRequests from
getting new auth pins.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Use rdlock_try() instead can_read() when path_traverse encounters
a NULL dentry. This can partly avoid infinitely waiting for the
dentry to become readable when the dentry is replica.
Strictly speaking, use rdlock_try() is still enough because auth
MDS may drop the REQRDLOCK message in some cases.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
discover_ino() has a parameter want_xlocked. The parameter indicates
if remote discover handler can proceed when xlocked dentry is
encountered. open_remote_ino() uses discover_ino() to find non-auth
inode, but always set 'want_xlocked' to false. This may cause
dead lock in some corner cases. For example:
we rename a inode's primary dentry to one of its remote dentry and
send slave request to one witness MDS. but before the slave request
reaches the witness MDS, the inode is trimmed from the witness MDS'
cache. Then when the slave request arrives, open_remote_ino() will
be called during traversing the destpath. open_remote_ino() calls
discover_ino() with 'want_xlocled=false' to find the inode.
discover_ino() sends MDiscover message to the inode's authority MDS.
The handler of MDiscover message finds the inode's primary dentry
is xlocked and it sleeps.
The fix is add a parameter 'want_xlocked' to open_remote_ino() and
make open_remote_ino() pass the parameter to discover_ino().
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
During export, it's possible to get cache expire messages in
DISCOVERING, FREEZING and PREPPING state.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
discover_ino() may return -ENOENT if it races with other FS activities.
so use C_MDC_RetryOpenRemoteIno instead of C_MDC_OpenRemoteIno as
onfinish callback.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
The clients may already send caps release message to the exporting
MDS, so the importing MDS waits for the release message forever.
consider revoking caps as issued can avoid this issue.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Locker::acquire_locks() skip auth pinning replica object if we only
request a rdlock and the lock is read-lockable. To get all locks,
we may call Locker::acquire_locks() several times, locks in replca
objects may become not read-lockable between calls. So it is
possible we need auth pin new objects after already take some locks.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Forwarding client request that was from MDS will trigger assertion
in MDS::forward_message_mds(). MDS only send client requests for
stray migration/reintegration, so it's safe to drop them.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
For an inode just changed authority, if the new auth MDS want to
change a lock in the inode from 'sync' to 'lock' state before caps
are exported. The lock in replica can be in 'sync->lock' state
because client caps prevent it from transitting to 'lock' state.
So we should call eval() after clearing client caps.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
So eval_gather() will not skip calling scatter_writebehind(),
otherwise the replica lock may be in flushing state forever.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Both CInode and CDentry's versionlocks are of type LocalLock.
Acquiring LocalLock in replica object is useless and problematic.
For example, if two requests try acquiring a replica object's
versionlock, the first request succeeds, the second request
is added to wait queue. Later when the first request finishes,
MDCache::request_drop_foreign_locks() finds the lock's parent is
non-auth, it skips waking requests in the wait queue. So the
second request hangs.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Unstable locks hold auth_pins on the object, it prevents the freezing
object become frozen and then unfreeze. So try_eval() should not wait
for freezing object
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
If we are failing a pipe, flush the incoming messages before we try to
reconnect. Similarly, flush queued messages on an existing pipe beore we
replace it. This ensures that when we get a socket failure and reconnect
the delayed messages are handled in the normal fashion.
Specifically, it fixes a situation like:
- read msg, update in_seq etc.
- delay msg
- pipe faults
- peer reconnects, we replace existing pipe, discard delayed msgs
- peer resends msgs
- we discard, because they are < in_seq
Signed-off-by: Sage Weil <sage@inktank.com>
We only queue the _applied_recovered_object callback on the primary for the
final push. It is this callback which decrements active_pushes. It's ok to
not increment active_pushes for the intermediate pushes since these only affact
a temp file.
Signed-off-by: Samuel Just <sam.just@inktank.com>
Only one caller can tolerate no segments; make a new
peek_current_segment() for them.
Motivated by paranoia tracking down a crash during client unmount, but
it wasn't this.
Signed-off-by: Sage Weil <sage@inktank.com>
Only request_cleanup() if the request isn't already committing. If it
is, wait for it to commit before we clean up.
It might fix all of #3531, #3210, #1947, and #1548. Maybe.
Signed-off-by: Sage Weil <sage@inktank.com>
We don't want to dump the cache every time an item is trimmed and the
mount_cond gets signaled; this can make umount crazy-slow when logging is
turned up.
Instead, only dump if we wait 5 seconds without making any progress on
shrinking the cache.
Signed-off-by: Sage Weil <sage@inktank.com>
- move all delay state into a single class
- create thread once and only once per Pipe
- adjust debug levels
- discard messages at the appropriate times
Signed-off-by: Sage Weil <sage@inktank.com>
Its life-cycle matches that of delay_queue, and the delayed_delivery
function respects it. For now queue_received is just setting it to
delay everything by 1 second.
Signed-off-by: Greg Farnum <greg@inktank.com>
After some brief thought, I believe deleting any messages in the
delay queue is correct -- we are trying to simulate line delays
in delivery and so anything still in the queue has supposedly
not arrived yet. So delete them when we stop the Pipe for
any reason.
Signed-off-by: Greg Farnum <greg@inktank.com>
The Pipe doesn't know the peer type in the constructor. It
doesn't always know in start_reader either, so this needs more work,
but at least it knows more frequently than it did.
Signed-off-by: Greg Farnum <greg@inktank.com>
When ms_inject_delay_type matches that of the incoming Connection,
the Pipe sets up a delay queue that it shuttles all Messages through.
This lets us check cleanup and some notification code but doesn't
actually generate any delays.
Signed-off-by: Greg Farnum <greg@inktank.com>
It doesn't actually interfere with publish_lock, and the current osdmap
ref.
Document what is going on.
Always preceed publish_map() with one or more pre_publish_map() calls.
Signed-off-by: Sage Weil <sage@inktank.com>
