Turned the ConcurrentIO class a template, so that we could use different
kind of containers that are needed for the different operations.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Instead of copy pasting the same code all over again, create a base
class for the needed concurrent IO operations.
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
This became conditional way back in 12e22b3d44eba51a70d8babebc2684f0c46575a7
for unclear reasons. It probably predates the in_use checks. In any case,
at this point, we should only arrive here if the PG was queued, implying
that there will always be an event to process.
Signed-off-by: Sage Weil <sage@redhat.com>
If we don't handle the event, we need to put the PG back into the peering
queue or else the event won't get processed until the next event is
queued, at which point we'll be processing events with a delay.
The queue_null is not necessary (and is a waste of effort) because the
event is still in pg->peering_queue and the PG is queued.
Note that this only triggers when we exceeed osd_map_max_advance, usually
when there is a lot of peering and recovery activity going on. A
workaround is to increase that value, but if you exceed osd_map_cache_size
you expose yourself to crache thrashing by the peering work queue, which
can cause serious problems with heavily degraded clusters and bit lots of
people on dumpling.
Backport: giant, firefly
Fixes: #10431
Signed-off-by: Sage Weil <sage@redhat.com>
Fixes: #10471
Backport: firefly, giant
We need to index the swift keys by the full uid:subuser when decoding
the json representation, to keep it in line with how we store it when
creating it through other mechanism.
Reported-by: hemant burman <hemant.burman@gmail.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
client/Client.cc: In member function 'bool Client::is_quota_bytes_exceeded(Inode*, uint64_t)':
client/Client.cc:10393:66: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (quota->max_bytes && (rstat->rbytes + new_bytes) > quota->max_bytes)
Signed-off-by: Sage Weil <sage@redhat.com>
Currently we make each monitor apply the incremental and encode the full
map locally. The original motivation was to save bandwidth, but the
savings are minimal to modest and the complexity associated with doing this
is huge.
This strategy also causes problems now that we have OSDMap crc's and old
mons/clusters may have diverging full OSDMaps due to mixed version
clusters. See #10422
Instead, include the encoded full map in the paxos transaction. We will
still apply the incremental and check the crc, but if it fails and we have
the correct version, reload it from disk and move on. If we don't, we
will continue as we have before--the primary mon doesn't have support for
crc's yet. When it does we will start verifying and/or get our
full map back into sync.
Fixes: #10422
Signed-off-by: Sage Weil <sage@redhat.com>
Currently we can have a race like so:
- send op in epoch X
- osd replies
- pool deleted in epoch X+1
- client gets X+1, sends map epoch check
- client gets reply
-> fails assert that there is no map check in flight
Avoid this situation by inferring that the pool is deleted when we see
that we previously sent the request but the pool is no longer present.
Since pool ids are not reused there is no point in doing a synchronous
map check at all.
Backport: giant
Fixes: #10372
Signed-off-by: Sage Weil <sage@redhat.com>
Users can easily forget this. It makes shutdown potentially block, but if
they have racing callbacks they get what they ask for.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 30678f6daf)
Add a call so that callers can make sure all queued callbacks have
completed before shutting down the ioctx. This avoids a segv triggered
by the LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1
test due to the ioctx being destroyed when the in-progress callback
does a notify_ack.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4ebd4b4280)
The notify completion has exactly one user, the librados caller which
does nothing but take a local (inner) lock and signal a Cond. Do this
in the fast-dispatch context for simplicity.
Notably, this makes the notify completion (and timeout) trigger a
notify2() return (with ETIMEDOUT) even when the finisher queue that
normally delivers notify is busy.. for example with a notify that is
being very slow. In our case, the unit test is doing a sleep(3) to
test timeouts but also prevented the ETIMEDOUT notification from
being delivered to the caller. This patch resolves that.
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9b78dafd4a)
