RGW: added a per topic configuration to control the notification persistency
Adding the configuration of persistency per topic that will override the global settings
Reviewed-by: yuvalif <ylifshit@redhat.com>
qa/config/crimson_qa_overrides: ignore POOL_APP_NOT_ENABLED
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
For snapshot-based mirroring, check that demote (or other mirror
snapshots) don't pile up. Nothing in particular to assert on for
journal-based mirroring but the test is still useful.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
After commit ac552c9b4d ("librbd: localize snap_remove op for mirror
snapshots"), rbd-mirror daemon no longer removes mirror snapshots when
it's done syncing them -- instead it only unlinks from them. However,
CreatePrimaryRequest state machine was not adjusted to compensate and
hence two cases were missed:
- primary demotion snapshot (rbd-mirror daemon unlinks from primary
demotion snapshots just like it does from regular primary snapshots);
this comes up when an image is demoted but then promoted on the same
cluster
- non-primary demotion snapshot (unlike regular non-primary snapshots,
non-primary demotion snapshots store peer uuids and rbd-mirror daemon
does unlinking just like in the case of primary snapshots); this
comes up when an image is demoted and promoted on the other cluster
Related is the case of orphan snapshots. Since they are dummy to begin
with, CreatePrimaryRequest would now clean up the orphan snapshot after
the creation of the force promote snapshot.
Fixes: https://tracker.ceph.com/issues/61707
Co-authored-by: Christopher Hoffman <choffman@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Despite being mirror snapshots, orphan snapshots don't have image
state: see CreateNonPrimaryRequest::write_image_state() for a similar
is_orphan() check. Attempting to remove image state generates bogus
"failed to read image state object" and "failed to remove image state"
errors.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
The tool is dedicated to create highly variable workloads.
The intended audience is
1) scraper testing - tool to sniff on OSD ops
2) CoDel testing - bandwidth/latency optimization algorithm in BS
Initial commit.
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
Address review comments that remained outstanding in
https://github.com/ceph/ceph/pull/53107:
- CephFS capitalization
- missing PR in the list of changes
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Boost stacktrace defines a few UUIDs that were recently added
to mingw as well [1], causing compilation errors [2]:
In file included from libs/stacktrace/build/../src/windbg.cpp:9:
./boost/stacktrace/detail/frame_msvc.ipp:31:5: error: redefinition of
'__mingw_uuidof_s<IDebugClient>'
__CRT_UUID_DECL(IDebugClient,0x27fe5639,...
We'll apply a fix that hasn't merged upsteam yet [3].
[1] ce5a9f624d
[2] https://github.com/boostorg/stacktrace/issues/133
[3] https://github.com/boostorg/stacktrace/pull/140
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
mingw-llvm can't handle the '--exclude-libs' linker flag, so
we'll have to skip it.
At the same time, cmake can't locate the boost shared libs as the
import libs are not generated when using mingw-llvm due to boost's
clang-linux.jam file. For now, we'll patch the cmake files, using
the dlls as import libs (which is allowed by mingw).
While at it, we'll avoid linking the static AND dynamic boost libs,
speeding the build.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Because of winpthreads issues, we had to use Boost's shared_mutex
implementation.
When using mingw-llvm, we can safely use libc++'s shared mutex
implementation.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
std::condition_variable::wait_for uses SleepConditionVariableSRW
on Windows, which has millisecond precision.
In order to avoid busy loops, we won't wait for less than one
millisecond on Windows.
Note that this situation is quite common since on Windows,
"wait_for" often returns ~1ms before the specified timeout.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
The monotonic clocks are commonly used for measuring time deltas,
which can be negative.
ceph::mono_clock and ceph::coarse_mono_clock currently use
unsigned duration types [1]. The difference operators are overloaded
in order to ensure that the result is signed [2][3].
However, we still have issues when unsigned timespans are compared.
For example, std::condition::wait_for can hang indefinitely due
to underflows [4][5]. It ends up using our unsigned type for a
negative timespan, which is then compared to
std::chrono::duration<Rep,Period>::zero.
In order to avoid such problems, we'll simply use a signed type
for monotonic clock durations.
With signed timespans, we can no longer assume that time_point::zero()
is equal to time_point::min(), so we're updating it accodingly.
[1] 4040f12347/src/common/ceph_time.h (L285)
[2] 4040f12347/src/common/ceph_time.h (L345-L380)
[3] 4040f12347/src/common/ceph_time.h (L466-L487)
[4] 91cff8a718/libcxx/include/__condition_variable/condition_variable.h (L178)
[5] 91cff8a718/libcxx/include/__condition_variable/condition_variable.h (L193)
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
We've been experiencing timer hangs with mingw-llvm.
std::condition_variable::wait_for was returning a few microseconds
before the requested time and then hanging when called with a
small interval (e.g. microseconds).
This was affecting the OSD periodic tick, which would hang after
a while (20m up to 2h).
The issue can be reproduced with a timer loop and a small interval
(e.g. 40us), in which case the timer is likely to hang after about
10s.
We're adding some tests, while the actual mingw-llvm issue will
be mitigated in a separate commit.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
winpthreads is a library that emulates the pthreads API using
Windows primitives. It's also used by the mingw/gcc libstdc++
for std::thread.
The issue is that winpthreads isn't well maintained. There
have been numerous bugs that haven't been addressed in years.
Specifically, we've been hitting deadlocks because of the
winpthreads rw lock implementation.
This change will allow building Ceph for Windows using mingw/llvm,
which uses libc++ and doesn't rely on winpthreads.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
We're checking a permission denied exception message that's
runtime specific:
/mnt/data/workspace/ceph_mingw_clang/src/test/dokan/dokan.cc:252: Failure
Expected equality of these values:
e.what()
Which is: "filesystem error: in remove: Permission denied
[\"Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5\"]"
exception_msg.c_str()
Which is: "filesystem error: cannot remove: No such device
[Z:\\ro_success_b76223c4-c590-45e0-ab78-4d281ac512b5]"
In order to support libc++, we'll drop the exception message
assertion and rely on the exception type.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
This test currently failes to build for Windows using llvm
due to unresolved symbols.
We'll address the issue by explicitly specifying the ceph-common
dependency.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Especially when targeting Windows, llvm may not necessarily
use pthreads for std::thread. In this case, we must not use the
"native" thread handle with the pthreads API.
We'll update the ceph_pthread_getname and ceph_pthread_setname
wrappers, adding a new one: ceph_pthread_kill.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
The "-Bsymbolic" and "-Bsymbolic-functions" flags only apply to ELF
binaries.
llvm errors out when targeting Windows, which is why we'll need
to skip those flags for Windows builds.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
clang errors out because of a few type mismatches that gcc
ignored through "-fpermissive".
We'll need to cast a few void pointers to the appropriate type.
There's also a function that doesn't have an explicit return type,
which was omitted by mistake.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Some symbols from the crc32, arch and fmt libs
are re-exported by libceph-common:
FAILED: bin/unittest_time.exe
ld.lld: error: fmt::v9::format_error::~format_error() was replaced
llvm throws errors because of the duplicate symbols.
One workaround is to use objects instead of static libs
for the libs. For libfmt we'll use the header-only version.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
