librados, pybind, tests: allow to list objects with the NUL character in names
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
It's needed to properly list an object with the NUL
character in the name. Such names can appear mostly
as a consequence of bugs in clients.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
At the moment, the Windows binaries can only be linked statically.
This is less than ideal since:
* the resulting binaries can be quite large, especially when
including debug symbols
* librados and librbd cannot be used directly by 3rd party apps
* the build duration is increased
In order to do a dynamic build, we'll:
* add an option to win32_build.sh
* fix link order
* dynamically link boost
* disable the "-pie" flag when using Mingw, which generates incorrect
executable entry points
* by default, cmake generates import libs for executables. The issue
is that for rados.exe, it generates librados.dll.a, thus overriding
the librados.dll import library, which breaks the build process.
We'll configure cmake to skip generating import libs for executables.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
symver isn't available on Windows. For now, we'll just pick the
default function version.
In the future, we may want to reconsider the versioning approach.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
The timeout was not initialized with the current config value during
RadosClient construction.
Fixes a8a23747aa
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Commit 306eebe05b added the RadosClient to
the config observer interface but forgot to add it to the observer list.
This meant that the RadosClient was never receiving callbacks to
handle_conf_change. Commit a8a23747aa
exposed this issue because the mon timeout was configured at run-time.
Fixes: https://tracker.ceph.com/issues/48030
Fixes: a8a23747aa
Fixes: 306eebe05b
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
boost::asio uses static member variables in a header-only library to
provide thread local storage. When boost::asio is used in multiple
independent shared libraries (e.g. librados and librbd), each module
will have its own version of the static variables and therefore will
cause inconsistencies when attempting to access the data.
Under GNU ELF extensions, static member variables are given global
unique symbol visibility. This was broken in librados due to the
symbol map, so manually export the static member variables so that
the dynamic linker can properly consolidate the variables.
See github.com boostorg/asio issue 150
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
* refs/pull/37529/head:
qa: set rados op timeouts for mds/ceph-fuse
qa: print debug info on mount cleanup
qa: remove redundant rmr
qa: use null mode to prevent undesired changes to mountpoint
qa: unmount all clients before deleting the file system
osdc: add timeout configs for mons/osds
common: accept timespan for SaferCond.wait_for
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
After deciding to always enable tracking log in early phase, there's no
need to keep "log_early" option here and remove it directly.
Suggested-by: Kefu Chai <kefu@redhat.com>
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
Have the Objecter track the rados_(mon|osd)_op_timeout configs so that
it can be configured at runtime/startup. This is useful for the
MDS/ceph-fuse so that we can avoid waiting forever for a response from
the Monitors that will never come (statfs on a deleted file system's
pools).
Also: make these configs take a time value rather than double. This is
simpler to deal with in the code and allows time units to be used (e.g.
"5m" for 5 minutes).
Fixes: https://tracker.ceph.com/issues/47734
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/35697/head:
doc: document cephfs mirroring developement status
cephfs-mirror: cephfs-mirror daemon
cephfs-mirror: FSMirror class to synchronize directory snaps
common: introduce configuration options for cephfs-mirror daemon
cephfs-mirror: InstanceWatcher class to register mirror instance
cephfs-mirror: MirrorWatcher class to watch cephfs-mirror object
cephfs-mirror: Watcher class to receive/ack watch notifications
cephfs-mirror: ClusterWatcher class for watching peer changes
librados: add API for fetching client addrs
common: add cephfs_mirror subsystem
cephfs-mirror: filesystem specification class
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
I would have expected ENOSYS or EOPNOTSUPP or similar, but the mon returns
EINVAL on an unrecognized command
Signed-off-by: Sage Weil <sage@newdream.net>
osd: refcounting chunks for snapshotted manifest object
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com> (portions Sam wrote)
Windows support [part 4]
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
There are a few name collisions between Windows headers and ceph.
Where possible, we'll update Ceph code to avoid redefinitions.
The following names overlap with Windows headers:
* ERROR, NO_ERROR
* DELETE
* LocalLock
* LOCK_SHARED, LOCK_EXCLUSIVE
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Client libraries like librbd cannot force the adaptation to neorados
and it would not be ideal to have two effective RADOS cluster
connections for librados and neorados.
The new neorados::RADOS::make_with_librados helper method will
allow neorados to re-use the existing thread pool, MGR, MON, and
objecter already created for librados::Rados.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
The motivation is that `claim(list&)` seems to actually be a pre-C++11
counterpart of the already available `operator=(list&&)`.
This commit deprecates the `claim()` method but doesn't drop it yet.
All occurrences of `buffer::list::claim(list&)` are switched to
* `list::operator=(list&&)` or
* reworked to use `list::list(list&&)` instead.
Changes are applied to: rgw, osdc, osd, os/memstore, os/filestore,
os/bluestore, os, msg, mgr, messages, mds, librbd, librados, crimson,
common, cls, mon.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
existing flush is the ReadOperation in order to flush older
clone that head.
Like existing flush, change tier-flush's type to ReadOperation
Signed-off-by: Myoungwon Oh <ohmyoungwon@gmail.com>
Thanks to Casey Bodley <cbodley@users.noreply.github.com> for
watch/notify fixes and Patrick Donnelly <pdonnell@redhat.com> for MDS
fix.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
some code coexist in crimson seastar environment and posix environment,
so add namespace to avoid same function conflict, for example add namespace
for CephContext, since the new namespace for classic ceph-osd,
need modify all files declare use CephContext by including "common_fwd.h"
which defined the namespace for each environment.
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
this would allow returning values from sync write operations
by passing the RETURNVEC flag to the sync versions of the APIs
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
* refs/pull/32831/head:
common, include: drop the copy{_in} from bufferlist entirely.
os/bluestore: switch copy_in() users to bufferlist::iterator.
osdc: switch users of bufferlist::copy{_in} to iterators.
osd: switch users of bufferlist::copy{_in} to iterators.
rgw: switch copy{_in} users to bufferlist::iterator.
ec: switch users of bufferlist::copy{_in} to iterators.
cls/queue: switch users of bufferlist::copy{_in} to iterators.
client: switch users of bufferlist::copy{_in} to iterators.
*: switch trivial users of bufferlist::copy{_in} to iterators.
test/bl: switch copy{_in} users to bufferlist::iterator.
common, include: kill the bl::last_p member.
common: encode for std::list<T> doesn't use bl::copy_in() anymore.
Reviewed-by: Kefu Chai <kchai@redhat.com>
This commits switches the most straightforward occurrences of:
* ceph::bufferlist::copy(off, len, dst) and
* ceph::bufferlist::copy_in(off, len, src)
to use iterators. After extending `bufferlist::begin()` with
the `offset` parameter and eradication of the `last_p` hint,
these method look rudimentary. There is no huge benefit from
bl.copy_in(off, len, dst)
instead of:
bl.begin(off).copy_in(len, dst)
especially when `off` is `0`.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
The mgr creates a per-module RADOS client connection for modules which
interact with RADOS (e.g. the volumes module). These clients should also
be blacklisted when the active mgr is failed; we don't want the former
active mgr to continue interacting with RADOS when the new one takes
over. This is particularly impactful for avoiding extraneous
"unresponsive client" warnings from the MDS when the mgr switches
(especially in testing). The MDS will pickup the new OSD blacklists
which include's the old mgr's libcephfs instance and blacklist/evict
that session quietly.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
before this change, the librados applications are responsible to call
`AioCompletion::release()` explicitly to release its internal pimpl
pointer. this is error prone and not intuitive.
after this change, the destructor of `AioCompletion` and
`PoolAsyncCompletion` will do this automatically. while
`AioCompletion::release()` and `PoolAsyncCompletion::release()` still
delete the instance as they did before. so this change is backward
compatible, as existing librados clients can still use `ptr->release()`
to free the completion instance, while new clients can just `delete
ptr`.
librados_test_stub is updated accordingly to match the new model
Signed-off-by: Kefu Chai <kchai@redhat.com>
