to list inconsistent snapsets of a given PG, this command exposes
get_inconsistent_snapsets() rados API to user.
Fixes: #13505
Signed-off-by: Kefu Chai <kchai@redhat.com>
to list inconsistent objects of a given PG, this command exposes
get_inconsistent_objects() rados API to user.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the inconsistent snapsets are identified in ReplicatedPG::_scrub()
after we compared the authorized objects with their replica/shards.
these inconsistent information are stored in the omap of objects
with prefix "SCRUB_SS_".
Signed-off-by: Kefu Chai <kchai@redhat.com>
persist inconsistent objects found when comparing the ScrubMaps
collected from replica/shards. the discrepancies between the auth
copy and the replica are identified as inconsistencies. and hence
encoded into the omap of an object of the temp coll of the PG in
question.
scrub_types.{h,cpp} are introduced to hide the details of how we
persist the scrub types from the librados client.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to list inconsistent PGs of a given pool. this command exposes
the underlying get_inconsistent_pgs() API to user.
Signed-off-by: Kefu Chai <kchai@redhat.com>
These will track whether local images should be mirrored, and map them
to a unique global id. There's a state field for safely disabling
mirroring while operating on multiple objects.
Fixes: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
We'll use this object only for mirroring-related purposes, not generic
settings on a pool.
Refs: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Tracking completions is critical for memory safety - if the
aio function succeeds, the completion must be tracked. However,
if a KeyboardInterrupt or similar arrives between the call and
the tracking, the completion will not be tracked.
Fix this by tracking the completion before the aio call, and
explicitly cleaning up in the failure case.
This leaves the opposite problem, where an unexpected exception
(other than simple error return from the aio function) will cause
the completion to not be freed until the Ioctx is destroyed, but
that is a relatively minor issue.
Signed-off-by: Hector Martin <marcan@marcan.st>
aio_read:
The reference to ret_s begins existing at PyBytes_FromStringAndSize and
is handed over to the callback if rados_aio_read succeeds. This creates
a lot of subtle scenarios where it might not be XDECREFed (e.g. if
a KeyboardInterrupt arrives at the wrong time). Instead, store the pointer
to that buffer in the completion object, and hand over responsibility for
the XDECREF to it. This guarantees that the "special" reference to this
incomplete object will be released when the completion object is
deallocated.
Also make sure we don't try to _PyBytes_Resize with a negative length.
Add a failure case to the aio_read test in test_rados.py
Completion: the wrapper methods weren't being called, which prevents
the completion objects from being freed until the Ioctx is. Fix this
and add a refcount check to the aio_read test.
Signed-off-by: Hector Martin <marcan@marcan.st>
When running the ceph-disk check suite, 'test_mark_init' & 'test_activate_dir' had a
permission denied of the freshly created stuff like in :
tests/ceph-disk.sh:237: test_mark_init: /bin/rm -fr /ceph/src/ceph-disk/test-ceph-disk/dir
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/snap_1 »: Permission denied
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/current »: Permission denied
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/snap_2 »: Permission denied
Those two tests are starting a mon with run_mon.
Trying to delete the files while the daemon are still alive is impossible.
This patch removes the explicit 'rm' and let the teardown doing the cleaning
stuff by stopping daemons & cleaning content.
By using this patch, make check is now sucessful on ceph-disk.
Signed-off-by: Erwan Velu <erwan@redhat.com>
When doing a make test, if your local setup was running a "dm-0" or "dm-1",
the make check was failing.
The detection of the local "dm-x" were leading to a wrong comparaison with
the expected simulated values.
The fix is using a dummy name for that "dm" to prevent any collision with the local setup.
Prior that patch, a typical output of that error looked like :
> assert expect == main.list_devices()
E assert [{'partitions...: '/dev/Xda'}] == [{'partitions'...: '/dev/Xda'}]
E At index 0 diff: {'partitions': [{'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'}, 'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d', 'path': '/dev/Xda1', 'is_partition': True, 'mount': None, 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2', 'type': 'data', 'state': 'unprepared', 'fs_type': None}], 'path': '/dev/Xda'} != {'partitions': [{'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d', 'whoami': None, 'path': '/dev/Xda1', 'is_partition': True, 'mount': '/var/cache/ccache', 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2', 'ceph_fsid': None, 'fs_type': 'btrfs', 'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'}, 'type': 'data', 'state': 'active'}], 'path': '/dev/Xda'}
E Full diff:
E - [{'partitions': [{'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'},
E + [{'partitions': [{'ceph_fsid': None,
E + 'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'},
E - 'fs_type': None,
E ? ^^^^
E + 'fs_type': 'btrfs',
E ? ^^^^^^^
E 'is_partition': True,
E - 'mount': None,
E + 'mount': '/var/cache/ccache',
E 'path': '/dev/Xda1',
E 'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d',
E - 'state': 'unprepared',
E ? ^^^^ -----
E + 'state': 'active',
E ? ^^^^^
E 'type': 'data',
E - 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2'}],
E ? --
E + 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2',
E + 'whoami': None}],
E 'path': '/dev/Xda'}]
tests/test_main.py:342: AssertionError
Signed-off-by: Erwan Velu <erwan@redhat.com>
When running run-tox.sh in a very simple env,
the test will fail if '/var/lib/ceph/tmp' doesn't exist.
This patch adds a check to create this directory if required as mkdtemp doesn't do it for you.
Prior this patch, the following behavior was seen :
tests/test_main.py:342:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ceph_disk/main.py:3753: in list_devices
fstype=fs_type, options='')
ceph_disk/main.py:1217: in mount
dir=STATEDIR + '/tmp',
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
suffix = '', prefix = 'mnt.', dir = '/var/lib/ceph/tmp'
def mkdtemp(suffix="", prefix=template, dir=None):
"""User-callable function to create and return a unique temporary
directory. The return value is the pathname of the directory.
Arguments are as for mkstemp, except that the 'text' argument is
not accepted.
The directory is readable, writable, and searchable only by the
creating user.
Caller is responsible for deleting the directory when done with it.
"""
if dir is None:
dir = gettempdir()
names = _get_candidate_names()
for seq in xrange(TMP_MAX):
name = names.next()
file = _os.path.join(dir, prefix + name + suffix)
try:
> _os.mkdir(file, 0700)
E OSError: [Errno 2] No such file or directory: '/var/lib/ceph/tmp/mnt.KoAV85'
/usr/lib64/python2.7/tempfile.py:333: OSError
Signed-off-by: Erwan Velu <erwan@redhat.com>
changes to WeightedPriorityQueue there is no strict round robin
dequeueing of classes. Removed that test from the unittest.
Signed-off-by: Robert LeBlanc <robert.leblanc@endurance.com>
