persist inconsistent objects found when comparing the ScrubMaps
collected from replica/shards. the discrepancies between the auth
copy and the replica are identified as inconsistencies. and hence
encoded into the omap of an object of the temp coll of the PG in
question.
scrub_types.{h,cpp} are introduced to hide the details of how we
persist the scrub types from the librados client.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to list inconsistent PGs of a given pool. this command exposes
the underlying get_inconsistent_pgs() API to user.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Split up the "ceph" package into four new packages:
1. ceph-mon
2. ceph-osd
3. ceph-mds (already done on Debian)
4. ceph-base (files shared among multiple servers)
and then:
5. Make "ceph" into a metapackage that depends on both -mon and -osd
(and -mds, for RPMs).
To describe the outcome of this change another way:
For RPMs:
- "ceph-{mon,osd,mds}" Require: ceph-base.
- "ceph" will become a metapackage that Requires: ceph-{mon,osd,mds}.
For DEBs:
- "ceph-{mon,osd,mds}" will Depends: ceph-base.
- "ceph" will become a metapackage that Depends: ceph-{mon,osd}.
- "ceph" will continue to Recommends: ceph-mds
New users should "yum install ceph-mon" or "yum install ceph-osd" (or
"apt-get install ceph-mon", etc) in order to install the exact daemons
that they need.
http://tracker.ceph.com/issues/10587Fixes: #10587
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
These will track whether local images should be mirrored, and map them
to a unique global id. There's a state field for safely disabling
mirroring while operating on multiple objects.
Fixes: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
We'll use this object only for mirroring-related purposes, not generic
settings on a pool.
Refs: #14419
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Tracking completions is critical for memory safety - if the
aio function succeeds, the completion must be tracked. However,
if a KeyboardInterrupt or similar arrives between the call and
the tracking, the completion will not be tracked.
Fix this by tracking the completion before the aio call, and
explicitly cleaning up in the failure case.
This leaves the opposite problem, where an unexpected exception
(other than simple error return from the aio function) will cause
the completion to not be freed until the Ioctx is destroyed, but
that is a relatively minor issue.
Signed-off-by: Hector Martin <marcan@marcan.st>
aio_read:
The reference to ret_s begins existing at PyBytes_FromStringAndSize and
is handed over to the callback if rados_aio_read succeeds. This creates
a lot of subtle scenarios where it might not be XDECREFed (e.g. if
a KeyboardInterrupt arrives at the wrong time). Instead, store the pointer
to that buffer in the completion object, and hand over responsibility for
the XDECREF to it. This guarantees that the "special" reference to this
incomplete object will be released when the completion object is
deallocated.
Also make sure we don't try to _PyBytes_Resize with a negative length.
Add a failure case to the aio_read test in test_rados.py
Completion: the wrapper methods weren't being called, which prevents
the completion objects from being freed until the Ioctx is. Fix this
and add a refcount check to the aio_read test.
Signed-off-by: Hector Martin <marcan@marcan.st>
When running the ceph-disk check suite, 'test_mark_init' & 'test_activate_dir' had a
permission denied of the freshly created stuff like in :
tests/ceph-disk.sh:237: test_mark_init: /bin/rm -fr /ceph/src/ceph-disk/test-ceph-disk/dir
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/snap_1 »: Permission denied
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/current »: Permission denied
/bin/rm: cannot remove « /ceph/src/ceph-disk/test-ceph-disk/dir/snap_2 »: Permission denied
Those two tests are starting a mon with run_mon.
Trying to delete the files while the daemon are still alive is impossible.
This patch removes the explicit 'rm' and let the teardown doing the cleaning
stuff by stopping daemons & cleaning content.
By using this patch, make check is now sucessful on ceph-disk.
Signed-off-by: Erwan Velu <erwan@redhat.com>
When doing a make test, if your local setup was running a "dm-0" or "dm-1",
the make check was failing.
The detection of the local "dm-x" were leading to a wrong comparaison with
the expected simulated values.
The fix is using a dummy name for that "dm" to prevent any collision with the local setup.
Prior that patch, a typical output of that error looked like :
> assert expect == main.list_devices()
E assert [{'partitions...: '/dev/Xda'}] == [{'partitions'...: '/dev/Xda'}]
E At index 0 diff: {'partitions': [{'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'}, 'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d', 'path': '/dev/Xda1', 'is_partition': True, 'mount': None, 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2', 'type': 'data', 'state': 'unprepared', 'fs_type': None}], 'path': '/dev/Xda'} != {'partitions': [{'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d', 'whoami': None, 'path': '/dev/Xda1', 'is_partition': True, 'mount': '/var/cache/ccache', 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2', 'ceph_fsid': None, 'fs_type': 'btrfs', 'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'}, 'type': 'data', 'state': 'active'}], 'path': '/dev/Xda'}
E Full diff:
E - [{'partitions': [{'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'},
E + [{'partitions': [{'ceph_fsid': None,
E + 'dmcrypt': {'holders': ['dm-0'], 'type': 'plain'},
E - 'fs_type': None,
E ? ^^^^
E + 'fs_type': 'btrfs',
E ? ^^^^^^^
E 'is_partition': True,
E - 'mount': None,
E + 'mount': '/var/cache/ccache',
E 'path': '/dev/Xda1',
E 'ptype': '4fbd7e29-9d25-41b8-afd0-5ec00ceff05d',
E - 'state': 'unprepared',
E ? ^^^^ -----
E + 'state': 'active',
E ? ^^^^^
E 'type': 'data',
E - 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2'}],
E ? --
E + 'uuid': '56244cf5-83ef-4984-888a-2d8b8e0e04b2',
E + 'whoami': None}],
E 'path': '/dev/Xda'}]
tests/test_main.py:342: AssertionError
Signed-off-by: Erwan Velu <erwan@redhat.com>
When running run-tox.sh in a very simple env,
the test will fail if '/var/lib/ceph/tmp' doesn't exist.
This patch adds a check to create this directory if required as mkdtemp doesn't do it for you.
Prior this patch, the following behavior was seen :
tests/test_main.py:342:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ceph_disk/main.py:3753: in list_devices
fstype=fs_type, options='')
ceph_disk/main.py:1217: in mount
dir=STATEDIR + '/tmp',
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
suffix = '', prefix = 'mnt.', dir = '/var/lib/ceph/tmp'
def mkdtemp(suffix="", prefix=template, dir=None):
"""User-callable function to create and return a unique temporary
directory. The return value is the pathname of the directory.
Arguments are as for mkstemp, except that the 'text' argument is
not accepted.
The directory is readable, writable, and searchable only by the
creating user.
Caller is responsible for deleting the directory when done with it.
"""
if dir is None:
dir = gettempdir()
names = _get_candidate_names()
for seq in xrange(TMP_MAX):
name = names.next()
file = _os.path.join(dir, prefix + name + suffix)
try:
> _os.mkdir(file, 0700)
E OSError: [Errno 2] No such file or directory: '/var/lib/ceph/tmp/mnt.KoAV85'
/usr/lib64/python2.7/tempfile.py:333: OSError
Signed-off-by: Erwan Velu <erwan@redhat.com>
changes to WeightedPriorityQueue there is no strict round robin
dequeueing of classes. Removed that test from the unittest.
Signed-off-by: Robert LeBlanc <robert.leblanc@endurance.com>
The mutex initializers in Linux are non-portable, as evidenced by their
names ending with _NP.
Use the POSIX compliant ways to initialize mutices instead.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
