If a previous remote snapshot was synced but the unlink failed,
ensure we retry the unlink so that the remote can cleanup the unused
snapshot.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Once a non-primary snapshot is no longer required for syncing, delete it
from the image.
Fixes: https://tracker.ceph.com/issues/44105
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
Previously only newly created user snapshots were included in the
non-primary snapshot snap-seq mapping table. However, we need to
retain a full history of the mapping table if we want to be able to
prune non-primary snapshots.
Failovers are a special case since we won't have a valid snap seq mapping
so it will need to be rebuilt. Luckily, both sides should be read-only
in the previous state so we can use the snapshot names to find matches.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
snapshot-based mirroring needs to be able to potentially delete a
demotion snapshot during the unlink process. Previously, these
snapshots have been left while the read-only error was ignored.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
A pending refresh could occur after setting the non-primary feature flag but
before the creation of the demotion snapshot. This would prevent the snapshot
from being created and would leave the image in a half-primary state.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
This will allow a remote rbd-mirror process to have a snapshot to use for
delta sync operations during failover.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
mgr/dashboard: use FQDN for failover redirection
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
batch should only return an error if some (but not all) devices are
filtered. When only some devices are filtered the resulting osd layout
could look very different from what a user expects. If all devies are
filtered just return success.
Fixes: https://tracker.ceph.com/issues/44994
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
in tasks/module_selftest.yaml, `TestModuleSelftest.test_telegraf()` is
called. but we fail to prepare a unix domain socket to which the telegraf
module can send stats. and telegraf module does not catch
FileNotFoundError exception, so the exception is populated to ceph-mgr
and is found by the test, hence the test is marked a failure whenever
telegraf is tested.
in this change,
* catch this exception, so it won't be caught by ceph-mgr
* whitelist the error message, so the test can pass
Signed-off-by: Kefu Chai <kchai@redhat.com>
we cannot rely on the order in which items are arranged in a dict, the
order varies from version to another. in Python2, it happens to work,
and we can always have the self-signed cert added first. but in Python3,
it does not. and an exception is thrown
```
teuthology.exceptions.ConfigError: ssl: ca root not found for
certificate rgw.client.0
```
in this change, before creating certs, the settings are reordered so
that the self-signed ones are created first.
Signed-off-by: Kefu Chai <kchai@redhat.com>
mgr/dashboard: fix errors related to frontend service subscriptions.
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
in 0437adc33a, we stop right before
reaching $top_srcdir, but we should stop at its parent directory.
in this change, instead of trying to be smart and to walk all the way
up to the root directory or $top_srcdir, we just check $top_srcdir/.git
directly, as we just know it's there or it does not exist at all.
Signed-off-by: Kefu Chai <kchai@redhat.com>
mon/MgrMonitor: show different error message when disabling a dne module
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
in boto, it tries to figure out the MIME type of a file by its name, if
the file-like objects has an attribute of "name". in Python2, the "name"
is always "<fdopen>", fortunately. while in Python3, `TemporaryFile` also
have a "name" which is its fd, and it is an integer now. so we have following
error when sending a `TemporaryFile` using
`upload_part_from_file()`:
```
2020-04-08T02:25:34.660 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last):
2020-04-08T02:25:34.661 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
2020-04-08T02:25:34.661 INFO:tasks.rgw_multisite_tests: self.test(*self.arg)
2020-04-08T02:25:34.662 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/rgw_multi/tests_ps.py", line 2567, in test_ps_creation_triggers
2020-04-08T02:25:34.662 INFO:tasks.rgw_multisite_tests: uploader.upload_part_from_file(fp, 1)
2020-04-08T02:25:34.663 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/multipart.py", line 260, in upload_part_from_file
2020-04-08T02:25:34.663 INFO:tasks.rgw_multisite_tests: query_args=query_args, size=size)
2020-04-08T02:25:34.664 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/key.py", line 1293, in set_contents_from_file
2020-04-08T02:25:34.664 INFO:tasks.rgw_multisite_tests: chunked_transfer=chunked_transfer, size=size)
2020-04-08T02:25:34.664 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/key.py", line 750, in send_file
2020-04-08T02:25:34.665 INFO:tasks.rgw_multisite_tests: chunked_transfer=chunked_transfer, size=size)
2020-04-08T02:25:34.665 INFO:tasks.rgw_multisite_tests: File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/virtualenv/lib/python3.5/site-packages/boto/s3/key.py", line 920, in
_send_file_internal
2020-04-08T02:25:34.666 INFO:tasks.rgw_multisite_tests: self.content_type = mimetypes.guess_type(self.path)[0]
2020-04-08T02:25:34.666 INFO:tasks.rgw_multisite_tests: File "/usr/lib/python3.5/mimetypes.py", line 289, in guess_type
2020-04-08T02:25:34.667 INFO:tasks.rgw_multisite_tests: return _db.guess_type(url, strict)
2020-04-08T02:25:34.667 INFO:tasks.rgw_multisite_tests: File "/usr/lib/python3.5/mimetypes.py", line 114, in guess_type
2020-04-08T02:25:34.667 INFO:tasks.rgw_multisite_tests: scheme, url = urllib.parse.splittype(url)
2020-04-08T02:25:34.668 INFO:tasks.rgw_multisite_tests: File "/usr/lib/python3.5/urllib/parse.py", line 881, in splittype
2020-04-08T02:25:34.668 INFO:tasks.rgw_multisite_tests: match = _typeprog.match(url)
2020-04-08T02:25:34.669 INFO:tasks.rgw_multisite_tests:TypeError: expected string or bytes-like object
```
to address this issue, in this change, a `NamedTemporaryFile` is used
instead of `TemporaryFile`. the former does have a "name" which is a
`str`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to address the test failures like
```
2020-04-07T15:44:58.693 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: ceph pg dump
pgs
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: pgid
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh: line 498: pgid: command not found
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
as the caller might want to `len(manager.get_osd_status()['raw'])`, and
`len()` does not accept a `filter` object.
also, the filtered osd statuses are printed out using `self.log()`, so
we should materialize the `filter` object before sending it to logging
facility. otherwise we will have something like:
```
2020-04-08T02:58:37.001 INFO:tasks.ceph.ceph_manager.ceph:<filter object at 0x7f5a080e1518>
```
in the logging message.
Signed-off-by: Kefu Chai <kchai@redhat.com>
