Commit Graph

2445 Commits

Author SHA1 Message Date
Jason Dillaman
42f00487b9 qa/workunits/rbd: simplify QCOW migration distro check
If the qemu-img command fails when attempting to use the QCOW (v1) format,
assume that the distro does not have support for the format enabled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 11:52:05 -05:00
Jason Dillaman
f3c44b083a librbd/migration: compute deltas between QCOW2 snapshots
Utilize the COPIED bit within the cluster offset to quickly determine
if the corresponding cluster block has been copied.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:35 -05:00
Jason Dillaman
1f4b5aeb0e librbd/migration: add support for reading from QCOW2 snapshots
Tweak the IO read path to now utilize the L1 table associated with
the specified snapshot id. This will cause the IO to properly read
from the specific snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:35 -05:00
Jason Dillaman
bd8db295ea librbd/migration: basic QCOW2 format handler
The initial implementation does not support backing files, compression,
snapshots, (deprecated) encryption, external data files, nor L2 subcluster
allocation. The former three features will be added in a future commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:35 -05:00
Jason Dillaman
83aa1a9e73 librbd/migration: basic QCOW v1 format handler
The initial implementation does not support backing files, compression, nor
(deprecated) encryption. The former two features will be added in a future commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:34 -05:00
Patrick Donnelly
37f7bba5c0
Merge PR #38776 into master
* refs/pull/38776/head:
	qa: get pool holding file data from file layout

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-01-08 11:27:08 -08:00
Sebastian Wagner
80cbe7ecaa
Merge pull request #38681 from pcuzner/enable-exporter
cephadm: enable exporter deployment at bootstrap

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-07 10:30:53 +01:00
Patrick Donnelly
d7cacef125
qa: get pool holding file data from file layout
Otherwise it always looks at the default data pool. For ec pools, this
may not be where the file data is.

Fixes: https://tracker.ceph.com/issues/48756
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 07:11:20 -08:00
Jason Dillaman
d22ca3d978 librbd/migration: require snapshot when importing from native source
Since we cannot mark the source image read-only when running in import-only
migration mode, we should require the user to provide a snapshot to ensure
that data cannot change while the migration is running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-02 09:26:53 -05:00
Paul Cuzner
bd8036a900 cephadm: update QA tests to include cephadm-exporter
Test updated to include deployment of a cluster with
cephadm-exporter active. Tests include varifying config
(token), the state of it's threads are all active and a
check that the placement pattern is for all hosts

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-12-23 12:24:52 +13:00
Ilya Dryomov
fb03fa3766
Merge pull request #38638 from idryomov/wip-krbd-stable-writes-attr
qa: krbd_stable_pages_required.sh: move to stable_writes attribute

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-12-18 10:30:34 +01:00
Ilya Dryomov
5adfc15b87 qa: krbd_stable_pages_required.sh: move to stable_writes attribute
bdi/stable_pages_required attribute was deprecated in 5.10 and now
always returns 0.  The replacement is queue/stable_writes.  (It is
also writeable, so we can simplify these test cases somewhat in the
future.)

Fixes: https://tracker.ceph.com/issues/48232
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-12-17 13:41:34 +01:00
Alfonso Martínez
5d7ee7c1f0 mgr/dashboard: CLI commands: read passwords from file
Fixes: https://tracker.ceph.com/issues/48355

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2020-12-17 08:36:57 +01:00
Mykola Golub
4939841633
Merge pull request #38379 from dillaman/wip-librbd-migration-5
librbd/migration: add thick-provisioned snapshot support to the raw format

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-12-04 17:43:59 +02:00
Jason Dillaman
80fe5a8735 qa/suites/rbd: new basic migration CLI test case
Pending a larger suite of tests for instant-restore image migration,
this test provides a basic sanity check for both the native and
raw image formats -- including basic snapshot tests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-12-03 08:30:52 -05:00
Ramana Raja
a232d273c1 qa/workunits/fs/snaps: allow tests to be run
... filesystems other than 'cephfs'.

It is not required to set  'allow_new_snaps' to True to allow snapshot
to be created on a filesystem. Remove that setting.

Remove 'fs/snaps/snaptest-0.sh' that is racy when running in parallel on
an another client that mounted the same file system. Include a similar
test in qa/tasks/cephfs/test_snapshots.py

Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-11-20 13:23:21 +05:30
Jason Dillaman
8d4a8735c3
Merge pull request #37939 from trociny/wip-rbd-nbd-wait-for-terminate
rbd-nbd: fixes and improvements for unmap/detach wait for process terminate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-11-16 17:10:17 -05:00
Kiefer Chang
7df12da184
mgr/dashboard: fix cephadm e2e test failure on deleting OSDs
- Adapt the OSD ID column change in this PR:
  https://github.com/ceph/ceph/pull/37506
- Turn off Cypress retries because it makes the failure log misleading.

Fixes: https://tracker.ceph.com/issues/48209
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-11-12 17:20:48 +08:00
Mykola Golub
9c9b14fab6 qa/workunits/rbd: improve regex for parsing ps output
On some platforms the pid may be indented with spaces.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-11-10 17:16:00 +00:00
Mykola Golub
4cda97a822 rbd-nbd: when unmapping or detaching by device try to find process
For `detach` failing to find the process is fatal while unmap
will still try to send disconnect to the device.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-11-10 17:15:59 +00:00
Mykola Golub
5e10672e4e
Merge pull request #37884 from dillaman/wip-48032
qa/workunits/rbd: fix permission issue when removing mirror peer

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-10-29 15:48:35 +02:00
Jason Dillaman
caaba8eabe qa/workunits/rbd: fix permission issue when removing mirror peer
Fixes: https://tracker.ceph.com/issues/48032
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-10-28 14:06:57 -04:00
Jason Dillaman
0db501b2e3
Merge pull request #37273 from trociny/wip-rbd-nbd-persist
rbd-nbd: allow to restart daemon without destroying device

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-10-26 07:57:47 -04:00
Kefu Chai
a40337c2d7
Merge pull request #37635 from jecluis/wip-46816
mon: have 'mon stat' output json as well

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-19 14:08:09 +08:00
J. Eric Ivancich
ff342f2f02
Merge pull request #37684 from ivancich/rgw-yummy-orphan-testing
rgw: use yum rather than dnf for teuthology testing of rgw-orphan-list

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-15 16:14:07 -04:00
J. Eric Ivancich
11a08a5bb8 rgw: use yum rather than dnf for testing rgw-orphan-list
The teuthology testing for rgw-orphan-list needs to install
`s3cmd`. Switch from using dnf to yum to work on a wider variety of
platforms.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-10-15 15:08:45 -04:00
Mykola Golub
3043d80168 rbd-nbd: make unmap/detach wait for rbd-nbd process to terminate
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-10-13 18:32:50 +01:00
Joao Eduardo Luis
122388429d
qa/cephtool: test 'mon stat' commands
Signed-off-by: Joao Eduardo Luis <joao@suse.com>
2020-10-13 08:31:24 +00:00
Mykola Golub
7d75c1d6b8 rbd-nbd: introduce new attach and detach commands
The commands allow to restart a daemon without destroying the nbd
device.

Now, if the netlink is used, a dead connection timeout is set on
the nbd device setup, so the device is not immediately released
if the rbd-nbd process terminates without disconnect (unmap).

The attach command just sends terminate signal to the rbd-nbd
process. The detach command starts a new process and connects to
the existing device.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-10-07 13:26:07 +01:00
Kiefer Chang
381e4ac5ac
qa/suites/rados/cephadm/dashboard: fix creating OSD failure
The iSCSI disks report a larger optimal I/O size, which causes
ceph-volume to miscalculate the extent count. Workaround this by
reporting 0.

Fixes: https://tracker.ceph.com/issues/47742
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-10-07 14:45:24 +08:00
Mykola Golub
b360186eb6 rbd-nbd: don't ignore namespace when unmapping by image spec
Fixes: https://tracker.ceph.com/issues/47665
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-09-27 17:59:49 +01:00
Ilya Dryomov
6827bbbcfb
Merge pull request #36927 from idryomov/wip-krbd-noudev
krbd: optionally skip waiting for udev events

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
2020-09-21 19:39:30 +02:00
Ilya Dryomov
d2884adb15 qa: add test for mapping and unmapping from a network namespace
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-09-21 13:54:08 +02:00
Neha Ojha
8ba0a61a51
Merge pull request #35906 from gregsfortytwo/wip-stretch-mode
Add a new stretch mode for 2-site Ceph clusters

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-18 14:31:45 -07:00
Joshua Schmid
4057f30187
Merge pull request #37059 from guits/guits-cephadm-shell-multiple-mounts
cephadm: support multiple mounts when running interactive shell
2020-09-18 10:57:52 +02:00
Volker Theile
7219377cfc cephadm: Introduce 'container' specification to deploy custom containers
Fixes: https://tracker.ceph.com/issues/46666

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-09-17 10:04:11 +02:00
Guillaume Abrioux
3ed3cff0d0 cephadm: support multiple mounts when running shell
This commit adds the multiple mounts support when running the
interactive shell.

ie:
```
--mount /foo /bar:/bar:z
```

Keeping default destination `/mnt` when no destination is passed for
backward compatibility. In the above example `/foo` will be mounted in
`/mnt/foo` and `/bar` in `/bar`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-15 16:44:47 +02:00
Greg Farnum
9506d09e3b Merge remote-tracking branch 'origin/master' into wip-stretch-mode
Conflicts:
	src/include/ceph_features.h

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-09-15 02:25:07 +00:00
Jason Dillaman
9929eff533
Merge pull request #37086 from trociny/wip-qa-rbd-nbd-unmap
qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-14 14:32:30 -04:00
Jason Dillaman
c5325306ed
Merge pull request #37038 from trociny/wip-rbd-nbd-quiesce-error
rbd-nbd: return quiesce hook exit code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-14 14:31:58 -04:00
Jason Dillaman
2fa9442dcc qa/workunits/mon: fixed excessively large pool PG count
Fixes: https://tracker.ceph.com/issues/47405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-09-14 08:58:52 -04:00
Lenz Grimmer
42a32baaaa
Merge pull request #36365 from bk201/wip-44637
mgr/dashboard: add e2e tests for Orchestrator components

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-09-14 10:56:41 +02:00
Greg Farnum
d02625331c Merge remote-tracking branch 'origin/master' into wip-stretch-mode 2020-09-14 02:32:19 +00:00
Mykola Golub
dc9ee9ba0b rbd-nbd: return quiesce hook exit code
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-09-11 14:51:51 +01:00
Mykola Golub
f0c69761c8 qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap
Previously it still could race when unmap_device returned success
because the device was not found in `rbd-nbd list-mapped` (the nbd
device was removed) but the test failed because the process was still
found in the ps table.

Fixes: https://tracker.ceph.com/issues/47394
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-09-10 17:13:26 +01:00
Ilya Dryomov
a1f8589679
Merge pull request #37050 from idryomov/wip-krbd-namespace-symlinks
rbd: include RADOS namespace in krbd symlinks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-10 08:04:10 +02:00
Ilya Dryomov
7ccd2c0dce qa: add test for krbd symlinks created by udev
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-09-09 09:21:54 +02:00
Samuel Just
4a73fd2251
Merge pull request #35899 from myoungwon/wip-fix-manifest-tier
osd, test: refactoring manfiest-tier

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-01 12:49:57 -07:00
Kiefer Chang
9c912d5449
mgr/dashboard: add e2e tests for the Orchestrator components
Fixes: https://tracker.ceph.com/issues/44637
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-08-31 16:50:16 +08:00
myoungwon oh
7217a6c00e osd, test: remove all things related to manifest_flush
Due to a plan to re-work based on CDC-based flush,
remove manifest_flush()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsumg.com>
2020-08-26 21:05:14 +09:00