Commit Graph

7802 Commits

Author SHA1 Message Date
Casey Bodley
b421128951
Merge pull request #38987 from cbodley/wip-qa-rgw-valgrind-supp-tls
valgrind: update suppression for _dl_allocate_tls

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-01-20 13:22:56 -05:00
Casey Bodley
ae8db357f7
Merge pull request #38966 from rzarzynski/wip-qa-rgw-no-valgrind-early-exit
qa/task/rgw: instruct Valgrind to don't exit on first error.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-01-20 11:08:38 -05:00
Casey Bodley
db17daf3b0 valgrind: update suppression for _dl_allocate_tls
in the rgw suite, there's an extra allocate_dtv() call in this stack.
add a ... to match it

  <stack>
    <frame>
      <ip>0x4C3321A</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>calloc</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>760</line>
    </frame>
    <frame>
      <ip>0x4012341</ip>
      <obj>/usr/lib64/ld-2.28.so</obj>
      <fn>allocate_dtv</fn>
    </frame>
    <frame>
      <ip>0x4012CD1</ip>
      <obj>/usr/lib64/ld-2.28.so</obj>
      <fn>_dl_allocate_tls</fn>
    </frame>
    <frame>
      <ip>0x101C7F32</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>pthread_create@@GLIBC_2.2.5</fn>
    </frame>
    <frame>
      <ip>0x6F14E64</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>Thread::try_create(unsigned long)</fn>
    </frame>

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-01-20 11:03:21 -05:00
Sebastian Wagner
bfa04adb65
Merge pull request #38945 from sebastian-philipp/qa-cephadm-smoke-yaml
qa/cephadm: Add yaml output to smoke test

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-01-20 12:48:50 +01:00
Jason Dillaman
85896a4164
Merge pull request #38944 from orozery/qa-qemu-rbd-encryption
qa/tasks/rbd: test qemu on top of rbd encryption

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-19 17:34:26 -05:00
Or Ozeri
3754c665a1 :qa/tasks/rbd: test qemu on top of rbd encryption
This commit adds new qemu xfstests workloads that run on top of librbd luks1/luks2 encryption.
This is currently done via nbd, instead of the qemu rbd driver.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-19 19:59:22 +00:00
Radoslaw Zarzynski
08adc42d60 qa/task/rgw: instruct Valgrind to don't exit on first error.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-01-19 15:31:37 +01:00
Casey Bodley
9e7a41f483
Merge pull request #38890 from cbodley/wip-qa-rgw-https-clients
qa/rgw: don't add a certificate for nonexistent rgw.client.1

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-01-18 15:02:10 -05:00
Sebastian Wagner
88c6c34e2b qa/cephadm: Add yaml output to smoke test
this will provide a more detailed output, like

```yaml
...snip...
service_type: node-exporter
service_name: node-exporter
placement:
  host_pattern: '*'
status:
  created: '2021-01-18T11:21:56.024810Z'
  last_refresh: '2021-01-18T11:23:24.477672Z'
  running: 0
  size: 1
events:
- "2021-01-18T11:23:09.602644Z service:node-exporter [ERROR] \"Failed while placing\
  \ node-exporter.ubuntuon ubuntu: cephadm exited with an error code: 1, stderr:Deploy\
  \ daemon node-exporter.ubuntu ...\nVerifying port 9100 ...\nTraceback (most recent\
  \ call last):\n  File \"<stdin>\", line 7274, in <module>\n  File \"<stdin>\", line\
  \ 1563, in _default_image\n  File \"<stdin>\", line 3698, in command_deploy\n  File\
  \ \"<stdin>\", line 2338, in deploy_daemon\n  File \"<stdin>\", line 1961, in create_daemon_dirs\n\
  AssertionError\""
...snip...
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-18 12:27:14 +01:00
Patrick Donnelly
e674f752b5
Merge PR #37297 into master
* refs/pull/37297/head:
	qa: add new client tests
	test: add client tests
	client: wire up alternate_name
	mds: fix alternate_name durability
	mds: add alternate_name feature support for dentries
	mds: add static encode/decode helpers for remote in CDentry
	mds: add "fscrypt" flag support for inode_t
	mds: add new CDentry tags support
	include: cleanup filepath cons
	mds: add comment on linkage durability
	client: constify some RWRef methods
	common: add strescape helper

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-16 10:40:51 -08:00
Patrick Donnelly
a6891a0c8a
qa: add new client tests
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-15 17:30:40 -08:00
Neha Ojha
5c11f40c12
Merge pull request #38856 from dzafman/wip-48789
test: Fix osd-scrub-scaps.sh to handle DB format change

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-15 16:27:59 -08:00
Neha Ojha
6fc9166af4
Merge pull request #38726 from ronen-fr/wip-ronenf-48720
qa/standalone/scrub/osd-recovery-scrub: handle primary change when waiting for scrub

Reviewed-by: David Zafman <dzafman@redhat.com>
2021-01-15 13:46:30 -08:00
Patrick Donnelly
60510411be
Merge PR #38160 into master
* refs/pull/38160/head:
	doc/mgr/volumes: Document 'fs subvolume authorized_list' cli
	qa: Add tests for list auth-ids of a subvolume
	mgr/volumes: Update the 'volumes' key to 'subvolumes' in auth metadata file
	mgr/volumes: Optionally authorize existing auth-ids
	mgr/volumes: Preserve existing caps while authorize/deauthorize auth-id
	mgr/volumes: Disallow authorize existing auth_id
	mgr/volumes: Add subvolume authorized_list command
	mgr/volumes: Add tenant_id option to subvolume authorize
	mgr/volumes: Persist auth and subvolume metadata

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-15 11:53:06 -08:00
Mykola Golub
aab9de639f
Merge pull request #38862 from dillaman/wip-librbd-migration-qcow
librbd/migration: basic QCOW format handler

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-01-15 20:58:30 +02:00
David Zafman
af9befb0f4 test: Fix osd-scrub-scaps.sh to handle DB format change
Caused by: f9c95fa7fc

Fixes: https://tracker.ceph.com/issues/48789

Signed-off-by: David Zafman <dzafman@redhat.com>
2021-01-15 10:35:30 -08:00
Kefu Chai
dc5d090edc
Merge pull request #38857 from dzafman/wip-48842
test: osd-recovery-prio.sh replace sleep with wait for both PGs recovering

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-16 01:46:43 +08:00
Jason Dillaman
652963e7df librbd/migration: address code review comments
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-15 09:33:40 -05:00
Kotresh HR
1b98e63e75 qa: Add tests for list auth-ids of a subvolume
Fixes: https://tracker.ceph.com/issues/44931
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-01-15 10:10:22 +05:30
Yuri Weinstein
9ad4a1f1e3
Merge pull request #38891 from yuriw/wip-yuriw-octopus-x-master
qa/tests: added stress-split-erasure-code-no-cephadm tests

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-14 14:11:33 -08:00
Josh Durgin
770369c04e qa: remove invalid characters from octopus-x/parallel
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-01-14 09:05:37 -08:00
Jason Dillaman
42f00487b9 qa/workunits/rbd: simplify QCOW migration distro check
If the qemu-img command fails when attempting to use the QCOW (v1) format,
assume that the distro does not have support for the format enabled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 11:52:05 -05:00
Yuri Weinstein
ffa3af3eb0 qa/tests: fixed yaml format
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-14 08:31:40 -08:00
Patrick Donnelly
78a7df1500
Merge PR #38846 into master
* refs/pull/38846/head:
	*: remove legacy ceph_volume_client.py library

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-14 08:10:19 -08:00
Patrick Donnelly
bc57344ded
Merge PR #38889 into master
* refs/pull/38889/head:
	qa: add delays only for osd/mds

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-01-14 08:07:40 -08:00
Jason Dillaman
f3c44b083a librbd/migration: compute deltas between QCOW2 snapshots
Utilize the COPIED bit within the cluster offset to quickly determine
if the corresponding cluster block has been copied.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:35 -05:00
Jason Dillaman
1f4b5aeb0e librbd/migration: add support for reading from QCOW2 snapshots
Tweak the IO read path to now utilize the L1 table associated with
the specified snapshot id. This will cause the IO to properly read
from the specific snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:35 -05:00
Jason Dillaman
bd8db295ea librbd/migration: basic QCOW2 format handler
The initial implementation does not support backing files, compression,
snapshots, (deprecated) encryption, external data files, nor L2 subcluster
allocation. The former three features will be added in a future commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:35 -05:00
Jason Dillaman
83aa1a9e73 librbd/migration: basic QCOW v1 format handler
The initial implementation does not support backing files, compression, nor
(deprecated) encryption. The former two features will be added in a future commit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-01-14 09:35:34 -05:00
Yuri Weinstein
489e60b9ee qa/tests: added stress-split-erasure-code-no-cephadm tests (not using cephadm)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-13 12:46:10 -08:00
Patrick Donnelly
d73cfde0b8
Merge PR #38835 into master
* refs/pull/38835/head:
	qa: add test for fs rm idempotency
	mon/FSCommands: restore idempotent behavior of fs rm
	qa: add ceph cmd helper
	qa: allow kwargs for raw_cluster_cmd

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-13 08:24:12 -08:00
Patrick Donnelly
825761176c
Merge PR #38848 into master
* refs/pull/38848/head:
	qa: ignore MDS_SLOW_METADATA_IO with osd thrasher

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-13 08:23:25 -08:00
Casey Bodley
e2f8f23d21 qa/rgw: don't add a certificate for nonexistent rgw.client.1
this override to the rgw task is causing the rgw/verify suite to start
an extra radosgw

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-01-13 10:58:42 -05:00
Patrick Donnelly
ef19481399
qa: add delays only for osd/mds
The delays were applied everywhere and needlessly interfere with test
commands sent to mons from the ceph admin command. Furthermore, the
delays would not affect the kernel client. Now the delays are performed
by the MDS on clients.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-13 07:54:09 -08:00
Kotresh HR
bd49b6409b mgr/volumes: Filter inherited snapshots while listing snapshots
Filter inherited snapshots resulted as part of a snapshot
at ancestor level while listing snapshots of a subvolume
and subvolumegroup

Also, fail the snapshot info on inherited snapshot.

Fixes: https://tracker.ceph.com/issues/48501
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2021-01-13 15:09:35 +05:30
Yuri Weinstein
dd0ad25540
Merge pull request #38806 from yuriw/wip-yuriw-octopus-x-master
qa/tests: added rbd_import_export workload

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-12 13:25:59 -08:00
Yuri Weinstein
6d5fb9fd6d qa/tests: added stress-split-no-cephadm tests
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-12 09:03:34 -08:00
Patrick Donnelly
1e9622cff9
Merge PR #38793 into master
* refs/pull/38793/head:
	qa: check for rank hole without racy joinable flag gymnastics
	qa: skip client mounts for TestResize

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-01-12 07:27:46 -08:00
Patrick Donnelly
780a1dd5ff
qa: add test for fs rm idempotency
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-12 07:24:33 -08:00
Patrick Donnelly
166bb4d551
qa: add ceph cmd helper
A more programmer friendly command to use.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-12 07:24:28 -08:00
Patrick Donnelly
c0907f99e8
qa: allow kwargs for raw_cluster_cmd
And refactor.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-12 07:24:28 -08:00
Patrick Donnelly
a3db265ad5
*: remove legacy ceph_volume_client.py library
This library is obsolete with the mgr volumes plugin since Nautilus.

The last remaining user of this library was Manila which will be using
the volumes plugin with Pacific and onwards.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-12 06:54:29 -08:00
David Zafman
4814648155 test: osd-recovery-prio.sh replace sleep with wait for both PGs
recovering

fixes: https://tracker.ceph.com/issues/48842

Signed-off-by: David Zafman <dzafman@redhat.com>
2021-01-11 17:30:00 -08:00
Yuri Weinstein
1e1c9c312a qa/tests: added rbd_import_export tests
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-11 10:51:14 -08:00
Patrick Donnelly
a59ecd3ed1
Merge PR #38640 into master
* refs/pull/38640/head:
	qa: add test for reserved keyword feature
	qa: use no client for required client feature tests
	mds: do not allow setting a reserved feature by name
	mds: return sv for efficiency

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-11 10:12:17 -08:00
Patrick Donnelly
b66ca823fe
qa: ignore MDS_SLOW_METADATA_IO with osd thrasher
Fixes: https://tracker.ceph.com/issues/48834
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-11 10:10:04 -08:00
Patrick Donnelly
318d3f4d80
Merge PR #38108 into master
* refs/pull/38108/head:
	doc, man: man page for `cephfs-top` utility
	doc: document `cephfs-top` utility
	test: selftest for `cephfs-top` utility
	spec, deb: package cephfs-top utility
	cephfs-top: top(1) like utility for Ceph Filesystem
	mgr/stats: include kernel version (for kclients) in `perf stats` command output
	mgr/stats: include version with `perf stats` output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-11 08:38:52 -08:00
Venky Shankar
0329d9b884 test: selftest for cephfs-top utility
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-01-11 06:15:53 -05:00
Patrick Donnelly
14b4787de1
client: add debug messages for osdmap wait
To help debug i47294.

See-also: https://tracker.ceph.com/issues/47294
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-10 21:11:06 -08:00
Patrick Donnelly
52ef68c681
Merge PR #38775 into master
* refs/pull/38775/head:
	qa: remove ceph file systems on completion

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-08 19:06:58 -08:00