Commit Graph

3592 Commits

Author SHA1 Message Date
Zack Cerza
752140aaa9
Merge pull request #48215 from zmc/osd-containers-tasks
qa/tasks: Updates to enable containerized loop-backed OSDs
2022-09-29 16:57:30 -06:00
Venky Shankar
187d73bf4c
Merge pull request #48174 from batrick/i57597
qa: output debugging for cephfs-journal-tool/cephfs-data-scan

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-09-29 18:14:23 +05:30
Rishabh Dave
fb25b0b17f
Merge pull request #47991 from kotreshhr/qa-fix-57446
qa: Fix test_subvolume_snapshot_info_if_orphan_clone

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-09-29 17:56:35 +05:30
Rishabh Dave
2ff34d1d60
Merge pull request #47985 from kotreshhr/qa-fix-57205
qa: Fix test_subvolume_group_ls_filter_internal_directories

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-09-29 17:54:24 +05:30
Zack Cerza
af67a7152c tasks/cephadm: Add 'osd_method' config item
Since the default of None gets interpreted as 'lvm', the only useful
value would be 'raw'. This will enable us to use loop devices inside
containers for OSDs.

Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-26 15:35:08 -06:00
Zack Cerza
74c3446c25 cephadm: Avoid TypeError in download_cephadm()
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-26 15:35:08 -06:00
Tobias Urdin
d6f5e49f3b qa/tasks/barbican: dont copy policy files
In newer versions the policies is inside the code
so we don't need these files for default policy
anymore.

Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-26 06:27:40 +00:00
Tobias Urdin
7c460bd2fb qa/tasks/barbican: sync barbican db and secret stores
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-23 06:31:06 +00:00
Tobias Urdin
cd3313531f qa/tasks/barbican: copy barbican policy json or yaml
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-23 06:31:06 +00:00
Zack Cerza
f094103726 tasks/nvme_loop: Skip on containerized testnodes
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-22 17:49:31 -06:00
Zack Cerza
1c8e7007ce tasks/nvme_loop: Don't modprobe if mod is loaded
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-22 17:48:42 -06:00
Adam King
2784a70271
Merge pull request #47756 from dparmar18/wip-dparmar-cephadm-after-revert
pybind/mgr/cephadm/upgrade: allow upgrades without reducing max_mds

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-22 09:54:06 -04:00
Samuel Just
ad87f822bd
Merge pull request #45369 from jyha200/wip-samplededup
ceph_dedup_tool: adding sample-dedup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 10:32:19 -07:00
Adam King
12235e92e6
Merge pull request #41855 from p-se/compile-cephadm-1
cephadm: compile cephadm

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2022-09-19 11:48:51 -04:00
Patrick Donnelly
acd395ace2
qa: output higher debugging for cephfs-journal-tool/cephfs-data-scan
Fixes: https://tracker.ceph.com/issues/57597
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-19 10:44:08 -04:00
myoungwon oh
537462a102 qa: remove unused values in deduplication.py
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 13:21:07 +09:00
Sungmin Lee
f81b1b61c9 qa: add validation stage for deduplication.py
To validate sample-dedup actually works, validate() runs
separated thread from sample-dedup and verifies
two following things.
1. check sample-dedup starts properly.
2. check references of all the chunk objects' in chunk tier
   exists in designated base pool.
This routune repeats for max_valication_cnt times while
sample-dedup is running. If it doesn't raise any fail while the loop,
we can pretend sample-dedup works accurately.
If not, assert() will stop this test.

In case that a reference of chunk object doesn't exist in base pool,
validate() gives a second chance after repairing it (chunk-repair op)
to deal with false-positive reference inconsistency.

Signed-off-by: Sungmin Lee <sung_min.lee@samsung.com>
2022-09-19 13:21:00 +09:00
myoungwon oh
cbe534e0b5 qa: add deduplication test under RGW workload (s3)
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 13:20:46 +09:00
Casey Bodley
2b888af746
Merge pull request #48054 from yuvalif/wip-yuval-fix-56572
tests: remove pubsub tests from multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kalpesh Pandya <kapandya@redhat.com>
2022-09-15 10:04:39 -04:00
Venky Shankar
7157008318
Merge pull request #47516 from nmshelke/fix-56632
qa: ignore disk quota exceeded failure in test

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-09-15 14:55:26 +05:30
Michael Fritch
95ead7401f qa/tasks/cephadm: conditionally pull cephadm binary from chacra
When a specific (older) version of cephadm is needed for a test, pull
it from chacra. If we need to bootstrap an older version of ceph, we
pull an older "single-script" binary using the older method.

Co-authored-by: Michael Fritch <mfritch@suse.com>
Co-authored-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00
Michael Fritch
72181fc14c qa/tasks: add chacra util
to query shaman and pull a binary from chacra

Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00
Sebastian Wagner
1ddd179054 qa/tasks: Drop downloading cephadm from the git source tree
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2022-09-13 12:17:20 -04:00
Adam King
78574ddad6
Merge pull request #47854 from phlogistonjohn/jjm-fix-57268
qa/tasks/kubeadm: set up tigera resources via kubectl create

Reviewed-by: Adam King <adking@redhat.com>
2022-09-12 15:56:43 -04:00
Yuval Lifshitz
acd6d4efea tests: remove pubsub tests from multisite
Fixes: https://tracker.ceph.com/issues/56572

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-09-12 18:54:46 +03:00
Kotresh HR
d382185f49 qa: Fix test_subvolume_snapshot_info_if_orphan_clone
The list of pending clones didn't match the expected.
The test fails occasionally. This is because the
clone would have finished before checking for this.
So increase the clone delay.

Fixes: https://tracker.ceph.com/issues/57446
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-09-06 18:03:14 +05:30
Kotresh HR
aafa754767 qa: Fix test_subvolume_group_ls_filter_internal_directories
Fixes: https://tracker.ceph.com/issues/57205
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-09-06 11:58:30 +05:30
Rishabh Dave
d5ab255225
Merge pull request #46988 from rishabh-d-dave/qa-suites-upgraded-client
qa/cephfs: clean up evicted client in 4-compat_client.yaml

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-02 17:51:25 +05:30
Kefu Chai
63744eb5c7
Merge pull request #44993 from kamoltat/wip-ksirivad-fix-bug-50089
mon/Elector: Added sanity check when pinging a peer monitor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2022-08-30 21:19:01 +08:00
John Mulligan
f8b0511d6c qa/tasks/kubeadm: set up tigera resources via kubectl create
Fixes: https://tracker.ceph.com/issues/57268

The tigera operator for the calico CNI has some pretty large resource
definitions. The length of the definitions can cause the "client side
apply", the default mode for `kubectl apply ....`, to fail due to the
length of the needed annotation that would result:

```
2022-08-22T20:24:55.636 INFO:teuthology.orchestra.run.smithi087.stdout:clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created
2022-08-22T20:24:55.670 INFO:teuthology.orchestra.run.smithi087.stdout:deployment.apps/tigera-operator created
2022-08-22T20:24:55.671 INFO:teuthology.orchestra.run.smithi087.stderr:The CustomResourceDefinition "installations.operator.tigera.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
2022-08-22T20:24:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1
```

There are two simple options for avoiding this error. One is to use
`kubectl create`. The create command will not make this lengthy
annotation. It will fail if any of the resources already exist. The
other option is to use server-side apply, via the `kubectl apply
--server-side ...` command. It is new in k8s 1.18. It will not create
the annotation either.

The block of code setting up the CNI already uses `kubectl create` to
create the custom resources that configure the tigera operator.
Therefore it should be safe to assume the block of code in question
doesn't need to be idempotent and we can also use `kubectl create`
elsewhere in the same block.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-08-29 10:03:01 -04:00
Nikhilkumar Shelke
41b0cb70e0 qa: ignore disk quota exceeded failure in test
Quota enforcement takes some time. It is documented.
Test should ignore 'Disk Quota Exceeded' failure.

Fixes: https://tracker.ceph.com/issues/56632
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-29 13:42:11 +05:30
dparmar18
089306e556 qa/tasks/fs: add cases in post_upgrade_checks() for fail_fs
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-08-26 20:48:11 +05:30
Rishabh Dave
e7b6c9d8c2
Merge pull request #46994 from rishabh-d-dave/qa-caps-helper-upgrade
qa/cephfs: upgrade caps_helper.py

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-08-26 18:43:52 +05:30
Venky Shankar
b17f0e5d1e
Merge pull request #46685 from Rethan/perf-limit-dump-loads-depth
mds/MDBalancer: add an arg to limit depth when dump loads for dirfrags

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-08-24 13:31:28 +05:30
Venky Shankar
d8ba839bcd
Merge pull request #46068 from neesingh-rh/change_perf_stats_o/p_structure
mgr/stats: change in structure of perf_stats o/p

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2022-08-24 11:13:07 +05:30
Venky Shankar
49e66ed0c6
Merge pull request #47079 from neesingh-rh/fix_56483
mgr/stats: missing clients in perf stats command output.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2022-08-24 09:49:47 +05:30
Venky Shankar
d1ca282f60
Merge pull request #45949 from neesingh-rh/feature-51434
mgr/volumes: Add volume info command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-23 21:33:09 +05:30
Kefu Chai
ccff328190
Merge pull request #46943 from kalaspuffar/python39
mgr/diskprediction_local: Support Python 3.9 for disk prediction module.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-08-23 23:26:07 +08:00
Venky Shankar
78871e59c0
Merge pull request #46599 from kotreshhr/rename_config_on_flush
mgr/volumes: Better handle config file on osd full scenario

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-08-23 13:25:02 +05:30
Venky Shankar
5e1625fa7c
Merge pull request #47063 from lxbsz/wip-56529
mds/client: fail the request if the peer MDS doesn't support getvxattr op

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2022-08-23 13:22:09 +05:30
Patrick Donnelly
a8a86789c7
Revert "Merge pull request #47092 from dparmar18/wip-dparmar-cephadm-simple-1"
This reverts commit 1c4da3dbd2, reversing
changes made to ee1e163b1e.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-08-22 13:10:27 -04:00
Neeraj Pratap Singh
74489c4f0b qa: add test_non_existing_mds_rank and modify test_query_mds_filter
Fixes: https://tracker.ceph.com/issues/56169
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-08-22 11:35:37 +05:30
Neeraj Pratap Singh
ff33cccf33 qa: Update the qa tests to be compatible with the new structure of 'perf stats' o/p.
test_client_metrics_and_metadataand other tests has been
updated as earlier it was checking according to the old structure
of perf stats o/p, which has been changed in this PR.

Fixes: https://tracker.ceph.com/issues/56162
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-08-22 11:31:52 +05:30
Kefu Chai
afe8058b6c qa/tasks/mgr/dashboard: accept headers as an option param
* add optional "headers" parameter to _request() and _get()
* correct the test of test_force_no_gzip, as the header in response
  should be "application/vnd.ceph.api.v1.0+json", not "application/json"

with this change, we are able to pass headers to _get() in dashboard api
tests.

without this change, we'd have following error when testing
`test_force_no_gzip` defined by `tasks.mgr.dashboard.test_requests.RequestsTest`:

raceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-api/src/pybind/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/jenkins-build/build/workspace/ceph-api/src/pybind/mgr/dashboard/controllers/_base_controller.py", line 263, in inner
    ret = func(*args, **kwargs)
TypeError: Summary.__call__() got an unexpected keyword argument 'headers'

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-22 14:01:16 +08:00
Rishabh Dave
c279b47ec9 qa/cephfs: lazy-force unmount clients that have been evicted
Before unmounting check if the client has been evicted and, if so, run
"umount -f -l" for the mount point of the client and cleanup the mount
right after it.

Attempting to unmount, cleanup or operate in any way over mount point
of a evicted client will hang the operation (and thereby our Python
code too). Lazy-force unmount prevents such hangs for our Python code
and also frees the mount point.

This commit also adds code to gather session info for kernel mounts
after mounting is successful. This is a necessity since network address
of session is needed to check if it is blocked by Ceph cluster.

Fixes: https://tracker.ceph.com/issues/56476
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-08-19 21:00:00 +05:30
Rishabh Dave
b79853fbe9 qa/cephfs: add a method to check if CephFS mounts have hanged
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-08-19 19:40:38 +05:30
Rishabh Dave
cef4c8d832
Merge pull request #45036 from rishabh-d-dave/qa-fix-is_mounted
qa/cephfs: upgrade is_mounted() in mount.py

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2022-08-19 18:43:59 +05:30
Rishabh Dave
775ed5fe64
Merge pull request #45869 from neesingh-rh/fix_55234
mgr/snap_schedule: replace .snap with the client configured snap dir name

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-19 18:38:51 +05:30
Kotresh HR
aece3b9b28 qa: Validate cleaning of the stale snapshot metadata
Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 18:27:54 +05:30
Xiubo Li
eae0a16002 qa: add a upgrade test suite from nautilus and test the new getvxattr op
For new OPs the old ceph may will crash the MDSs when receiving
unknow OPs. As a workaround the new clients will check the sessions
feature bits to decide to send the new OPs or not.

Fixes: https://tracker.ceph.com/issues/56529
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-08-16 08:30:52 +08:00