Commit Graph

9773 Commits

Author SHA1 Message Date
Ilya Dryomov
8d57d55738
Merge pull request #50230 from idryomov/wip-58833
librbd: call apply_changes() after setting librados_thread_count

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
2023-02-27 14:15:31 +01:00
Venky Shankar
fb0c668e3b Merge PR #44240 into main
* refs/pull/44240/head:
	qa/cephfs: upgrade xfstests_dev.py for FUSE mounts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-02-27 11:06:01 +05:30
Ronen Friedman
dac0fce773
Merge pull request #50236 from ronen-fr/wip-rf-total-dump
test/osd-scrub-dump.sh: fix scrub chunk size

Reviewed-by: Samuel Just <sjust@redhat.com>-
2023-02-26 21:30:30 +02:00
Ronen Friedman
ce7e132e7d test/osd-scrub-dump.sh: fix scrub chunk size
The test performs shallow scrubs, intentionally using small chunk
sizes to allow dump commands time to check specific details.
Following commit ffda64119f
(PR#44749), shallow scrubs chunks are controlled by a separate
configuration parameter. This PR fixes the test to use the
correct parameter.

An additional minor change is an adjustment to the test loop sleep time:
it is now reduced to guarantee that a dump followed by a counter
increase will be performed in more-or-less the scrubs frequency.

Fixes: https://tracker.ceph.com/issues/58797
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-02-26 15:06:19 +02:00
Rishabh Dave
b88e917461 qa/cephfs: upgrade xfstests_dev.py for FUSE mounts
Add support for testing CephFS mounted with FUSE using xfstests-dev.

Fixes: https://tracker.ceph.com/issues/55354
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-25 16:03:45 +05:30
Rishabh Dave
d8749faf21
Merge pull request #50142 from rishabh-d-dave/fs-xfstests_dev-distro
qa/cephfs: add 'rhel' to family of RH OS in xfstest_dev.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-02-24 22:51:20 +05:30
Samuel Just
acedd169e4
Merge pull request #48516 from athanatos/sjust/wip-57801
crimson,mon: add guards to avoid accidental crimson deployment and to avoid usage of unsupported features with crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2023-02-23 18:39:27 -08:00
Rishabh Dave
fec9be184d qa/cephfs: add 'rhel' to family of RH OS in xfstest_dev.py
The string returned by get_system_type() for RHEL 8.4 recently wasn't
one of the strings that xfstests_dev.py expects. The string returned
this time was instead "rhel". Add this string to the list of strings
that xfstests_dev.py expects for Red Hat OS family so that this failure
won't occur again in future. Also log the detected distro and version so
that it is easier to debug when this issue occurs next time.

Fixes: https://tracker.ceph.com/issues/58726
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-23 22:31:27 +05:30
Ilya Dryomov
b21a379c5b librbd: call apply_changes() after setting librados_thread_count
Otherwise the setting doesn't take effect.  While at it, replace
home-grown stringify() with standard to_string().

Fixes: https://tracker.ceph.com/issues/58833
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-23 12:50:45 +01:00
Rishabh Dave
4017bfa2c5
Merge pull request #46574 from rishabh-d-dave/xfstests-dev-improvements
qa/cephfs: improvements for xfstests_dev.py

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-23 17:08:53 +05:30
Radoslaw Zarzynski
6668814a17
Merge pull request #49528 from NitzanMordhai/wip-nitzan-filestore-removal
Reef: filestore removal

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-23 12:20:49 +01:00
Venky Shankar
da7928adab Merge PR #50196 into main
* refs/pull/50196/head:
	qa: export DIFF_LENGTH env parameter to print the full git diff

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-23 11:16:21 +05:30
Laura Flores
39df3b8827 qa/crontab: temporarily disable teuthology cronjobs
Temporarily disable teuthology cron jobs before
the Reef dev freeze. This commit is intended to
lessen the load on the teuthology queue as we
test final patches for Reef, but should be reverted
after the dev freeze.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-02-22 23:17:19 +00:00
Rishabh Dave
92aaaf7b57 qa/cephfs: save results dir from xfstests-dev repo
The "results" directory in xfstests-dev repo contains logs that are
created by it during execution of its tests. Copying this directory to
Ceph log directory saves it from being erased during tear down conducted
by teuthology test runner. These logs allow deeper inspection in to the
cause of failure of tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:44:29 +05:30
Rishabh Dave
053a1fb851 qa/cephfs: don't create symlinks for xfstests-dev
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
18465b8f56 qa/cephfs: modify user creation code for xfstests-dev
Create user "fsgqa2" and, since xfstests-dev's README uses option "-m"
while creating user "fsgqa", do the same in our codebase.

Creating user "fsgqa2" is optional along with other users according to
xfstests-dev's README. Create all these users is a preventive step since
tests needing user/users maybe skipped during execution.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
392e278749 qa/cephfs: log xfstests-dev local.config
Log contents of xfstests-dev repo config file "local.config" to make
debugging easier.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
4d75730200 qa/cephfs: minor changes in xfstests_dev.py
In xfstests-dev repo, local.config options "SCRATCH_DEV" and
"SCRATCH_MNT" were enabled in commit
2ba6b87105 but TODO comment for the
wasn't deleted. Let's delete that and use format string instead of
format function on regular string.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:29:02 +05:30
Rishabh Dave
49c7a79f8a qa/cephfs: make it easier to use xfstests-dev using vstart_runner
The intention of this re-arrangement is to make it easier for developers
to use an already cloned and built xfstests-dev repo  with
vstart_runner.py.

The re-arrangements are -

* move tearDown() closer to setUp() (so that devs can
  enable/disable parts of them easily),

* carve out separate methods (so that these codeblocks can be disabled
  in a single go) for -
  ** deleting groups and users,
  ** deleting xfstests-dev and xfsprogs-dev repo and
  ** building xfstests-dev repo

* carve out method for deleting xfstests-dev and xfsprogs-dev repos

* add self.xfstests_repo_path comment (for devs to set path to
  xfstests-dev repo), and,

* add a note showing how to use already cloned and built xfstests-dev
  repo

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:07:48 +05:30
Rishabh Dave
4a00b2bbb1 qa/cephfs: use standard name for logging variable
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-22 20:03:05 +05:30
Venky Shankar
ae6f1b18be Merge PR #50175 into main
* refs/pull/50175/head:
	cephfs: upgrade cephfs-shell's path wherever necessary

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-02-22 19:47:43 +05:30
Xiubo Li
0f9d3f2d3d qa: export DIFF_LENGTH env parameter to print the full git diff
From the xfstests-dev's README doc set DIFF_LENGTH to 0 will print
the full diff instead of the default 10.

This will be very helpful to debug the test failures, or we will
lose many valuable debug infomation.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-22 14:15:14 +08:00
Venky Shankar
86b90286cd Merge PR #49934 into main
* refs/pull/49934/head:
	qa: add test_fscrypt_dummy_encryption test case support
	qa: add 'options' parameter support for write_local_config
	qa: add ceph.exclude file to exclude individual tests
	qa: add require_kernel_mount helper support
	qa: rename test_fscrypt to test_fscrypt_encrypt

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-22 11:41:44 +05:30
Rishabh Dave
48ef044477 cephfs: upgrade cephfs-shell's path wherever necessary
Commit dc69033763 moves cephfs-shell from
"<CEPH-REPO-ROOT>/src/tools/cephfs/" to
"<CEPH-REPO-ROOT>/src/tools/cephfs/shell" but cephfs-shell's location in
src/vstart.sh and qa/tasks/cephfs/test_cephfs_shell.py is left
un-updated. This produces a broken vstart_environment.sh and broken
export command in test_cephfs_shell.py.

Introduced-by: dc69033763
Fixes: https://tracker.ceph.com/issues/58795
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-02-21 19:16:27 +05:30
Xiubo Li
b420db8dba qa: add test_fscrypt_dummy_encryption test case support
This will test around 130 test cases with the test_dummy_encryption
option enabled.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-21 16:15:12 +08:00
Xiubo Li
306b8bea4a qa: add 'options' parameter support for write_local_config
This will allow the fscrypt test case to add dedicated options.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-21 12:41:33 +08:00
Xiubo Li
002d45b589 qa: add ceph.exclude file to exclude individual tests
Skip some tests, which will take to much time or will fail, for now.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-21 12:41:30 +08:00
Venky Shankar
347acdf738 Merge PR #49154 into main
* refs/pull/49154/head:
	qa: add tests for cephfs-top
	doc: updates --dump and --dumpfs
	cephfs-top: dump values to stdout

Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
2023-02-21 09:33:18 +05:30
Laura Flores
46d0cbf033
Merge pull request #49910 from myoungwon/fix-58587
qa/workunits/rados/test_dedup_tool.sh: reset dedup tier during tests
2023-02-20 14:24:03 -06:00
Casey Bodley
2a7301dbbb
Merge pull request #49950 from cbodley/wip-qa-ragweed-tox
qa/rgw: run ragweed tests with tox

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2023-02-20 09:29:50 -05:00
Casey Bodley
cc39576353 qa/rgw: update boto3 example path in notification_tests.py
fallout from https://github.com/ceph/ceph/pull/50102

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-18 09:30:55 -05:00
Ilya Dryomov
f4edd7728a
Merge pull request #49614 from isodude/wip-librbd-misalign-discard
librbd: Fix local rbd mirror journals growing forever

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-17 18:09:39 +01:00
Casey Bodley
b4250ba40b qa/rgw: add tox task before ragweed
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-16 15:37:58 -05:00
Casey Bodley
fc81fe705c qa/ragweed: run ragweed tests with tox
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-16 15:37:58 -05:00
Lucian Petrut
3fdad23e63 quincy: qa: update test_rbd_wnbd.py to report latency in seconds
At the moment, the latency results are reported in nanoseconds.
In order to improve readability, we'll convert it to seconds.

While at it, we'll fix the fio duration report, which we're
wrongfully dividing by 1000 twice.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-02-16 12:11:58 +00:00
Stefan Chivu
fa62b2add6 qa: Added latency to results table in test_rbd_wnbd.py
Added data regarding completion latency (ns) to the fio results
table, such as min, max, mean, stddev, etc.

Signed-off-by: Stefan Chivu <schivu@cloudbasesolutions.com>
2023-02-16 12:11:48 +00:00
Ilya Dryomov
fcfef0a19e qa/workunits/rbd-nbd: work around "rbd feature disable" hang
"rbd feature disable" appears to reliably hang if the corresponding
remote request is proxied to rbd-nbd (because rbd-nbd happens to own
the exclusive lock after a series of blkdiscard calls) [1].  Work
around it here by enabling journaling before the image is mapped
and disabling it after the image is unmapped.

Also, don't assert on the output of "rbd journal inspect --verbose"
having a certain number of entries.  This is racy: if the script gets
delayed after the last blkdiscard call for some reason, there may be
fewer entries present in the journal or none at all.

[1] https://tracker.ceph.com/issues/58740

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-16 13:05:05 +01:00
Jos Collin
ddd20d4d4d
qa: add tests for cephfs-top
* Updated the existing tests
* Added new qa tests for --dump and --dumpfs options
* Updated the cluster configuration to 2 mds, 2 clients

Fixes: https://tracker.ceph.com/issues/57014
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-02-16 15:13:19 +05:30
Xiubo Li
0ee2f80367 qa: add require_kernel_mount helper support
This will simplify the code later.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:43:39 +08:00
Xiubo Li
b699307a34 qa: rename test_fscrypt to test_fscrypt_encrypt
The test_fscrypt_encrypt will only run the 'encrypt' related test
cases without 'test_dummy_encryption' option enabled. This will
test the filename and content verification.

After this I will add the whole test cases with 'test_dummy_encryption'
option.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:43:39 +08:00
Ronen Friedman
4cc70b709d
Merge pull request #44749 from ronen-fr/wip-rf-large-chunk
osd/scrub: use separate chunk size configuration for shallow scrubs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2023-02-15 12:06:58 +02:00
Samuel Just
588dec7c72 qa/config/crimson_qa_overrides: set crimson experimental feature
Signed-off-by: Samuel Just <sjust@redhat.com>
2023-02-13 22:36:36 -08:00
Samuel Just
d79c22c564 qa/config/crimson_qa_overrides: add overrides.ceph.flavor=crimson
teuthology/suite/placeholder.py sets it to default otherwise.

Fixes: https://tracker.ceph.com/issues/57873
Signed-off-by: Samuel Just <sjust@redhat.com>
2023-02-13 22:36:36 -08:00
Samuel Just
f954e63985 qa/config/crimson_qa_overrides: set osd_pool_default_crimson
Signed-off-by: Samuel Just <sjust@redhat.com>
2023-02-13 22:36:35 -08:00
Samuel Just
0449a4093e qa/task/ceph.py: set-allow-crimson for crimson clusters
Signed-off-by: Samuel Just <sjust@redhat.com>
2023-02-13 22:36:35 -08:00
Ronen Friedman
ffda64119f osd/scrub: create a separate chunk size conf for shallow scrubs
Using the existing common default chunk size for scrubs that are
not deep scrubs is wasteful: a high ratio of inter-OSD messages
per chunk, while the actual OSD work per chunk is minimal.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-02-14 07:58:01 +02:00
Ilya Dryomov
6d8fb57156
Merge pull request #50071 from idryomov/wip-58688
qa/suites/rbd: fix sporadic "rx-only direction" test failures

Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-02-13 12:54:26 +01:00
Nitzan Mordechai
e682b60617 test: Remove all filestore tests and use
- test_trans convert FileStore to BlueStore test
 - xattr_bench convert FileStore to BlueStore usage
 - remove test_idempotent_sequence tests
 - remove test_idempotent
 - remove test_filejournal
 - removing filestore tests from store_test
 - remove rep_read_unfound test for filestore only
 - remove osd-dup convert filestore to bluestore
 - osd-scrub-repair start only bluestore osd
 - osd-pool-create remove filestore expected_num_object test
 - Remove chain_xattr and LFNIndex uneeded test

Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
2023-02-12 06:11:29 +00:00
Nitzan Mordechai
5008369a3e ceph_manager: removing filestore objectstore-tools file apply-layout-setting
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2023-02-12 06:11:29 +00:00
Nitzan Mordechai
d8202bc67c rados: upgrade suite test filestore removal
When upgrading osd with filestore to reef, restart should not be possible
the osd won't boot and error message will be showed in the osd log

Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
2023-02-12 06:11:29 +00:00