Commit Graph

10284 Commits

Author SHA1 Message Date
Casey Bodley
de55616429
Merge pull request #53607 from cfsnyder/wip-cfsnyder-bucket-check-bug
rgw: fix radosgw-admin bucket check stat calculation bug

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-27 18:38:42 +01:00
Kamoltat (Junior) Sirivadhna
dc5d6ffa42
Merge pull request #52442 from kamoltat/wip-ksirivad-fix-tracker-61922
pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2023-09-25 17:57:38 -04:00
Kamoltat
0972dbf225 qa/workunits: modified tests for noautoscale flag change
modified:

`qa/workunits/mon/test_noautoscale_flag.sh`
`qa/workunits/cephtool/test.sh`

adding test coverage to files mentioned above

Fixes: https://tracker.ceph.com/issues/61922

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2023-09-25 12:46:36 +00:00
Casey Bodley
3cdfef2cf6
Merge pull request #53097 from mkogan1/rgw-wip-t62141-valg-supp
valgrind: UninitCondition under __run_exit_handlers suppression

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-23 16:11:38 +01:00
Cory Snyder
340522f9ae rgw: add test case to reproduce bucket check stats bug for versioned bucket
Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-09-22 21:17:55 +00:00
Venky Shankar
8a84a6cc70 Merge PR #53279 into main
* refs/pull/53279/head:
	qa: descrease pgbench scale factor to 32 for postgresql database test

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2023-09-22 20:11:12 +05:30
Venky Shankar
8f9fc152c5 Merge PR #53418 into main
* refs/pull/53418/head:
	qa: use exisitng ignorelist override list for fs:mirror[-ha]

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2023-09-22 20:09:44 +05:30
Rishabh Dave
954ed30f3c
Merge pull request #48027 from lxbsz/wip-57154
mds/client: check the cephx mds auth access in client side

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-09-22 20:08:10 +05:30
Casey Bodley
9b7cd7fc7d
Merge pull request #52576 from cfsnyder/wip-62075
rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-22 15:33:25 +01:00
Rishabh Dave
c4f968b931
Merge pull request #52042 from rishabh-d-dave/better-errmsg-for-perm
MDSAuthCaps: print better error message for perm flag in MDS caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-09-22 20:02:40 +05:30
Rishabh Dave
3234c696de
Merge pull request #47533 from rishabh-d-dave/ceph-fuse-cmd
qa/cephfs: cleanup code for ceph-fuse command

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-09-22 20:01:13 +05:30
Casey Bodley
503c855f86
Merge pull request #52259 from cbodley/wip-61859
qa/rgw: disable 'rgw lc debug interval' in most suites

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2023-09-22 15:08:36 +01:00
Ronen Friedman
5ecd20f620
Merge pull request #53531 from ronen-fr/wip-rf-squeue2
osd/scrub: extract scrub initiation code out of the OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-09-22 16:46:38 +03:00
Ilya Dryomov
f0c48dfcd6
Merge pull request #53311 from idryomov/wip-62711
qa/suites/{rbd,krbd}: disable POOL_APP_NOT_ENABLED health check

Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-09-22 15:44:35 +02:00
Rishabh Dave
104fd674b0
Merge pull request #53528 from rishabh-d-dave/cephfs-qa-mdtest
qa/cephfs: fix build failure for mdtest project

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-22 11:56:03 +05:30
Casey Bodley
b975270d4e qa/rgw: enable s3tests for 'storage classes'
other than the rgw/lifecycle subsuite, all of the other subsuites were just
adding storage classes to the rgw task, but not the s3tests task that
enables the storage class test cases

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-21 17:51:59 -04:00
Casey Bodley
d46af6d3de qa/rgw: disable 'rgw lc debug interval' in most suites
this has a performance cost, and the teuthology environment can
already be sluggish. if lifecycle expiration/transition testing
was limited to the rgw/lifecycle and rgw/cloud-transition suites,
the others would probably run significantly faster

Fixes: https://tracker.ceph.com/issues/61859

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-21 17:51:59 -04:00
Casey Bodley
c7f2bc42d9 qa/s3tests: use cloudtier_tests and lifecycle_tests to control filters
when `extra_attrs` was made additive, these tests were adding both
'not lifecycle_expiration' and 'lifecycle_expiration' to the command
line. use a yaml flag instead to control whether or not the s3tests task
adds the 'not' filter

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-21 17:51:59 -04:00
Casey Bodley
62245cb5f3 qa/rgw: remove duplicate overrides in rgw/lifecycle
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-09-21 17:51:59 -04:00
Ilya Dryomov
0b68a8b4c0 qa/suites/krbd: disable POOL_APP_NOT_ENABLED health check
... same as for rbd suite.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-21 23:36:07 +02:00
Ilya Dryomov
9e884ddeec qa/suites/rbd: drop POOL_APP_NOT_ENABLED from ignorelists
With "mon warn on pool no app = false" in the config, it's obviously
redundant.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-21 23:36:07 +02:00
Ilya Dryomov
e64830eb8e qa/suites/rbd: disable POOL_APP_NOT_ENABLED health check
Commit 990806e635 ("mon, qa: issue pool application warning even
if pool is empty") made it impossible to create a pool without raising
a (bogus) health alert.  See [1] for details.

[1] https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/ZTDYC5HN677RR26EB4P6PORN6L2IFH4R/

Fixes: https://tracker.ceph.com/issues/62711
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-21 23:36:07 +02:00
Mark Kogan
03616b4330 valgrind: UninitCondition under __run_exit_handlers suppression
reqiered in CentOS / RHEL 9 & Ubuntu 22.04.1 LTS

Fixes: https://tracker.ceph.com/issues/62141

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2023-09-21 16:55:30 +03:00
Rishabh Dave
8b62329195 qa/cephfs: add "set -x" in mdtest.yaml
Set the flag for printing the commands that will be executed so that
it's easier to go through teuthology.log

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-21 18:16:21 +05:30
Rishabh Dave
6a2e94b323 qa/cephfs: fix build failure for mdtest project
To fix the mdtest job failure (which happens because building mdtest
project fails) do -

1. Use ior projects intead of mdtest project bcecause latter was merged
   into former. See:
   https://github.com/MDTEST-LANL/mdtest/blob/master/README.md

2. Purge mpich package and then install it again. This is a vital step
   that's needed to build ior project on Ubuntu 22.04.

Fixes: https://tracker.ceph.com/issues/61574
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-21 18:15:16 +05:30
Patrick Donnelly
0737936248
Merge PR #53149 into main
* refs/pull/53149/head:
	qa: lengthen shutdown timeout for thrashed MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-09-20 08:31:17 -04:00
Ronen Friedman
0af4fc4261 osd/scrub: extract restrictions_on_scrubbing()
from ScrubQueue::select_pg_and_scrub().

Clearing the path to moving some ScrubQueue methods into
OscScrub. Starting here with the CPU load tracker.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-20 01:39:10 -05:00
Milind Changire
29eb1ca580
Merge PR #52892 into main
* refs/pull/52892/head:
	qa: add test to validate periodic checks by async threads
	mgr/volumes: periodically check for async work

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2023-09-19 12:53:29 +05:30
Milind Changire
a94089bde2
Merge PR #52692 into main
* refs/pull/52692/head:
	qa/tasks/cephfs: reset the client_inject_fixed_oldest_tid after test

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-19 12:51:52 +05:30
Patrick Donnelly
6c993ee3bb
Merge PR #53175 into main
* refs/pull/53175/head:
	qa: increase the http postBuffer size and disable sslVerify

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-18 08:37:16 -04:00
Cory Snyder
b79dcf640a qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-09-15 18:43:17 +00:00
Cory Snyder
9b2042a075 rgw: add radosgw-admin bucket check olh/unlinked commands
Adds commands to radosgw-admin for checking for and fixing leftover
entries in the bucket index (and associated RADOS objects).

Fixes: https://tracker.ceph.com/issues/62075
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-09-15 18:43:17 +00:00
Patrick Donnelly
6c387554d8
Merge PR #52638 into main
* refs/pull/52638/head:
	mds: flush monc log before abort
	qa: check for expected cluster log message
	qa: ignore expected cluster warning from damage tests

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-09-15 14:38:02 -04:00
Venky Shankar
98758eee89 Merge PR #53070 into main
* refs/pull/53070/head:
	PendingReleaseNotes: add note about 'm' and 'M' periods
	doc/cephfs: add note about monthly scheduling and retention period
	qa: update minute period multiplier to m
	qa: change minute duration specifier from M to m
	mgr/snap_schedule: change minute duration specifier from M to m

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-14 15:39:07 +05:30
Venky Shankar
99adb599e7 Merge PR #51891 into main
* refs/pull/51891/head:
	qa: fix cephfs-top tests for the new fields
	cephfs-top: include the missing fields in --dump output

Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2023-09-14 09:29:07 +05:30
Venky Shankar
a728d170cc qa: use exisitng ignorelist override list for fs:mirror[-ha]
The custom ignorelist yaml would also require `POOL_APP_NOT_ENABLED`
to be ignored. The custom yaml is unnecessary anyway.

Fixes: http://tracker.ceph.com/issues/62482
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-09-13 12:04:43 +05:30
Venky Shankar
59d1f7655c Merge PR #53153 into main
* refs/pull/53153/head:
	Revert "mds: align quota.max_bytes to 4MB or 4KB"

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-09-13 11:10:59 +05:30
Jos Collin
0241a82b4d
qa: fix cephfs-top tests for the new fields
Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-09-12 17:35:30 +05:30
Milind Changire
9c88234791
qa: update minute period multiplier to m
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 14:11:03 +05:30
Milind Changire
e7218e03ae
qa: change minute duration specifier from M to m
Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 13:58:26 +05:30
Rishabh Dave
7d1f6d70f1 qa/cephfs: set omit_sudo for cmd mounting FUSE conn dir
vstart_runner.py removes sudo from the command arguments when "omit_sudo"
is not explicitly set to "False". This leads to 2 error messages in output
of vstart_runner.py which are copied at the end of this commit message.
Setting "omit_sudo=False" for the execution of command should remove both
of these error messages.

This also fixes the potential crash of vstart_runner.py when the FUSE
connections directory is not mounted on the machine and it requires
superuser privileges to mount it.

The 2 error messages copied as it is from the log -

DEBUG:__main__:"sudo" was omitted from the following cmd args before execution and logging using function overriding; check vstart_runner.py for more details.
DEBUG:__main__:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections
mount: /sys/fs/fuse/connections: must be superuser to use mount.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-12 13:26:00 +05:30
Rishabh Dave
c4b2ccd9d6 qa/cephfs: check before mounting fuse connections directory
Check if "/sys/fs/fuse/connections" is already mounted before attempting
to mount it again. Doing so leads to unnecessary error messages in the
vstart_runner.py's output (and most likely in logs for teuthology test
runs too) which confuses the users. See end for the error message.

Also, remove "check_status=False" so that the when the commands crashes
the testing is halted immediately. Initially, this must've been added to
ignore errors generated from re-mounting FUSE connections directory
repeatedly.

The error message -

DEBUG:__main__:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections
mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-12 13:25:45 +05:30
Adam King
2b839838f7
Merge pull request #52881 from adk3798/upgrade-test-start-squid
qa/cephadm: start upgrade tests from quincy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2023-09-11 11:29:38 -04:00
Adam King
a45ade8c15
Merge pull request #52708 from vshankar/wip-62236
qa: move nfs (mgr/nfs) related tests to fs suite

Reviewed-by: Adam King <adking@redhat.com>
2023-09-11 10:56:20 -04:00
Ramana Raja
a04044670c qa/tasks/cephfs/test_admin: run root_squash tests only for FUSE client
kclient doesn't have CEPHFS_FEATURE_MDS_AUTH_CAPS required to
enforce root_squash. Run root_squash tests only for FUSE client.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2023-09-11 09:29:46 +08:00
Ramana Raja
c8bae07f70 qa/tasks/cephfs: Add reproducer for https://tracker.ceph.com/issues/56067
A kernel CephFS client with MDS root_squash caps is able to write to a
file as non-root user. However, the data written is lost after clearing
the kernel client cache, or re-mounting the client. This issue is not
observed with a FUSE CephFS client.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2023-09-11 09:29:46 +08:00
Xiubo Li
28023f84d7 qa: add test for checking access in client side of root_squash
Test the 'chown' and 'truncate', which will call the setattr and
'cat' will open the files. Before each testing will open the file
by non-root user and keep it to make sure the Fxw caps are issued,
and then user the 'sudo' do to the tests, which will set the uid/gid
to 0/0.

Fixes: https://tracker.ceph.com/issues/57154
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-11 09:29:46 +08:00
Xiubo Li
4af6a3f744 qa: add sudo paramter for read_file()
Fixes: https://tracker.ceph.com/issues/57154
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-11 09:29:46 +08:00
Patrick Donnelly
7264120b26
Merge PR #53331 into main
* refs/pull/53331/head:
	Revert "Revert "Merge PR #53077 into main""

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2023-09-09 10:50:22 -04:00
Rishabh Dave
8e1e322054 AuthMonitor: no need to check permission in MDS caps
For "fs authorize" command, AuthMonitor.cc checks if permissions is "r"
or begins with "rw". This check is redundant now.
AuthMonitor::valid_caps() runs MDSAuthCaps.parse() which now runs same
check for the MDS caps, regardless of the command.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-08 20:17:37 +05:30