Commit Graph

2792 Commits

Author SHA1 Message Date
Casey Bodley
de55616429
Merge pull request #53607 from cfsnyder/wip-cfsnyder-bucket-check-bug
rgw: fix radosgw-admin bucket check stat calculation bug

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-27 18:38:42 +01:00
Kamoltat (Junior) Sirivadhna
dc5d6ffa42
Merge pull request #52442 from kamoltat/wip-ksirivad-fix-tracker-61922
pybind/mgr/pg_autoscaler: noautoscale flag retains individual pool configs
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2023-09-25 17:57:38 -04:00
Kamoltat
0972dbf225 qa/workunits: modified tests for noautoscale flag change
modified:

`qa/workunits/mon/test_noautoscale_flag.sh`
`qa/workunits/cephtool/test.sh`

adding test coverage to files mentioned above

Fixes: https://tracker.ceph.com/issues/61922

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2023-09-25 12:46:36 +00:00
Cory Snyder
340522f9ae rgw: add test case to reproduce bucket check stats bug for versioned bucket
Reproduces a regression where radosgw-admin bucket check incorrectly counts
objects that started as unversioned and later transitioned to versioned.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-09-22 21:17:55 +00:00
Casey Bodley
9b7cd7fc7d
Merge pull request #52576 from cfsnyder/wip-62075
rgw: add radosgw-admin bucket check olh/unlinked commands

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-22 15:33:25 +01:00
Patrick Donnelly
6c993ee3bb
Merge PR #53175 into main
* refs/pull/53175/head:
	qa: increase the http postBuffer size and disable sslVerify

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-18 08:37:16 -04:00
Cory Snyder
b79dcf640a qa/workunits/rgw: add tests that reproduce bucket stats inconsistency bugs
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-09-15 18:43:17 +00:00
Cory Snyder
9b2042a075 rgw: add radosgw-admin bucket check olh/unlinked commands
Adds commands to radosgw-admin for checking for and fixing leftover
entries in the bucket index (and associated RADOS objects).

Fixes: https://tracker.ceph.com/issues/62075
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-09-15 18:43:17 +00:00
Ilya Dryomov
153df2d64b qa: add "failover / failback loop" test for rbd-mirror
For snapshot-based mirroring, check that demote (or other mirror
snapshots) don't pile up.  Nothing in particular to assert on for
journal-based mirroring but the test is still useful.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-01 19:05:36 +02:00
Xiubo Li
cafbe1cf15 qa: increase the http postBuffer size and disable sslVerify
The git repo size is large and maybe the postBuffer size is not
large enough and this will cause git commands' failure of '128'.

Fixes: https://tracker.ceph.com/issues/59413
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-08-28 17:13:27 +08:00
Casey Bodley
7d7218ee3c
Merge pull request #52007 from tcoldrick-bb/fix-invalid-json
rgw/rgw_admin.cc: Handle pg_ver and source_zone explicitly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-08-02 13:42:22 -04:00
Ilya Dryomov
d49df8d74c qa/workunits/rbd: use jammy version of qemu-iotests for centos 9
It's the one we are using for all recent distros.

While at it, get rid of custom bin directory -- it appears that both
v2.3.0 and v2.11.0 tests are happy with just symlinks in the current
directory.

Fixes: https://tracker.ceph.com/issues/61565
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-07-25 14:00:04 +02:00
Tom Coldrick
5e7c4a52ca rgw_admin.cc: Handle pg_ver and source_zone explicitly
At present, the pg_ver and source_zone attrs are handled as if they're
strings, but are in fact unsigned integer types.  This means that when
dumped they yield invalid UTF-8 characters.  As well as hiding the
meaning of these attributes, this also means that we're not compliant
with RFC 8259 for JSON, which specifies that externally visible JSON
must be UTF-8 encoded [0].

[0]: https://rfc-editor.org/rfc/rfc8259#section-8.1

Signed-off-by: Tom Coldrick <tcoldrick@bloomberg.com>
2023-07-25 10:32:24 +01:00
Casey Bodley
3281eb85ce
Merge pull request #52143 from cbodley/wip-61567
test/pybind: replace nose with pytest

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-07-20 14:51:05 -04:00
Cory Snyder
aa1f40e80d rgw: fix/improve test_rgw_versioning.py tests
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-07-14 12:50:55 +00:00
Casey Bodley
0d0128f50e test/pybind/rados: convert from nose to pytest
* replace nose.Skiptest with pytest.skip()
* replace nose attrs with @pytest.mark
* replace nose assertions
* replace setUp/tearDown with setup/teardown_method

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-07-06 11:02:37 -04:00
Casey Bodley
0bd813f069 test/pybind/cephfs: convert from nose to pytest
* use a fixture for setup_test()
* invoke test_cephfs.py with pytest

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-07-06 11:02:35 -04:00
Casey Bodley
af04457a43 test/pybind/rbd: convert from nose to pytest
* use fixtures for temporary images and groups
* use pytest.skip instead of nose.SkipTest
* replace setUp/tearDown with setup/teardown_method
* add @pytest.mark.skip_if_crimson
* replace nose assertions

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-07-06 11:02:11 -04:00
Cory Snyder
38133e59d2 rgw: cancel olh modification by removing pending xattrs in set_olh error scenarios
Makes request threads clean up after themselves and remove their pending
xattrs from OLH objects before they return an error from set_olh.

Fixes: https://tracker.ceph.com/issues/59663
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-07-04 09:44:04 +00:00
Cory Snyder
38e027a8c3 qa/workunits/rgw: add test for PUT response during reshard
Tests that a PUT succeeds when the bucket is actively resharding.

Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2023-07-04 09:31:59 +00:00
J. Eric Ivancich
fb9d481188
Merge pull request #50667 from ivancich/wip-add-reupload-testing
qa/rgw: test that multipart re-upload does not leave any orphans

Reviewed-by: Cory Snyder <csnyder@iland.com>
2023-06-30 11:30:27 -04:00
Adam King
54413fd570
Merge pull request #51738 from phlogistonjohn/jjm-cephadm-new-deploy
cephadm: create a new version of the deploy command

Reviewed-by: Adam King <adking@redhat.com>
2023-06-22 11:39:49 -04:00
Yuri Weinstein
b1ed114a66
Merge pull request #51450 from batrick/i59716
pybind/rados: keep byte representation if decode fails

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-06-21 17:12:08 -04:00
John Mulligan
f890d82478 qa/workunits/cephadm: update script use use cephadm _orch deploy
cephadm _orch deploy will eventually be the only deploy method so
we need to update the test script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-06-21 13:19:16 -04:00
Ilya Dryomov
ca20127c70
Merge pull request #52109 from idryomov/wip-61616
librbd: avoid object map corruption in snapshots taken under I/O

Reviewed-by: Mykola Golub <mgolub@suse.com>
2023-06-21 11:43:26 +02:00
Ilya Dryomov
acb270a3dd qa/workunits/rbd: make continuous export-diff test actually work
The current version is pretty useless:

- "rbd bench" writes the same byte (0xff) over and over again, so
  almost all checksumming is in vain
- snapshots are taken in a steady state (i.e. not under I/O), so no
  race conditions can get exposed
- even with these caveats, it's not wired up into the suite

Redo this workunit to be a reliable reproducer for the issue fixed
in the previous commit and wire it up for both krbd and rbd-nbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-06-20 22:14:39 +02:00
Adam King
8e5a983a73
Merge pull request #51267 from phlogistonjohn/jjm-cephadm-versioning
Support cephadm "binary" versioning

Reviewed-by: Adam King <adking@redhat.com>
2023-06-14 14:23:27 -04:00
Casey Bodley
4167b5c797
Merge pull request #48879 from samarahu/wip-d4n
RGW: WIP D4N

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-06-08 11:44:55 -04:00
Nizamudeen A
098f175d71
Merge pull request #51955 from rhcs-dashboard/teuth-e2e-fix
qa/workunits: fix test_dashboard_e2e.sh: no spec files found

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-06-08 14:18:14 +05:30
Nizamudeen A
bec3140b20 qa/workunits: fix test_dashboard_e2e.sh: no spec files found
Fixes: https://tracker.ceph.com/issues/61578
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-06-07 14:04:33 +05:30
Patrick Donnelly
1d3a549ec2
qa: test first-damage.py with binary file name
Specifically, one which cannot be decoded as utf8.

Fixes: https://tracker.ceph.com/issues/59716
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-06-06 21:39:42 -04:00
Patrick Donnelly
b6867bb255
qa: add option to set python executable
To support systems like mine with multiple python installs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-06-06 21:39:41 -04:00
Samarah
cd1d249678 QA: Add D4N teuthology suite
Signed-off-by: Samarah <samarah.uriarte@ibm.com>
2023-06-06 14:04:47 -04:00
J. Eric Ivancich
aeffd1b598 qa/rgw: test that multipart re-upload does not leave any orphans
Runs a boto script that reuploads one part multiple times before
completing and then we check for any orphans.

Original boto script contributed by Matt Benjamin
<mbenjami@redhat.com> on top of which modifications were made.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-06-02 17:39:26 -04:00
Nizamudeen A
7c5d92ad48 mgr/dashboard: fix test_dashboard_e2e.sh failure
The qa e2e is failing because the script is not adapted with cypress 10.

Fixes: https://tracker.ceph.com/issues/61519
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-05-31 11:09:25 +05:30
John Mulligan
bd59b80919 qa/workunits/cephadm: align test_cephadm.sh with new cephadm version
The `cephadm version` command no longer bases the output on the
container images, rather it uses a special python file added to the
zipapp during the build to report on the version of cephadm (the
binary).

The other option was to preserve this behavior and add a new version
command or make it behave differently depending on what options were
provided. I discussed the options with AMK in person and we decided that
changing the tests was preferable.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-05-22 13:26:07 -04:00
Matan
653b97e472
Merge pull request #51388 from Matan-B/wip-matanb-c-enable-rbd-tests
qa/suites/crimson: Enhance rbd api testing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-05-11 16:28:55 +02:00
Ramana Raja
a2f15d4b2f qa/workunits/rbd: Add tests for rbd_support module recovery
... after the module's RADOS client is blocklisted.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2023-05-08 16:45:41 -04:00
Matan Breizman
5823c04542 qa/suites/crimson: Skip unsupported tests (Crimson)
Align with `rbd_api_tests` and skip deep_copy and breaklock tests
in Crimson.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-05-08 10:57:06 +00:00
Adam King
2f3afa76ee
Merge pull request #51226 from jsoref/spelling-orchestrator
orchestrator: Fix spelling

Reviewed-by: Adam King<adking@redhat.com>
2023-05-03 17:31:04 -04:00
Josh Soref
965ee91d3f rbd: fix spelling errors
* acquire
* are
* asynchronous
* attempt
* bootstrap
* concurrent
* consume
* couldn't
* cumulative
* disable
* disabling
* disaster
* disconnected
* endianness
* entries
* exclusive
* filesystem
* flag
* generic
* github
* image
* information
* initiating
* latency
* limitations
* metadata
* modify
* namespace
* noautoconsole
* ourselves
* prefetch
* propagate
* protection
* recorder
* recover
* release
* replicated
* reserved
* selection
* sentinel
* several
* snapshot
* source
* specifying
* suppress
* synchronize
* the
* transfer
* triggering
* unknown
* validation
* version
* visible
* write log entries

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-26 09:30:53 -04:00
Josh Soref
94ade0cd16 orchestrator: fix spelling errors
* a new
* accommodated
* adopted
* appended
* because
* bootstrap
* bootstrapping
* brackets
* classes
* cluster
* compatible
* completely
* confusion
* daemon
* daemons
* dashboard
* enclosure
* existing
* explicit
* following
* format
* host
* implementation
* inferred
* keepalived
* kubic
* maintenance
* necessarily
* necessary
* network
* notifier
* octopus
* permanent
* presenting
* related
* see
* snapshot
* stateful
* the
* track
* version
* wasn't
* weird

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-26 09:21:42 -04:00
Venky Shankar
6684f3e55e Merge PR #50909 into main
* refs/pull/50909/head:
	qa/workunit: print the detail commands excuted in the scripts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-04-14 15:59:12 +05:30
Yuri Weinstein
23a958d647
Merge pull request #47893 from kotreshhr/ceph-mgr-finisher-block
mgr: Add one finisher thread per module

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-04-06 09:23:24 -07:00
Xiubo Li
907bf5214d qa/workunit: print the detail commands excuted in the scripts
This will make it to be much easier when debugging the qa tests failures.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-04-06 10:38:14 +08:00
Mark Kogan
5846a9c267 rgw : fix python script using s3cmd with error code 403 ubuntu 20.04
Fixes: https://tracker.ceph.com/issues/54104

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2023-04-03 15:18:55 +03:00
Adam King
c36a07d425
Merge pull request #49824 from adk3798/iscsi-etc-hosts
cephadm: mount host /etc/hosts for daemon containers in podman deployments

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-03-30 12:24:32 -04:00
Venky Shankar
e84f319c66 Merge PR #50053 into main
* refs/pull/50053/head:
	libcephfs: move ClearSetuid to suidsgid.cc
	libcephfs: add test cases for dropping the suid/sgid in write/truncate
	libcephfs: add test cases for dropping the suid/sgid in fallocate
	libcephfs: fix ClearSetuid incorrectly using SETATTR_MODE mask
	client: switch to clear_suid_sgid for ftruncate
	client: switch to clear_suid_sgid for _write()
	mds/client: clear the suid/sgid in fallocate path
	client: allow unprivileged users to clear suid/sgid

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2023-03-30 14:48:26 +05:30
Rishabh Dave
877f6be743 qa/workunit/fs: print commands for making debugging easier
Print the commands and their arguments as they are being executed for
kernel_untar_tar.sh so that it's easier to debug when a teuthology
failure occurs due to it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-03-29 18:06:58 +05:30
Kotresh HR
2c2ef6d56b qa: Add test for per-module finisher thread
Fixes: https://tracker.ceph.com/issues/51177
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2023-03-29 11:34:07 +05:30