Commit Graph

204 Commits

Author SHA1 Message Date
Casey Bodley
0d0128f50e test/pybind/rados: convert from nose to pytest
* replace nose.Skiptest with pytest.skip()
* replace nose attrs with @pytest.mark
* replace nose assertions
* replace setUp/tearDown with setup/teardown_method

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-07-06 11:02:37 -04:00
Laura Flores
46d0cbf033
Merge pull request #49910 from myoungwon/fix-58587
qa/workunits/rados/test_dedup_tool.sh: reset dedup tier during tests
2023-02-20 14:24:03 -06:00
myoungwon oh
b2534d90ec qa/workunits/rados/test_dedup_tool.sh: reset dedup tier during tests
Dedup_tier option needs to be reset whenever every test begins,
because dedup_tier(dedup_chunk_pool) is deleted at the end of each
test---if not, this causes error in case dedup_tier's pool id is
changed.
This commit adds a set command to refresh newly added dedup_tier
pool id.

fixes: https://tracker.ceph.com/issues/58587

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-01-30 11:10:15 +09:00
Tim Serong
93c0456159 qa/workunits/rados/test_crash: chown crash files to ceph user
Fixes: https://tracker.ceph.com/issues/58098
Signed-off-by: Tim Serong <tserong@suse.com>
2022-12-08 12:02:40 +11:00
Laura Flores
bf30531f01 qa/workunits/rados: specify redirect in curl command
This workunit currently grabs files in html form, since
there is a redirect that occurs. If we specify `-L` in the
curl command, it handles redirects and graps the files
appropriately.

Fixes: https://tracker.ceph.com/issues/58046
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-11-18 00:31:59 +00:00
Laura Flores
80893b4e51
Merge pull request #47900 from Matan-B/wip-matanb-librados-20
examples/librados: Compile with c++20 on Ubuntu
2022-09-21 10:06:12 -05:00
Samuel Just
ad87f822bd
Merge pull request #45369 from jyha200/wip-samplededup
ceph_dedup_tool: adding sample-dedup

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-09-19 10:32:19 -07:00
Matan Breizman
be838652fa examples: Compile with c++20
This change will apply to ubuntu only.
Fixes: https://tracker.ceph.com/issues/57163

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-09-19 13:18:20 +00:00
JinyongHa
4e0add1b9a qa: add test case for sample_dedup operation of dedup_tool
Signed-off-by: JinyongHa <jy200.ha@samsung.com>
2022-08-26 00:52:01 +00:00
Laura Flores
d6b46a2c77 qa/workunits/rados: pull librados files from "main" instead of "master"
Fixes: https://tracker.ceph.com/issues/57122
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-08-16 18:07:51 +00:00
myoungwon oh
ea749df517 osd/,rados/: modify TIER_FLUSH to implicitly make an object a manifest object
Existing object-dedup command always creates temporary object and
perform set-chunk to make the input object manifest---this is because
current tier-flush, which is called by object-dedup, works only if
the target object is manifest, otherwise it just return 0.

So, object-dedup may not work if the target is already manifest.
Moreover, it causes unnecessary overhead if the target object is not manifest.

To solve these, this commit makes existing tier-flush to
change object's state to manifest implicitly.

With this change, the only required operation when calling object-dedup
is a tier-flush and object-dedup can work with the manifest object.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-07-28 10:22:11 +09:00
myoungwon oh
565320f04d tool/ceph-dedup-tool: add performing dedup option on cloned object
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-06-22 14:47:23 +09:00
myoungwon oh
16e7d5578c qa: add object-dedup test
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-12-14 13:49:45 +09:00
myoungwon oh
e7e875c547 qa: add chunk-dedup test
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-12-14 12:27:43 +09:00
myoungwon oh
dd1d563ff2 qa/test_dedup_tool: fix test cases for ceph-dedup-tool
1. add a delay after pool creation
2. fix checking output due to changed output format
3. fix wrong variable to test chunk-repair

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-11-12 13:22:48 +09:00
myoungwon oh
c1d2119d5e qa: add a test for chunk-repair command
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-10-16 01:49:30 +09:00
Mykola Golub
7311f6656f qa/suites/rados: add crushdiff test
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2021-08-27 17:45:40 +03:00
Kefu Chai
056065c7e6
Merge pull request #42429 from neha-ojha/wip-51638-cleanup
qa/*/test_envlibrados_for_rocksdb.sh: remove OS specific configuration

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-21 11:24:40 +08:00
Neha Ojha
76466da6f4 qa/*/test_envlibrados_for_rocksdb.sh: remove OS specific configuration
Details in https://tracker.ceph.com/issues/51638#note-3

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-07-16 17:20:34 +00:00
Danny Abukalam
4eb7cc5ce2
qa: Add SoftIron Linux as valid debian-based distro in test scripts
Signed-off-by: Danny Abukalam <danny@softiron.com>
2021-07-15 07:16:59 -04:00
Neha Ojha
345cb641ed qa/*/test_envlibrados_for_rocksdb.sh: install libarchive-3.3.3
To workaround the libarchive dependency issue seen with centos 8, which
has been causing consistent failures like

```
2021-06-04T04:52:51.147 INFO:tasks.workunit.client.0.smithi071.stdout:Installed:
2021-06-04T04:52:51.148 INFO:tasks.workunit.client.0.smithi071.stdout:  cmake-3.18.2-9.el8.x86_64                 cmake-data-3.18.2-9.el8.noarch
...
2021-06-04T04:52:57.554 INFO:tasks.workunit.client.0.smithi071.stderr:+ cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_TESTS=ON -DWITH_LIBRADOS=ON -DWITH_SNAPPY=ON -DWITH_GFLAGS=OFF -DFAIL_ON_WARNINGS=OFF ..
2021-06-04T04:52:57.579 DEBUG:teuthology.orchestra.run:got remote process result: 127
2021-06-04T04:52:57.580 INFO:tasks.workunit.client.0.smithi071.stderr:cmake: symbol lookup error: cmake: undefined symbol: archive_write_add_filter_zstd
```
More details in https://tracker.ceph.com/issues/51101#note-5

Fixes: https://tracker.ceph.com/issues/51101
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-07-13 15:17:54 +00:00
Kefu Chai
caf364db5e
Merge pull request #38120 from kiizawa/wip-cls-remote-read
osd: allow remote read by calling cls method from within cls context

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-12 16:42:52 +08:00
Ken Iizawa
5a71708cbd osd: add an object class to invoke remote reads along with a test to exercise it
Fixes: https://tracker.ceph.com/issues/48182
Signed-off-by: Ken Iizawa <iizawa.ken@fujitsu.com>
2021-04-04 01:42:47 +09:00
myoungwon oh
ed24df159c qa/workuints: add ceph_dedup_tool test for scrub refcount
Based on CDC flush, a test, which calls get/put reference,
is added.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-03-29 17:13:28 +09:00
Patrick Donnelly
54b3c8f7a7
test/libcephsqlite,qa: add tests for libcephsqlite
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:55 -07:00
Kefu Chai
6364748328 qa/workunits/rados/test_envlibrados_rocksdb: s/PowerTools/powertools/
see also 7a68845579

Fixes: https://tracker.ceph.com/issues/49787
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-13 17:03:32 +08:00
Samuel Just
4a73fd2251
Merge pull request #35899 from myoungwon/wip-fix-manifest-tier
osd, test: refactoring manfiest-tier

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-01 12:49:57 -07:00
myoungwon oh
7217a6c00e osd, test: remove all things related to manifest_flush
Due to a plan to re-work based on CDC-based flush,
remove manifest_flush()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsumg.com>
2020-08-26 21:05:14 +09:00
Kefu Chai
af93f56203 qa/workunits/rados: quote optional args
otherwise they will be escaped by shell

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-22 16:11:36 +08:00
Kefu Chai
6ad87dbbdc qa/workunits/rados: pass options down to test_rados.py
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-21 16:41:31 +08:00
Sage Weil
84f9603d8d ceph-dedup-tool: get-chunk-ref -> dump-chunk-refs
Signed-off-by: Sage Weil <sage@newdream.net>
2020-05-27 10:44:31 -05:00
Sage Weil
a92fa51b9b ceph-dedup-tool: --add-chunk-ref -> --chunk-get-ref (and -put-ref)
Signed-off-by: Sage Weil <sage@newdream.net>
2020-05-27 10:44:31 -05:00
Kefu Chai
1b72cb4b6a
Merge pull request #35033 from badone/wip-test-the-right-rocksdb-version
qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-27 21:58:16 +08:00
Sage Weil
c828e5d77c common/rabin: remove
The implementation is buggy, and slower than FastCDC.

Signed-off-by: Sage Weil <sage@newdream.net>
2020-05-27 07:48:18 -05:00
Brad Hubbard
9d2632a953 qa/workunits/rados/test_envlibrados_rocksdb: Remove Xenial cmake3 requirements
It's no longer necessary to handle Xenial as a special case.

Fixes: https://tracker.ceph.com/issues/45561

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-05-15 16:50:00 +10:00
Brad Hubbard
1361913f59 qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb
We should be building the version of rocksdb the release is pinned to,
not master. Let's just update the rocksdb submodule and clone that.

Fixes: https://tracker.ceph.com/issues/44981

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-05-13 12:45:31 +10:00
Sage Weil
de20c7bc61 Merge PR #34105 into master
* refs/pull/34105/head:
	Merge PR #34042 into octopus
	Merge PR #33959 into octopus
	Merge PR #34067 into octopus
	mgr/DaemonServer: add explicit check that acting matches for merge
	Merge pull request #34040 from dillaman/wip-44396-partial-fix
	Merge PR #34098 into octopus
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	Merge PR #34082 into octopus
	Merge PR #34068 into octopus
	cephadm: relabel /etc/ganesha mount
	Merge PR #34046 into octopus
	Merge PR #34092 into octopus
	Merge pull request #33719 from ukernel/wip-44416
	rbd-mirror: leader watcher should not cancel get locker if locker is invalid
	rbd-mirror: snapshot sync request needs to check for interruption
	librbd: request exclusive lock when moving to trash
	rbd-mirror: basic integration with sync throttling
	rbd-mirror: don't prematurely finish snapshot replay loop
	rbd-mirror: pass InstanceWatcher to snapshot Replayer
	doc/releases/octopus.rst: add note about ec recovery below min_size
	mgr/cephadm: configure rgw_frontends for rgw service
	cephadm: switch grafana image to the ceph repo
	Merge PR #34034 into octopus
	qa/suites/rados/cephadm/upgrade: update starting version
	Merge PR #33540 into octopus
	Merge PR #34023 into octopus
	Merge PR #34044 into octopus
	Merge PR #34030 into octopus
	doc/orchestrator: update rgw creation
	mgr/cephadm: clean up client.crash.* container_image settings after upgrade
	cephadm: make add-repo --release and --version independent
	cephadm: env over last used
	mgr/orch: accept port and ssl flags to 'apply rgw'
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	remove ceph_test_rados_watch_notify
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors
	mgr/rook: fix 'orch ps' for osds
	qa: fix all the fsx.sh-invoking yaml files to install dependencies
	mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-23 08:24:06 -05:00
Sage Weil
282932867a remove ceph_test_rados_watch_notify
This is an old test, we have good watch/notify coverage in the newer
tests, and it is buggy.

Fixes: https://tracker.ceph.com/issues/43861
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 17:45:17 -05:00
Kefu Chai
78991864f6 tools/rados: use object-locator in user-visible outputs
we normalize object-locator to object_locator when parsing command line
options. but object-locator is more consistent with other options
suppored by "rados" cli, and "-" is easier to type than "_". it's also
more widely used in command line options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-12 11:19:47 +08:00
Deepika Upadhyay
21508bd9dd mon/OSDMonitor: add flag --yes-i-really-mean-it for setting pool size 1
Adds option `mon_allow_pool_size_one` which will be disabled by default
to ensure pools are not configured without replicas.
If the user still wants to use pool size 1, they will have to change the
value of `mon_allow_pool_size_one` to true and then have to pass flag
`--yes-i-really-mean-it` to cli command:

Example:
`ceph osd pool test set size 1 --yes-i-really-mean-it`

Fixes: https://tracker.ceph.com/issues/44025
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-03-09 23:27:36 +05:30
Kefu Chai
f05696ebeb qa: enable CRB repo for RHEL8
instead of enabling PowerTools repo, we need to enable CodeReady Builder
repo for RHEL8

also, since we are moving to RHEL8, there is no need to install cmake3
specifically for CentOS. CentOS 8 comes with cmake3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-06 23:41:23 +08:00
Sage Weil
f70e206e13 qa/workunits/rados/test_large_omap_detection: py3-ify
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 13:55:08 -06:00
Sage Weil
4a566bdc2d Merge PR #32362 into master
* refs/pull/32362/head:
	qa/packages/packages: el8 has granular -debuginfo
	qa/tasks/cbt: include py2 deps on ubuntu for now
	src/test: misc python -> python3
	qa/suites/rados/singleton-flag/valgrind-leaks: run on latest centos
	qa/workunits: env python -> env python3
	qa/suites/rados/dashboard/tasks/dashboard: whitelist OSDMAP_FLAGS
	qa/suites/rados/verify: ping to specific centos
	qa/workunits/rados/test_envlibrados_rocksdb: enable el8 PowerTools
	qa/workunits/rados/test_pool_access.sh: python -> python3
	mgr/crash: fix signature for py3

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-12-20 09:29:47 -06:00
Sage Weil
481a09f6b9 qa/workunits/rados/test_envlibrados_rocksdb: enable el8 PowerTools
We need it for snappy-devel

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 07:17:10 -06:00
Sage Weil
bfbad54b55 qa/workunits/rados/test_pool_access.sh: python -> python3
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 07:17:10 -06:00
Sage Weil
253a557f10 qa/workunits: python -> python3 throughout
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 12:38:47 -06:00
Sage Weil
240154f114 qa: use python3 for venvs etc
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-13 08:47:28 -06:00
Kefu Chai
5a3d093545 qa/workunits/rados/test_envlibrados_for_rocksdb: accomodate rocksdb cmake changes
since
bac38c992a,

tests are not enabled for non-debug builds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-10 14:58:00 +08:00
Kefu Chai
8384a7fd8c
Merge pull request #31693 from tchaikov/wip-42782
qa/workunits/rados/test_librados_build.sh: download from current branch

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-25 00:56:09 +08:00
Sage Weil
5b809cd330 qa/workunits/rados/test_python: make sure rbd pool exists
The ceph2 task doesn't create it for you.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-21 10:46:54 -06:00