Commit Graph

178 Commits

Author SHA1 Message Date
Samuel Just
4a73fd2251
Merge pull request #35899 from myoungwon/wip-fix-manifest-tier
osd, test: refactoring manfiest-tier

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-09-01 12:49:57 -07:00
myoungwon oh
7217a6c00e osd, test: remove all things related to manifest_flush
Due to a plan to re-work based on CDC-based flush,
remove manifest_flush()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsumg.com>
2020-08-26 21:05:14 +09:00
Kefu Chai
af93f56203 qa/workunits/rados: quote optional args
otherwise they will be escaped by shell

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-22 16:11:36 +08:00
Kefu Chai
6ad87dbbdc qa/workunits/rados: pass options down to test_rados.py
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-21 16:41:31 +08:00
Sage Weil
84f9603d8d ceph-dedup-tool: get-chunk-ref -> dump-chunk-refs
Signed-off-by: Sage Weil <sage@newdream.net>
2020-05-27 10:44:31 -05:00
Sage Weil
a92fa51b9b ceph-dedup-tool: --add-chunk-ref -> --chunk-get-ref (and -put-ref)
Signed-off-by: Sage Weil <sage@newdream.net>
2020-05-27 10:44:31 -05:00
Kefu Chai
1b72cb4b6a
Merge pull request #35033 from badone/wip-test-the-right-rocksdb-version
qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-27 21:58:16 +08:00
Sage Weil
c828e5d77c common/rabin: remove
The implementation is buggy, and slower than FastCDC.

Signed-off-by: Sage Weil <sage@newdream.net>
2020-05-27 07:48:18 -05:00
Brad Hubbard
9d2632a953 qa/workunits/rados/test_envlibrados_rocksdb: Remove Xenial cmake3 requirements
It's no longer necessary to handle Xenial as a special case.

Fixes: https://tracker.ceph.com/issues/45561

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-05-15 16:50:00 +10:00
Brad Hubbard
1361913f59 qa/workunits/rados/test_envlibrados_rocksdb: Build correct rocksdb
We should be building the version of rocksdb the release is pinned to,
not master. Let's just update the rocksdb submodule and clone that.

Fixes: https://tracker.ceph.com/issues/44981

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-05-13 12:45:31 +10:00
Sage Weil
de20c7bc61 Merge PR #34105 into master
* refs/pull/34105/head:
	Merge PR #34042 into octopus
	Merge PR #33959 into octopus
	Merge PR #34067 into octopus
	mgr/DaemonServer: add explicit check that acting matches for merge
	Merge pull request #34040 from dillaman/wip-44396-partial-fix
	Merge PR #34098 into octopus
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	Merge PR #34082 into octopus
	Merge PR #34068 into octopus
	cephadm: relabel /etc/ganesha mount
	Merge PR #34046 into octopus
	Merge PR #34092 into octopus
	Merge pull request #33719 from ukernel/wip-44416
	rbd-mirror: leader watcher should not cancel get locker if locker is invalid
	rbd-mirror: snapshot sync request needs to check for interruption
	librbd: request exclusive lock when moving to trash
	rbd-mirror: basic integration with sync throttling
	rbd-mirror: don't prematurely finish snapshot replay loop
	rbd-mirror: pass InstanceWatcher to snapshot Replayer
	doc/releases/octopus.rst: add note about ec recovery below min_size
	mgr/cephadm: configure rgw_frontends for rgw service
	cephadm: switch grafana image to the ceph repo
	Merge PR #34034 into octopus
	qa/suites/rados/cephadm/upgrade: update starting version
	Merge PR #33540 into octopus
	Merge PR #34023 into octopus
	Merge PR #34044 into octopus
	Merge PR #34030 into octopus
	doc/orchestrator: update rgw creation
	mgr/cephadm: clean up client.crash.* container_image settings after upgrade
	cephadm: make add-repo --release and --version independent
	cephadm: env over last used
	mgr/orch: accept port and ssl flags to 'apply rgw'
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	remove ceph_test_rados_watch_notify
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors
	mgr/rook: fix 'orch ps' for osds
	qa: fix all the fsx.sh-invoking yaml files to install dependencies
	mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-23 08:24:06 -05:00
Sage Weil
282932867a remove ceph_test_rados_watch_notify
This is an old test, we have good watch/notify coverage in the newer
tests, and it is buggy.

Fixes: https://tracker.ceph.com/issues/43861
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-18 17:45:17 -05:00
Kefu Chai
78991864f6 tools/rados: use object-locator in user-visible outputs
we normalize object-locator to object_locator when parsing command line
options. but object-locator is more consistent with other options
suppored by "rados" cli, and "-" is easier to type than "_". it's also
more widely used in command line options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-12 11:19:47 +08:00
Deepika Upadhyay
21508bd9dd mon/OSDMonitor: add flag --yes-i-really-mean-it for setting pool size 1
Adds option `mon_allow_pool_size_one` which will be disabled by default
to ensure pools are not configured without replicas.
If the user still wants to use pool size 1, they will have to change the
value of `mon_allow_pool_size_one` to true and then have to pass flag
`--yes-i-really-mean-it` to cli command:

Example:
`ceph osd pool test set size 1 --yes-i-really-mean-it`

Fixes: https://tracker.ceph.com/issues/44025
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-03-09 23:27:36 +05:30
Kefu Chai
f05696ebeb qa: enable CRB repo for RHEL8
instead of enabling PowerTools repo, we need to enable CodeReady Builder
repo for RHEL8

also, since we are moving to RHEL8, there is no need to install cmake3
specifically for CentOS. CentOS 8 comes with cmake3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-06 23:41:23 +08:00
Sage Weil
f70e206e13 qa/workunits/rados/test_large_omap_detection: py3-ify
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 13:55:08 -06:00
Sage Weil
4a566bdc2d Merge PR #32362 into master
* refs/pull/32362/head:
	qa/packages/packages: el8 has granular -debuginfo
	qa/tasks/cbt: include py2 deps on ubuntu for now
	src/test: misc python -> python3
	qa/suites/rados/singleton-flag/valgrind-leaks: run on latest centos
	qa/workunits: env python -> env python3
	qa/suites/rados/dashboard/tasks/dashboard: whitelist OSDMAP_FLAGS
	qa/suites/rados/verify: ping to specific centos
	qa/workunits/rados/test_envlibrados_rocksdb: enable el8 PowerTools
	qa/workunits/rados/test_pool_access.sh: python -> python3
	mgr/crash: fix signature for py3

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-12-20 09:29:47 -06:00
Sage Weil
481a09f6b9 qa/workunits/rados/test_envlibrados_rocksdb: enable el8 PowerTools
We need it for snappy-devel

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 07:17:10 -06:00
Sage Weil
bfbad54b55 qa/workunits/rados/test_pool_access.sh: python -> python3
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-20 07:17:10 -06:00
Sage Weil
253a557f10 qa/workunits: python -> python3 throughout
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 12:38:47 -06:00
Sage Weil
240154f114 qa: use python3 for venvs etc
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-13 08:47:28 -06:00
Kefu Chai
5a3d093545 qa/workunits/rados/test_envlibrados_for_rocksdb: accomodate rocksdb cmake changes
since
bac38c992a,

tests are not enabled for non-debug builds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-10 14:58:00 +08:00
Kefu Chai
8384a7fd8c
Merge pull request #31693 from tchaikov/wip-42782
qa/workunits/rados/test_librados_build.sh: download from current branch

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-25 00:56:09 +08:00
Sage Weil
5b809cd330 qa/workunits/rados/test_python: make sure rbd pool exists
The ceph2 task doesn't create it for you.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-21 10:46:54 -06:00
Kefu Chai
ad194519d4 qa/workunits/rados/test_librados_build.sh: use branch name printed by librados-config
Fixes: https://tracker.ceph.com/issues/42782
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 01:29:52 +08:00
Yuval Lifshitz
b8b43fe59d allow for multiple "build" dirs to exist
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-09-06 09:38:32 +03:00
Kefu Chai
52277bfa52 qa/workunits/rados/test_envlibrados_for_rocksdb: install newer cmake
rocksdb needs CMake 3.5.1 since
https://github.com/facebook/rocksdb/pull/5662

Fixes: https://tracker.ceph.com/issues/41253
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-18 01:32:25 +08:00
Sage Weil
0c0093e450 Merge PR #29034 into master
* refs/pull/29034/head:
	doc/mgr/crash: document missing commands, options
	qa/suites/rados/singleton/all/test-crash: whitelist RECENT_CRASH
	qa/suites/rados/mgr/tasks/insights: whitelist RECENT_CRASH
	qa/tasks/mgr/test_insights: crash module now rejects bad crash reports
	mgr/telemetry: fix remote into crash do_ls()
	mgr/crash: don't make these methods static
	mgr/BaseMgrModule: handle unicode health detail strings
	mgr/crash: verify timestamp is valid
	qa/suites/mgr: whitelist RECENT_CRASH
	mgr/crash: remove unused var
	mgr/crash: remove unused import 'six'
	qa/workunits/rados/test_crash: health check
	mgr/crash: improve validation on post
	mgr/crash: automatically prune old crashes after a year
	mgr/crash: raise RECENT_CRASH warning for recent (new) crashes
	mgr/crash: add 'crash ls-new'
	mgr/crash: add option and serve infra
	mgr/crash: keep copy of crashes in memory
	mgr/pg_autoscaler: adjust style to match built-in tables
	mgr/crash: make 'crash ls' a nice table with a NEW column
	mgr/crash: nicely format 'crash info' output
	mgr/crash: add 'crash archive <id>', 'crash archive-all' commands

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-24 17:20:21 -05:00
Kefu Chai
1a379bc202
Merge pull request #28765 from myoungwon/fix-chunk-scrub-dedup-tool
src/tools/ceph-dedup-tool: Fix chunk scrub

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-24 09:59:58 +08:00
Sage Weil
f6e0894902 qa/workunits/rados/test_crash: health check
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-19 14:45:47 -05:00
Sage Weil
c691f2f876 mgr/crash: make 'crash ls' a nice table with a NEW column
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-19 09:43:04 -05:00
Sage Weil
09b7eddad6 qa/workunits/rados/test_cache_pool.sh: s/forward/proxy/
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-10 17:39:19 -05:00
myoungwon oh
c2c29293cd src/tools: fix chunk_scrub
This commit makes chunk_scrub as a chunk_pool-based operation.
chunk_scrub procedure:
1. find references in the chunk_pool
2. check whether the reference exists in a base_pool
3. fix references

Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2019-06-27 22:39:06 +09:00
Kefu Chai
b76a6c729f qa/workunits/rados/test_envlibrados_for_rocksdb: support SUSE distros
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 17:46:46 +08:00
Kefu Chai
be89e66f74 qa/workunits/rados/test_librados_build.sh: install build deps
to build helloworld of librados, we need to install the build
dependencies.

Fixes: https://tracker.ceph.com/issues/40155
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 17:46:46 +08:00
Kefu Chai
0072d487aa qa/workunits/rados/test_envlibrados_for_rocksdb: use helper script
instead of having its own copy of `install()`, try to leverage the
helper in ceph-helpers-root.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 17:46:46 +08:00
Nathan Cutler
f9f824448a qa: add version number sanity singleton to rados suite
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-05-08 11:31:34 +02:00
myoungwon oh
8a8d5e89bb src/test: remove unused objects to calculate accurate dedup ratio.
Signed-off-by: Myoungwon Oh <ommw@sk.com>
2019-04-24 18:22:37 +09:00
myoungwon oh
d72cc5a0ee src/tools: fix dedup tool's name
Signed-off-by: Myoungwon Oh <ommw@sk.com>
2019-04-08 19:19:09 +09:00
myoungwon oh
a095765f2f qa/workunit: add rabin chunk test
Signed-off-by: Myoungwon Oh <ommw@sk.com>
2019-04-04 23:58:57 +09:00
Jason Dillaman
9683869442 Revert "qa: update test_envlibrados_for_rocksdb.sh for libradospp split"
This reverts commit 0475dfdea3.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-21 08:28:43 -05:00
myoungwon oh
e11fb7ce4d src/tools: add stats (fixed objects,total objects)
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2019-01-26 12:11:51 +09:00
Vikhyat Umrao
8a694fc2f9 qa: specify filestore for misc tests
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 13:09:19 -06:00
Sage Weil
d8d16ba6aa qa/workunits/rados/test_health_warnings: prevent out osds
Fixes: http://tracker.ceph.com/issues/37776
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-31 11:18:05 -06:00
Igor Fedotov
25664c4ccf test: add test coverage for rados df reports
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Nathan Cutler
6c6ae9e30d qa/workunits/rados/test_librados_build.sh: grab files from explicit git branch
Fixes: https://tracker.ceph.com/issues/37392
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-27 00:34:17 +01:00
Kefu Chai
60dc1c2a32 qa: patch rocksdb the right way
quote from patch(1)

>       but usually just
>
>       patch -pnum <patchfile

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 13:03:00 +08:00
Kefu Chai
2dc0f86227 qa: use FOUND_VAR to be backward compatible with cmake 2.8.12
before this change, we assume that the variable set if rados::radospp is
found will be radospp_FOUND, but this is a feature cmake 3, see
https://cmake.org/cmake/help/v3.3/module/FindPackageHandleStandardArgs.html

while the cmake shipped by centos is cmake 2.8.12, where the variable
name will be <UPPERCASED_NAME>_FOUND, see
https://cmake.org/cmake/help/v2.8.12/cmake.html#module:FindPackageHandleStandardArgs

in the test of test_envlibrados_for_rocksdb.sh, we are using cmake not
the cmake3 offered by EPEL7, so RADOSPP_FOUND will be set instead. that's why
executable env_librados_test will fail to link against rados::radospp.
as rados::radospp won't be defined if radospp_FOUND is not defined/set.

after this change, the 2nd mode of FIND_PACKAGE_HANDLE_STANDARD_ARGS()
is used instead to ensure that radospp_FOUND is defined even if cmake
2.8.12 is used.

also, the message() commands for debugging purpose are removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:57:52 +08:00
Kefu Chai
0e5bdd9d56 librados: remove rados_tmap_*
we have switched from tmap to omap long ago.

but keep the server side implementation around, in case ancient
client is still using these tmap APIs.

also, tmap_update() is kept, because librbd is using it for v1 image
backward compatibility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-07 14:05:25 -08:00
Kefu Chai
0475dfdea3 qa: update test_envlibrados_for_rocksdb.sh for libradospp split
include a patch so rocksdb can use libradospp instead of librados. will
upstream the patch and make it work for both pre-nautilus librados and
nautilus libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00