Commit Graph

4275 Commits

Author SHA1 Message Date
Sage Weil
ba49aacee2 Merge PR #25111 into master
* refs/pull/25111/head:
	test: Add test for requested scrub priority
	osd: Prioritize user specified scrubs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-19 13:15:21 -06:00
Sage Weil
d69e8d8de8 Merge PR #14092 into master
* refs/pull/14092/head:
	mgr/DaemonServer: fix session leak
	mon/MonClient: ignore new mon commands while stopping
	mgr/DeviceState: fix DeviceState initial refcount
	qa/suites: valgrind ceph-mgr too

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-16 07:11:44 -06:00
Lenz Grimmer
34a5ac0b19
Merge pull request #25084 from s0nea/wip-dashboard-add-missing-test-suites
mgr/dashboard/qa: add missing dashboard suites

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2018-11-16 11:16:42 +01:00
Sage Weil
13d1f62538 Merge PR #24973 into master
* refs/pull/24973/head:
	mgr: Separate diskprediction cloud plugin from the diskprediction plugin
2018-11-16 00:23:14 -06:00
Sage Weil
d941b28089 Merge PR #19987 into master
* refs/pull/19987/head:
	qa/tasks/ceph: ignore failure when chown ceph:ceph

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-16 00:22:17 -06:00
hsiang41
9799eb67eb mgr: Separate diskprediction cloud plugin from the diskprediction plugin
Separate diskprediction local cloud from the diskprediction plugin.
Devicehealth invoke device prediction function related on the global
configuration "device_failure_prediction_mode".

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
2018-11-16 00:15:41 -06:00
Tatjana Dehler
ef62c5eb4e mgr/dashboard/qa: add missing dashboard suites
Fixes: https://tracker.ceph.com/issues/37152
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-11-15 11:45:56 +01:00
David Zafman
1841928e28 test: Add test for requested scrub priority
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-14 23:57:20 -08:00
Kefu Chai
a04691031c
Merge pull request #24858 from jecluis/wip-pr-19983-tests
qa/cephtool: test bounds on pool's `hit_set_*`

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-15 14:12:09 +08:00
Kefu Chai
60dc1c2a32 qa: patch rocksdb the right way
quote from patch(1)

>       but usually just
>
>       patch -pnum <patchfile

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 13:03:00 +08:00
Kefu Chai
2dc0f86227 qa: use FOUND_VAR to be backward compatible with cmake 2.8.12
before this change, we assume that the variable set if rados::radospp is
found will be radospp_FOUND, but this is a feature cmake 3, see
https://cmake.org/cmake/help/v3.3/module/FindPackageHandleStandardArgs.html

while the cmake shipped by centos is cmake 2.8.12, where the variable
name will be <UPPERCASED_NAME>_FOUND, see
https://cmake.org/cmake/help/v2.8.12/cmake.html#module:FindPackageHandleStandardArgs

in the test of test_envlibrados_for_rocksdb.sh, we are using cmake not
the cmake3 offered by EPEL7, so RADOSPP_FOUND will be set instead. that's why
executable env_librados_test will fail to link against rados::radospp.
as rados::radospp won't be defined if radospp_FOUND is not defined/set.

after this change, the 2nd mode of FIND_PACKAGE_HANDLE_STANDARD_ARGS()
is used instead to ensure that radospp_FOUND is defined even if cmake
2.8.12 is used.

also, the message() commands for debugging purpose are removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:57:52 +08:00
Kefu Chai
ae36a61446 qa/suites: add librados2 to "extra_packages" for upgrade tests
we use the playbook of "testnodes.yml" defined by ceph-cm-ansible for
initializing test nodes, and the role of "testnode" is used by
testnodes.yml. "testnode" requires "qemu-system-x86" or "qemu-kvm"
package to be installed. the qemu in turn depends on librbd1 and
librados2.

before librados3 was introduced, this worked perfectly. because in ceph
repo, qa/packages/packages.yaml defines the default set of packages the
"install" tasks should install. and in that yaml file, librados2 was
listed. so the package management system will overwrite the librados2
installed by ansible playbook with the version specified by the
"install" task, as apt/yum thinks this is what user requires explicitly,
so it's fine to install a different version of librados2.

after librados3 was introduced, librados2 was removed from
qa/packages/packages.yaml. because, by default, we need to install
librados3 instead of librados2 for ready a nautilus cluster. but the
problem is, the packge list also applies to "install" tasks installing
releases before nautilus, where we still need to replace the librados2
installed by ansible.

so, to address this issue, "librados2" is added to "extra_packages" of
the "install" tasks of tests installing old releases to install
librados2 explicitly instead of as a dependency of other ceph packages
like librbd1.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:56:24 +08:00
Nathan Cutler
0185abfeb4 tests: make ceph-admin-commands.sh log what it does
Fixes: http://tracker.ceph.com/issues/37089
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-13 19:46:24 +01:00
Kefu Chai
c89e0715cb
Merge pull request #25037 from tchaikov/wip-qa-upgrade
qa: add librados3 to exclude_packages for ugprade tests

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-13 06:38:21 +08:00
Yuri Weinstein
5127d0166e qa/tests: added "-n 7" to make sure mimic-x runs on built master branch
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-10 10:18:49 -08:00
Kefu Chai
f10fc004bc qa: add librados3 to exclude_packages for ugprade tests
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-10 07:17:24 -08:00
Jason Dillaman
6ac41ccbaf
Merge pull request #24214 from iridescent-rsy/bursting_io
librbd: reduce the TokenBucket fill cycle and support bursting io configuration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 19:53:55 -05:00
Josh Durgin
fd2a4c5733
Merge pull request #22476 from dzafman/wip-23875
Removal of snapshot with corrupt replica crashes osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-09 15:15:01 -08:00
Lenz Grimmer
5b0a957961
Merge pull request #24627 from Devp00l/wip-manage-ec-profiles
mgr/dashboard: Adds ECP management to the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 16:29:43 +01:00
Sage Weil
03908113b4 qa/suites: valgrind ceph-mgr too
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-09 08:52:07 -06:00
Lenz Grimmer
3ba874004e
Merge pull request #24489 from ricardoasmarques/wip-saml2
mgr/dashboard: SSO - SAML 2.0 support

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 13:11:22 +01:00
Stephan Müller
3a7f85809a mgr/dashboard: Adds ECP info endpoint
The new info endpoint will provide the frontend with the necessary
information it needs to create new profiles.

Fixes: https://tracker.ceph.com/issues/25156
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-11-09 09:40:39 +01:00
David Zafman
a159f162c5 test: osd-scrub-snaps.sh: After snapshot removal wait for snaptrim to complete
Due to deliberate corruptions snaptrim_error means snaptrim is done

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:20 -08:00
David Zafman
e37f95ac27 test: osd-scrub-snaps.sh: Testing with new --rmtype in ceph-objectstore-tool
Use --rmtype snapmap with new obj16 to remove snapmap only, check for repair message
Use --rmtype nosnapmap to remove obj5 while leaving snapmap behind

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:20 -08:00
David Zafman
f43faf4ad7 test: cleanup: Remove redundant cat of log and handle errors in create_scenario()
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:19 -08:00
Sage Weil
c8a8dc21fd Merge PR #24828 into master
* refs/pull/24828/head:
	qa/osd-bluefs-volume-ops: use ceph-bluestore-tool for fsck
	qa/osd-bluefs-volume-ops: reduce space usage for the test case

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-11-08 16:26:52 -06:00
Kefu Chai
0966f43d98
Merge pull request #24896 from tchaikov/wip-librados-cleanup
librados,rpm,deb: various fixes to address librados3 transition and cleanups in librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-08 10:19:57 -08:00
Ricardo Marques
04f4d5053e mgr/dashboard: SAML 2.0 support
Fixes: https://tracker.ceph.com/issues/24268

Signed-off-by: Ricardo Dias <rdias@suse.com>
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-11-08 15:27:37 +00:00
Lenz Grimmer
9a0a06592a
Merge pull request #24947 from votdev/bug_36708
mgr/dashboard: tasks.mgr.dashboard.test_osd.OsdTest failures

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-11-08 14:41:38 +01:00
Kefu Chai
0e1ec8dc20 qa: install libradospp-dev for librados_hello_world.yaml
libradospp-{dev,devel} is necessary for compiling sources in
examples/librados/hello_world.cc

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-07 14:13:48 -08:00
David Zafman
02964703de
Merge pull request #24749 from dzafman/wip-36474
Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-07 14:05:55 -08:00
Kefu Chai
0e5bdd9d56 librados: remove rados_tmap_*
we have switched from tmap to omap long ago.

but keep the server side implementation around, in case ancient
client is still using these tmap APIs.

also, tmap_update() is kept, because librbd is using it for v1 image
backward compatibility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-07 14:05:25 -08:00
Volker Theile
e6f130d470 mgr/dashboard: backend api tests: tasks.mgr.dashboard.test_osd.OsdTest failures
- Fix bug in Dashboard QA unit test framework. Don't set the application type header manually, this is done by the requests library if required.
- Enhance QA unit test helper: Print the response of the API request if it fails. This should help to identify the problem more easily.
- Fix bug in the OSD controller. A parameter needs to be converted to integer.
- Take care that the params of the request object are not modified.

The issue was introduced by PR https://github.com/ceph/ceph/pull/24475. The CherryPy json_in plugin disclosed the errorneous unit test helper implementation.

Fixes: https://tracker.ceph.com/issues/36708

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-11-07 17:36:53 +01:00
Shiyang Ruan
fd613b1c32 librbd: make the minimum of schedule tick for QoS configurable.
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
2018-11-07 13:28:56 +08:00
Shiyang Ruan
761393a955 librbd: support burst limit configuration.
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
2018-11-07 13:25:53 +08:00
Sage Weil
5b9be42bf5 Merge PR #15047 into master
* refs/pull/15047/head:
	tool/ceph_objectstore_tool: add new op that reset last_complete to last_update

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-06 10:47:18 -06:00
Jason Dillaman
cc592823fa
Merge pull request #24717 from trociny/wip-rbd-mirror-status-instance
rbd: show info about mirror daemon instance in image mirror status output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-06 11:11:49 -05:00
Mykola Golub
7d2ffd981b rbd: show info about mirror instance in image mirror status output
It is particularly useful when running multiple rbd-mirror instances
in Active-Passive or Active-Active mode.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-11-05 22:34:39 +02:00
Mykola Golub
472abc2ed9 test/librbd: migration supporting namespace tests
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-11-03 19:36:44 +02:00
Casey Bodley
cf9a0a28ea
Merge pull request #24895 from votdev/feature_36681
rgw: Return tenant field in bucket_stats function

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-02 16:28:26 -04:00
Yuri Weinstein
a24793ed6c
Merge pull request #22923 from ceph/wip-fix-latest-distro
qa/tests: update links for centos latest to point to 7.5

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-02 12:30:41 -07:00
Volker Theile
9440776f06 rgw: Return tenant field in bucket_stats function
Fixes: https://tracker.ceph.com/issues/36681

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-11-02 13:26:10 +01:00
Vasu Kulkarni
67bfc94f17 qa/tests: update links for centos latest to point to 7.5
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-11-01 11:09:12 -07:00
Kefu Chai
b4a5f76243 qa: add librados3 to exclude list of pre-nautilus install tasks
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
083b846198 qa: s/librados2/librados3/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
0475dfdea3 qa: update test_envlibrados_for_rocksdb.sh for libradospp split
include a patch so rocksdb can use libradospp instead of librados. will
upstream the patch and make it work for both pre-nautilus librados and
nautilus libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
3730d10623 librados: move C++ APIs into libradospp
the goal is to decouple C++ API from C API, and to version them
differently, as they are targeting different consumers.

this allows us to change the C++ API and bumping up its soversion
without requiring consumer to recompile the librados client for
using the new librados. in this way, C++ API can move faster than
C API. for example, if bufferlist interface is changed for better
performance, and this breaks existing API/ABI, we can bump up
the C++ library's soversion, and and the C library's version unchanged
but ship the new librados's C binding. so the librados client linked
against librados's C library will be able to take advantage of
the improvement in C++ library. while the librados client
linked against C++ library won't break at runtime due to unresolved
symbol or changed structure layout.

this is massive change, the genereal idea is to

* split librados.cc into two source files: librados_c.cc and
  librados_cxx.cc, the former for implementing C APIs, the later
  for C++ APIs.
* extract the C++ API in librados into librados-cxx, the library
  name will be libradospp. but we can change it before nautilus
  is released.
* link these librados libraries with static libraries which it
  depends on, so "-Wl,--exclude-libs,ALL" link flags can help
  hide the non-public symbols.
* extract the tests exercising librados' C++ API into a different
  source file named *_cxx.cc. for instance, to move the C++ tests
  in aio.cc into aio_cxx.cc
* extract the shared helper functions which do not use any librados
  or librados-cxx APIs into test_shared{.cc,h}. the "shared" here
  means, *shared* by C++ and C tests.
* extract the test fixtures, i.e., the subclasses of testing::Test,
  for testing C++ APIs into testcase_cxx.cc.
* update qa/workunits/rados/test.sh accordingly to add the splitted
  tests
* update the consumers of librados to link against librados-cxx
  instead, if they are using the C++ API.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:30 +08:00
Joao Eduardo Luis
256ced3586 qa/cephtool: test bounds on pool's hit_set_*
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-10-31 16:05:40 +00:00
Sage Weil
9ab9dcfc0d Merge PR #24809 into master
* refs/pull/24809/head:
	os/bluestore: omit redundant '/' in OSD path for ceph-bluestore-tool if
	os/bluestore: improve error handling for migrate ops in
	qa/standtalone/osd-bluefs-volume-ops: remove redundant code.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-30 15:09:45 -05:00
David Zafman
3f621a1190 test: Set any value for osd_delete_sleep to guarantee we are testing even SSD
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-10-30 11:42:05 -07:00