Commit Graph

1018 Commits

Author SHA1 Message Date
Sage Weil
6e1598bf22 Merge PR #26485 into master
* refs/pull/26485/head:
	qa/suites/upgrade/luminous-x: force clone v1 format for final rbd python test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-19 08:30:41 -06:00
Patrick Nawracay
cfbefbf169 mgr/dashboard: Add support for managing RBD QoS
Fixes: http://tracker.ceph.com/issues/36191

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-02-19 11:08:59 +01:00
Mykola Golub
04cc884df0 qa/suites/upgrade/luminous-x: force clone v1 format for final rbd python test
Fixes: https://tracker.ceph.com/issues/38346
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-02-18 16:23:13 +00:00
Sage Weil
a60f332c32 Merge PR #24805 into master
* refs/pull/24805/head:
	qa/suite: add dedup test
	src/tools: fix compile error (master version issue)
	src/tools: add stats (fixed objects,total objects)
	src/tools: make room for cdc
	src/tools: make enhacned stats and interface class
	src/tools: set timelimit and add signal handler to check progress
	src/tools: use the slice thing and make parallel (chunk_scrub)
	src/test: add max-thread test in test_dedup_tool.sh
	src/tools: use the slice thing and make parallel
	src/test: add chunk-scrub test in test_dedup_tool.sh
	src/tools: add chunk-scrub op in dedup tool
	src/cls/cas: add has_chunk op
	src/test: add test_dedup_tool.sh
	src/tools: initial works for dedup tool

Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-17 08:18:06 -06:00
Sage Weil
19be155c3e qa/suites/upgrade/mimic-x/stress-split: drop pglog_hardlimit test
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-15 16:06:52 -06:00
Sage Weil
20beafab36 qa/suites/upgrade/mimic-x/stress-split: update for msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-15 16:06:52 -06:00
Sage Weil
6207c0b3f4 qa/suites/upgrade/mimic-x/parallel: update for msgr v2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-15 16:06:52 -06:00
Kefu Chai
9e561e9fb4
Merge pull request #26380 from tchaikov/wip-split-dashboard-package
deb,rpm,qa: split dashboard package

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-14 16:59:58 +08:00
Lenz Grimmer
c1f6e0e867
Merge pull request #25989 from votdev/feature_36488
mgr/dashboard: Add UI to configure the telemetry mgr plugin

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-02-14 09:43:15 +01:00
Sage Weil
b20862c846 qa/suites/upgrade/luminous-x/parallel: enable all classes
Otherwise it's annoying because the class list changes between luminous and nautilus,
and we don't want to futz around with changing this setting during the upgrade.

The problematic classes are 'cas' (added) and 'sdk' (not enabled by default but
included by the cls/ workunit.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 14:35:18 -06:00
Sage Weil
525f3f7e34 qa/suites/upgrade/luminous-x/parallel/5-final-workload/rados_mon_thrash: use x branch
The luminous version is (1) not what we want and (2) will fail because
ceph_test_rados_api_tier no longer exists in master.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 14:35:18 -06:00
Sage Weil
28546e065d qa/suites/upgade/luminous-x: pglog_hardlimit succeeds now on luminous due to backport
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 11:41:21 -06:00
Sage Weil
5947146423 qa/suites/upgrade/luminous-x/stress-split: mons on separate hosts, enable msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 11:41:21 -06:00
Sage Weil
d260ba9f84 qa/suites/upgrade/luminous-x/parallel: mon per host, msgr2
- mon per host
- enable msgr2 at the end
- include the nautilus.yaml

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 11:41:21 -06:00
Kefu Chai
3d128d040e qa: exclude new packages when installing old versions
ceph-mgr-dashboard is introduced in nautilus, so exclude it when
installing old versions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-13 21:37:30 +08:00
Sage Weil
d93abd8c53 qa/suites/upgrade/luminous-x: whitelist 'slow request'
The luminous warnings say slow request.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 07:20:51 -06:00
Sage Weil
96f58cd548 Merge PR #26265 into master
* refs/pull/26265/head:
	qa/suites/rados/upgrade: whitelist 'slow request'

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-13 06:40:41 -06:00
Volker Theile
b9177e43c0 mgr/dashboard: Add UI to configure the telemetry mgr plugin
Fixes: tracker.ceph.com/issues/36488

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-02-12 16:36:56 +01:00
Sage Weil
059844f9b9 Merge PR #26336 into master
* refs/pull/26336/head:
	qa/tasks/keystone.py: no need for notcmalloc in example
	qa/suites/rgw/tempest/tasks/rgw_tempest: no need for notcmalloc

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2019-02-11 15:47:29 -06:00
Ilya Dryomov
400686efd9
Merge pull request #26339 from idryomov/wip-krbd-namespaces-test
qa: krbd namespaces test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-11 16:00:32 +01:00
Sage Weil
b4fa47390d Merge PR #25977 into master
* refs/pull/25977/head:
	qa/suites: exclude new packages when installing old versions
	rpm: add dependency on python-kubernetes module to ceph-mgr-rook package
	rpm,deb: add rbd_support module to ceph-mgr
	packaging: split ceph-mgr diskprediction and rook plugins into own packages

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-09 13:46:51 -06:00
myoungwon oh
89f41ad9ba qa/suite: add dedup test
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2019-02-09 12:45:10 +09:00
Sage Weil
5cbdf32430 qa/suites: exclude new packages when installing old versions
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-08 12:59:22 -06:00
Sage Weil
b8b398e1ab qa/suites/rgw/tempest/tasks/rgw_tempest: no need for notcmalloc
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-08 09:11:07 -06:00
Ilya Dryomov
7fdb879004 qa: krbd namespaces test
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-08 15:29:20 +01:00
Sage Weil
d3766916a7 Merge PR #26059 into master
* refs/pull/26059/head:
	mon/MonClient: fix keepalive with v2 auth
	msg/async/ProtocolV2: reject peer_addrs of -
	msg/async/ProtocolV2: clean up feature management
	mon/MonClient: set up rotating_secrets, etc before msgr ready
	msg/async: let client specify preferred order of modes
	msg/async/ProtocolV2: include entity_name, features in reconnect
	msg/async/ProtocolV2: fix write_lock usage around AckFrame
	qa/suites/rados/verify/validator/valgrind: debug refs = 5
	qa/standalone/ceph-helpers: fix health_ok test
	auth/AuthRegistry: only complain about disabling cephx if cephx was enabled
	auth/AuthRegistry: fix locking for get_supported_methods()
	auth: remove AUTH_UNKNOWN weirdness, hardcoded defaults.
	msg/async/ProtocolV2: remove unused get_auth_allowed_methods
	osd: set up messener auth_* before setting dispatcher (and going 'ready')
	mon/AuthMonitor: request max_global_id increase from peon in tick
	mon: prime MgrClient only after messengers are initialized
	qa/suites/rados/workloads/rados_api_tests.yaml: debug mgrc = 20 on mon
	auth: document Auth{Client,Server} interfaces
	auth: future-proof AUTH_MODE_* a bit in case we need to change the encoding byte
	mon/MonClient: request monmap on open instead of ping
	mgr/PyModuleRegistry: add details for MGR_MODULE_{DEPENDENCY,ERROR}
	crimson: fix build
	mon/MonClient: finsih authenticate() only after we get monmap; fix 'tell mgr'
	mon: add auth_lock to protect auth_meta manipulation
	ceph-mon: set up auth before binding
	mon: defer initial connection auth attempts until initial quorum is formed
	mon/MonClient: make MonClientPinger an AuthCleint
	ceph_test_msgr: use DummyAuth
	auth/DummyAuth: dummy auth server and client for test code
	mon/Monitor: fix leak of auth_handler if we error out
	doc/dev/cephx: re-wordwrap
	doc/dev/cephx: document nautilus change to cephx
	vstart.sh: fix --msgr2 option
	msg/async/ProtocolV2: use shared_ptr to manage auth_meta
	auth/Auth{Client,Server}: pass auth_meta in explicitly
	mon/MonClient: behave if authorizer can't be built (yet)
	osd: set_auth_server on client_messenger
	common/ceph_context: get_moduel_type() for seastar cct
	auth: make connection_secret a std::string
	auth,msg/async/ProtocolV2: negotiate connection modes
	auth/AuthRegistry: refactor handling of auth_*_requred options
	osd,mgr,mds: remove unused authorize registries
	switch monc, daemons to use new msgr2 auth frame exchange
	doc/dev/msgr2: update docs to match implementation for auth frames
	auth/AuthClientHandler: add build_initial_request hook
	msg/Messenger: attach auth_client and/or auth_server to each Messenger
	auth: introduce AuthClient and AuthServer handlers
	auth: codify AUTH_MODE_AUTHORIZER
	msg/Connection: track peer_id (id portion of entity_name_t) for msgr2
	auth/AuthAuthorizeHandler: add get_supported_methods()
	auth/AuthAuthorizeHandler: fix args for verify_authorizer()
	auth: constify bufferlist arg to AuthAuthorizer::add_challenge()
	auth/cephx: share all tickets and connection_secret in initial reply
	msg/async,auth: add AuthConnectionMeta to Protocol
	auth/AuthClientHandler: pass in session_key, connection_secret pointers
	auth/AuthServiceHandler: take session_key and connection_secret as args
	auth/cephx: pass more specific type into build_session_auth_info
	mon/Session: separate session creation, peer ident, and registration
	mon/AuthMonitor: bump max_global_id from on_active() and tick()
	mon/AuthMonitor: be more careful with max_global_id
	mon: only all ms_handle_authentication() if auth method says we're done
	mon/AuthMonitor: fix "finished with auth" condition check
	auth: clean up AuthServiceHandler::handle_request() args
	auth: clean up AuthServiceHandler::start_session()
	mon/AuthMonitor: drop unused op arg to assign_global_id()
	msg/async: separate TAG_AUTH_REQUEST_MORE and TAG_AUTH_REPLY_MORE
	msg/async: consolidate authorizer checks
	msg/async: move get_auth_allowed into ProtocolV2.cc
	mon/MonClient: trivial cleanup

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-02-07 16:51:39 -06:00
Sage Weil
65e81e6eb4 qa/suites/rados/verify/validator/valgrind: debug refs = 5
If we detect a leak, let's include logging so we can find it.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
Sage Weil
ee59743a1a qa/suites/rados/workloads/rados_api_tests.yaml: debug mgrc = 20 on mon
Seeing some hangs when the mon is forwarding mgr commands (pg deep-scrub)
to the mgr.  This is a buggy test (it should send it to the mgr directly)
but it is helpful to verify the mon forwarding behavior works.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
Yuri Weinstein
46e617926b
Merge pull request #26068 from badone/wip-ceph-ansible-switch-to-ansible-2.7
qa/tests: Move ceph-ansible tests to ansible version 2.7

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-02-07 06:20:59 -08:00
Ilya Dryomov
711df71790 qa: objectstore snippets for krbd
krbd was being tested with filestore, up until recently when the
default for osd_objectstore was changed to bluestore.  This broke
rbd_simple_big.yaml because bluestore_block_size defaults to 10G.
Pick up the sepia setting of 90G from bluestore-bitmap.yaml.

Run fsx subsuite with both filestore and bluestore.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-05 11:23:42 +01:00
Sage Weil
3cb57037b9 qa/suites/rados/upgrade: whitelist 'slow request'
failure_reason: '"2019-02-03 22:52:41.561332 osd.10 (osd.10) 790 : cluster [WRN] slow
  request 30.154662 seconds old, received at 2019-02-03 22:52:11.406639: osd_op(client.56148.0:39092
  8.9 8.70387d99 (undecoded) ondisk+retry+write+known_if_redirected e1372) currently
  waiting for peered" in cluster log'

We're restarting OSDs, and may see slow requests in the process.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-03 23:28:54 -06:00
Ilya Dryomov
04f5b343f9 qa: update krbd tests for zeroout
Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-03 10:57:07 +01:00
Ilya Dryomov
031bbea739 qa: krbd discard with alloc_size vs zeroout tests
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-02 18:33:32 +01:00
Patrick Donnelly
ef46216d8d
mds: recall caps incrementally
As with trimming, use DecayCounters to throttle the number of caps we recall,
both globally and per-session.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Neha Ojha
4ef94e89c8 qa/suites/rados/thrash: change crush_tunables to jewel in rados_api_tests
Fixes: http://tracker.ceph.com/issues/38042
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-24 16:54:29 -08:00
Matt Benjamin
d4c4082cca
Merge pull request #25774 from yehudasa/wip-rgw-tiering-3
rgw: rados tiering
2019-01-22 12:42:21 -05:00
Brad Hubbard
bf3fb55a90 qa/tests: Move ceph-ansible tests to ansible version 2.7
Required by ceph-ansible master

Fixes: http://tracker.ceph.com/issues/37973

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-01-22 15:23:17 +10:00
Casey Bodley
92ea0a6873
Merge pull request #25996 from yehudasa/wip-37885
qa/rgw/tempest: either force os_type or select random distro

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-01-18 11:45:29 -05:00
David Zafman
99ddd3666b
Merge pull request #22797 from dzafman/wip-19753
osd: Deny reservation if expected backfill size would put us over bac…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-18 07:42:00 -08:00
Sage Weil
4f48314a7a Merge PR #25937 into master
* refs/pull/25937/head:
	qa: specify filestore for misc tests
	osd/objectstore: change osd_objectstore default to bluestore

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-17 18:51:16 -06:00
Sage Weil
b5e5ee6f40 Merge PR #25849 into master
* refs/pull/25849/head:
	qa/suites/rados/upgrade: one mon per node, and enable-msgr2 at end
	qa/rados/thrash-old-clients: avoid msgr2
	mon: make bootstrap rank check more robust
	mon: clean up probe debug output a bit
	msg/async: use v1 for v1 <-> [v2,v1] peers
	msg/async/AsyncMessenger: drop single-use _send_to
	mon/HealthMonitor: raise MON_MSGR2_NOT_ENABLED if mons not bound to msgr2
	doc/rados/operations/health-checks: document MON_* health warnings
	mon/MonMapMonitor: add 'mon enable-msgr2' command
	mon: respawn if rank addr changes
	mon/MonMap: calc_addr_mons() after setting rank addrvec

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-17 11:04:30 -06:00
Sage Weil
7a89787cde qa/suites/rados/upgrade: one mon per node, and enable-msgr2 at end
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 10:03:14 -06:00
Sage Weil
f7b6a4b440 qa/rados/thrash-old-clients: avoid msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 10:03:14 -06:00
Xie Xingguo
18b3f73ecb
Merge pull request #25941 from xiexingguo/wip-warn-enable-upmap
mgr/balancer: blame if upmap won't actually work

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-17 18:22:28 +08:00
Patrick Donnelly
ac302de7b7
qa: silence read-only WRN for damage testing
Fixes: http://tracker.ceph.com/issues/37944

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-16 11:55:54 -08:00
Yehuda Sadeh
0c238c2704 qa/rgw/tempest: either force os_type or select random distro
Fixes: https://tracker.ceph.com/issues/37885

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2019-01-16 11:39:29 -08:00
Vikhyat Umrao
8a694fc2f9 qa: specify filestore for misc tests
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 13:09:19 -06:00
xie xingguo
c7356c66b0 mgr/balancer: blame if upmap won't actually work
With automatic balancing on, and if mode is set to upmap,
balancer will fail silently if min_compat_client is lower than
luminous.
You can't figure out that unless you take a closer look at the
mgr log, which is super annoying..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-16 09:19:54 +08:00
Sage Weil
6ba8db68cd mon/HealthMonitor: raise MON_MSGR2_NOT_ENABLED if mons not bound to msgr2
If the ms_bind_msgr2 option is enabled, and all mons are nautilus,
raise a health alert if any mons aren't bound to msgr2 addresses.

Whitelist tests that mon_bind_addrvec=false or mon_bind_msgr2=false.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-15 10:42:29 -06:00
Yehuda Sadeh
dab1424588 qa/rgw/tempest: either force os_type or select random distro
Fixes: https://tracker.ceph.com/issues/37885

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2019-01-14 17:05:25 -08:00