Commit Graph

4432 Commits

Author SHA1 Message Date
xie xingguo
c7356c66b0 mgr/balancer: blame if upmap won't actually work
With automatic balancing on, and if mode is set to upmap,
balancer will fail silently if min_compat_client is lower than
luminous.
You can't figure out that unless you take a closer look at the
mgr log, which is super annoying..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-16 09:19:54 +08:00
Sage Weil
b73ef78363 Merge PR #25902 into master
* refs/pull/25902/head:
	qa/standalone/mon/mon-handle-forward: fix grep path and check return results
	mon/MonClient: be less aggressive/paranoid about reopening mon connections

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-12 10:26:32 -06:00
Sage Weil
d0bf18379c Merge PR #25917 into master
* refs/pull/25917/head:
	qa/suites/rados/multimon/tasks/mon_recovery: whitelist PG_AVAILABILITY

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-12 10:25:57 -06:00
Sage Weil
9a32d5ba1f Merge PR #25886 into master
* refs/pull/25886/head:
	qa/tasks/rebuild_mondb: use monmap to properly name the mons
	ceph-monstore-tool: add --monmap <path> argument to rebuild command
	ceph-mon: dump monmap to debug logs
	Revert "mon/MonMap: no noname- mon name prefix when for_mkfs"
	mon/MonMap: improve/fix initial monmap generation for mkfs
	ceph-mon: make membership check with public_addr more robust
	ceph-mon: make mkfs check whether I am in monmap more robust

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-12 05:44:00 -06:00
Sage Weil
c18a5d2e1c qa/tasks/rebuild_mondb: use monmap to properly name the mons
We used to rely on the monmap bootstrap code to magically create a valid
monmap with named mons because our old-style ceph.conf had mon_addr
values in each mon.foo section.  Instead, just feed it a real monmap
from pre-destruction.

In practice, a user can manually generate this monmap, or rename the
mons after the fact with --inject-monmap, or whatever.  Out of scope
for this test, so we just do the simplest thing to make the rebuild test
work.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-11 16:10:14 -06:00
Sage Weil
af435783b4 qa/suites/rados/multimon/tasks/mon_recovery: whitelist PG_AVAILABILITY
The mgr creates a pool for device health, and mons may be thrashing and
make peering slow.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-11 09:43:07 -06:00
Jason Dillaman
e920693ff6
Merge pull request #25529 from Songweibin/wip-fix-rbd-migration
librbd: disallow trash restoring when image being migrated

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-01-11 09:03:58 -05:00
Sage Weil
221afb0e28 Merge PR #25840 into master
* refs/pull/25840/head:
	qa/msgr: add async-v1only case

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-10 17:20:10 -06:00
Sage Weil
2762955576 qa/standalone/mon/mon-handle-forward: fix grep path and check return results
This makes the test more strict and less confusing.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-10 17:18:38 -06:00
Yuri Weinstein
4d7d0f53f3 qa/tests: reduced nubmer of runs on 'ovh'
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-01-10 09:03:16 -08:00
Lenz Grimmer
7fe4820c60
Merge pull request #25791 from rhcs-dashboard/36740-pool-stats-read-write-columns
mgr/dashboard: add columns to Pools table

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2019-01-10 09:58:05 +01:00
Josh Durgin
a05f9ebaa6
Merge pull request #25816 from neha-ojha/wip-36686
osd/mon: fix upgrades for pg log hard limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil  <sage@redhat.com>
2019-01-09 13:17:30 -08:00
Lenz Grimmer
26de63e5af
Merge pull request #25813 from rhcs-dashboard/updated-test-full-health
mgr/dashboard: updated health API test
2019-01-09 14:32:40 +01:00
alfonsomthd
d9404d6813 mgr/dashboard: add columns to Pools table
* Added columns:
Usage, Read bytes, Write bytes, Read ops, Write ops.

* TableKeyValueComponent:
Bugfix in method: _insertFlattenObjects

* Updated jest global mock:
Now window.getComputedStyle returns an object
that has getPropertyValue defined
(otherwise jest pool tests involving chart.js threw error). See:
https://developer.mozilla.org/en-US/docs/Web/API/Window/getComputedStyle

Fixes: https://tracker.ceph.com/issues/37717
Fixes: https://tracker.ceph.com/issues/34320

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-09 08:59:47 +01:00
Patrick Donnelly
03990857fa
Merge PR #25621 into master
* refs/pull/25621/head:
	mds: allow boot on read-only
	mds: setup readonly mode for PurgeQueue
	mds: return string_view for type str
	mds: add missing locks for PurgeQueue methods
	mds: delete on_error context on des

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-08 15:56:16 -08:00
Patrick Donnelly
4e4db26c9c
Merge PR #24556 into master
* refs/pull/24556/head:
	tools/cephfs: make 'cephfs-data-scan scan_links' update snaptable
	tools/cephfs: make 'cephfs-data-scan scan_links' update inotable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-08 15:10:28 -08:00
Sage Weil
4c69fe2d3b qa/msgr: add async-v1only case
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-08 13:22:39 -06:00
Casey Bodley
1b2b885518
Merge pull request #25381 from cbodley/wip-qa-rgw-cls
qa/rgw: add cls_lock/log/refcount/version tests to verify suite

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-08 13:05:11 -05:00
Sage Weil
587b548043 Merge PR #25801 into master
* refs/pull/25801/head:
	qa/tasks/radosbench: default to 64k writes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-08 08:30:25 -06:00
songweibin
1f2eef7f1f librbd: disallow trash restoring when image being migrated
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2019-01-08 10:26:06 +08:00
Neha Ojha
c0da26505f qa/suites/upgrade/*-x/stress-split: set pglog_hardlimit flag
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-07 09:42:54 -08:00
Neha Ojha
24c3e2d669 qa/suites/upgrade/luminous-x: add pg log settings
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-07 09:42:54 -08:00
Yuri Weinstein
45af678d3d qa/tests: added pg log settings to mimic-x
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-07 09:42:22 -08:00
Sage Weil
78070b96bf qa/tasks/radosbench: default to 64k writes
The default rados bench is 4MB writes, but this tends to fill up our
tiny test clusters.  Use 64KB instead.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-07 07:22:29 -06:00
alfonsomthd
dcf46cfe94 mgr/dashboard: updated health API test
Updated test_full_health to reflect changes in
mgr_map / mon_status payloads.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-07 12:49:25 +01:00
Yan, Zheng
b593e5a881 tools/cephfs: make 'cephfs-data-scan scan_links' update snaptable
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-01-07 16:50:04 +08:00
Yan, Zheng
01089652d3 tools/cephfs: make 'cephfs-data-scan scan_links' update inotable
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-01-07 16:49:31 +08:00
Sage Weil
1688d8fd92 qa/suites/rados/thrash-old-clients: no async-v2only
Old clients don't support the v2 protocol.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-05 12:26:56 -06:00
Sage Weil
e069c30cb3 Merge remote-tracking branch 'private/wip-mon-kv-fix' into wip-mimic-4 2019-01-04 14:03:56 -06:00
Sage Weil
251f667ef8 Merge PR #25009 into master
* refs/pull/25009/head:
	librbd: stringify locker name with get_legacy_str()
	osdc/Objecter: fix list_watchers addr rendering to match legacy
	test/crimson: disable unittest_seastar_messenger test
	msg/msg_types: encode entity_addr_t TYPE_ANY as TYPE_LEGACY for pre-nautilus
	client: make blacklist detection handle TYPE_ANY entries
	mon/OSDMonitor: maintain compat output for 'blacklist ls'
	client: maintain compat for {inst,addr}_str in status dump
	qa/tasks/ceph_manager: compare osd flush seq #'s as ints
	qa/suites/fs: make use of simple.yaml where appropriate
	qa/msgr: move msgr factet into generic re-usable dir
	crimson: fix monmap build for seastar
	doc/start/ceph.conf: trim the sample ceph.conf file
	doc/rados/operations: only describe --public-{addr,network} method for adding mons
	PendingReleaseNotes: deprecate 'mon addr'
	doc: fix some 'mon addr' references
	doc/rados/configuration: fix some 'mon addr' references
	doc/rados/configuration/network-config-ref: revise network docs somewhat
	doc/rados/configuration/network-config-ref: remove totally obsolete section
	qa/suites/rados: replace mon_seesaw.py task with a small bash script
	qa/suites/fs/upgrade: don't bind to v2 addrs
	qa/tasks/mon_thrash: avoid 'mon addr' in mon section
	mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set
	osd/OSDMap: maintain compat addr fields
	msg/msg_types: add get_legacy_str()
	mds/MDSMap.h: maintain compat addr field
	mon/MgrMap: maintain compat active_addr field
	mon/MonClient: reconnect to mon if it's addrvec appears to have changed
	qa/tasks/ceph.conf.template: increase mon_mgr_mkfs_grace
	msg/async/ProtocolV2: fill in IP for all peer_addrs
	msg/async: print all addrs on debug lines
	mon/MonMap: no noname- mon name prefix when for_mkfs
	ceph-monstore-tool: print initial monmap
	msg/async/ProtocolV2: advertise ourselves as a v2 addr when using v2 protocol
	msg/async: assert existing protocol matches current protocol
	msg/async: add missing modelines
	mon/MonMap: add missing modeline
	vstart.sh: put mon addrs in mon_host, not 'mon addr'
	msg/async: better debug around conn map lookups and updates
	mon/MonClient: dump initial monmap at debug level 10
	qa/standalone/osd/osd-fast-mark-down: use v1 addr w/ simplemessenger
	qa/tasks/ceph: set initial monmap features with using addrvec addrs
	monmaptool: add --enable-all-features option
	qa/tasks/ceph: only use monmaptool --addv if addr has [,:v]
	qa/tasks/ceph_manager: make get_mon_status use mon addr
	qa/tasks/ceph: keep mon addrs in ctx namespace
	mon/OSDMonitor: log all osd addrs on boot
	msg/simple: behave when v2 and v1 addrs are present at target
	mon/MonClient: warn if global_id changes
	msg/Connection: add warning/note on get_peer_global_id
	mds/MDSDaemon: clean up handle_mds_map debug output a bit
	qa/suites/rados/upgrade: debug mds
	mds/MDSRank: improve is_stale_message to handle addrvecs
	msg/async: make loopback detect when sending to one of our many addrs
	qa/suites/rados/upgrade: no aggressive pg num changes
	mon/OSDMonitor: require nautilus mons for require_osd_release=nautilus
	mon/OSDMonitor: require mimic mons for require_osd_release=mimic
	qa/suites/rados/thrash-old-clients: use legacy addr syntax in ceph.conf
	msg/async: preserve peer features when replacing a connection
	qa/tasks/ceph.py: move methods from teuthology.git into ceph.py directly; support mon bind * options
	mon/MonMap: adjust build_initial behavior for mkfs vs probe
	mon/MonMap: improve ambiguous addr behavior
	qa/suites/rados/upgrade: spread mons a bit
	qa/rados/thrash-old-clients: keep mons on separate hosts
	qa/standalone/mon/misc.sh: tweak test to be more robust
	qa/tasks/mon_seesaw: expect v1/v2 prefix in addr
	osd/OSDMap: fix is_blacklisted() check to assume type ANY
	mon/OSDMonitor: use ANY addr type for blacklisting
	mon/msg_types: TYPE_V1ORV2 -> TYPE_ANY
	qa/workunits/cephtool: fix blacklist test
	qa/suites/upgrade: install old version with only v1 addrs
	common/options: by default, bind to both msgr v1 and v2 addresses
	vstart.sh: add --msgr1, --msgr2, --msgr21 options
	msg/async/ProtocolV2: be flexible with server identity check
	msg/msg_types: fix entity_addrvec_t::parse() with null end arg
	qa/suites/rados/basic/msgr: no msgr2 addrs in initial monmaps
	qa/tasks/ceph: add 'mon_bind_addrvec' and 'mon_bind_msgr2' options
	monmaptool: add --addv argument to pass in addrvec directly
	qa/suites/rados/basic/msgr: do not use msgr2 with simplemessenger
	qa/suites/rados/basic/msgr: async is not experimental
	messages/MOSDBoot: fix compat with pre-nautilus
	mon/MonMap: allow v1 or v2 to be explicitly specified along with part
	msg/msg_types: allow parsing of IPs without assuming v1 vs v2
	msg/msg_types: default parse to v2 addrs
	msg: standarize on v1: and v2: prefixes for *all* entity_addr_t's
	vstart.sh: use msgr2 by default
	mon/MonMap: remove get_addr() methods
	ceph-mon: adjust startup/bind/join sequence to use addrs
	mon: use MonMap::get_addrs() (instead of get_addr())
	mon/MonClient: change pending_cons to addrvec-based map
	mon/MonMap: fix set_addr() caller, kill wrapper
	mon/MonMap: remove addr-based add()
	monmaptool: fix --add to do either legacy or msgr2+legacy
	monmaptool: clean up iterator use a bit
	mon/MonMap: handle ambiguous mon addrs by trying both legacy and msgr
	mon/MonMap: take addrvec for set_initial_members
	mon/MonMap: use addrvecs for test instances
	mon: pass addrvec via MMonJoin
	mon/MonmapMonitor: fix 'mon add' to populate addrvec
	mon/MonMap: addr -> addrvec
	msg/async/ProtocolV2: only update socket_addr if we learned our addr
	osd: go active even if mon only accepted our v1 addr
	test/msgr: add test for msgr2 protocol
	msg/async/ProtocolV2: share socket_addr and all addrs during handshake
	msg/async: print socket_addr for the connection
	msg/async: msgr2 protocol placeholder
	msg/async: move ProtocolV1 class to its own source file
	msg/async: keep listen addr in ServerSocket, pass to new connections
	msg/async/AsyncMessenger: fix set_addr_unknowns

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-04 13:42:09 -06:00
Neha Ojha
645e465f6e
Merge pull request #25781 from neha-ojha/wip-fix-upgrade-rhel
qa/suites/upgrade/mimic-x: fix rhel runs

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2019-01-04 08:01:17 -08:00
Ricardo Marques
695976705f
Merge pull request #25210 from dillaman/wip-dashboard-rbd-mirroring
dashboard: support configuring block mirroring pools and peers
2019-01-04 10:39:09 +00:00
Neha Ojha
a74129d26d qa/suites/upgrade/mimic-x: fix rhel runs
The following fragment was required for rhel on ovh
overrides:
    ansible.cephlab:
      skip_tags: entitlements,packages,repos

Since, this suite runs on smithi in our nightlies, we should not need
this.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-03 13:39:30 -08:00
Patrick Donnelly
ee7d253494
Merge PR #24635 into master
* refs/pull/24635/head:
	test: add scrub control command tests
	mds: scrub abort/pause/resume/status control commands
	test: switch using "scrub start" tell interface to initiate scrub
	mds: introduce "scrub start" tell interface to initiate scrub
	mds: dump scrub formatted output when context completion
	mds: generate random scrub tag when empty
	mds: introduce C_ExecAndReply context completion class
	mds: use CInode::scrub_is_in_progress() wherever required

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-03 10:15:31 -08:00
Sage Weil
0d4c4db3c0 qa/tasks/ceph_manager: compare osd flush seq #'s as ints
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
8a3d90199d qa/suites/fs: make use of simple.yaml where appropriate
There's more needed than just ms_type=simple now.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
d518eb6cac qa/msgr: move msgr factet into generic re-usable dir
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
16980bd12f qa/suites/rados: replace mon_seesaw.py task with a small bash script
The teuthology test did not like the change to remove 'mon addr' from
ceph.conf.  The standalone script is easier to test.

Note that it avoids mon names 'a', 'b', 'c' since the MonMap::build_initial
uses those.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
f857c70c9c qa/suites/fs/upgrade: don't bind to v2 addrs
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
caa3a82ada qa/tasks/mon_thrash: avoid 'mon addr' in mon section
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
c4ae3554e1 qa/tasks/ceph.conf.template: increase mon_mgr_mkfs_grace
The grace starts with the monmap creation stamp, and ceph.py does a lot
of work between creating that map and actually starting daemons (e.g.,
preparing all of the osd devices), leading to occasional MGR_DOWN errors.
Double the grace period.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
b92be2ca9b qa/standalone/osd/osd-fast-mark-down: use v1 addr w/ simplemessenger
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
a19b8e5b14 qa/tasks/ceph: set initial monmap features with using addrvec addrs
The --add option will only infer a bare IP to include a v2 addr if the
NAUTILUS feature is there, and that isn't normally present on a freshly
generate monmap.  Add it if we are doing addrvecs!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
241d402d7c qa/tasks/ceph: only use monmaptool --addv if addr has [,:v]
Otherwise, we want the --add path, which has the logic to infer ports,
v2+v1, etc.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
ac2430a43d qa/tasks/ceph_manager: make get_mon_status use mon addr
We don't have the 'mon addr' config property any more.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
545df766be qa/tasks/ceph: keep mon addrs in ctx namespace
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
d980907fc4 qa/suites/rados/upgrade: debug mds
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
68913080b5 qa/suites/rados/upgrade: no aggressive pg num changes
We now run with mixed mons and old mgrs, so this won't work.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
576b6a77f1 qa/suites/rados/thrash-old-clients: use legacy addr syntax in ceph.conf
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
1ab352dd31 qa/tasks/ceph.py: move methods from teuthology.git into ceph.py directly; support mon bind * options
Having these live in teuthology.git is silly, since they are only consumed
by the ceph task, and it is hard to revise the behavior.

Revise the behavior by adding mon_bind_* options.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00