Commit Graph

300 Commits

Author SHA1 Message Date
Sage Weil
2ad02fbfe3 qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-09 09:34:49 -06:00
Sage Weil
10b9626ea7 qa/standalone/scrub/osd-scrub-repair: fix unfound grep
It's now "1/2 unfound":

             1/2 objects unfound (50.000%)

..presumably due to the rbd pool init creating the rbd_directory.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 18:23:48 -06:00
Sage Weil
30fc7f5e97 qa/standalone/ceph-helpers: fix test_wait_for_clean
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 18:07:10 -06:00
Sage Weil
1e2b0c7252 qa/standalone/ceph-helpers.sh: fix test_run_mon
- Only create each osd once
- forget the first osdmap dump test; it's pointless

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 17:43:00 -06:00
Sage Weil
bf74c1adc4 qa/standalone/osd/osd-rep-recov-eio: fix better
- no need for the default pool size
- no initial osds or it will collide with setup_osds later
- no need for rbd pool at all

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 17:41:11 -06:00
Sage Weil
62136d381a Merge PR #26794 into master
* refs/pull/26794/head:
	mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS
	qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
	mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
	Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
	mon/OSDMonitor: use legacy_equals to compare osd addrs
	msg/msg_types: make legacy_equals() symmetrical
	mon/MDSMonitor: stop using get_orig_source_inst()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-07 22:12:52 -06:00
Sage Weil
c939eefa16 qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-07 16:35:35 -06:00
Sage Weil
b59ff3860f qa/standalone/osd/osd-force-create-pg: create more pgs
Avoid warnings about too few pgs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:27:56 -06:00
Sage Weil
cba0483b09 qa/standalone: make sure an osd is running before create_rbd_pool
'rbd pool init' now does IO.  Drop the pool, or change the pool size to 1.

Fixes: http://tracker.ceph.com/issues/38585
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:27:56 -06:00
Sage Weil
01316aa7bd qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap
This test introduces a map gap.  What *should* happen is that when there is
such a gap, we cannot import.  Previously, the test didn't reliably produce
a map gap at all, and didn't check that import failed--it verified that it
passed.

Fix the test so that it reliably produces a gap *and* reports
min_last_epoch_clean to the mon so we can trim.  Then verify we fail to
import, but can with --force.  But remove the pg again, because if we
force an import with a map gap the osd will refuse to start.

Fixes: http://tracker.ceph.com/issues/38525
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-03 10:23:27 -06:00
Sage Weil
c6a7b2cbd1 qa/standalone/osd/osd-markdown: disable CLI command dups
The markdown test is based on marking down a specific number of times, but
the duplicate commands from the CLI may not get absorbed/batched by the
mon, breaking the test.  Override the default qa/tasks/workunit.py
behavior of sending dups.

Fixes: http://tracker.ceph.com/issues/38359
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-18 15:02:25 -06:00
David Zafman
64beabc4c6 test: Limit loops waiting for force-backfill/force-recovery to happen
Fixes: http://tracker.ceph.com/issues/38309

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-13 17:44:53 -08:00
David Zafman
910a95b9c8 test: osd-backfill-stats.sh Fix check of multi backfill OSDs, skip remapped test
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-07 20:05:58 -08:00
David Zafman
690ff9a21f
Merge pull request #26213 from dzafman/wip-38041
osd: Fix recovery and backfill priority handling

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-02-07 17:26:34 -08:00
David Zafman
ca5cf14fa8 test: Add scripts to test backfill/recovery priority handling
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-07 15:46:23 -08:00
Sage Weil
dcdca44aa4 qa/standalone/ceph-helpers: fix health_ok test
Stopping the osd daemon won't reliably get you HEALTH_WARN or ERR; you have
to make sure it is also marked down.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
David Zafman
36e305c4b6 test: Ignore kill_daemons() error
Workaround for: http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-05 11:31:32 -08:00
David Zafman
bca4fe98b1 test: Fix kill_daemon() to check after last large sleep
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-05 11:30:04 -08:00
David Zafman
cc6339c0cd test: Increase timeouts in osd-backfill-space.sh because of failure seen
Fixes: http://tracker.ceph.com/issues/38027

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-05 11:29:32 -08:00
David Zafman
70b5136208 test: Add option to wait_for_clean() to execute at every sleep
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-30 09:35:51 -08:00
David Zafman
553d83dd24
Merge pull request #25403 from liyichao/rdigest
tools: Add clear-data-digest command to objectstore tool.

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-01-30 09:30:23 -08:00
David Zafman
894bdf080e
Merge pull request #26158 from dzafman/wip-38053
Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-30 09:29:01 -08:00
Kefu Chai
8d5ddb5817
Merge pull request #26091 from tchaikov/wip-36737
cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

Tested-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2019-01-30 21:35:52 +08:00
Kefu Chai
94a84b6f5a test: listen on random port in tests which start ceph-mon
See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-27 21:16:54 +08:00
David Zafman
388e54d906 test: ceph-objectstore-tool cut down on large run
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-26 19:35:11 -08:00
David Zafman
07e4273c6a test: ceph-objectstore-tool: Add test for EC object dump to check hinfo section
Fixes: http://tracker.ceph.com/issues/38053

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-26 19:35:11 -08:00
David Zafman
786b39f18f test: ceph-objectstore-tool: Fix EC code handling so it doesn't skip EC objects
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-26 19:35:11 -08:00
David Zafman
0753dc2b17 test: Remove unnecessary shell code that breaks this python test
Caused by: 8a694fc2f9

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-26 19:35:11 -08:00
David Zafman
ef2dc05de0 osd, test: Add test case with osd support for overdue PG scrubs and deep scrubs
Add trigger_deep_scrub osd command for testing
Publish stats when trigger_scrub/trigger_deep_scrub is used for testing
Add optional argument to trigger_scrub/trigger_deep_scrub
for amount of extra time to change last scrub stamps

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-23 16:49:33 -08:00
David Zafman
879d89aace test: Correct typo trying to call flush_pg_stats
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-23 16:49:33 -08:00
David Zafman
99ddd3666b
Merge pull request #22797 from dzafman/wip-19753
osd: Deny reservation if expected backfill size would put us over bac…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-18 07:42:00 -08:00
liyichao
da5832b2b4 tools: Add clear-data-digest command to objectstore tool.
There may be a situation where data digest in object info is
inconsistent with that computed from object data, then deep-scrub
will fail even though all three repicas have the same object data.

Fixes: https://tracker.ceph.com/issues/37935

Signed-off-by: Li Yichao <liyichao.good@gmail.com>
2019-01-17 11:03:51 +08:00
Vikhyat Umrao
8a694fc2f9 qa: specify filestore for misc tests
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 13:09:19 -06:00
Sage Weil
2762955576 qa/standalone/mon/mon-handle-forward: fix grep path and check return results
This makes the test more strict and less confusing.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-10 17:18:38 -06:00
Sage Weil
251f667ef8 Merge PR #25009 into master
* refs/pull/25009/head:
	librbd: stringify locker name with get_legacy_str()
	osdc/Objecter: fix list_watchers addr rendering to match legacy
	test/crimson: disable unittest_seastar_messenger test
	msg/msg_types: encode entity_addr_t TYPE_ANY as TYPE_LEGACY for pre-nautilus
	client: make blacklist detection handle TYPE_ANY entries
	mon/OSDMonitor: maintain compat output for 'blacklist ls'
	client: maintain compat for {inst,addr}_str in status dump
	qa/tasks/ceph_manager: compare osd flush seq #'s as ints
	qa/suites/fs: make use of simple.yaml where appropriate
	qa/msgr: move msgr factet into generic re-usable dir
	crimson: fix monmap build for seastar
	doc/start/ceph.conf: trim the sample ceph.conf file
	doc/rados/operations: only describe --public-{addr,network} method for adding mons
	PendingReleaseNotes: deprecate 'mon addr'
	doc: fix some 'mon addr' references
	doc/rados/configuration: fix some 'mon addr' references
	doc/rados/configuration/network-config-ref: revise network docs somewhat
	doc/rados/configuration/network-config-ref: remove totally obsolete section
	qa/suites/rados: replace mon_seesaw.py task with a small bash script
	qa/suites/fs/upgrade: don't bind to v2 addrs
	qa/tasks/mon_thrash: avoid 'mon addr' in mon section
	mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set
	osd/OSDMap: maintain compat addr fields
	msg/msg_types: add get_legacy_str()
	mds/MDSMap.h: maintain compat addr field
	mon/MgrMap: maintain compat active_addr field
	mon/MonClient: reconnect to mon if it's addrvec appears to have changed
	qa/tasks/ceph.conf.template: increase mon_mgr_mkfs_grace
	msg/async/ProtocolV2: fill in IP for all peer_addrs
	msg/async: print all addrs on debug lines
	mon/MonMap: no noname- mon name prefix when for_mkfs
	ceph-monstore-tool: print initial monmap
	msg/async/ProtocolV2: advertise ourselves as a v2 addr when using v2 protocol
	msg/async: assert existing protocol matches current protocol
	msg/async: add missing modelines
	mon/MonMap: add missing modeline
	vstart.sh: put mon addrs in mon_host, not 'mon addr'
	msg/async: better debug around conn map lookups and updates
	mon/MonClient: dump initial monmap at debug level 10
	qa/standalone/osd/osd-fast-mark-down: use v1 addr w/ simplemessenger
	qa/tasks/ceph: set initial monmap features with using addrvec addrs
	monmaptool: add --enable-all-features option
	qa/tasks/ceph: only use monmaptool --addv if addr has [,:v]
	qa/tasks/ceph_manager: make get_mon_status use mon addr
	qa/tasks/ceph: keep mon addrs in ctx namespace
	mon/OSDMonitor: log all osd addrs on boot
	msg/simple: behave when v2 and v1 addrs are present at target
	mon/MonClient: warn if global_id changes
	msg/Connection: add warning/note on get_peer_global_id
	mds/MDSDaemon: clean up handle_mds_map debug output a bit
	qa/suites/rados/upgrade: debug mds
	mds/MDSRank: improve is_stale_message to handle addrvecs
	msg/async: make loopback detect when sending to one of our many addrs
	qa/suites/rados/upgrade: no aggressive pg num changes
	mon/OSDMonitor: require nautilus mons for require_osd_release=nautilus
	mon/OSDMonitor: require mimic mons for require_osd_release=mimic
	qa/suites/rados/thrash-old-clients: use legacy addr syntax in ceph.conf
	msg/async: preserve peer features when replacing a connection
	qa/tasks/ceph.py: move methods from teuthology.git into ceph.py directly; support mon bind * options
	mon/MonMap: adjust build_initial behavior for mkfs vs probe
	mon/MonMap: improve ambiguous addr behavior
	qa/suites/rados/upgrade: spread mons a bit
	qa/rados/thrash-old-clients: keep mons on separate hosts
	qa/standalone/mon/misc.sh: tweak test to be more robust
	qa/tasks/mon_seesaw: expect v1/v2 prefix in addr
	osd/OSDMap: fix is_blacklisted() check to assume type ANY
	mon/OSDMonitor: use ANY addr type for blacklisting
	mon/msg_types: TYPE_V1ORV2 -> TYPE_ANY
	qa/workunits/cephtool: fix blacklist test
	qa/suites/upgrade: install old version with only v1 addrs
	common/options: by default, bind to both msgr v1 and v2 addresses
	vstart.sh: add --msgr1, --msgr2, --msgr21 options
	msg/async/ProtocolV2: be flexible with server identity check
	msg/msg_types: fix entity_addrvec_t::parse() with null end arg
	qa/suites/rados/basic/msgr: no msgr2 addrs in initial monmaps
	qa/tasks/ceph: add 'mon_bind_addrvec' and 'mon_bind_msgr2' options
	monmaptool: add --addv argument to pass in addrvec directly
	qa/suites/rados/basic/msgr: do not use msgr2 with simplemessenger
	qa/suites/rados/basic/msgr: async is not experimental
	messages/MOSDBoot: fix compat with pre-nautilus
	mon/MonMap: allow v1 or v2 to be explicitly specified along with part
	msg/msg_types: allow parsing of IPs without assuming v1 vs v2
	msg/msg_types: default parse to v2 addrs
	msg: standarize on v1: and v2: prefixes for *all* entity_addr_t's
	vstart.sh: use msgr2 by default
	mon/MonMap: remove get_addr() methods
	ceph-mon: adjust startup/bind/join sequence to use addrs
	mon: use MonMap::get_addrs() (instead of get_addr())
	mon/MonClient: change pending_cons to addrvec-based map
	mon/MonMap: fix set_addr() caller, kill wrapper
	mon/MonMap: remove addr-based add()
	monmaptool: fix --add to do either legacy or msgr2+legacy
	monmaptool: clean up iterator use a bit
	mon/MonMap: handle ambiguous mon addrs by trying both legacy and msgr
	mon/MonMap: take addrvec for set_initial_members
	mon/MonMap: use addrvecs for test instances
	mon: pass addrvec via MMonJoin
	mon/MonmapMonitor: fix 'mon add' to populate addrvec
	mon/MonMap: addr -> addrvec
	msg/async/ProtocolV2: only update socket_addr if we learned our addr
	osd: go active even if mon only accepted our v1 addr
	test/msgr: add test for msgr2 protocol
	msg/async/ProtocolV2: share socket_addr and all addrs during handshake
	msg/async: print socket_addr for the connection
	msg/async: msgr2 protocol placeholder
	msg/async: move ProtocolV1 class to its own source file
	msg/async: keep listen addr in ServerSocket, pass to new connections
	msg/async/AsyncMessenger: fix set_addr_unknowns

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-04 13:42:09 -06:00
Sage Weil
16980bd12f qa/suites/rados: replace mon_seesaw.py task with a small bash script
The teuthology test did not like the change to remove 'mon addr' from
ceph.conf.  The standalone script is easier to test.

Note that it avoids mon names 'a', 'b', 'c' since the MonMap::build_initial
uses those.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
b92be2ca9b qa/standalone/osd/osd-fast-mark-down: use v1 addr w/ simplemessenger
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
7559a47f5b qa/standalone/mon/misc.sh: tweak test to be more robust
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
David Zafman
554ea73cb5 test: Disable duplicate request command test during scrub testing
Scrub testing requires an orderly control of scrubbing.  Most but not
all the time, the duplicate scrub request is ignored because the first
request hasn't finished.  Teuthology enables this environment variable
in the workunit handling.

Fixes: https://tracker.ceph.com/issues/36525

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-21 18:28:23 -08:00
David Zafman
094d39aa09 test: Add testing for erasure code backfill out of space detection
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-18 09:30:44 -08:00
David Zafman
3b8f86c8b0 test: Add testing for backfill out of space detection
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-18 09:30:44 -08:00
David Zafman
975dbc5841 test: Minor improvement to create_ec_pool()
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-10 20:16:01 -08:00
Igor Fedotov
79fd227639 qa: replace raw_bytes_used field access in QA test cases
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
d07c10dfc0 os/bluestore: add main device expand capability.
One can do that via ceph-bluestore-tool's bluefs-bdev-expand command

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-11-29 12:48:20 +03:00
David Zafman
1841928e28 test: Add test for requested scrub priority
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-14 23:57:20 -08:00
Josh Durgin
fd2a4c5733
Merge pull request #22476 from dzafman/wip-23875
Removal of snapshot with corrupt replica crashes osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-09 15:15:01 -08:00
David Zafman
a159f162c5 test: osd-scrub-snaps.sh: After snapshot removal wait for snaptrim to complete
Due to deliberate corruptions snaptrim_error means snaptrim is done

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:20 -08:00
David Zafman
e37f95ac27 test: osd-scrub-snaps.sh: Testing with new --rmtype in ceph-objectstore-tool
Use --rmtype snapmap with new obj16 to remove snapmap only, check for repair message
Use --rmtype nosnapmap to remove obj5 while leaving snapmap behind

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:20 -08:00
David Zafman
f43faf4ad7 test: cleanup: Remove redundant cat of log and handle errors in create_scenario()
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-11-08 14:48:19 -08:00
Sage Weil
c8a8dc21fd Merge PR #24828 into master
* refs/pull/24828/head:
	qa/osd-bluefs-volume-ops: use ceph-bluestore-tool for fsck
	qa/osd-bluefs-volume-ops: reduce space usage for the test case

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-11-08 16:26:52 -06:00