Commit Graph

3144 Commits

Author SHA1 Message Date
Sage Weil
f67bd47d0a
Merge pull request #20096 from myoungwon/wip-manifest-two-pool-test
src/test/osd: add two pool test for manifest objects

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-26 14:41:14 -06:00
myoungwon oh
a1d6304442 qa/suites/rados/thrash/workloads: add paramters to support two pools
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2018-01-26 16:20:01 +09:00
Kefu Chai
4233cc02d4
Merge pull request #19651 from yanghonggang/master
mon/OSDMonitor.cc: fix expected_num_objects interpret error

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-26 14:34:11 +08:00
Mykola Golub
023759c7c1
Merge pull request #20124 from dillaman/wip-devstack
qa/workunits/rbd: switch devstack to pike release

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-01-26 07:41:00 +02:00
Jason Dillaman
6253490445
Merge pull request #19996 from Songweibin/wip-rbd-deep-cp
rbd: add deep cp CLI method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-25 21:59:31 -05:00
Jason Dillaman
7ff1e0bc21 qa/workunits/rbd: switch devstack to pike release
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-25 18:44:20 -05:00
Kefu Chai
52bf90bb10
Merge pull request #20047 from jcsp/wip-prometheus-qa
qa: add new prometheus test to rados/mgr suite

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-25 23:43:07 +08:00
John Spray
b3e6b8e1e2 qa: always dump corrupt messages in rados/upgrade
Previously only a service with debug_ms>=1 would dump
corrupt messages: in an upgrade test we're *alway*
interested in a corrupt message.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:21 -05:00
John Spray
d9a47181c4 mgr: add health checks for failed modules
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
John Spray
f95b079c21 qa/mgr: add test for command execution errors
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
songweibin
0551029242 rbd: add deep cp CLI method
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-01-24 09:04:33 +08:00
Theofilos Mouratidis
b9e4aa94b7 rbd trash: replace cli delay option, add rbd trash purge command
Replaced the delay argument for the trash move
command with a string acceptable by /bin/date, e.g.:

$ rbd trash move --pool foo --image bar --expires-in "2 weeks"

Added a "rbd trash purge" command that deletes any expired
image from the trash, has also a command to alter the current
expiration date with the "--older-than" argument which accepts
again a valid argument for /bin/date, e.g.:

$rbd trash purge mypool --older-than "2017-08-20"

There is also the "threshold" argument which tries to remove the
oldest trashed images (by deferment end time) until the pool space
is freed up to a percentage point, e.g.:

$ rbd trash purge mypool --threshold 0.9

If mypool uses 1GB it will try to remove trashed images until the
pool usage becomes equal to or lower than 900MB.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
2018-01-23 11:41:54 +01:00
John Spray
e2c68d5e25 qa: assign prometheus ports during selftest
This was throwing IOError("Port 9283 not free on '::'",)
when trying to serve, since merging https://github.com/ceph/ceph/pull/19744

It's because the standbys (on the same node as the active) are
now trying to listen too.

Fixes: https://tracker.ceph.com/issues/22755
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-23 10:23:39 +00:00
John Spray
dd4f322592 qa: add new prometheus test to rados/mgr suite
Added in https://github.com/ceph/ceph/pull/19744

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-23 10:23:39 +00:00
Sage Weil
12e6166397
Merge pull request #20046 from tchaikov/wip-22711
test: reduce the chance to have degraded PGs

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-22 17:46:30 -06:00
Sage Weil
fc8bbaf002
Merge pull request #19362 from myoungwon/wip-manifest-rados-op-promote
osd, librados: add a rados op (TIER_PROMOTE)

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-22 08:21:24 -06:00
John Spray
c05d96330d
Merge pull request #19744 from jan--f/mgr-prometheus-standby-mondown
pybind/mgr/prometheus: add StandbyModule and handle failed MON cluster

Reviewed-by: John Spray <john.spray@redhat.com>
2018-01-22 13:29:48 +00:00
Kefu Chai
74c9cfc466 test: reduce the chance to have degraded PGs
in test_mon_osd_misc(), there is good chance that the cluster chooses
to use an unbalanced weight because of the data distribution at that moment.
but this setting could prevent the CRUSH from choosing enough number of
OSDs for test_mon_cephdf_commands(), where 32 PGs are to be created. so
it's more likely that the CRUSH fails to pick enough OSDs for all PGs.
that's why we have a curr_object_copies_rate = 0.5.

so, in this change, pg=pgp=1 is specified for the new pool.

Fixes: http://tracker.ceph.com/issues/22711
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-22 21:24:58 +08:00
Jan Fajerski
4a45b02539 qa/tasks/mgr: add test_prometheus; smoke tests for prometheus module
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-01-22 13:21:45 +01:00
Yang Honggang
c24f2baec9 mon/OSDMonitor.cc: fix expected_num_objects interpret error
Fixes: http://tracker.ceph.com/issues/22530
Signed-off-by: Yang Honggang <joseph.yang@xtaotech.com>
2018-01-21 21:00:17 -05:00
Jason Dillaman
4e0dc282e4
Merge pull request #19937 from trociny/wip-rbd-showmapped
rbd: adjusted "showmapped" JSON and XML formatted output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-19 10:32:39 -05:00
Sage Weil
c2d28e2750
Merge pull request #18971 from liewegas/wip-pg-scrub-preempt
osd/PG: allow scrub preemption

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-18 12:37:48 -06:00
Sage Weil
5ac3bfa34c qa/suites/rados/singleton/all/divergent_priors*: unsquelch osd debug
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-16 21:52:09 -06:00
David Zafman
7ccb7b7023
Merge pull request #19850 from dzafman/wip-calc-stats
osd/PG: re-write of _update_calc_stats and improve pg degraded state

Fixes: http://tracker.ceph.com/issues/20059

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-16 11:58:49 -08:00
David Zafman
9f103f013c tests: recovery-unfound-found test needs to account for correct misplaced calculations
The test expected HEALTH_OK when in a state with misplaced objects therefore HEALTH_WARN

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-16 10:52:21 -08:00
Kefu Chai
0c38eb87e2
Merge pull request #19673 from zouaiguo/wip_erase_code
mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools.

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-15 18:34:34 +08:00
Kefu Chai
7aba57b9b4
Merge pull request #18191 from hjwsm1989/osd-mark-down
qa/standalone/osd/osd-mark-down: create pool to get updated osdmap faster

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-15 11:09:02 +08:00
David Zafman
88ce0c1a91 test: Verify stat calculations during backfill
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
David Zafman
f5af1af6d3 test: Verify stat calculations during recovery
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
David Zafman
aeba36a660 ceph-helpers.sh: Add flush_pg_stats() to wait_for_clean() to make it reliable
osd-scrub-repair.sh: Fixes for omap keys landing on different OSDs due to flush

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
David Zafman
c77941f593 qa: Ignore degraded PGs when injecting random eio errors
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
Sage Weil
df5a37091a
Merge pull request #19946 from liewegas/wip-22677
qa/workunits/rados/test_rados_tool: fix stray |, race
2018-01-14 13:40:54 -06:00
Sage Weil
3f922e79c3 qa/tasks/ceph: disable osd_debug_deep_scrub_sleep in case it is set
Otherwise the final scrub may take too long.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:47:24 -06:00
Sage Weil
ff9fa835da qa/workunits/rados/test_rados_tool: remove stray |, fix race
Fixes: http://tracker.ceph.com/issues/22676
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:35:14 -06:00
Victor Denisov
838cae45af rbd: Add group snapshot rename commands
Signed-off-by: Victor Denisov <denisovenator@gmail.com>
2018-01-13 14:00:31 -05:00
Sage Weil
ea2c99ec87 qa/workunits/rados/test_rados_tool.sh: -x
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-13 10:26:24 -06:00
Yuri Weinstein
a7b97806fc
Merge pull request #19793 from badone/wip-blocking-deep-scrub
ceph.in: Add blocking mode for scrub and deep-scrub

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-12 14:47:30 -08:00
Patrick Donnelly
e592c0210f
Merge PR #19885 into master
* refs/pull/19885/head:
	qa: increase osd count for ec testing
	qa: add missing openstack configs

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-12 14:37:25 -08:00
Mykola Golub
de97a3be48 rbd: adjusted "showmapped" JSON and XML formatted output
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-01-12 21:36:00 +02:00
Jason Dillaman
619b0c0244 rbd: adjusted "lock list" JSON and XML formatted output
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-12 11:31:07 -05:00
myoungwon oh
93e986c064 qa/suites/rados/thrash: add tier_promote op
1.add tier_promote op for redirect and chunked cases.
2.rename set-chunk.yaml due to current chunked object
only for the read case.

Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2018-01-12 14:38:57 +09:00
Sage Weil
c1ec2752ee
Merge pull request #19180 from theanalyst/tools/rados-omap-clear
tools: rados add a cli option to clear omap keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-11 15:50:12 -06:00
Jason Dillaman
31a382f1ec rbd: utilize '<name>@<snap>' format when creating/removing group snaps
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-11 10:38:23 -05:00
Victor Denisov
d95b791e8d qa/workunits/rbd: Add tests for consistency groups
Signed-off-by: Victor Denisov <denisovenator@gmail.com>
2018-01-11 10:38:23 -05:00
Brad Hubbard
637bf125c3 qa: Add blocking versions of scrub and deep-scrub
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-01-10 14:05:55 +10:00
Patrick Donnelly
f8c275b8b6
qa: increase osd count for ec testing
Missing in d0732fc96f.

Fixes: http://tracker.ceph.com/issues/22646
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-01-09 16:48:18 -08:00
Patrick Donnelly
e8a0f1c554
qa: add missing openstack configs
First introduced in: ec6fb28eaf

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-01-09 16:48:02 -08:00
Casey Bodley
f473d57013 librados: add async interfaces for use with Networking TS
Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-09 09:58:18 -05:00
Sage Weil
0cdb6a74e7
Merge pull request #19616 from ifed01/wip-ifed-revert-extents
osd: get rid off extent map in object_info

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-09 08:44:52 -06:00
zouaiguo
a61e62cf44 mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools.
when we create a pool specify a rule, for example "ceph osd pool create foo replicated 10 rule_foo",
we will set pool foo erasure-code-profile to rule_foo,
if there has an erasure-code-profile names rule_foo, use "ceph osd erasure-code-profile rule_foo" will fail,
"Error EBUSY: foo pool(s) are using the erasure code profile 'rule_foo'", this is wrong.

we should do:
1. set erasure-code-profile to "" when create replicated pools
2. whether erasure-code-profile is used by pool not only judge pool erasure_code_profile property and also the pool is_erasure

Signed-off-by: zouaiguo <zou.aiguo@zte.com.cn>
2018-01-09 14:14:06 +08:00