Commit Graph

3130 Commits

Author SHA1 Message Date
Theofilos Mouratidis
b9e4aa94b7 rbd trash: replace cli delay option, add rbd trash purge command
Replaced the delay argument for the trash move
command with a string acceptable by /bin/date, e.g.:

$ rbd trash move --pool foo --image bar --expires-in "2 weeks"

Added a "rbd trash purge" command that deletes any expired
image from the trash, has also a command to alter the current
expiration date with the "--older-than" argument which accepts
again a valid argument for /bin/date, e.g.:

$rbd trash purge mypool --older-than "2017-08-20"

There is also the "threshold" argument which tries to remove the
oldest trashed images (by deferment end time) until the pool space
is freed up to a percentage point, e.g.:

$ rbd trash purge mypool --threshold 0.9

If mypool uses 1GB it will try to remove trashed images until the
pool usage becomes equal to or lower than 900MB.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
2018-01-23 11:41:54 +01:00
Sage Weil
12e6166397
Merge pull request #20046 from tchaikov/wip-22711
test: reduce the chance to have degraded PGs

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-22 17:46:30 -06:00
Sage Weil
fc8bbaf002
Merge pull request #19362 from myoungwon/wip-manifest-rados-op-promote
osd, librados: add a rados op (TIER_PROMOTE)

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-22 08:21:24 -06:00
John Spray
c05d96330d
Merge pull request #19744 from jan--f/mgr-prometheus-standby-mondown
pybind/mgr/prometheus: add StandbyModule and handle failed MON cluster

Reviewed-by: John Spray <john.spray@redhat.com>
2018-01-22 13:29:48 +00:00
Kefu Chai
74c9cfc466 test: reduce the chance to have degraded PGs
in test_mon_osd_misc(), there is good chance that the cluster chooses
to use an unbalanced weight because of the data distribution at that moment.
but this setting could prevent the CRUSH from choosing enough number of
OSDs for test_mon_cephdf_commands(), where 32 PGs are to be created. so
it's more likely that the CRUSH fails to pick enough OSDs for all PGs.
that's why we have a curr_object_copies_rate = 0.5.

so, in this change, pg=pgp=1 is specified for the new pool.

Fixes: http://tracker.ceph.com/issues/22711
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-22 21:24:58 +08:00
Jan Fajerski
4a45b02539 qa/tasks/mgr: add test_prometheus; smoke tests for prometheus module
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-01-22 13:21:45 +01:00
Jason Dillaman
4e0dc282e4
Merge pull request #19937 from trociny/wip-rbd-showmapped
rbd: adjusted "showmapped" JSON and XML formatted output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-19 10:32:39 -05:00
Sage Weil
c2d28e2750
Merge pull request #18971 from liewegas/wip-pg-scrub-preempt
osd/PG: allow scrub preemption

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-18 12:37:48 -06:00
Sage Weil
5ac3bfa34c qa/suites/rados/singleton/all/divergent_priors*: unsquelch osd debug
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-16 21:52:09 -06:00
David Zafman
7ccb7b7023
Merge pull request #19850 from dzafman/wip-calc-stats
osd/PG: re-write of _update_calc_stats and improve pg degraded state

Fixes: http://tracker.ceph.com/issues/20059

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-16 11:58:49 -08:00
David Zafman
9f103f013c tests: recovery-unfound-found test needs to account for correct misplaced calculations
The test expected HEALTH_OK when in a state with misplaced objects therefore HEALTH_WARN

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-16 10:52:21 -08:00
Kefu Chai
0c38eb87e2
Merge pull request #19673 from zouaiguo/wip_erase_code
mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools.

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-15 18:34:34 +08:00
Kefu Chai
7aba57b9b4
Merge pull request #18191 from hjwsm1989/osd-mark-down
qa/standalone/osd/osd-mark-down: create pool to get updated osdmap faster

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-15 11:09:02 +08:00
David Zafman
88ce0c1a91 test: Verify stat calculations during backfill
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
David Zafman
f5af1af6d3 test: Verify stat calculations during recovery
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
David Zafman
aeba36a660 ceph-helpers.sh: Add flush_pg_stats() to wait_for_clean() to make it reliable
osd-scrub-repair.sh: Fixes for omap keys landing on different OSDs due to flush

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
David Zafman
c77941f593 qa: Ignore degraded PGs when injecting random eio errors
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-01-14 18:17:23 -08:00
Sage Weil
df5a37091a
Merge pull request #19946 from liewegas/wip-22677
qa/workunits/rados/test_rados_tool: fix stray |, race
2018-01-14 13:40:54 -06:00
Sage Weil
3f922e79c3 qa/tasks/ceph: disable osd_debug_deep_scrub_sleep in case it is set
Otherwise the final scrub may take too long.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:47:24 -06:00
Sage Weil
ff9fa835da qa/workunits/rados/test_rados_tool: remove stray |, fix race
Fixes: http://tracker.ceph.com/issues/22676
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:35:14 -06:00
Victor Denisov
838cae45af rbd: Add group snapshot rename commands
Signed-off-by: Victor Denisov <denisovenator@gmail.com>
2018-01-13 14:00:31 -05:00
Sage Weil
ea2c99ec87 qa/workunits/rados/test_rados_tool.sh: -x
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-13 10:26:24 -06:00
Yuri Weinstein
a7b97806fc
Merge pull request #19793 from badone/wip-blocking-deep-scrub
ceph.in: Add blocking mode for scrub and deep-scrub

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-12 14:47:30 -08:00
Patrick Donnelly
e592c0210f
Merge PR #19885 into master
* refs/pull/19885/head:
	qa: increase osd count for ec testing
	qa: add missing openstack configs

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-12 14:37:25 -08:00
Mykola Golub
de97a3be48 rbd: adjusted "showmapped" JSON and XML formatted output
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-01-12 21:36:00 +02:00
Jason Dillaman
619b0c0244 rbd: adjusted "lock list" JSON and XML formatted output
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-12 11:31:07 -05:00
myoungwon oh
93e986c064 qa/suites/rados/thrash: add tier_promote op
1.add tier_promote op for redirect and chunked cases.
2.rename set-chunk.yaml due to current chunked object
only for the read case.

Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2018-01-12 14:38:57 +09:00
Sage Weil
c1ec2752ee
Merge pull request #19180 from theanalyst/tools/rados-omap-clear
tools: rados add a cli option to clear omap keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-11 15:50:12 -06:00
Jason Dillaman
31a382f1ec rbd: utilize '<name>@<snap>' format when creating/removing group snaps
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-11 10:38:23 -05:00
Victor Denisov
d95b791e8d qa/workunits/rbd: Add tests for consistency groups
Signed-off-by: Victor Denisov <denisovenator@gmail.com>
2018-01-11 10:38:23 -05:00
Brad Hubbard
637bf125c3 qa: Add blocking versions of scrub and deep-scrub
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-01-10 14:05:55 +10:00
Patrick Donnelly
f8c275b8b6
qa: increase osd count for ec testing
Missing in d0732fc96f.

Fixes: http://tracker.ceph.com/issues/22646
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-01-09 16:48:18 -08:00
Patrick Donnelly
e8a0f1c554
qa: add missing openstack configs
First introduced in: ec6fb28eaf

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-01-09 16:48:02 -08:00
Casey Bodley
f473d57013 librados: add async interfaces for use with Networking TS
Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-09 09:58:18 -05:00
Sage Weil
0cdb6a74e7
Merge pull request #19616 from ifed01/wip-ifed-revert-extents
osd: get rid off extent map in object_info

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-09 08:44:52 -06:00
zouaiguo
a61e62cf44 mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools.
when we create a pool specify a rule, for example "ceph osd pool create foo replicated 10 rule_foo",
we will set pool foo erasure-code-profile to rule_foo,
if there has an erasure-code-profile names rule_foo, use "ceph osd erasure-code-profile rule_foo" will fail,
"Error EBUSY: foo pool(s) are using the erasure code profile 'rule_foo'", this is wrong.

we should do:
1. set erasure-code-profile to "" when create replicated pools
2. whether erasure-code-profile is used by pool not only judge pool erasure_code_profile property and also the pool is_erasure

Signed-off-by: zouaiguo <zou.aiguo@zte.com.cn>
2018-01-09 14:14:06 +08:00
Yuri Weinstein
c30eca51a4
Merge pull request #19765 from liewegas/wip-force-backfill-cleanups
qa: tolerate failure to force backfill

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-08 12:50:18 -08:00
Igor Fedotov
1653bcca3e qa/standalone/scrub/osd-scrub-repair.sh: remove extents flag from object_info_t
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-01-08 20:10:16 +03:00
Jason Dillaman
50ef9aff9d
Merge pull request #19704 from trociny/wip-nbd-format
rbd-nbd: output format support for list-mapped command

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-07 09:04:35 -05:00
Kefu Chai
e7097593a7 qa/standalone: remove osd-map-max-advance related tests
this setting was removed in 8967b73

Fixes: http://tracker.ceph.com/issues/22596
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-06 19:40:15 +08:00
Patrick Donnelly
f6e1a797d4
Revert "Merge PR #19369 into master"
This reverts commit 3189ba19a7, reversing
changes made to b7620de020.

Despite the change in json format being positive, the unfortunate side-effect
is that it broke upgrade testing (because the QA framework must handle the
transition of mdsmap["info"] to a list from object) and the ceph-mgr.

Fixes: http://tracker.ceph.com/issues/22527
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-01-04 09:42:37 -08:00
Sage Weil
819a3578fa Merge tag 'v13.0.1' 2018-01-03 10:04:20 -06:00
Sage Weil
8651e15c93 qa/tasks/ceph_manager: tolerate failure to force backfill/recoery
The pool may have been deleted out from underneath us.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-03 08:37:02 -06:00
Mykola Golub
e372e9492b rbd-nbd: output format support for list-mapped command
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-01-03 14:44:10 +02:00
Mykola Golub
a5fd0e71d1
Merge pull request #19724 from dillaman/wip-18480
rbd: support cloning an image from a non-primary snapshot

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-01-03 14:11:25 +02:00
Jason Dillaman
e4b8ae348b
Merge pull request #19666 from trociny/wip-nbd-unmap-by-image
rbd-nbd: allow to unmap by image or snap spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-02 20:35:30 -05:00
Yuri Weinstein
10fc85089c qa/tests - Added options to use both cases: mon.a and installer.0
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-01-02 10:12:54 -08:00
Kefu Chai
0cd700e64d
Merge pull request #19634 from jcsp/wip-22514
qa: configure zabbix properly before selftest

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-03 00:09:37 +08:00
Jason Dillaman
c7c15b681c librbd: permit creation of clones against non-primary images
Fixes: http://tracker.ceph.com/issues/18480
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-02 10:11:56 -05:00
Sage Weil
6dc41a2bb1
Merge pull request #19605 from ceph/wip-use-installer0-master
qa/tests: run ceph-ansible task on installer.0 role/node
2017-12-29 10:45:06 -06:00