Commit Graph

1765 Commits

Author SHA1 Message Date
Sage Weil
13738cd752
Merge pull request #19939 from liewegas/wip-osd-new-class
mon,osd: do not use crush_device_class file to initalize class for new osds

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-02-12 11:17:14 -06:00
Jason Dillaman
82ef376e80 librbd: auto-remove trash snapshots when image is deleted
Fixes: http://tracker.ceph.com/issues/22873
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-09 13:34:55 -05:00
Mykola Golub
57baca0166 qa/workunits/rbd/cli_generic: fix test_purge for rbd default format 1
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-02-09 16:04:43 +02:00
songweibin
21c3bc3e69 test/rbd: cli_generic fails if v1 image format or deep-flatten disabled
Fixes: http://tracker.ceph.com/issues/22950

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-02-09 16:17:27 +08:00
Sage Weil
59bdada5d5 mon/OSDMonitor: accept crush_device_class in 'osd new' json input
If provided, set the OSD device_class at OSD creation time.  This is
simpler than writing a file that the OSD has to read in and use to
set its initial device class, and also avoids a bit of sticky state
at the OSD that will make it keep trying to reset its device class on
startup if it ever gets cleared.

Note that we now ignore json input fields we don't understand, so remove
a test case.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-08 12:34:49 -06:00
Josh Durgin
eae6e64034
Merge pull request #20173 from liewegas/wip-os-ch
os: unify Sequencer and CollectionHandle

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-02-07 14:04:08 -08:00
Sage Weil
5749434ad6 test/bench: remove smallio bench tools
These are stale and superceded by the fio ObjectStore (and other)
backends.  Shed the weight!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-06 08:21:29 -06:00
Jason Dillaman
57559a47f6 qa/workunits/rbd: force use of v1 clone when testing orphan
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-05 11:12:00 -05:00
Jason Dillaman
3ac57a7eb7
Merge pull request #19711 from trociny/wip-20762
rbd: unified way to map images using different drivers

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-02-02 13:56:13 -05:00
Jason Dillaman
bba63db1b4
Merge pull request #20170 from trociny/wip-22803
qa/workunits/rbd: remove "trash purge --threshold" test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-02-02 08:32:36 -05:00
Mykola Golub
be60490f15 qa/workunits/rbd: remove "trash purge --threshold" test
It is difficult to make it work reliably in different environments.

Fixes: http://tracker.ceph.com/issues/22803
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-02-01 17:36:21 +02:00
songweibin
c3e119d0d3 librbd: fix deep copy a child-image
* tweak create a cloned image when the source image is
  a clone (or at least one of its snapshots is a clone).

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-02-01 08:17:19 +08:00
Mykola Golub
6a57358add rbd: unified way to map images using different drivers
Instead of "rbd map|unmap|showmapped", "rbd ndb map|unmap|list",
"rbd ggate map|unmap|list" commands, provide:

 rbd device -t krbd|nbd|ggate|... map|unmap|list

which gives interface consistent between drivers.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-01-29 09:34:11 +02:00
Mykola Golub
023759c7c1
Merge pull request #20124 from dillaman/wip-devstack
qa/workunits/rbd: switch devstack to pike release

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-01-26 07:41:00 +02:00
Jason Dillaman
6253490445
Merge pull request #19996 from Songweibin/wip-rbd-deep-cp
rbd: add deep cp CLI method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-25 21:59:31 -05:00
Jason Dillaman
7ff1e0bc21 qa/workunits/rbd: switch devstack to pike release
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-25 18:44:20 -05:00
songweibin
0551029242 rbd: add deep cp CLI method
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-01-24 09:04:33 +08:00
Theofilos Mouratidis
b9e4aa94b7 rbd trash: replace cli delay option, add rbd trash purge command
Replaced the delay argument for the trash move
command with a string acceptable by /bin/date, e.g.:

$ rbd trash move --pool foo --image bar --expires-in "2 weeks"

Added a "rbd trash purge" command that deletes any expired
image from the trash, has also a command to alter the current
expiration date with the "--older-than" argument which accepts
again a valid argument for /bin/date, e.g.:

$rbd trash purge mypool --older-than "2017-08-20"

There is also the "threshold" argument which tries to remove the
oldest trashed images (by deferment end time) until the pool space
is freed up to a percentage point, e.g.:

$ rbd trash purge mypool --threshold 0.9

If mypool uses 1GB it will try to remove trashed images until the
pool usage becomes equal to or lower than 900MB.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
2018-01-23 11:41:54 +01:00
Kefu Chai
74c9cfc466 test: reduce the chance to have degraded PGs
in test_mon_osd_misc(), there is good chance that the cluster chooses
to use an unbalanced weight because of the data distribution at that moment.
but this setting could prevent the CRUSH from choosing enough number of
OSDs for test_mon_cephdf_commands(), where 32 PGs are to be created. so
it's more likely that the CRUSH fails to pick enough OSDs for all PGs.
that's why we have a curr_object_copies_rate = 0.5.

so, in this change, pg=pgp=1 is specified for the new pool.

Fixes: http://tracker.ceph.com/issues/22711
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-01-22 21:24:58 +08:00
Kefu Chai
0c38eb87e2
Merge pull request #19673 from zouaiguo/wip_erase_code
mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools.

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-15 18:34:34 +08:00
Sage Weil
df5a37091a
Merge pull request #19946 from liewegas/wip-22677
qa/workunits/rados/test_rados_tool: fix stray |, race
2018-01-14 13:40:54 -06:00
Sage Weil
ff9fa835da qa/workunits/rados/test_rados_tool: remove stray |, fix race
Fixes: http://tracker.ceph.com/issues/22676
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:35:14 -06:00
Victor Denisov
838cae45af rbd: Add group snapshot rename commands
Signed-off-by: Victor Denisov <denisovenator@gmail.com>
2018-01-13 14:00:31 -05:00
Sage Weil
ea2c99ec87 qa/workunits/rados/test_rados_tool.sh: -x
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-13 10:26:24 -06:00
Yuri Weinstein
a7b97806fc
Merge pull request #19793 from badone/wip-blocking-deep-scrub
ceph.in: Add blocking mode for scrub and deep-scrub

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-01-12 14:47:30 -08:00
Jason Dillaman
619b0c0244 rbd: adjusted "lock list" JSON and XML formatted output
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-12 11:31:07 -05:00
Sage Weil
c1ec2752ee
Merge pull request #19180 from theanalyst/tools/rados-omap-clear
tools: rados add a cli option to clear omap keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-11 15:50:12 -06:00
Jason Dillaman
31a382f1ec rbd: utilize '<name>@<snap>' format when creating/removing group snaps
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-11 10:38:23 -05:00
Victor Denisov
d95b791e8d qa/workunits/rbd: Add tests for consistency groups
Signed-off-by: Victor Denisov <denisovenator@gmail.com>
2018-01-11 10:38:23 -05:00
Brad Hubbard
637bf125c3 qa: Add blocking versions of scrub and deep-scrub
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-01-10 14:05:55 +10:00
Casey Bodley
f473d57013 librados: add async interfaces for use with Networking TS
Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-09 09:58:18 -05:00
zouaiguo
a61e62cf44 mon/OSDMonitor.cc : set erasure-code-profile to "" when create replicated pools.
when we create a pool specify a rule, for example "ceph osd pool create foo replicated 10 rule_foo",
we will set pool foo erasure-code-profile to rule_foo,
if there has an erasure-code-profile names rule_foo, use "ceph osd erasure-code-profile rule_foo" will fail,
"Error EBUSY: foo pool(s) are using the erasure code profile 'rule_foo'", this is wrong.

we should do:
1. set erasure-code-profile to "" when create replicated pools
2. whether erasure-code-profile is used by pool not only judge pool erasure_code_profile property and also the pool is_erasure

Signed-off-by: zouaiguo <zou.aiguo@zte.com.cn>
2018-01-09 14:14:06 +08:00
Mykola Golub
e372e9492b rbd-nbd: output format support for list-mapped command
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-01-03 14:44:10 +02:00
Mykola Golub
a5fd0e71d1
Merge pull request #19724 from dillaman/wip-18480
rbd: support cloning an image from a non-primary snapshot

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-01-03 14:11:25 +02:00
Jason Dillaman
e4b8ae348b
Merge pull request #19666 from trociny/wip-nbd-unmap-by-image
rbd-nbd: allow to unmap by image or snap spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-02 20:35:30 -05:00
Jason Dillaman
c7c15b681c librbd: permit creation of clones against non-primary images
Fixes: http://tracker.ceph.com/issues/18480
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-01-02 10:11:56 -05:00
Mykola Golub
be87e8b118 rbd-ggate: allow to unmap by image or snap spec
(to match the behaviour of krbd unmap)

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-12-27 20:29:58 +00:00
Mykola Golub
8e0f40aae1 rbd-ggate: extended info when listing devices
This matches nbd and krbd drivers output.

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-12-27 20:29:58 +00:00
Mykola Golub
02e211c282 rbd-nbd: allow to unmap by image or snap spec
(to match the behaviour of krbd unmap)

Signed-off-by: Mykola Golub <mgolub@suse.com>
2017-12-25 17:19:49 +02:00
Mykola Golub
47e38ca1b9 qa/workunits/rbd: bench-write is part of exclusive option test
Signed-off-by: Mykola Golub <mgolub@suse.com>
2017-12-25 17:19:49 +02:00
Jason Dillaman
8c2ee6dd0f qa/workunits/rbd: simplify split-brain test to avoid potential race
Fixes: http://tracker.ceph.com/issues/22485
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-12-19 16:30:25 -05:00
Abhishek Lekshmanan
83631d9bd8 qa: add test for clearomap cli option
Fixes:http://tracker.ceph.com/issues/22255
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-12-05 11:06:22 +01:00
John Spray
967fc9cf8a
Merge pull request #18649 from b-ranto/wip-restful-fixes
mgr/restful: A couple of restful fixes

Reviewed-by: John Spray <john.spray@redhat.com>
2017-11-29 12:11:58 +00:00
Kefu Chai
b3c159e9fb qa/ceph-disk: enlarge the simulated SCSI disk
100MB will be allocated for journal, and the remaining 100MB is for data
device. taking the inode into consideration, there will be approximately
87988 kB available for the activated OSD. and it will complain with a
"nearfull" state.

Fixes: http://tracker.ceph.com/issues/22136
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-28 14:55:33 +08:00
Kefu Chai
aa34321a4a qa/workunits/ceph-disk: do not redirect stderr to stdout
normally, if we care about the output of ceph-disk, we expect a json
string, and ceph-disk sends the output to stdout, and errors/warnings
to stderr. so everything works as expected. and the test should also
follow this tradition. for example, if deprecated warnings are printed,
the warning message should not be collected along with the json string.

see also: d44334f3

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-28 14:55:19 +08:00
Boris Ranto
059588e758 qa/restful: Test pg_num/pgp_num modifications
Signed-off-by: Boris Ranto <branto@redhat.com>
2017-11-27 14:35:26 +01:00
Mykola Golub
7fb9920316 qa/workunits/rbd: wait for status propagated only if daemon started
Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-11-21 22:29:16 +02:00
Jason Dillaman
0502cb1573
Merge pull request #19073 from trociny/wip-qa-demote
qa/workunits/rbd: wait for demote status is propagated

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-11-21 12:52:08 -05:00
Kefu Chai
d44334f317 qa/workunits: silence py warnings for ceph-disk tests
ceph-disk now prints "depreacted" warning message when it starts. but
the tests parses its stdout and stderr for a json string. so we need to
silence the warnings for the tests.

Fixes: http://tracker.ceph.com/issues/22154
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-11-21 21:49:48 +08:00
Mykola Golub
f4b88b2c9d qa/workunits/rbd: wait for demote status is propagated
(before promoting the image on another cluster)

Signed-off-by: Mykola Golub <to.my.trociny@gmail.com>
2017-11-21 13:22:50 +02:00