Commit Graph

50483 Commits

Author SHA1 Message Date
Loic Dachary
94c5df4567 tests: verify ceph-disk lockbox activation scenarii
Simulate the cases where the activation (via udev running trigger)
sequences are:

* journal then lockbox
* data then lockbox
* lockbox

All of them must end with the OSD verfied to be up.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
0b4ab0d329 ceph-disk: improve trigger verbosity
The ceph-disk activate errors were ignored and not displayed. Capture
stdout/stderr and display it if the exit code is non zero. Also fail
when an activate fails.

Pass the --verbose flag to activate, if given to trigger.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
ed56cef1f1 tests: make ceph-disk workunit resilient to non ascii
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
48c21ced5f ceph-disk: implement deactivate --once
It is mostly useful for test purposes.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
869eb3af80 tests: verify ceph-disk activate-lockbox
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
b36859ce72 ceph-disk: protect list with activate lock
list may try to mount partitions to figure out the OSD id and other
details. If it does so while the OSD is activated, it will race and lead
to errors, either for activation or for list.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
4d42a56797 ceph-disk: implement list for lockbox
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
d355a383d1 doc: update ceph-disk to refer to ceph-disk --help
The ceph-disk page is often obsolete, mostly because maintaining
it requires a significant amount of copy/paste and re-formatting.
Now that the --help of ceph-disk has been updated to include a more
verbose explanation of each subcommand, simplify the man page to
give an overview of the subcommands and suggest the --help is used
to get more information.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
67ef329af4 ceph-disk: add a description for each subcommand
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
1ec58fcfc8 ceph-disk: implement lockbox key management
Instead of storing the dmcrypt keys in the /etc/ceph/dmcrypt-keys
directory, they are stored in the monitor. If a machine with
OSDs created with ceph-disk prepare --dmcrypt is lost, it does
not contain the key that would allow to decrypt their content.

The dmcrypt key is retrieved from the monitor using a different keyring
for each OSD. It is stored in a small partition called the lockbox. At
boot time the lockbox is mounted

    /var/lib/ceph/osd-lockbox/$uuid

and used when the $uuid partition is detected by udev to map it with
cryptsetup.

The OSDs that were prepared prior to the lockbox implementation are
supported by looking up the key found in /etc/ceph/dmcrypt-keys before
looking in /var/lib/ceph/osd-lockbox/$uuid.

http://tracker.ceph.com/issues/14669 Fixes: #14669

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
bd4bd5c65e ceph-disk: simplify trigger
The ceph-disk trigger deals with dmcrypt mapping which is redundant with
what ceph-disk activate-* does when the --dmcrypt flag is set. Remove
the dmcrypt mapping code and add the --dmcrypt flag to ceph-disk
activate-* where relevant.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
67f9e8dace ceph-disk: re-using an OSD partition bugous test
When re-using an OSD partition, the test of the partition type is done
against a non-existent variable.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
15db7feb8d ceph-disk: make dmcrypt_unmap idempotent
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
977bf038a4 ceph-disk: destroy does not unmap block/journal
The dmcrypt boolean needs to be set for destroy to ensure the
journal/block partitions are 'cryptsetup remove'.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
0186062d29 ceph-disk: display the function name with the log messages
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Yehuda Sadeh
155a53d649 Merge pull request #7786 from ceph/wip-rgw-indexless
rgw: indexless buckets (Yehuda Sadeh)

 - can define a policy, for which buckets are indexless
 - users can then create buckets under the specified placement target
 - indexless buckets will not be synced across zones
 - does not work with (s3) versioned buckets

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-03-03 14:18:25 -08:00
Yehuda Sadeh
b7d022f61a rgw: indexless buckets
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-03-03 14:06:19 -08:00
Yehuda Sadeh
f1ac0de5bf rgw: configurable index type
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-03-03 14:04:19 -08:00
Yehuda Sadeh
bc60ba3ea9 Merge pull request #7764 from rzarzynski/wip-rgw-etag-dlo
rgw: multiple improvements regarding etag calculation for SLO/DLO of Swift API.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-03-03 13:43:27 -08:00
Sage Weil
be3544f162 Merge remote-tracking branch 'gh/jewel' 2016-03-03 13:35:13 -05:00
Sage Weil
ea45099808 v10.0.4
Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-03 13:34:46 -05:00
Loic Dachary
3a1b307d4d Merge pull request #7897 from branch-predictor/bp-bench-errors-fatal
common/obj_bencher.cc: make verify error fatal

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-03-03 23:15:45 +07:00
Piotr Dałek
658bba85c4 common/obj_bencher.cc: make verify error fatal
When run without "--no-verify", all verification errors are noted,
but they are not forwarded/reported anywhere else but to cerr, which
will cause automated testing to ignore them. Make seq_read_bench and
rand_read_bench return -EIO on any verification error which will,
in turn, return it back to caller.

Fixes: #14971
Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-03-03 16:04:12 +01:00
Sage Weil
799ea24b42 Merge pull request #7757 from ukernel/jewel-14684
mon/MDSMonitor.cc: properly note beacon when health metrics changes

Reviewed-by: John Spray <john.spray@redhat.com>
2016-03-03 09:37:50 -05:00
Sage Weil
9483cae681 Merge pull request #7882 from branch-predictor/bp-unconfuse-health-1
mon: unconfuse object count skew message

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-03 09:15:29 -05:00
Piotr Dałek
f792a3f613 PGMonitor: unconfuse object count skew message
"Pool <pool> has too few pgs" is okay assuming it does not take other
pools into account. And since it does, it is confusing in the following
scenario:

1. Create two pools, one with small pg count and one with large
   pg count
2. Put a whole lot of objects in smaller pool, resulting in "too few
   pgs" warning on that pool, which is expected behavior.
3. Put a whole lot of objects in larger pool, warning goes away.
   Suddenly smaller pool has plenty of PGs?

Current message suggests adding more nodes (or PGs) to pool, when
actually it's warning about significantly more objects in that
particular pool than in the other pools.

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-03-03 15:01:56 +01:00
John Spray
353edf44d1 Merge pull request #7898 from oritwas/wip-cmake-add-fs_types
cmake: add common/fs_types.cc to libcommon

Reviewed-by: John Spray <john.spray@redhat.com>
2016-03-03 13:54:25 +00:00
Sage Weil
8ce41ccc91 Merge pull request #7881 from aclamk/ceph-df-used-fix
mon: fix calculation of %USED

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-03 08:51:50 -05:00
Sage Weil
9127a4c707 Merge pull request #7892 from majianpeng/bluestore-leak-unwritten-extent
osd: bluestore: fix check for write falling within the same extent

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-03 08:49:17 -05:00
Sage Weil
a9db6ee7e5 Merge pull request #7886 from stiopaa1/blueStoreOverrides
osd: bluestore: add 'override' to virtual functions

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-03 08:41:21 -05:00
Sage Weil
84747dcbf4 Merge pull request #7895 from majianpeng/bluestore-trim-overlay
os/bluestore/BlueStore: Don't leak trim overlay data before write.
2016-03-03 08:40:21 -05:00
Jianpeng Ma
fcc459c8af os/bluestore/BlueStore: Don't leak trim overlay data before write.
Suppose: bluestore_overlay_max_length=bluestore_min_alloc_size;
bluestore_overlay_max = 2;
For the following ops:
write(off=0, len=4096) --->write into overlay
write(off=4096, len=4096)-->write into overlay
write(off=0, len=bluestore_min_alloc_size)-->because overlay_map.size()
>=2, it allocate a extent.

It should trim overlay data(0,4096) &(4096, 4096),and then write(0,
bluestore_min_alloc_size).
But the original code don't trim overlay data.
This make the later read data is orignal data rather that new data.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-03-03 21:46:55 +08:00
Orit Wasserman
a7a65e7add cmake: add common/fs_types.cc to libcommon
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2016-03-03 14:17:21 +01:00
Jianpeng Ma
b0b4b6de36 os/bluestore/BlueStore: Fix bug when calc offset & end whether locate in the a extent.
Suppose: bluestore_overlay_max_length == bluestore_min_alloc_size
The orignal code which calc content of written whether locate in a
extent:
(offset / min_alloc_size) == (offset + length) /min_alloc_size
This will make the case which offset=0 & length =min_alloc_size locate
in the different extent.
In fact, this content is in the same extent.

Change end = offset + length - 1 make work.

Fixes: #14954

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-03-03 18:58:04 +08:00
Piotr Dałek
cd4961581a common/obj_bencher.cc: use more readable constant instead of magic number
When clean_up_slow() fails, it returns "-5" which is equal to -EIO.
Change it in source, so it's not confusing for someone who does not
remember all error codes (functionality remains the same).

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-03-03 11:22:57 +01:00
Gregory Farnum
22417925fe Merge pull request #7098 from liewegas/wip-file-layout
mds, client: add namespace to file_layout_t (previously ceph_file_layout)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-03-02 19:15:18 -08:00
Michal Jarzabek
83e77cf165 Bluestore.h:Added 'override' to virtual functions
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-03-02 22:14:30 +00:00
Casey Bodley
045ad3d2a5 Merge pull request #7849 from ceph/wip-cmake
Wip cmake
2016-03-02 14:08:53 -05:00
Ali Maredia
d8cfef9d30 cmake: simplified heap_profiler_objs dependencies
The heap_profiler object library only needs to be
added to mon,mds, and osd instead of the numerous
targets it is part of

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-03-02 13:37:40 -05:00
Sage Weil
3e41cfde77 client/Inode: fix has_dir_layout
Compare to default file_layout_t() -- not zeroed memory.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-02 12:29:12 -05:00
branto1
fb1b291ef6 Merge pull request #7867 from SUSE/wip-14940
RPM: move %post(un) ldconfig calls to ceph-base

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
2016-03-02 18:26:13 +01:00
Orit Wasserman
e09863b494 Merge pull request #7869 from rzarzynski/wip-rgw-aws4-puthash
rgw: calculate payload hash in RGWPutObj_ObjStore only when necessary.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-03-02 18:08:48 +01:00
Sage Weil
2786f0bf11 Merge remote-tracking branch 'gh/jewel' 2016-03-02 09:22:28 -05:00
Sage Weil
67696f0bec Merge pull request #7612 from liewegas/wip-buffer-combined
buffer: raw_combined allocations buffer and ref count together

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-03-02 08:31:08 -05:00
Sage Weil
c4f68ade15 Merge pull request #7690 from xiexingguo/xxg-wip-14804
common/TrackedOp: fix inaccurate counting for slow requests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-03-02 08:30:26 -05:00
Sage Weil
b7a3217b45 Merge pull request #7704 from xiexingguo/xxg-wip-14814
osd: filestore: fix wrong scope of result code for error cases during mkfs

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-02 08:29:54 -05:00
Sage Weil
c25a8a22a7 Merge pull request #7812 from xiexingguo/xxg-wip-14872
osd: fix wrong return type of find_osd_on_ip()

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-02 08:29:09 -05:00
Haomai Wang
8889d0c7da Merge pull request #7815 from branch-predictor/bp-conditional-crc
msg/async: don't calculate msg header crc when not needed

Reviewed-by: Haomai Wang <haomai@xsky.com>
2016-03-02 21:28:40 +08:00
Sage Weil
1137c4ea8c Merge pull request #7711 from xiexingguo/xxg-wip-14817
osd: filestore: fix result code overwritten for clone

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-02 08:28:03 -05:00
Sage Weil
9576b0c22b Merge pull request #7790 from tchaikov/wip-log-segv
log: segv in a portable way

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2016-03-02 08:27:36 -05:00