Commit Graph

67887 Commits

Author SHA1 Message Date
Matt Benjamin
deb2c1ea98 rgw_file: add timed namespace invalidation
With change, librgw/rgw_file consumers can provide an invalidation
callback, which is used by the library to invalidate directories
whose contents should be forgotten.

The existing RGWLib GC mechanism is being used to drive this.  New
configuration params have been added.  The main configurable is
rgw_nfs_namespace_expire_secs, the expire timeout.

Updated post Yehuda review.

Fixes: http://tracker.ceph.com/issues/18651

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-01-24 08:21:33 -05:00
Haomai Wang
b2df030a50 Merge pull request #12956 from liuhongtong/wip-nvmedevice
os/bluestore: fix NVMEDevice::open failure if serial number ends with a …

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Pan liu <liupan1111@gmail.com>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-01-24 18:07:05 +08:00
Wido den Hollander
8fe6a0303b
osd: Return correct osd_objectstore in OSD metadata
Do not simply read the configuration value as it might have changed
during OSD startup by reading the type from disk.

Fixes: http://tracker.ceph.com/issues/18638

Signed-off-by: Wido den Hollander <wido@42on.com>
2017-01-24 08:22:34 +01:00
Haomai Wang
8558a0fe52 Merge pull request #13055 from optimistyzy/2017123_random_read
os/bluestore/NVMEDevice.cc: fix the random read issue.

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-01-24 12:01:27 +08:00
optimistyzy
c02e3e5266 os/bluestore/NVMEDevice.cc: use aio_wake in both read and read_random
There are some issues with random read, it seems that
we did not call aio_wake.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-01-24 11:35:45 +08:00
Sage Weil
7eddf919b4 Merge pull request #12929 from wjwithagen/wip-wjw-empty-cluster_network
common/pick_address.cc: Copy public_netw to cluster_netw if cluster empty

Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-23 21:16:21 -06:00
Sage Weil
d0740a3514 Merge branch 'master' into wip-wjw-empty-cluster_network 2017-01-23 21:16:09 -06:00
Sage Weil
6390ea7312 os/bluestore: include logical object offset in crc error message
This is more useful to the user.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-23 18:51:06 -06:00
Yuri Weinstein
f9f9b63e09 Merge pull request #12901 from yuyuyu101/wip-msgr-client-bind
msg: client bind

Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-23 15:00:26 -08:00
Nathan Cutler
f848235d19 Merge pull request #13036 from SUSE/wip-michal-koutny
mailmap: Michal Koutny affiliation

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Michal Koutný <mkoutny@suse.com>
2017-01-23 22:25:34 +01:00
Jason Dillaman
20da91e881 Merge pull request #12326 from vshankar/wip-15028
librbd: create fewer empty objects during copyup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-01-23 10:39:16 -05:00
Sage Weil
8f5fcfd857 os/bluestore/BlueFS: tolerate longer tail
The important requirement here is that we have enough buffer to write the
whole page.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-23 10:24:40 -05:00
Sage Weil
c776710a8e os/bluestore/BlueFS: increase size threshold before we flush (and generate io)
Having this too high means you might be more bursty. In practice,
though, the commit path is doing explicit syncs on small chunks
anyway.  And compaction work should probably stay reasonably chunky.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-23 10:21:09 -05:00
Sage Weil
7a53d4389e os/bluestore/BlueFS: preallocate larger write buffer
Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-23 10:21:09 -05:00
Ramesh Chander
506a06a71e unittest_bit_alloc race fix
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-23 10:19:48 -05:00
Haomai Wang
2b2ab914b2 Merge pull request #13056 from optimistyzy/123_logic
NVMEDevice.cc: cleanup the logic in data_buf_next_sge

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-01-23 22:44:07 +08:00
Kefu Chai
1e8ca9bb90 Merge pull request #12938 from LiumxNL/fix-process-osd-failure
mon/OSDMonitor: fix process osd failure

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-23 20:07:35 +08:00
Kefu Chai
4f13f3e9b5 Merge pull request #9419 from songbaisen/a4
osdmaptool: show all the pg map to osds info

Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-23 19:43:35 +08:00
Kefu Chai
d54f75d244 Merge pull request #12828 from donglinpeng/dlppull
common/buffer: close pipe fd if set nonblocking fails.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-23 19:42:14 +08:00
optimistyzy
edb0e7d4c6 NVMEDevice.cc: cleanup the logic in data_buf_next_sge
Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-01-23 11:00:02 +08:00
Sage Weil
65f5187e42 Merge pull request #12980 from liewegas/wip-osdop-printer
osd/osd_types: clean up OSDOp printers

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-22 10:45:37 -06:00
Sage Weil
1bda01736f osd/osd_types: clean up OSDOp printers
Specify which ones get offset~length instead of blacklisting those that
don't.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-22 11:44:46 -05:00
Sage Weil
55b96a3b65 Merge pull request #12950 from liewegas/wip-omap-getkeys
osd: extend OMAP_GETKEYS and GETVALS to include a 'more' output field

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-01-22 10:44:04 -06:00
Sage Weil
9a6a276aa2 Merge pull request #12961 from songbaisen/s10
crush:  when osd_location_hook does not exist, we should exit error.


Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-22 10:43:27 -06:00
Sage Weil
5dccac8697 Merge pull request #12975 from wonzhq/fix-pgstats-down
mon: don't set last_osd_report when the pg stats msg is ignored

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-22 10:43:00 -06:00
Sage Weil
c0b4ced7b1 Merge pull request #12976 from LiumxNL/wip-170118
osd/OSDMap: get_previous_up_osd_before() may run into endless loop

Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-22 10:42:33 -06:00
Sage Weil
4bad4bd86a Merge pull request #12981 from liewegas/wip-crush-move-osd
mon/OSDMonitor: make 'osd crush move ...' work on osds

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-22 10:42:08 -06:00
Sage Weil
7495487e56 Merge pull request #13008 from athanatos/wip-clone-range
osd,librados: remove clone_range and associated multi-object cruft

Reviewed-by: Sage Weil <sage@redhat.com>
2017-01-22 10:41:55 -06:00
Mykola Golub
fce69f6348 Merge pull request #13033 from dillaman/wip-18618
journal: don't hold future lock during assignment

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-01-22 14:52:32 +02:00
Hongtong Liu
97aa6ec290 bluestore: Release segs before delete task when write fails
Fix assert failure in ~Task().

Signed-off-by: Hongtong Liu <hongtong.liu@istuary.com>
2017-01-22 17:26:33 +08:00
Hongtong Liu
9caff5e455 os/bluestore: fix NVMEDevice::open failure if serial number ends with a number
buf in effect is the serial number in ceph.conf and
the serial number consists of 16 hexadecimal characters.
1. In order to avoid ignoring the numbers, scan buf
with isxdigit.
2. In order to ignore all the potential garbage,
scan buf from the beginning.

Signed-off-by: Hongtong Liu <hongtong.liu@istuary.com>
2017-01-22 17:25:04 +08:00
Kefu Chai
5647d5ee8e common/BackTrace: demangle on FreeBSD also
the output on FreeBSD/clang looks like:

1: 0x44bfb3 <_Z3foov+0x413> at /usr/srcs/Ceph/work/ceph/build/bin/unittest_back_trace
2: 0x44c23e <_ZN20BackTrace_Basic_Test8TestBodyEv+0x1e> at /usr/srcs/Ceph/work/ceph/build/bin/unittest_back_trace
3: 0x4d068a <_ZN7testing8internal38HandleSehExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc+0x7a> at /usr/srcs/Ceph/work/ceph/build/bin/unittest_back_trace
4: 0x4b5977 <_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc+0x77> at /usr/srcs/Ceph/work/ceph/build/bin/unittest_back_trace
...

and update the test accordingly, as FreeBSD/clang uses '<>' to enclose
the mangled function and offset.

also, only demangle the C++ mangled names. those names always start with
"_Z". on FreeBSD, after demangling, "main" is turned into "unsigned
long", which does not make sense.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-01-22 16:14:56 +08:00
Xie Xingguo
24413051d6 Merge pull request #13051 from chuanhong-wang/dev_conf_clean
common: delete unused conf "filestore_debug_disable_sharded_check"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-22 13:35:50 +08:00
chuanhong.wang
57c1542ed4 common: delete unused conf "filestore_debug_disable_sharded_check"
Signed-off-by: Chuanhong Wang <chuanhong.wang@163.com>
2017-01-22 12:23:40 +08:00
Mingxin Liu
0de7ac79b5 test: cannot create pool snap on tier
Signed-off-by: Mingxin Liu <mingxin@xsky.com>
2017-01-22 01:26:43 +08:00
Mingxin Liu
4ea68a4bf3 OSDMonitor: make pool snaps on cache tier should be not allowed
Signed-off-by: Mingxin Liu <mingxin@xsky.com>
2017-01-22 01:26:34 +08:00
Mykola Golub
11dce465f0 Merge pull request #13042 from dillaman/wip-librbd-compile-warning
librbd: fixed initializer list ordering

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-01-21 17:52:42 +02:00
Jason Dillaman
0f21ceef83 journal: don't hold future lock during assignment
It's possible that the future raced with its owner and reaches
an empty reference count. This was resulting in the future being
destructed while its lock was still held.

Fixes: http://tracker.ceph.com/issues/18618
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-01-21 09:48:14 -05:00
Jason Dillaman
4da2326252 librbd: fixed initializer list ordering
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-01-21 09:17:36 -05:00
Venky Shankar
ac4e6cc2b6 test / librbd: create few empty objects during flatten
Fixes: http://tracker.ceph.com/issues/15028
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2017-01-21 16:29:27 +05:30
Venky Shankar
8419036659 test / librbd: Write ones instead of zeroes to test images
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2017-01-21 16:29:27 +05:30
Venky Shankar
c455711c2f librbd: Create few empty objects during copyup
This is based out of Doug's (@fullerdj) work (PR #9329)
as an attempt to avoid creating empty objects when
flattening an image and otherwise whenever unnecessary.

This gives good optimization benefit when a parent image
is sparsely populated. Moreover, this change is required
for correct behavior when checking disk usage of a clone
(which used to report fully allocated image due to all,
including empty objects being created during flatten).

Signed-off-by: Douglas Fuller dfuller@redhat.com
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2017-01-21 16:29:27 +05:30
Venky Shankar
ad0dcff47f librbd: make has_parent() prone to callers from copyup
This is required when CopyupRequest would need to invoke
pre_object_map_update() as part of upcoming changes to
create fewer child image objects whenever possible.

CopyupRequest constructor accepts image extents as an
rvalue forcing the caller to transfer ownership to it
and leaving the original variable in an unspecified
stated making has_parent() return incorrect state when
invoked from CopyupRequest. Therefore, introduce a
private tracking state that can be used in place of
checking emptiness of parent image extents.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2017-01-21 16:29:27 +05:30
Sage Weil
1f150f8a52 PendingReleaseNotes: mention old clients vs new OSDs
Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:18 -05:00
Sage Weil
2d9cfadf71 rados: use bare omap_get_keys op
This handles the client-side looping on 'more' if the OSD limits
the response size.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:18 -05:00
Sage Weil
faec324f64 ceph_test_rados: pass nullptr for pmore for omap fetch
Assume OSD limits are hugh enough.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:18 -05:00
Sage Weil
35671cedb2 rados: omap bench: pass null for pmore for omap fetch
Assume OSD limits are high enough for us.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:18 -05:00
Sage Weil
b9f4194306 ceph_test_rados_*: use new omap methods
Make these tests pass nullptr for pmore.  This codifies our
assumption that the OSD limits are high enough.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:18 -05:00
Sage Weil
fbebd83537 librados: add omap_get_{keys,vals}2 with pmore output arg
Expose public methods that include a new output argument to indicate
whether there are more keys to fetch or not.

Mark the old interfaces deprecated.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:17 -05:00
Sage Weil
cfa6f7932e osdc/Objecter: infer ptruncated on old OSDs via max_entries
If we do not get an explicit 'more' value from the OSD, infer it by
checking whether we got the max requested entries.  On old OSDs, which
don't enforce a limit, this will work.  On new OSDs, we will get the
explicit result.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-01-20 17:52:17 -05:00