Commit Graph

53032 Commits

Author SHA1 Message Date
Jianpeng Ma
c82e08692e common/buffer: Change rebuild_aligned_* & rebuild_page_aligned API.
Make those func return value from void to bool. Using the return value
we can know whether really rebuild content in order to optimize .

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-05-08 06:41:24 +08:00
Jianpeng Ma
3161ab1c07 os/bluestore/BlueFS: remove the duplicate bufferlist.rebuild().
In the later BlockDevice::aio_write(), it will rebuild again if
bufferlist need rebuild.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-05-08 06:41:21 +08:00
Sage Weil
646a6a28c3 Merge pull request #8912 from markhpc/wip-promote-throttle-fix
osd: fix reversed promote throttle default parameters

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-05-06 15:13:56 -04:00
Sage Weil
cdae4a3011 Merge pull request #8902 from xiexingguo/xxg-wip-cheanup-osdmon
mon/OSDMonitor: kill dead code

Reviewed-by: Sage Weil <sage@redhat.com>
2016-05-06 15:13:26 -04:00
Sage Weil
83ae5d6ab3 Merge pull request #8290 from yangdongsheng/chain_xattr
Fix unittest for chain_xattr.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2016-05-06 15:13:01 -04:00
Sage Weil
702f523228 Merge pull request #8173 from ceph/buffer-api-visibility
buffer: fix iterator_impl visibility through typedef

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-05-06 15:12:08 -04:00
Sage Weil
a99fae9dc1 Merge pull request #8289 from majianpeng/bluestore-zero-clear-tail-cache
os/bluestore/BlueStore: For _zero, check whether clear tail_bl.

Reviewed-by: Sage Weil <sage@redhat.com>
2016-05-06 15:11:31 -04:00
Samuel Just
b158fa6f5a Merge pull request #8977 from liewegas/wip-ceph-tool
qa/workunits/rados/test_rados_tool: fix copy paste error

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-05-06 12:01:35 -07:00
Gregory Farnum
8cbfb00d4c Merge pull request #8388 from dreamhost/wip-buildfix-mds-test
mds: build fix for LIBMDS consumer.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-05-06 11:57:44 -07:00
Sage Weil
7f1bc1b038 qa/workunits/rados/test_rados_tool: fix copy paste error
Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 14:54:41 -04:00
Jason Dillaman
a122569dfd tests/rbd: add option to leave behind fsx test data
tests/rbd: add option to leave behind fsx test data

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-05-06 13:42:04 -04:00
Douglas Fuller
c33d0cbcd0 tests/rbd: add option to leave behind fsx test data
Previously, fsx deleted all test data upon successful completion. Add
an option to leave the data behind for further analysis.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2016-05-06 10:36:00 -07:00
Kefu Chai
73e22784dd Merge pull request #8936 from bstillwell/manual-install-doc-fixes1
doc: install/manual-deployment: use sudo on some commands

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-05-07 00:55:21 +08:00
Sage Weil
9fb8b103de Merge pull request #8950 from stiopaa1/osd_passSharedByRef_shouldsharemap
osd/OSD: pass shared pointer by const reference

Reviewed-by: Sage Weil <sage@redhat.com>
2016-05-06 11:49:36 -04:00
Kefu Chai
80b6660abd Merge pull request #8962 from beess/doc-fix1
doc: Fixes three hyphen errors and a syntax error.

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-05-06 22:03:11 +08:00
Sage Weil
c73b2f408c Merge pull request #8934 from liewegas/wip-bluestore-extent-flags
os/bluestore: drop transient extent flags

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-05-06 09:25:11 -04:00
Sage Weil
4e534ab8cf Merge pull request #8607 from ceph/wip-15372
msg/async: add missing DelayedDelivery and delay injection
2016-05-06 08:21:14 -04:00
Boris Ranto
3637676bfa Merge branch 'sharkcz-optional-tcmalloc'
Reviewed-by: Boris Ranto <branto@redhat.com>
2016-05-06 13:46:25 +02:00
Dan Horák
efa7f7b365 fix tcmalloc handling in spec file
- there is no gperftools/tcmalloc on s390(x) yet
- explicitly disable tcmalloc when built without

Signed-off-by: Dan Horák <dan@danny.cz>
2016-05-06 13:45:30 +02:00
xie xingguo
4c58365ca3 doc/rgw: fix hyperlinks which can't work
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-05-06 13:44:19 +02:00
xie xingguo
0210745309 doc/rgw: hyperlink of "Ceph Object Gateway" doesn't work
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-05-06 13:44:19 +02:00
Brad Hubbard
f8f260133b cmake: Remove redundant call to get_processors
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-05-06 13:44:19 +02:00
Sage Weil
efedfffe04 osd: remove useless set_role from split completion
This is done by PG::split_into().

Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:19 +02:00
Sage Weil
c8be0f7129 osd: respect shard when setting role on new pgs
This matches PG::start_peering_interval().

Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:18 +02:00
Sage Weil
33a2e3797d osd: pay attention to shard when setting role
This matches start_peering_interval().

Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:18 +02:00
Chendi.Xue
dbe0e470de create and check omap fsid
1. write osd_uuid to omap dir when doing filestore mkfs
2. check if omap fsid matches osd fsid when doing filestore mount
   (if there is no osd_uuid under omap, assume this as match)

Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
2016-05-06 13:44:18 +02:00
Chendi Xue
9a5109a6ab Added option to deploy omap to a separate device(path)
In the HDD as OSD, SSD as journal test, we saw a great
throughput improvement if moving omap to a SSD device at
randwrite case.

This patch aim to add a config option 'filestore_omap_backend_path'
for users to configurate omap path before deploy.

Signed-off-by: Chendi Xue <chendi.xue@intel.com>
2016-05-06 13:44:18 +02:00
Igor Fedotov
520ef71e0a ObjctStore: More accurate disk space usage calculation when considering 'whiteout'/empty/small objects.
This also adds cache protection against potential overflow with removed objects

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-05-06 13:44:18 +02:00
Yan, Zheng
63e44e3297 msg/simple/Pipe: avoid infinite loop in Pipe::do_recv()
I found that, sometimes an OSD thread uses 100% CPU after cutting network
between OSD and client. recv(2) in Pipe::do_recv() keeps returning -EAGAIN,
which causes infinite loop. the call trace is:

 Pipe::do_recv (...)
 Pipe::buffered_recv (...)
 Pipe::tcp_read_nonblocking (...)
 Pipe::tcp_read (...)

Pipe::tcp_read() first calls Pipe::tcp_read_wait() to check if data is
avaliable. If there are prefetched data, Pipe::tcp_read_wait() return
immediately. Pipe::buffered_recv() is called, which reads data from the
prefetched data. If prefetched data isn't enough, Pipe::buffered_recv()
calls Pipe::do_recv() to read data from socket. But it's possble that
socket has no data at this time, so Pipe::do_recv() keeps retry.

The fix is simple, just not retry when recv(2) return -EAGAIN.

Fixes: #14120
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2016-05-06 13:44:18 +02:00
songbaisen
f086f93bcb show the total pg on each osd.
ceph pg dump osds
dumped osds in format plain
osdstat kbused kbavail kb hb in hb out pg sum
0 143304204 79623420 234882056 [1,2] [] 330
1 143304204 79623420 234882056 [0,2] [] 295
2 143304148 79623476 234882056 [0,1] [] 303
sum 429912556 238870316 704646168

Fixes:#14183
Signed-off-by: songbaisen song.baisen@zte.com.cn
2016-05-06 13:44:18 +02:00
Jianpeng Ma
f8fa44a14a os/bluestore/KernelDevice: For buffer mode no need do bufferlist::rebuild.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-05-06 13:44:17 +02:00
songbaisen
6f046c6c78 osd: copy blacklist in better way.
Signed-off-by: song baisen song.baisen@zte.com.cn
2016-05-06 13:44:17 +02:00
xie xingguo
0f7029b92a mon/PGMonitor: don't get error code overwritten by 0
dump_stuck_pg_stats() method returns either 0 on success or
an negative error code on failure. So it is wrong to reset
"r" to 0.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-05-06 13:44:17 +02:00
xie xingguo
037cf3f6a8 mon/PGMonitor: fix inaccurate warn of unscrubbed pgs
1. we automatically perform deep-scrub at osd_deep_scrub_interval,
   not mon_scrub_interval(1week vs. 1day).

2. theoretically a pg can be in a combination state of both unscrubbed
   and un-deep-scrubbed.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-05-06 13:44:17 +02:00
Sage Weil
01c0f429dd test/mon/mkfs.sh: clean up kv_backend file
Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:17 +02:00
Sage Weil
efc3510f12 mon/MonitorDBStore: remember kv backend type
If it is specified, use it; otherwise, go by the config
option.

Record the type when creating the store.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:17 +02:00
Sage Weil
0c0dc6a465 mon/MonitorDBStore: {read,write}_meta
These are lifted from ObjectStore.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:17 +02:00
Sage Weil
3f1dee1925 mon/MonitorDBStore: move backend instantiation out of ctor
Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:17 +02:00
Sage Weil
be6279de2e mon/MonitorDBStore: fix bug in dump output file
Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:17 +02:00
Sage Weil
0ac671ece2 mon: remove bad rocksdb option
Signed-off-by: Sage Weil <sage@redhat.com>
2016-05-06 13:44:17 +02:00
Kefu Chai
7e8bfef44d os/FileStore::sync_entry check for stop in after wait
there is chance that the sync_entry() gets signaled in the
WaitInterval(max_interval) call because of sync_cond.Signal() call after
"stop = true" in umount(), so without this change, sync_entry() will
continue wait until min_interval is reached. this forces umount() to
wait even it has called d_force_sync(), and hence slows down the
progress of umount(). so we need to check for `stop` if we are not
signalled because of `force_sync`.

Fixes: http://tracker.ceph.com/issues/15695
Reported-by: Haomai Wang <haomaiwang@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-05-06 13:44:16 +02:00
Boris Ranto
efe4e620bf rpm: Fix SELinux relabel on fedora
The SELinux userspace utilities stopped providing versions when they
switched to CIL language. We need to use a different technique to
relabel the files.

Fixes: #15725
Signed-off-by: Boris Ranto <branto@redhat.com>
2016-05-06 13:44:16 +02:00
Abhishek Varshney
e6384c785b rgw: introduce rgw_list_bucket_min_readahead parameter for bucket listing from rados
Signed-off-by: Abhishek Varshney <abhishek.varshney@flipkart.com>
2016-05-06 13:44:16 +02:00
Radoslaw Zarzynski
4baf488edc rgw: fix printing wrong X-Storage-Url in Swift's TempAuth.
If the option "rgw_swift_account_in_url" is being set to true,
both user and subuser IDs will be present in the X-Storage-Url
header generated by the implementation of Swift API's TempAuth.
This patch rectifies this behavior and makes that only the user
identifier will be placed there.

Fixes: http://tracker.ceph.com/issues/15667
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-05-06 13:44:16 +02:00
Nathan Cutler
71cbc832de Test-enable XIO messenger and IB/RDMA support
Fixes: http://tracker.ceph.com/issues/15472

Signed-off-by: Lars Marowsky-Bree <lmb@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-05-06 13:44:16 +02:00
David Zafman
79669c4b06 rados, test: Restrict usage of --prefix in cleanup
As a precaution to using cleanup for mass deletion of other
objects, only allow --prefix which begins with "benchmark_data."

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-05-06 13:44:16 +02:00
David Zafman
4e0b0d9a7e test: Test rados cleanup command with and without --all
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-05-06 13:44:16 +02:00
David Zafman
0aa29f3765 rados: Support --all for rados bench cleanup
Remove duplicate code in generate_object_name()
Need to handle multiple different pids during cleanup
Add same message from clean_up_slow() in clean_up()

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-05-06 13:44:16 +02:00
David Zafman
2951cc3920 common: Fix ObjBencher clean_up when no prefix specified
By default remove all objects named benchmark_data_{hostname} when
there is no benchmark_last_metadata present.

Fixes: http://tracker.ceph.com/issues/15338

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-05-06 13:44:16 +02:00
David Zafman
1976ee7fc2 test: Test rados ls with --all option
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-05-06 13:44:16 +02:00