Commit Graph

50602 Commits

Author SHA1 Message Date
Sage Weil
ef80690622 buffer: clean up raw_combined construction
Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 09:09:22 -05:00
Sage Weil
b6ed4d3ebf buffer: size append_buffer so that it fits into page-multiple allocations
We drop some unittest assertions about alloc buffer size.  Sorry!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 09:09:22 -05:00
Sage Weil
f2c0d5cd10 rbd-replay: s/CEPH_BUFFER_APPEND_SIZE/CEPH_PAGE_SIZE/
Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 09:09:22 -05:00
Sage Weil
ce3e5a3347 buffer: alloc right-sized buffer from read_fd
This may as well fit the input; this doesn't relate to the
append buffer.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:29 -05:00
Sage Weil
73dcd26fc8 buffer: use raw_combined for certain allocations
If the alignment is on a page boundary, or the allocation is big,
a separate buffer::raw goes faster.  The rest of the time,
a raw_combined does.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:29 -05:00
Sage Weil
6be3b99d5d buffer: align unspecified allocations to a word
This will let us put policy create_aligned.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:28 -05:00
Sage Weil
724a493adc buffer: combine data and buffer::raw into single allocation
Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:28 -05:00
Sage Weil
04482ae902 buffer: add front(), back(), get_num_buffers() methods
These eliminate most callers of buffers(), which exposes the
internal list<ptr>.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:28 -05:00
Sage Weil
69bcbe104d unittest_bufferlist: benchmark some allocations
Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:28 -05:00
Sage Weil
08c0d98bc1 unittest_bufferlist: fix append_bench
- fix source
- include larger sizes

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:28 -05:00
Sage Weil
7723d294a5 unittest_bufferlist: fix ptr move test
Do not assume there is a trailing null the terminate the string.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-03-01 08:47:28 -05:00
Sage Weil
149f01f3d8 Merge pull request #7739 from xiexingguo/xxg-wip-14387
osd: fix race condition for heartbeat_need_update

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-01 08:46:08 -05:00
Sage Weil
ef5973324b Merge pull request #7703 from xiexingguo/xxg-wip-statfs
osd: filestore: fast abort if statfs encounters ENOENT

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-01 08:45:39 -05:00
Sage Weil
57b838f4b8 Merge pull request #7766 from adamemerson/wip-portable-mutex
common: Do not use non-portable constants in mutex_debug

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2016-03-01 08:45:00 -05:00
Sage Weil
5f429178b5 Merge pull request #7799 from xiexingguo/xxg-wip-fixnvme
osd: bluestore: NVMEDevice: fix error handling

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-01 08:44:21 -05:00
Sage Weil
46bd7bbb8b Merge pull request #7804 from jjhuo/fixes
osd: kstore: fix a race condition in _txc_finish()

Reviewed-by: Sage Weil <sage@redhat.com>
2016-03-01 08:43:49 -05:00
Sage Weil
0edaae6305 Merge pull request #7816 from marcan/build-fixes
configure.ac: boost_iostreams is required, not optional

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-03-01 08:43:25 -05:00
Jason Dillaman
34dd39c034 librbd: use aio watch/unwatch during open/close state machines
Removed the place holders in the open/close async state machines with
the newly added aio watch/unwatch methods.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 08:36:17 -05:00
Jason Dillaman
eac7e7a101 librados_test_stub: added aio_watch/aio_unwatch/aio_watch_flush
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 08:36:17 -05:00
Jason Dillaman
feb2fc0240 test: adjust rbd test case guards to handle new defaults
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 08:32:39 -05:00
Jason Dillaman
0b2847e63a test: use default RBD features in test cases
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:55:06 -05:00
Jason Dillaman
ed83975957 librbd: permit deep-flatten to be dynamically disabled
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:55:06 -05:00
Jason Dillaman
d24883e497 rbd: update default image features
Exclusive lock, object map, fast-diff, and deep-flatten have been
enabled by default for all new images.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:55:04 -05:00
Kefu Chai
6eff39af79 qa/workunits/cephtool/test.sh: wait longer in ceph_watch_start()
"ceph --watch-debug" and "ceph tell mon.foo version" could connect
to different monitors, and there is chance that "ceph --watch-debug"
is not connected yet when "ceph tell" completes, and hence the former
fails to collect the cluster log including the "ceph tell" related
message. this renders test_mon_tell() unreliable. so, in
ceph_watch_start(), we should wait until the "ceph" cli connects to the
monitor and receives messages from it.

Fixes: #14910
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-03-01 20:48:24 +08:00
Piotr Dałek
42d81bffc4 common/obj_bencher.cc: faster object name generation
generate_object_name() uses std::ostringstream to build object name
and calls generate_object_prefix() (which uses another std::ostringstream),
making it quite inefficient and actually being one of limiting factors
in small-object benchmarks. New version that does everything on one char
array and within one function is up to 3,8x faster (in my testing bumping
throughput from 35MB/s to 40MB/s on rados bench rand -t 128 on 1KB objects).

Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
2016-03-01 13:47:10 +01:00
Jason Dillaman
e4f73b3bc7 librbd: apply orphaned maintenance ops after journal replay
If a client recorded a maintenance op to the journal but crashed
before writing the op finish event, the image will be in an
inconsistent state.  Therefore, once the end of the journal is
reached, attempt to apply all queued ops.

Fixes: #14822
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:39:42 -05:00
Jason Dillaman
c4387301c1 librbd: only cancel replay of journal op events upon error
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:39:41 -05:00
Jason Dillaman
9c9963715a librbd: delay commit of op start event
If the start event is flagged as committed before the op is
actually executed, librbd won't be able to replay the event
should a crash occur.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:38:06 -05:00
Jason Dillaman
b0dc047e36 journal: permit fire-and-forget aio commit position flush
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:38:05 -05:00
Jason Dillaman
ed9de5fc5e librbd: support replay of maintenance ops
Ignore errors that are to be expected when replaying
a maintenance op two or more times.

Fixes: #14822
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-03-01 07:38:05 -05:00
Abhishek Lekshmanan
ea9cb4fffa doc: rgw explain keystone's verify ssl switch
Adding info about `rgw keystone verify ssl` to configuration reference,
also adding a note in rgw keystone to explain the usage.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2016-03-01 12:49:51 +01:00
Loic Dachary
d5c9349ea8 Merge pull request #7858 from jdurgin/wip-rbd-formatted-output-progress
test/cli-integration/rbd: disable progress output

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-03-01 17:58:25 +07:00
John Spray
3d8fc94381 Merge pull request #7745 from sileht/sileht/rados-cython
pybind: move cephfs to Cython

Reviewed-by: John Spray <john.spray@redhat.com>
2016-03-01 10:40:21 +00:00
Haomai Wang
4ea3d68368 Merge pull request #7831 from yuyuyu101/wip-14912
AsyncMessenger: fix several bugs
2016-03-01 17:54:03 +08:00
Haomai Wang
48a5f2d835 Merge pull request #7852 from xiexingguo/xxg-wip-14928
msg: async: start over after failing to bind a port in specified range

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kefu@redhat.com>
2016-03-01 17:53:20 +08:00
xie xingguo
8ffc4e8001 msg: async: start over after failing to bind a port in specified range
The original report associated with this fix can be found at
http://tracker.ceph.com/issues/13002. I reopen one in case it
needs to be backported or something alike.

Fixes: #14928
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-03-01 22:45:14 +08:00
Josh Durgin
acc640542c test/cli-integration/rbd: disable progress output
This test is intended to check machine-readable output and usage, not
human-readable output.

Fixes: #14931
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-02-29 22:57:56 -08:00
Josh Durgin
4f98ba25d0 Merge pull request #7778 from marcan/pybind-fixes
pybind/rados: fix object lifetime issues and other bugs in aio

Reviewed-by: Mehdi Abaakouk <sileht@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-29 22:48:53 -08:00
Kefu Chai
cbe8dfbdbc Merge pull request #7855 from tchaikov/wip-fix-ftbfs
test/TestPGLog: fix the FTBFS

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-03-01 14:39:29 +08:00
Kefu Chai
203987331e test/TestPGLog: fix the FTBFS
the new pure virtual method was introduced by e7edf20.

Fixes: #14930
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-03-01 13:20:29 +08:00
Josh Durgin
abd708d858 Merge pull request #7781 from ceph/wip-fix-rbd-cli-tests
rbd/run_cli_tests.sh: Reflect test failures

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-29 17:30:09 -08:00
James Liu
6cd5e2049e fixed a bug for write bench with rados
Signed-off-by: James Liu <james.liu@ssi.samsung.com>
2016-02-29 16:25:42 -08:00
Ken Dreyer
1a84b8c4c1 Merge pull request #7826 from SUSE/wip-move-requires
RPM: move runtime dependencies to ceph-base and fix other packaging issues

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2016-02-29 16:25:38 -07:00
Yehuda Sadeh
cf812eadde rgw: aggregate usage by payer
If bucket has requester-payer property set, aggregate the usage data
on it by the user that did the operation.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
2016-02-29 15:19:19 -08:00
Yehuda Sadeh
ff8305a973 rgw: fix permission check for request payer
If request_payer auth check returns false, we need to abort.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-29 15:19:02 -08:00
Samuel Just
440e9b5d56 Merge pull request #7769 from somnathr/wip-make-shared
Use make_shared while creating shared_ptr

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-02-29 15:04:28 -08:00
Samuel Just
554b643aa4 test_filejournal: reserve throttle as needed
This is awkward, if there are later users other than this test
and JournalingObjectStore, it'll be worth making it a bit less
clunky and error prone.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-29 11:23:58 -08:00
Samuel Just
30f548e9fb config_opts: make filestore_queue_max_(ops|bytes) U64
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-02-29 10:52:00 -08:00
Samuel Just
306c7bf40e Merge pull request #7765 from athanatos/wip-lost-unfound-repop
Repop and lost-unfound overhaul

Reviewed-by: Sage Weil <sage@redhat.com>
2016-02-29 10:42:01 -08:00
Jason Dillaman
622748ae15 rbd: deprecate image format 1
The rbd cli will warn about the deprecation when attempting to create
image format 1 images.  librbd will log an error message when opening
a format 1 RBD image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-29 13:22:32 -05:00