Commit Graph

98020 Commits

Author SHA1 Message Date
letterwuyu
ecb88dce63 osd: change pg_missing_item encode and decode
since luminous, recovery_deletes is always true, and octopus won't
maintain compatibility with jewel OSDs

Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 21:30:06 +08:00
letterwuyu
20eb8a2e95 osd: fix add_next_event about clean_regions
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:44 +08:00
letterwuyu
6ab1e39941 osd: fix ReplicatedBackend::submit_push_data log output
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:44 +08:00
letterwuyu
e12f8cf93d osd: add a comment to pg_missing_set::add
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
7b581ed9ad osd: fix indentation
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
2e935df15c osd: make ObjectCleanRegions::dump consistent
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
6ec07190da osd: using SERVER_OCTOPUS everywhere
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
7148dda076 osd: supplement doc
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
letterwuyu
e9f328ef08 osd: fix ObjectRecoveryInfo decode
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
lishuhao
b7a0002888 fix the type of _size in class interval_set, int64_t -> uint64_t
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
lishuhao
cb927925af osd: mark ObjectCleanRegions dirty for related operations
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:42 +08:00
lishuhao
2455e08347 osd: change build_push_op and submit_push_data base recovery strategy
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:32 +08:00
lishuhao
28af7820f2 osd: add ObjectCleanRegions for pg log
we use merge ObjectCleanRegions to calculate the data that needs recovery when merge log

Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00
lishuhao
19e71a2f89 test: add unittest case for ObjectCleanRegion
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00
lishuhao
3a8164ac35 osd: define ObjectCleanRegions for recovery object
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00
lishuhao
20e26e8395 osd: partial recovery add doc
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00
Kefu Chai
1b414e8281
Merge pull request #27538 from iotcg/radosgw
init-radosgw: use ceph-conf to get cluster configuration value

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-05-06 13:29:46 +08:00
Kefu Chai
ba970314fe
Merge pull request #27885 from b-ranto/wip-restful-perf-counters
restful: Expose perf counters

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-05-04 13:28:13 +08:00
Kefu Chai
382a3efddf
Merge pull request #27891 from tchaikov/wip-mgr-pgp-num-change-throttling
mgr/DaemonServer: refactor pgp_num changes throttling

Reviewed-by: Sage Weil <sage@redhat.com>
2019-05-04 13:26:39 +08:00
Yuri Weinstein
bb4bceb3b2
Merge pull request #27960 from yuriw/wip-yuriw-crontab-master
qa/tests: removed all runs on ovh
2019-05-03 13:36:56 -07:00
J. Eric Ivancich
a7b4667954
Merge pull request #27896 from ivancich/wip-add-rgw-period-comments
rgw: add some comments to rgw code to help explain functionality

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-05-03 16:12:48 -04:00
Yuri Weinstein
e96f61a205
Merge pull request #27956 from yuriw/wip-yuriw-smoke-master
qa/tests - cleaned up distro settings
2019-05-03 13:06:11 -07:00
Yuri Weinstein
e699d9026d qa/tests: removed all runs on ovh
and consolidated `smoke` runs re: https://github.com/ceph/ceph/pull/27956

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-05-03 12:40:59 -07:00
J. Eric Ivancich
8111dd712b rgw: add some comments to rgw code to help explain functionality
Comments to explain the role of RGWPeriod and the use of prefixes in
gc log entries are added. Some other comments are reformatted for 80
columns.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-05-03 15:29:59 -04:00
Casey Bodley
10e9fa3df0
Merge pull request #27838 from yuvalif/wip-yuval-pubsub_teuthology-new
rgw/pubsub: add multisite pubsub tests to teuthology

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-05-03 14:49:25 -04:00
Yuri Weinstein
52fdf23026 qa/tests - cleaned up distro settings
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-05-03 08:21:42 -07:00
Lenz Grimmer
9b99ed9157
Merge pull request #27448 from ricardoasmarques/upgrade-ceph-iscsi-version
mgr/dashboard: Support ceph-iscsi config v9

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-05-03 13:10:23 +02:00
Nathan Cutler
9b5815f505
Merge pull request #27887 from smithfarm/wip-bci-len255plus
script/backport-create-issue: handle long Redmine issue names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-05-03 13:06:08 +02:00
Mykola Golub
7f80660068
Merge pull request #27895 from dillaman/wip-librbd-zero-copy
librbd: support zero-copy writes via the C API

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-05-03 10:19:06 +03:00
Kefu Chai
dfdb1d0810
Merge pull request #16929 from amitkumar50/cov-rgw-1
rgw: Initialize member variables in rgw_sync.h, rgw_rados.h

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-05-03 13:25:42 +08:00
Kefu Chai
88d2f975af
Merge pull request #27927 from tchaikov/wip-util.collect_sys_info-is-optional
test/common/test_util: skip it if /etc/os-release does not exist

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-05-03 10:34:49 +08:00
Casey Bodley
7a28272c34
Merge pull request #27897 from ivancich/wip-add-editor-comments-rgw-services
rgw: add editor directive comments to rgw services source files

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-05-02 14:23:33 -04:00
Abhishek L
ae8fe24b6f
Merge pull request #27820 from smithfarm/wip-39361
rgw: drop cloud sync module logs attrs from the log

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-05-02 19:52:57 +02:00
Kefu Chai
66de7bd8cc test: do not conditionize on GTEST_HAS_PARAM_TEST
value-parameterized is supported on all platforms, so drop this check

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-03 01:38:27 +08:00
Kefu Chai
71f5f3e8da test: s/INSTANTIATE_TEST_CASE_P/INSTANTIATE_TEST_SUITE_P/
the former is deprecated

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-03 01:35:04 +08:00
Jason Dillaman
ade792ee03 qa/valgrind.supp: added false-positive in boost::queue::lockfree
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 12:59:57 -04:00
Kefu Chai
45629975ac test/rbd_mirror: add "using ::testing::DoAll"
we should `using` it before calling it using `DoAll`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-03 00:53:17 +08:00
Kefu Chai
61e4a521b2 test/common/test_util: skip it if /etc/os-release does not exist
some GNU/Linux distros do not ship this file, and we should not fail the
test on them.

inspired by
http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/ceph-skip-collect-sys-info-test.patch?id=48f19e60c4677e392ee2c23f28098cfcaf9d1710

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-03 00:09:15 +08:00
Kefu Chai
d4387d33ef cmake: do not install googletest
the new googletest added an option to allow its consumer to skip this,
so instead of patching googletest, let's use this option.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-02 23:52:45 +08:00
Kefu Chai
9acf335464 googletest: pick up GTEST_SKIP() changes
so we can use it to disable test at runtime

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-05-02 23:52:25 +08:00
Yuval Lifshitz
51fc8a40e3 rgw/pubsub: fix more test issues with teuthology failures
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-05-02 18:51:59 +03:00
Jason Dillaman
4876450faf librbd: use zero-copy on write via the C API when cache is disabled
It will also be disabled when using synchronous write API methods.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
308fcc3456 common/deleter: added missing headers
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
2856777613 librbd: allow AioCompletion objects to be blocked
This will be used when user-provided memory is wrapped into a
ceph::buffer::raw pointer to prevent its release prior to the
drop of its last reference internally.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
e2ee3cd50c librbd: AioCompletion fail path should complete in clean thread
Ensure that no internal librbd locks are being held when the
callbacks are invoked for AioCompletion::fail.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
4a2e2e08c0 librbd: simplify AioCompletion reference counting for 'fail' case
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
ef9d74720f librbd: switch to lock-free queue for event poll IO interface
'perf' shows several percent of CPU being wasted on lock contention
in the event poll interface. The 'fio' RBD engine uses this poll
IO interface by default when available.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
002afa0fe3 librbd: avoid using lock within AIO completion where possible
'perf' shows several percent of CPU is being utilized handling the
heavyweight locking semantics of AIO completion. With these changes,
the lock contention disappears.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
b5fc7ecaf7 librbd: remove special case for starting AioCompletion ops
All ops can be immediately started now that flush ops won't
accidentally block themselves.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00
Jason Dillaman
09e4127d5d librbd: simplify IO flush handling through AsyncOperation
Allow ImageFlushRequest to directly execute a flush call through
AsyncOperation. This will allow the flush to be directly linked
to its preceeding IOs.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-05-02 09:30:45 -04:00