Commit Graph

55706 Commits

Author SHA1 Message Date
Josh Durgin
becdbe2917 ReplicatedPG: clear log update waiters during shutdown
This prevents leaking repops that are referenced by LogUpdateCtx for
updates that were in flight.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
fbf3b79cae test/librados: add test that requires correct dup error detection
This only works reliably with the
objecter_retry_writes_after_first_reply setting, so make it part of
the test setup.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
56f7115632 test/librados: add a way to pass ceph config options
This way individual tests or testcases can change settings

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
6faa449fc6 Objecter: add option for testing osd dup handling
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
fa8aff9a25 ReplicatedPG: require kraken feature bit on osdmap to record write errors in the pg log
Older OSDs can't handle the error entries.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
112eab06fb OSDMonitor: add kraken feature bit
This is needed to turn on persisting write errors, since older OSDs
won't be able to handle them.

Other features for kraken could potentially use this as well.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
4c709e9b6d ReplicatedPG: return error codes for dups from the pg log
This prevents reordering guarded writes or deletes.

Without this, the following sequence:

delete foo -> -ENOENT
write foo -> success
(client connection fails)
resend delete foo -> success, object deleted
resend write foo -> success - dup op, so no write performed

results in the object not existing, instead of containing data. After
this change, both delete and write are detected as dups and the
original ordering is preserved.

Fixes: http://tracker.ceph.com/issues/14468
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
ed33ffcbd3 ReplicatedPG: skip stat invalidation when recording write errors
This is only needed for the lost/unfound use of submit_log_entries() etc.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
cf8e466f34 ReplicatedPG: persist write errors in the pg log
This is required to prevent re-ordering of guarded writes or deletes
in the presence of network failures and resends.

Use the existing submit_log_entries() method to initiate a repop that
only updates the pg log.

Keep the write error semantics close to the existing implementation -
if we have a buffer, return it, but do not persist the buffer for now.

Refs: http://tracker.ceph.com/issues/14468
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
41861fae08 PGLog: skip indexing errors by object
Dup detection only needs them indexed by version, and keeping them out
of the object index prevents error entries from contributing to the
missing set during recovery.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
ea66e5ca4d PGLog: ignore error entries when constructing the missing set
Errors should only be used for dup detection.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
2b8062ebc1 osd_types: add an error pg log entry type
This will store write error codes for use in dup op detection.
A few places use checks assuming is_update() or is_delete() are
opposites - fix those to ignore or consider errors, as appropriate.

Refs: http://tracker.ceph.com/issues/14468
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
b675726c0e ReplicatedPG: removed unused obc check
The one place this was set was removed by e7edf20fb7

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
62864beebd ReplicatedPG: removed unused OnComplete struct
Added but never used in e7edf20fb7

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:12 -07:00
Josh Durgin
2609dd922f ReplicatedPG: remove unused mark_object_lost()
The only caller was removed in e7edf20fb7

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:12 -07:00
Sage Weil
181f6c67da Merge pull request #10201 from xiexingguo/xxg-wip-bluestore-2016-07-08
os/bluestore: misc cleanups

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-08 18:29:50 -04:00
Casey Bodley
0273d5595c Merge pull request #10157 from cbodley/wip-rgw-sync-error-repo
rgw: fix error_repo segfault in data sync
2016-07-08 16:12:26 -04:00
Casey Bodley
28609029cf rgw: fix error_repo segfault in data sync
RGWDataSyncShardCR will only allocate an error_repo if it's doing
incremental sync, so RGWDataSyncSingleEntryCR needs to guard against a
null error_repo

also, RGWDataSyncShardCR::stop_spawned_services() was dropping the last
reference to the error_repo before calling drain_all(), which meant that
RGWDataSyncSingleEntryCR could still be holding a pointer. now uses a
boost::intrusive_ptr in RGWDataSyncSingleEntryCR to account for its
reference

Fixes: http://tracker.ceph.com/issues/16603

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 15:53:57 -04:00
vasukulkarni
299a5210e3 Merge pull request #10184 from ktdreyer/doc-rados-operations-systemd
doc: rm SysV instructions, add systemd
2016-07-08 11:56:50 -07:00
Yehuda Sadeh
ed1be58a86 Merge pull request #9570 from zaitcev/wip-5073-multen-doc
rgw: Add documentation for the Multi-tenancy feature

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-08 11:46:48 -07:00
Yehuda Sadeh
6489be299c Merge pull request #10151 from oritwas/wip-rgw-fix-cls-sync-stats
rgw: register the correct handler for cls_user_complete_stats

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-08 11:40:01 -07:00
Kefu Chai
21402181f5 Merge pull request #9934 from tchaikov/wip-build-on-armf
common: instantiate strict_si_cast<long> not strict_si_cast<int64_t>

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-07-08 22:58:28 +08:00
Kefu Chai
5ba22025a8 Merge pull request #10204 from ceph/wip-cmake
cmake: fixes for pypi changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 22:51:48 +08:00
Kefu Chai
31db4c5f9f common: instantiate strict_si_cast<long> not strict_si_cast<int64_t>
this fixes the build on armf.

on 32bit platforms, cstdint is very likely to

 typedef long long int int64_t;

this results in compilation error like

 `common/strtol.cc:190:75: error: duplicate explicit instantiation of 'T
 strict_si_cast(const char, std::string) [with T = long long int;
 std::string = std::basic_string]'

 [-fpermissive]
 template int64_t strict_si_cast(const char *str, std::string *err);
 ^`

we can address this by instantiate the primitive type of `long long`
instead of `in64_t`.

Fixes: http://tracker.ceph.com/issues/16398
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-08 21:20:53 +08:00
Kefu Chai
37dbe98a31 Merge pull request #8189 from chardan/wip-chardan-15100
librados examples: link and include from current source tree by default.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-08 21:10:35 +08:00
xie xingguo
ff0329634a os/bluestore: release compressor if comp_mode turned out to be none
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-08 20:44:42 +08:00
xie xingguo
ed5f7c4df7 os/bluestore: drop unnecessary increase of iterator
We are going to break anyway.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-08 20:44:42 +08:00
xie xingguo
359c3863a5 os/bluestore: assert to confirm that we don't access violation
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-08 20:44:42 +08:00
xie xingguo
a7228eb8b9 os/bluestore: drop unused member _bytes
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-08 20:44:42 +08:00
xie xingguo
72fe4df3f8 os/bluestore: check against that we don't access violation
Some day we might want make this interface more applicable by handling
the abnormal cases more gracefully...

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-08 20:44:42 +08:00
Igor Fedotov
5c2a1bbb1f Merge pull request #10137 from xiexingguo/xxg-wip-bluestore-2016-07-05-02
os/bluestore: change algorithm of compression header from string to int

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2016-07-08 13:43:33 +02:00
xie xingguo
1209cb3d12 os/bluestore: change algorithm of compression header from string to int
The literal description of compression algorithm can vary from
various compression types and thus increases the complexity of
en/decoding, which as a result can cause chaos. Also it can be
more efficient by translating it into a fixed-length type.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-08 19:39:17 +08:00
Orit Wasserman
1a8e72cb2d rgw: register the correct handler for cls_user_complete_stats
Fixes: http://tracker.ceph.com/issues/16624
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2016-07-08 10:54:00 +02:00
Kefu Chai
ce1f9c6e6e pybind: put temp .o files in $TMPDIR
otherwise they will go to $PWD/$TMPDIR instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-08 15:03:45 +08:00
Kefu Chai
edd7c9d709 cmake: creage egg_info files into build directory
otherwise they will go to the source tree, and "git ls-files" will list
them as untracked files, which annoy gitbuilder-ceph-tarball*-cmake
gitbuilders. like

+ echo 'error: Added files:'
error: Added files:
+ cat .git/added-files
src/pybind/rados/rados.egg-info/PKG-INFO
...

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-08 15:02:59 +08:00
Kefu Chai
07387eb9af Merge pull request #10186 from ceph/wip-cmake-no-with-mds
cmake: remove WITH_MDS option

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-08 11:37:04 +08:00
Sage Weil
33fb3bc63f Merge pull request #10163 from stiopaa1/msg_asyncmess_removeUnusedFunction_create_anon_connection
msg/AsyncMessenger: remove unused function
2016-07-07 16:33:21 -04:00
Ali Maredia
235a72a8f4 Merge pull request #10178 from ceph/wip-cmake
cmake: use stock Find* modules.

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-07 15:24:45 -04:00
Ali Maredia
cfcf0706f2 cmake: remove option
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2016-07-07 14:31:32 -04:00
Ken Dreyer
99e0ee1223 doc: rm SysV instructions, add systemd
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2016-07-07 11:59:18 -06:00
Sage Weil
956c769969 Merge pull request #10128 from xiexingguo/xxg-wip-bluestore-2016-07-05
os/bluestore: introduce power 2 macros for block alignment and rounding

Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-07 13:40:42 -04:00
Casey Bodley
d1a10c5416 Merge pull request #10180 from theanalyst/mrgw/cmake-fixes
mrgw: search for cmake build dir.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-07 13:00:32 -04:00
Abhishek Lekshmanan
eeb394e04c
mrgw: search for cmake build dir.
Similar to mstart and mstop, mrgw also needs to find the correct build
dir as we pass in the pidfile and asokfile which otherwise falls back to
src/run dir.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2016-07-07 18:22:27 +02:00
Ali Maredia
3ed34b58bd Merge pull request #10116 from tchaikov/wip-cmake-make-check
test: make check using cmake

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-07 12:13:46 -04:00
Casey Bodley
7dac5c1e21 Merge pull request #10097 from yehudasa/wip-mrun-cmake
mrun, mstart.sh, mstop.sh: search for cmake build directory

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-07 12:03:09 -04:00
David Zafman
d3bcc7dc4d Merge pull request #9489 from athanatos/wip-16156
ReplicatedPG: call op_applied for submit_log_entries based repops

Reviewed-by: David Zafman <dzafman@redhat.com>
2016-07-07 08:57:53 -07:00
Kefu Chai
b5bb77d566 cmake: use the stock FindCURL.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 23:26:02 +08:00
Kefu Chai
3646fa5c6b cmake: use the stock FindEXPAT.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 23:26:02 +08:00
Kefu Chai
d053705c03 qa/workunits/ceph-helpers.sh: disable test_pg_scrub()
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 23:17:11 +08:00
Kefu Chai
0250214529 Merge pull request #10139 from SUSE/wip-cmake-really-fix-fcgi-include-dir
cmake: Really add FCGI_INCLUDE_DIR to include_directories for rgw

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-07 21:45:52 +08:00