Commit Graph

55877 Commits

Author SHA1 Message Date
Kefu Chai
58cec42f50 Merge pull request #9117 from wjin/clean
osd: remove duplicated function

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-11 11:54:15 +08:00
Kefu Chai
35ce55d204 Merge pull request #9252 from vumrao/wip-vumrao-15983
mon: change osdmap flags set and unset messages

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-11 11:51:36 +08:00
Kefu Chai
e15c8efa8e Merge pull request #9452 from mslovy/wip-preserve-alloc-hint
osd:preserve allocation hint attribute during recovery

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2016-07-11 11:50:48 +08:00
Kefu Chai
90a18d822e Merge pull request #9698 from LiumxNL/fix-clones-flush-evict
rados: add option to include clones when doing flush or evict

Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-11 11:49:27 +08:00
Kefu Chai
e9e3a2398c Merge pull request #9703 from badone/wip-cryptoKey-encrypt-segfault
cephx: Fix multiple segfaults due to attempts to encrypt or decrypt

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-11 11:48:26 +08:00
Kefu Chai
b2decbfc78 Merge pull request #9717 from songbaisen/a7
mon: do not recalculate 'to_remove' when it's known

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-11 11:47:43 +08:00
Kefu Chai
654d86fe2b Merge pull request #9763 from dx9/wip-pthread-np
compat: add abstractions for non portable pthread name funcs

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-11 11:46:47 +08:00
Kefu Chai
afb1935374 ceph_test_rados_api_tmap_migrate: remove test for tmap_upgrade
the tested feature was removed in ad6cc41

> tmap_upgrade was only for Jewel, so remove it for Kraken.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-11 11:30:38 +08:00
Michal Jarzabek
22bf7e98f9 msg/AsyncMessenger: change return type to void
Change the return type of function Processor::start to void. It
doesn't make sense to return constant value(zero).

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-07-10 19:26:00 +01:00
xie xingguo
af82dbcbeb os/bluestore: add support for allocating 0-sized extent
For testing, we may choose randomized length of extent to allocate,
which means we could trim down the length to 0.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-10 21:25:52 +08:00
xie xingguo
39458dd308 os/bluestore: fix allocation of discontiguous blocks
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-10 21:25:50 +08:00
xie xingguo
6147ea0f29 os/bluestore: add sanity check for get_free() method
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-10 21:25:49 +08:00
xie xingguo
0c86d5ec0f os/bluestore: simplify is_exhausted() method
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-10 21:25:47 +08:00
xie xingguo
b7d09ed137 os/bluestore: make zone/span size of bitmap-allocator configurable
Perviously as zone_size_block and span_size are identical, so we could use
span_size only to simplify the calcuation.

Now that the zone_size_block and span_size are both configurable, so
it is necessary to treat them respectively.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-10 21:25:45 +08:00
Kefu Chai
89ba840a6c Merge pull request #10190 from ktdreyer/doc-delete-pool
doc: more details for pool deletion

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-10 19:57:07 +08:00
Brad Hubbard
3ad56cbf61 makefile: change librgw_file_* as check_PROGRAMS
These targets depend on libgmock_main.la which is not built during
a default build.

Fixes: http://tracker.ceph.com/issues/16646
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-07-10 16:42:19 +10:00
Michal Jarzabek
72f75d9f63 msg/Event.h: add const to member function
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
2016-07-09 08:38:20 +01:00
Josh Durgin
becdbe2917 ReplicatedPG: clear log update waiters during shutdown
This prevents leaking repops that are referenced by LogUpdateCtx for
updates that were in flight.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
fbf3b79cae test/librados: add test that requires correct dup error detection
This only works reliably with the
objecter_retry_writes_after_first_reply setting, so make it part of
the test setup.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
56f7115632 test/librados: add a way to pass ceph config options
This way individual tests or testcases can change settings

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
6faa449fc6 Objecter: add option for testing osd dup handling
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
fa8aff9a25 ReplicatedPG: require kraken feature bit on osdmap to record write errors in the pg log
Older OSDs can't handle the error entries.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:14 -07:00
Josh Durgin
112eab06fb OSDMonitor: add kraken feature bit
This is needed to turn on persisting write errors, since older OSDs
won't be able to handle them.

Other features for kraken could potentially use this as well.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
4c709e9b6d ReplicatedPG: return error codes for dups from the pg log
This prevents reordering guarded writes or deletes.

Without this, the following sequence:

delete foo -> -ENOENT
write foo -> success
(client connection fails)
resend delete foo -> success, object deleted
resend write foo -> success - dup op, so no write performed

results in the object not existing, instead of containing data. After
this change, both delete and write are detected as dups and the
original ordering is preserved.

Fixes: http://tracker.ceph.com/issues/14468
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
ed33ffcbd3 ReplicatedPG: skip stat invalidation when recording write errors
This is only needed for the lost/unfound use of submit_log_entries() etc.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
cf8e466f34 ReplicatedPG: persist write errors in the pg log
This is required to prevent re-ordering of guarded writes or deletes
in the presence of network failures and resends.

Use the existing submit_log_entries() method to initiate a repop that
only updates the pg log.

Keep the write error semantics close to the existing implementation -
if we have a buffer, return it, but do not persist the buffer for now.

Refs: http://tracker.ceph.com/issues/14468
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
41861fae08 PGLog: skip indexing errors by object
Dup detection only needs them indexed by version, and keeping them out
of the object index prevents error entries from contributing to the
missing set during recovery.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
ea66e5ca4d PGLog: ignore error entries when constructing the missing set
Errors should only be used for dup detection.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
2b8062ebc1 osd_types: add an error pg log entry type
This will store write error codes for use in dup op detection.
A few places use checks assuming is_update() or is_delete() are
opposites - fix those to ignore or consider errors, as appropriate.

Refs: http://tracker.ceph.com/issues/14468
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
b675726c0e ReplicatedPG: removed unused obc check
The one place this was set was removed by e7edf20fb7

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:13 -07:00
Josh Durgin
62864beebd ReplicatedPG: removed unused OnComplete struct
Added but never used in e7edf20fb7

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:12 -07:00
Josh Durgin
2609dd922f ReplicatedPG: remove unused mark_object_lost()
The only caller was removed in e7edf20fb7

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-07-08 18:33:12 -07:00
Sage Weil
181f6c67da Merge pull request #10201 from xiexingguo/xxg-wip-bluestore-2016-07-08
os/bluestore: misc cleanups

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-08 18:29:50 -04:00
Casey Bodley
0273d5595c Merge pull request #10157 from cbodley/wip-rgw-sync-error-repo
rgw: fix error_repo segfault in data sync
2016-07-08 16:12:26 -04:00
Casey Bodley
28609029cf rgw: fix error_repo segfault in data sync
RGWDataSyncShardCR will only allocate an error_repo if it's doing
incremental sync, so RGWDataSyncSingleEntryCR needs to guard against a
null error_repo

also, RGWDataSyncShardCR::stop_spawned_services() was dropping the last
reference to the error_repo before calling drain_all(), which meant that
RGWDataSyncSingleEntryCR could still be holding a pointer. now uses a
boost::intrusive_ptr in RGWDataSyncSingleEntryCR to account for its
reference

Fixes: http://tracker.ceph.com/issues/16603

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 15:53:57 -04:00
vasukulkarni
299a5210e3 Merge pull request #10184 from ktdreyer/doc-rados-operations-systemd
doc: rm SysV instructions, add systemd
2016-07-08 11:56:50 -07:00
Yehuda Sadeh
ed1be58a86 Merge pull request #9570 from zaitcev/wip-5073-multen-doc
rgw: Add documentation for the Multi-tenancy feature

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-08 11:46:48 -07:00
Yehuda Sadeh
6489be299c Merge pull request #10151 from oritwas/wip-rgw-fix-cls-sync-stats
rgw: register the correct handler for cls_user_complete_stats

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-08 11:40:01 -07:00
Daniel Gryniewicz
395f2c5ffe CMake - stop pip checking for updates
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2016-07-08 13:41:27 -04:00
Casey Bodley
16976eedb7 rgw: remove datalog keys from error repo on ENOENT
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:26 -04:00
Casey Bodley
6f65d192cd test/rgw: add optional --tenant flag to test_multi.py
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:26 -04:00
Casey Bodley
3cf715c682 rgw: add tenant id to GetBucketInstanceInfo
use the rgw_bucket overload of get_bucket_instance_info() so it can
supply the tenant id

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:26 -04:00
Casey Bodley
d945e20995 rgw: carry tenant id with data sync
use rgw_bucket_shard to track buckets, which includes tenant id

Fixes: http://tracker.ceph.com/issues/16469

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:26 -04:00
Casey Bodley
2a1e1a776c rgw: add tenant to url for RGWRESTStreamWriteRequest
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:26 -04:00
Casey Bodley
fc0df4802b rgw: add tenant to rgw_bucket json format
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:26 -04:00
Casey Bodley
d5ac140040 rgw: add rgw_bucket_parse_bucket_key()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:25 -04:00
Casey Bodley
c8aa668fc3 rgw: add tenant name to datalog entries
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:25 -04:00
Casey Bodley
3eae201c8a rgw: convert bucket instance listings back to metadata key format
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:25 -04:00
Casey Bodley
a0befa1e7b rgw: use tenant/ for bucket instance metadata keys
to work around the ambiguity of parsing tenant: and :shard in the same
bucket instance metadata key, use tenant/ instead

to preserve backward compatibility with existing objects, new helper
function rgw_bucket_instance_key_to_oid() converts this / back to a :
before being used as an object name

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:25 -04:00
Casey Bodley
51ff8ef7c6 rgw: add get_key() methods to format rgw_buckets
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-07-08 12:20:25 -04:00