Commit Graph

60114 Commits

Author SHA1 Message Date
Samuel Just
9f17e2b535 osd,mon: remove FLAG_DEBUG_FAKE_EC_POOL
This was used in the past as scaffolding while the ec pools were being
developed.  There should be no legitimate users.

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Tomy Cheru <tomy.cheru@sandisk.com>
2016-11-17 10:40:17 -08:00
Samuel Just
1dc6b4b694 osd_types: remove unused fill_in_setattrs
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:17 -08:00
Samuel Just
c723f5417a osd_types::ObjectModDesc: remove claim_append
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:17 -08:00
Samuel Just
2c99f864df osd/: switch all users of PGTransaction to use the new structure
This patch removes ReplicatedBackend::PGTransaction and implemenations
and switches over all users.  Happily, do_osd_ops loses the mod_desc
cruft and OpContext::pending_attrs.  PGTransaction doesn't really
have a natural way to implement append, however.  In reality, I think
this is probably an improvement, but it does mean that copy_from's
final transaction is now filled in by a lambda rather than by
appending a transaction fragment.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:17 -08:00
Samuel Just
20204b642a osd/: introduce PGTransaction
ECBackend is going to need a transaction representation which reduces
the operational representation from the OSDOp to a descriptive one
which makes questions like "what is the largest offest written" and
"does this transaction delete the object?" simple to answer.  At the
same time, we're going to eliminate the PGBackend::PGTransaction
interface since I don't think writing directly to an
ObjectStore::Transaction is buying us enough to offset the irritation
of having to update both implemenations.

A happy consequence of this design will be that we can fill in the
pg_log_entry_t::mod_desc member after submission in the backend
rather than inline in do_osd_ops.  We can also dispense with having
to maintain OpContext::pending_attrs separately from the ongoing
PGTransaction.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:17 -08:00
Samuel Just
6ae520da55 common/: add interval_map
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:17 -08:00
Samuel Just
49275c2d27 inline_variant: simplify it a lot, enable perfect forwarding
The previous implementation was a bit more baroque than it
needed to be.  Also, it made copies of the lambdas in a
few places.  Finally, it caused segfaults.  Not actually
sure why.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
05ddacb192 common/: add match() utilities for boost::variant
C++ doesn't have a sum type with nice pattern matching syntax.
Fortunately, someone on stack overflow fixed that.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
9c65dee274 osd_types: update_snaps should take a const argument
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
de3c22b0ee Context: add [Gen]LambdaContext and some related helpers
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
1934874e44 hobject: add helper typedefs
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
eb3b2024f2 PGBackend: add DoutPrefixProvider to parent interface
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
9499cdcd87 FileStore::_do_copy_range: tolerate short reads on replay
Consider a sequence like:

0. foo object size is 15
1. clone_range foo -> foo.0 5~5
2. write foo 5~5
3. clone_range foo -> foo.1 10~5
4. write 10~5 foo
5. rename foo -> foo.1
6. remove foo.0
7. remove foo.1
8. remove foo.2

If this sequence is interupted after 8 and replayed from 1, by the time
it gets to 3 the object will only have size 10 and no replay guard
(since 1 was skipped and 2 recreated the object with size 10 resulting
in a short read.  This should only happen if the replay guard is
missing, which should only happen if the object gets deleted later
in the sequence.

Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
bcb5a0da21 store_test::col_split_test: send bounded size transactions
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:16 -08:00
Samuel Just
313e55d948 vstart: ratchet down the osd_copyfrom_max_chunk to make multiple chunks likely
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-11-17 10:40:15 -08:00
Loic Dachary
654ad753c9 Merge pull request from dachary/wip-16014-cot
tests: use shorter directories for tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 17:37:27 +01:00
Loic Dachary
77fe9af960 Merge pull request from tchaikov/wip-osd-scrub-repair-cleanup
test/osd-scrub-repair.sh: use repair() instead of "ceph pg repair"

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-11-17 16:54:53 +01:00
Sage Weil
6662c80b46 Merge pull request from xiexingguo/xxg-wip-bluestreo-1117
os/bluestore: fix bufferspace stats leak due to blob splitting

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2016-11-17 09:25:03 -06:00
Sage Weil
12b12e6023 Merge pull request from ifed01/wip-bluestore-drop-blob_depth
os/bluestore: remove garbage collector staff

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 09:19:57 -06:00
Loic Dachary
a9c07e7e1b tests: use shorter directories for tests
So that jenkins can use longer directories. We can't have both otherwise
the limit UNIX domain socket path length limit triggers errors such as:

... client.admin.12750.asok is too long! The maximum length on this system is 107

Fixes: http://tracker.ceph.com/issues/16014

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-11-17 16:08:52 +01:00
Casey Bodley
1cf0f1e3a7 Merge pull request from tianshan/fix-RGWSimpleRadosLockCR
rgw: fix RGWSimpleRadosLockCR set_description()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-11-17 09:57:12 -05:00
Casey Bodley
7bab807821 Merge pull request from ceph/wip-rgw-fix-period-commit
rgw: fix missing master zone for a single zone zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-11-17 09:56:18 -05:00
Jason Dillaman
eb0248c6e3 Merge pull request from leseb/glance-mitaka
doc: openstack glance mitaka uses show_multiple_locations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-11-17 09:31:22 -05:00
Orit Wasserman
0870b00c77 rgw: fix missing master zone for a single zone zonegroup
Fixes: http://tracker.ceph.com/issues/17364
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2016-11-17 14:51:35 +01:00
Igor Fedotov
adddaf49a7 os/bluestore: remove garbage collector staff
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-11-17 13:34:17 +00:00
Kefu Chai
e445953379 Merge pull request from renhwztetecs/renhw-wip-mgr
mgr: init() return when connection daemons failed && add some err info

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 19:40:53 +08:00
Igor Fedotov
7814549c74 Merge pull request from xiexingguo/xxg-wip-bluestore-init-csum
os/bluestore: avoid unnecessary call to init_csum()

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 14:07:40 +03:00
Kefu Chai
1be2eb114b test/osd-scrub-repair.sh: use repair() instead of "ceph pg repair"
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-11-17 18:51:52 +08:00
Kefu Chai
15649431a0 Merge pull request from tchaikov/wip-ceph-objectstore-tool
test: ceph-objectstore-tool: should import platform before using it

Reviewed-by: Owen Synge <osynge@suse.com>
2016-11-17 18:13:47 +08:00
Sébastien Han
def903a7ef doc: openstack glance mitaka uses show_multiple_locations
As of the Mitaka release show_image_direct_url is not needed, but
instead show_multiple_locations should be used.
Adding the necessary guidance for Mitaka release.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-11-17 10:57:36 +01:00
Kefu Chai
e3de5f4a08 test: ceph-objectstore-tool: should import platform before using it
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-11-17 17:20:57 +08:00
Kefu Chai
2c7f08b849 Merge pull request from dzafman/wip-16064
common osd: Improve scrub analysis, list-inconsistent-obj output and osd-scrub-repair test

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 15:48:32 +08:00
Kefu Chai
7c471eaf5a Merge pull request from wjwithagen/wip-wjw-tests-ceph_objectstore_tool
ceph_objectstore_tool.py: Don't use btrfs on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 14:57:51 +08:00
Kefu Chai
3a00c50a72 Merge pull request from liupan1111/wip-code-clean-ups
os/filestore: handle EINTR returned by io_getevents()

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 14:54:49 +08:00
Kefu Chai
761f193282 Merge pull request from bassamtabbara/wip-crushtool-fix
mon: if crushtool config is empty use internal crush test

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 14:53:28 +08:00
xie xingguo
3bf051d6f9 os/bluestore: fix bufferspace stats leak due to blob splitting
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-17 14:51:31 +08:00
Kefu Chai
5445a3dd0e Merge pull request from dzafman/wip-test-17830
test: fix test-erasure-eio and osd-scrub-repair races (17830)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-11-17 14:39:17 +08:00
Xie Xingguo
b24bd44149 Merge pull request from branch-predictor/bp-rmxattr-bl
librados: remove unused bufferlist from rados_write_op_rmxattr

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-17 10:24:08 +08:00
David Zafman
d0410062e2 test: Add sleep 1 so that we look for 60 seconds
This failed because test blew through 60 requests in 8 seceons before
the scrub repair even started on Jenkins.

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 17:58:09 -08:00
David Zafman
b7657c5431 test: Fix duplicate port in test-erasure-code-plugins.sh
Add comments about uniqueness of port number required

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 17:56:17 -08:00
Sage Weil
20b58ba6c9 Merge pull request from liewegas/wip-bluestore-warning
os/bluestore: fix warning and uninit variable

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-11-16 18:29:52 -06:00
David Zafman
006ab8dfe7 mon: Fix compiler warnings, missed _dump() change
Caused by: af720cc87b

Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 15:40:41 -08:00
David Zafman
db7be41c1e test: Re-enable osd-scrub-repair.sh
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 15:40:40 -08:00
David Zafman
d30056a16a test: In osd-scrub-repair.sh fix use of wait_for_clean()
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-11-16 15:40:40 -08:00
Loic Dachary
88ce18da90 Merge pull request from dachary/wip-17849-bootstrap-osd
mon,ceph-disk: add lockbox permissions to bootstrap-osd

Reviewed-by: Sage Weil <sage@redhat.com>
2016-11-17 00:40:25 +01:00
Sage Weil
ee84e78f90 include/mempool: silence warnings
/home/sage/src/ceph4/src/include/mempool.h:421:68: warning: no return statement in function returning non-void [-Wreturn-type]
   void *operator new[](size_t size) { assert(0 == "no array new"); } \

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-16 17:14:10 -06:00
Sage Weil
28ebc35c89 os/bluestore/KernelDevice: initialize oldest
Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-16 17:14:10 -06:00
Gregory Farnum
41408ec921 Merge pull request from batrick/mon-_dump-bugfix
mon: remove utime_t param in _dump

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-11-16 13:58:38 -08:00
Piotr Dałek
7b3643decc librados: remove unused bufferlist from rados_write_op_rmxattr
rados_write_op_rmxattr declares the bl bufferlist variable, then doesn't make
any use of it. Get rid of it, as it's probably an effect of copy+paste.

Signed-off-by: Piotr Dałek <git@predictor.org.pl>
2016-11-16 22:52:53 +01:00
Gregory Farnum
b017355626 Merge pull request from batrick/client-redundant-init
client: remove redundant initialization

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-11-16 13:38:55 -08:00