Commit Graph

79385 Commits

Author SHA1 Message Date
Sage Weil
6a91d20936 ceph-objectstore-tool: fix import of post-split pg from pre-split ancestor
If we export a PG, it later splits, and then we try to import a
post-split portion of it, we need to generate the PastIntervals
properly (instead of crashing).

Fixes: http://tracker.ceph.com/issues/21753
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-10 17:02:56 -05:00
Patrick Donnelly
7feb6b28f0
Merge PR #18223 into master
* refs/pull/18223/head:
	githubmap: define merge conflict resolution

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-10 14:33:59 -07:00
Ken Dreyer
0fcdba5339 doc: rm stray ")" character from mds config ref
I accidentally introduced this in
4fb89a6317

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-10-10 15:11:46 -06:00
Adam C. Emerson
b3118cabb8 rgw: Remove assertions in IAM Policy
A couple of them could be triggered by user input.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-10 16:21:48 -04:00
Patrick Donnelly
e0e348eedf
githubmap: define merge conflict resolution
Use "union" merge to automatically resolve trivial append merge conflicts from
using ptl-tool.py.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-10 12:06:34 -07:00
Matt Benjamin
65b7543390 Merge pull request #17882 from ZVampirEM77/wip-rgw-encryption
rgw: clean up and fix some bugs for encryption
2017-10-10 14:56:08 -04:00
Adam C. Emerson
bbefcc2f0b config: Remove _get_val
I added this originally to prevent a deadlock when reading a new-style
configuration variable from a called observer. I didn't realize at the
time that the config mutex was recursive, so this change was
superfluous.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-10 14:43:17 -04:00
Sage Weil
85055e81d7 Merge pull request #17759 from liewegas/wip-21407
osd: fix waiting_for_peered vs flushing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-10-10 13:17:01 -05:00
Dan Mick
6309b81804 Merge pull request #18208 from jan--f/mgr-prometheus-fix-undef-metrics
pybind/mgr/prometheus: fix metric type undef -> untyped

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: John Spray <jspray@redhat.com>
2017-10-10 10:38:49 -07:00
Mykola Golub
069c8dc2f4 Merge pull request #18136 from dillaman/wip-21561
rbd-mirror: primary image should register in remote, non-primary image's journal

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-10 18:01:44 +03:00
Mykola Golub
1dbf7f37fe Merge pull request #18158 from dillaman/wip-21711
librbd: refresh image after applying new metadata

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-10 18:00:52 +03:00
Kefu Chai
8dc100d3f3 ceph-kvstore-tool: use unique_ptr<> manage the lifecycle of bluestore and db
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-10 22:28:42 +08:00
myoungwon oh
07d460c17c osd: generalize PGSrub, PGRecovery, PGSnapTrim
This commit will allow PGScrub, PGRecovery, PGSnapTrim
to be created anywhere (e.g. TestMClockClientQueue.cc)

Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 23:18:22 +09:00
myoungwon oh
e9416d524f osd: use get_pgid() instead of info.pgid
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 23:18:03 +09:00
myoungwon oh
3d608c1857 osd: use global id instead of entity_inst_t
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 23:01:07 +09:00
Kefu Chai
b679abcfc5 osd: move pgid into OpQueueItem
this pave the road to generalize OpWQ

We're going to want to be able to queue things that are not ordered by
the PG lock.  To that end, this patch genearlizes OSD::ShardedOpWQ to
use a type which can specify an ordering token and locking structure
other than a PG.

There is a lot of collateral damage which I didn't feel was worth
separating out into other commits.  The code in ShardedOpWQ itself got
some superficial cleanup.  Also, the item being queued has been switched
to not use a boost::variant.  It was a cute way before to make the type
easily copyable, but adding more visitors for the locking support would
have been annoying.  Instead, the variant is a unique_ptr to an
interface.  This makes the queue item type no longer copyable, which is
just as well since we don't really want to be copying queue items anyway
(duplicates would be most likely a bug)

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 23:00:45 +09:00
Sage Weil
47a5fcc404 osd/PG: fix clang build vs private state events
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-10 08:29:54 -05:00
Jason Dillaman
ad01a883c5 librbd: refresh image after applying new/removing old metadata
Fixes: http://tracker.ceph.com/issues/21711
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-10 09:10:26 -04:00
Kefu Chai
260f87a9f1 mon/PGMap: let pg_string_state() return boost::optional<>
better to be specific, so we don't run into the same problem even we
are using the MSB of uint64_t for a pg state bit in future. we could,
however use uint64_t(-1) to indicate the pg_string_state()'s failure to
parse the state string, because pg_string_state() only translate a
single state a time. but it's always better to be explicit than
implicit.

Fixes: http://tracker.ceph.com/issues/21609
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-10 20:41:48 +08:00
Kefu Chai
90b9591c7d mon/PGMap: use uint64_t for pg state
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-10 20:41:48 +08:00
myoungwon oh
9e12935240 src/test: fix move semantics
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 21:37:10 +09:00
Kefu Chai
90e71c5ff8 common,osd: enforce move semantics in WorkQueue
enable the WorkQueue to support move-only template types.

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 21:32:42 +09:00
Kefu Chai
0c9c5ffe0f cmake: set supported language the right way
the second param of enable_launage() is not used, we should call it
multiple times to enable more than one language. switch to project()
command for simplicity.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-10 19:54:31 +08:00
Yan, Zheng
9d5c92a336 mds: update client metadata for already open session
session opened by Server::prepare_force_open_sessions() has no
client metadata.

Fixes: http://tracker.ceph.com/issues/21746
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-10-10 17:54:30 +08:00
lu.shasha
3189edc28e rgw: don't change rados object's mtime when update olh
null instance and olh share rados object. Null instance using the rados object mtime as its mtime. When olh updated, the rados object mtime will change.
Thus the null instance will be incorrect. When list objects in bucket, the mtime is correct, down the null instance the last modified time is incorrect.
So when update olh, using the previous mtime, thus the null instance mtime will be correct.

Fixes: http://tracker.ceph.com/issues/21743

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
2017-10-10 16:54:00 +08:00
Kefu Chai
f29d304af4 log: fix build on osx
we can not assume that the the `rep` type is identical to `time_t` and
`susecond_t`, on osx they are `int`, not `int64_t`. so cast they as
necessary.

this fixes the error and warning of

LogClock.h:112:7: error: non-constant-expression cannot be narrowed from type 'rep' (aka 'long long') to '__darwin_suseconds_t' (aka 'int') in initializer list [-Wc++11-narrowing]
             std::chrono::duration_cast<std::chrono::microseconds>(
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/kefu/dev/ceph/src/log/LogClock.h:112:7: note: insert an explicit cast to silence this issue
             std::chrono::duration_cast<std::chrono::microseconds>(
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LogClock.h:139:46: warning: format specifies type 'long' but the argument has type 'int' [-Wformat]
                      bdt.tm_hour, bdt.tm_min, bdt.tm_sec, tv.tv_usec / 1000);
                                                           ^~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-10 15:41:56 +08:00
Jan Fajerski
6306392492 pybind/mgr/prometheus: fix metric type undef -> untyped
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2017-10-10 08:40:31 +02:00
Kefu Chai
6cd9c71de6 Merge pull request #18205 from yaozongyou/fix-bluestore-cache-kv-max-type
doc: correct wrong bluestore config types

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-10 12:49:19 +08:00
Yao Zongyou
7966d3cd1d doc: fix bluestore config type error.
Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
2017-10-10 11:34:26 +08:00
Kefu Chai
526246db3e Merge pull request #18161 from Liuchang0812/wip-bluestore-ebusy
os/bluestore: do not core dump when we try to open kvstore twice

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-10 10:55:07 +08:00
Kefu Chai
7d155c4cd0 Merge pull request #18149 from tchaikov/wip-cleanups
common, osd: various cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-10 10:53:52 +08:00
Kefu Chai
a753282796 Merge pull request #18177 from tchaikov/wip-kv-kill-warning
kv: unify {create_and_,}open() methods

Reviewed-by: Sage Weil <sage@redhat.com>
2017-10-10 10:52:53 +08:00
Jason Dillaman
d774d4702f rbd-mirror: removed duplicate client registration logic
The client will now be registered before the bootstrap state machine
is invoked.

Fixes: http://tracker.ceph.com/issues/21561
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-09 20:53:51 -04:00
Jason Dillaman
d74b95dc10 rbd-mirror: local primary images should still register to remote images
Fixes: http://tracker.ceph.com/issues/21561
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-09 20:53:50 -04:00
John Spray
980f18d319 Merge pull request #18195 from ktdreyer/doc-mds-blacklist-interval-manual
doc: "mds blacklist interval" vs manually blacklisting
2017-10-09 23:31:09 +01:00
Greg Farnum
165b61a7df common: by default, do not assert on leaks in the shared_cache code
Update the standard qa suite cluster configs so that we continue
asserting in our nightlies, but users don't hit this.

Fixes: http://tracker.ceph.com/issues/21737

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2017-10-09 14:16:05 -07:00
J. Eric Ivancich
e98cd6f2e0 rgw: consolidate code that implements hashing algorithms
Copies of the same hashing algorithm are scattered throughout the rgw
codebase. Consolidate them.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2017-10-09 16:29:52 -04:00
Sage Weil
8f7dc8b0d8 osd: fix waiting_for_peered vs flushing
on_flush() requeues waiting_for_peered, but we flush twice on the
primary during peering, and we don't want to requeue the first one
(when we have the master pg log merged).

Fix by moving waiting_for_peered to waiting_for_flush if we aren't
already flush on _activate_committed.  If we get an op and are
peered but not flushed, queue ops there.  (We can simplify this
check a bit since pgbackend inactive message handling doesn't care
about flushed or not flushed.)  When flushed, we requeue
waiting_for_flush.

The waiting_for_flush, waiting_for_peered, and waiting_for_active
lists are all mutually exclusive, so this mostly serves to
clarify what we are waiting for (not to keep items separate). And
it means that on_flushed() will only requeue things that were
waiting for it specifically.

Fixes: http://tracker.ceph.com/issues/21407
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-09 15:27:51 -05:00
Kefu Chai
d28506d7ff Merge pull request #18188 from jschmid1/wip-man-osd-config-ref-syntax
doc: fixes syntax in osd-config-ref

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-10 01:01:00 +08:00
Kefu Chai
8c5fea68cb enforce move semantics of Request type
Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-10 01:32:29 +09:00
Kefu Chai
b0ab76a80f OSD: s/PGQueueable/OpQueueItem/g
PGQueueable was always a pretty bad name, and going forward, we won't
necessarily be scheduling things ordered only by the PG lock, so let's
change it.

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 01:31:51 +09:00
Samuel Just
b912def13e PG.h: remove some unnecessary virtual declarations
PG users do not need these public methods.

Signed-off-by: Samuel Just <sjust@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2017-10-10 01:06:51 +09:00
Joshua Schmid
a657717449
doc: fix syntax in rst
* Fix syntax in osd max object size section.

Fixes: http://tracker.ceph.com/issues/21733
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2017-10-09 17:31:45 +02:00
Ken Dreyer
4fb89a6317 doc: "mds blacklist interval" vs manually blacklisting
The "mds blacklist interval" setting has no effect on the time that
the "ceph osd blacklist" command will use by default. Clarify this in
the docs.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-10-09 09:28:38 -06:00
Casey Bodley
f390fc1df9 rgw: remove error handling for get_random_bytes
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-09 10:42:25 -04:00
Casey Bodley
c678cc574c cls: remove error handling for get_random_bytes
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-09 10:42:25 -04:00
Casey Bodley
1ae1cdb752 common: add CryptoRandom to CephContext
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-09 10:42:25 -04:00
Casey Bodley
b605b5033b auth: add CryptoRandom wrapper for random impl
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-09 10:42:25 -04:00
Casey Bodley
513fa79f2d cmake: detect presence of getentropy
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-09 10:42:25 -04:00
Casey Bodley
6acfeeff53 random: use ceph::util for non-cryptographic randomness
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-10-09 10:42:24 -04:00