Commit Graph

97755 Commits

Author SHA1 Message Date
Yuval Lifshitz
10b8b57d72 rgw/pubsub: revert the RGWSysObjectCtx change
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-05-06 23:08:48 +03:00
Yuval Lifshitz
b4005db82d rgw/pubsub: cleanup tests for multiple notifications
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-05-05 11:31:20 +03:00
Casey Bodley
c0b9d10901 rgw: RGWUserPubSub::write invalidates its obj_ctx
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-05-05 10:06:48 +03:00
Yuval Lifshitz
3279a786ba rgw/pubsub: fix doc on updates. attempt to fix multi-notifications
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-05-02 20:33:34 +03:00
Yuval Lifshitz
0e8ac0618c
Merge pull request #27671 from yuvalif/fix_arn_issue
rgw/pubsub: fix topic arn. tenant support to multisite tests
2019-04-28 11:43:02 +03:00
Mykola Golub
ab1c8043b0
Merge pull request #27757 from dillaman/wip-39021-2
librbd: copyup read stats were incorrectly tied to child

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-27 18:58:53 +03:00
Sage Weil
cf45294686 Merge PR #18694 into master
* refs/pull/18694/head:
	Merge branch 'master' into sk-add_current_num_in_quota_info
	mon/OSDMonitor.cc: Add current numbers of objects and bytes

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-27 10:41:18 -05:00
Sage Weil
f5314cfb33 Merge PR #22411 into master
* refs/pull/22411/head:
	common/WorkQueue: narrow ThreadPool::_lock in func worker.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-04-27 10:41:01 -05:00
Sage Weil
320acd45a0 Merge PR #27821 into master
* refs/pull/27821/head:
	common/options: allow osd_pool_default_pg_autoscale_mode to update a runtime

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-27 10:40:44 -05:00
Sage Weil
f4c4e112b0 Merge PR #27823 into master
* refs/pull/27823/head:
	msg/async/ProtocolV1: generate authorizer for mon to mon connections
	mon/Monitor: require authorizer from peer monitors

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-27 10:40:29 -05:00
Neha Ojha
f4684eb864
Merge pull request #27798 from neha-ojha/wip-39441
osd/PG: do not use approx_missing_objects pre-nautilus

Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-04-26 16:00:53 -07:00
David Zafman
a8b8ce5c75
Merge pull request #27209 from dzafman/wip-38840
osd: Don't evict after a flush if intersecting scrub range

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 15:23:37 -07:00
David Zafman
66b041fa4a
Merge pull request #27769 from dzafman/wip-39333
osd-backfill-space.sh test failed in TEST_backfill_multi_partial()

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 11:55:04 -07:00
Sage Weil
d1b8093174 Merge PR #27732 into master
* refs/pull/27732/head:
	common/assert: include ceph_abort_msg(arg) arg in log output

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-04-26 13:24:01 -05:00
Sage Weil
d2520707e0 Merge PR #27729 into master
* refs/pull/27729/head:
	osd: take heartbeat_lock when calling heartbeat()

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-26 13:23:36 -05:00
Sage Weil
fe3ead1802 Merge PR #26176 into master
* refs/pull/26176/head:
	os/bluestore: do not check osd_max_object_size in _open_path()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-26 13:23:21 -05:00
Sage Weil
8de8eadc6e Merge PR #27786 into master
* refs/pull/27786/head:
	qa/suites/upgrade/*-x-singleton: enable bluestore debugging settings

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 13:23:04 -05:00
Sage Weil
b9a04409aa msg/async/ProtocolV1: generate authorizer for mon to mon connections
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-26 13:18:42 -05:00
Sage Weil
e2f32372a4 mon/Monitor: require authorizer from peer monitors
Mon to mon links use authorizers.  It's only non-mons connecting to mons
that use MAuth messages.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-26 13:07:18 -05:00
David Zafman
9931023457 test: osd-backfill-spsace.sh doesn't matter which PG wins the race
Fixes: http://tracker.ceph.com/issues/39333

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-26 10:11:00 -07:00
Sage Weil
23e6b0c5ea common/options: allow osd_pool_default_pg_autoscale_mode to update a runtime
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-26 11:26:21 -05:00
Sage Weil
c25d26375c Merge PR #27789 into master
* refs/pull/27789/head:
	os/bluestore: correctly measure deferred writes into new blobs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-04-26 10:40:47 -05:00
Sage Weil
908ea39b3b Merge PR #27655 into master
* refs/pull/27655/head:
	common/options: flag misc ms_* options STARTUP
	common/options: flag misc options STARTUP
	common/options: mark cluster log options with FLAG_RUNTIME, use get_val
	common/options: mark a bunch of options with FLAG_STARTUP

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 10:40:31 -05:00
Jason Dillaman
483074ef7a
Merge pull request #27724 from idryomov/wip-rbd-copyup-zeros
librbd: snapshot object maps can go inconsistent during copyup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-26 08:11:58 -04:00
Jason Dillaman
334241998e
Merge pull request #27755 from trociny/wip-rbd-io-scheduler-leak
librbd: leak on canceling simple io scheduler timer task

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-26 08:11:17 -04:00
J. Eric Ivancich
8a4a6ab196
Merge pull request #27411 from ivancich/wip-clean-up-logging
rgw: clean up some logging

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-04-25 18:18:57 -04:00
J. Eric Ivancich
1b33207372
Merge pull request #27456 from wjwithagen/wjw-fix-rgw_amqp.cc
rgw: use the compatibilty function for pthread_setname

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-25 18:17:21 -04:00
J. Eric Ivancich
a553edbb4b
Merge pull request #27617 from cbodley/wip-39295
rgw: bucket stats report mtime in UTC

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2019-04-25 18:14:59 -04:00
David Zafman
39cc14bdc1
Merge pull request #27503 from dzafman/wip-39099
osd: Give recovery for inactive PGs a higher priority

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-25 15:06:56 -07:00
Neha Ojha
4c617ecf1c osd/PG: do not use approx_missing_objects pre-nautilus
We changed async recovery cost calculation in nautilus to also take
into account approx_missing_objects in ab241bf7e9
This commit depends on https://github.com/ceph/ceph/pull/23663, hence
wasn't backported to mimic.

Mimic only uses the difference in length of logs as the cost. Due to this,
the same OSD might have different costs in a mixed mimic and nautilus(or above)
cluster. This can lead to choose_acting() cycling between OSDs, when trying
to select the acting set and async_recovery_targets.

Fixes: https://tracker.ceph.com/issues/39441
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-04-25 14:44:15 -07:00
Neha Ojha
5f34369c71
Merge pull request #27764 from neha-ojha/wip-fix-mimic-x-singleton
qa/suites/upgrade: use correct branch names

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-25 14:42:24 -07:00
Sage Weil
de92bc877e common/options: flag misc ms_* options STARTUP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 16:26:06 -05:00
Sage Weil
0c944c0f51 common/options: flag misc options STARTUP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 16:26:06 -05:00
Sage Weil
5dcf3676a6 common/options: mark cluster log options with FLAG_RUNTIME, use get_val
We already have observers set up for these; switch to using the
runtime-safe accessor and flag them.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 16:26:06 -05:00
David Zafman
71d254647a test: osd-recovery-scrub.sh ignore error from kill_daemons()
Another work around for http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
796794b38b osd: Add new object_location_counts to pg_stat_t
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
97f3b7c2e9 mgr: If the requested OSD is down don't trust osd_stat info
If we have a down AND out OSD it may contains osd_stat with num_pgs == 0.
When all PGs aren't active+clean we need an accurate value or we consider
the osd missing stat info.

Fixes: https://tracker.ceph.com/issues/38930

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
69eaaaadd0 mon, mgr: Improve PG count by changing up to up_no_acting in pg_count
By not adding a field to pg_count, we don't have to worry about
versioning or backwards compatibility.  A mixed cluster won't
be any worse than before.

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
9750061d5d mgr: Improve ok-to-stop by using the avail_no_missing for recovery
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
42497780db osd: Prevent priority from overflowing in the next base level
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
71d82dbeb9 test: Add tests for pool recovery priority conversion
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
444aa9f9fe osd, mon: New pool recovery priority range -10 to 10
Use OSD_POOL_PRIORITY_MAX and OSD_POOL_PRIORITY_MIN constants
Scale legacy priorities if exceeds maximum

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
Abhishek L
b0ecd0a3b7
Merge pull request #27781 from theanalyst/rgw-admin-delete-at
rgw: admin: handle delete_at attr in object stat output

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2019-04-25 19:40:47 +02:00
Sage Weil
6fe8540b57 os/bluestore: correctly measure deferred writes into new blobs
Writes into new blobs were all counted as write_small_new, but those can
still be deferred later in _do_alloc_write if they are <= than the
prefer_deferred setting.

See http://tracker.ceph.com/issues/38816

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 10:49:04 -05:00
Ilya Dryomov
5b9d85c136 librbd: the first post-migration snapshot isn't always dirty
Currently, the first post-migration snapshot is always marked EXISTS
(i.e. dirty).  This is wrong, because the data can be inherited from
a pre-migration snapshot, handled by deep copy.

Mark all post-migration snapshots EXISTS_CLEAN in this case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-04-25 17:15:12 +02:00
Ilya Dryomov
4456dc3939 librbd: don't update snapshot object maps if copyup data is all zeros
If the data read from the parent is all zeros, deep copyup isn't
performed.  However snapshot object maps are updated unconditionally,
causing inconsistencies where nonexistent objects are marked
OBJECT_EXISTS or OBJECT_EXISTS_CLEAN.

Fixes: http://tracker.ceph.com/issues/39435
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-04-25 17:15:12 +02:00
Sage Weil
0fe0ec3343 qa/suites/upgrade/*-x-singleton: enable bluestore debugging settings
We default to bluestore already; use the yaml explicitly so that we get
debugging turned up.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 08:33:56 -05:00
Abhishek Lekshmanan
90999a4de8 rgw: admin: handle delete_at attr in object stat output
Fixes: https://tracker.ceph.com/issues/39481

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-04-25 14:13:33 +02:00
Kefu Chai
878e488be3
Merge pull request #27574 from iotcg/iwarp_rocev2
msg/async/rdma: fix memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-25 16:35:07 +08:00
Changcheng Liu
d775a46959 msg/async/rdma: use ibv_device_attr object type in Device class
1. Avoid to do memory management without using pointer to operate the
allocated space.
2. Since member type has been changed in class Device, it use member
domain operator "." to access to the sub-member in object.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-25 14:08:00 +08:00