Commit Graph

97724 Commits

Author SHA1 Message Date
Sage Weil
8de8eadc6e Merge PR #27786 into master
* refs/pull/27786/head:
	qa/suites/upgrade/*-x-singleton: enable bluestore debugging settings

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 13:23:04 -05:00
Sage Weil
c25d26375c Merge PR #27789 into master
* refs/pull/27789/head:
	os/bluestore: correctly measure deferred writes into new blobs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-04-26 10:40:47 -05:00
Sage Weil
908ea39b3b Merge PR #27655 into master
* refs/pull/27655/head:
	common/options: flag misc ms_* options STARTUP
	common/options: flag misc options STARTUP
	common/options: mark cluster log options with FLAG_RUNTIME, use get_val
	common/options: mark a bunch of options with FLAG_STARTUP

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 10:40:31 -05:00
Jason Dillaman
483074ef7a
Merge pull request #27724 from idryomov/wip-rbd-copyup-zeros
librbd: snapshot object maps can go inconsistent during copyup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-26 08:11:58 -04:00
Jason Dillaman
334241998e
Merge pull request #27755 from trociny/wip-rbd-io-scheduler-leak
librbd: leak on canceling simple io scheduler timer task

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-26 08:11:17 -04:00
J. Eric Ivancich
8a4a6ab196
Merge pull request #27411 from ivancich/wip-clean-up-logging
rgw: clean up some logging

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-04-25 18:18:57 -04:00
J. Eric Ivancich
1b33207372
Merge pull request #27456 from wjwithagen/wjw-fix-rgw_amqp.cc
rgw: use the compatibilty function for pthread_setname

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-25 18:17:21 -04:00
J. Eric Ivancich
a553edbb4b
Merge pull request #27617 from cbodley/wip-39295
rgw: bucket stats report mtime in UTC

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2019-04-25 18:14:59 -04:00
David Zafman
39cc14bdc1
Merge pull request #27503 from dzafman/wip-39099
osd: Give recovery for inactive PGs a higher priority

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-25 15:06:56 -07:00
Neha Ojha
5f34369c71
Merge pull request #27764 from neha-ojha/wip-fix-mimic-x-singleton
qa/suites/upgrade: use correct branch names

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-25 14:42:24 -07:00
Sage Weil
de92bc877e common/options: flag misc ms_* options STARTUP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 16:26:06 -05:00
Sage Weil
0c944c0f51 common/options: flag misc options STARTUP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 16:26:06 -05:00
Sage Weil
5dcf3676a6 common/options: mark cluster log options with FLAG_RUNTIME, use get_val
We already have observers set up for these; switch to using the
runtime-safe accessor and flag them.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 16:26:06 -05:00
David Zafman
71d254647a test: osd-recovery-scrub.sh ignore error from kill_daemons()
Another work around for http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
796794b38b osd: Add new object_location_counts to pg_stat_t
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
97f3b7c2e9 mgr: If the requested OSD is down don't trust osd_stat info
If we have a down AND out OSD it may contains osd_stat with num_pgs == 0.
When all PGs aren't active+clean we need an accurate value or we consider
the osd missing stat info.

Fixes: https://tracker.ceph.com/issues/38930

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
69eaaaadd0 mon, mgr: Improve PG count by changing up to up_no_acting in pg_count
By not adding a field to pg_count, we don't have to worry about
versioning or backwards compatibility.  A mixed cluster won't
be any worse than before.

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
9750061d5d mgr: Improve ok-to-stop by using the avail_no_missing for recovery
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
42497780db osd: Prevent priority from overflowing in the next base level
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
71d82dbeb9 test: Add tests for pool recovery priority conversion
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
David Zafman
444aa9f9fe osd, mon: New pool recovery priority range -10 to 10
Use OSD_POOL_PRIORITY_MAX and OSD_POOL_PRIORITY_MIN constants
Scale legacy priorities if exceeds maximum

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
Abhishek L
b0ecd0a3b7
Merge pull request #27781 from theanalyst/rgw-admin-delete-at
rgw: admin: handle delete_at attr in object stat output

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2019-04-25 19:40:47 +02:00
Sage Weil
6fe8540b57 os/bluestore: correctly measure deferred writes into new blobs
Writes into new blobs were all counted as write_small_new, but those can
still be deferred later in _do_alloc_write if they are <= than the
prefer_deferred setting.

See http://tracker.ceph.com/issues/38816

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 10:49:04 -05:00
Ilya Dryomov
5b9d85c136 librbd: the first post-migration snapshot isn't always dirty
Currently, the first post-migration snapshot is always marked EXISTS
(i.e. dirty).  This is wrong, because the data can be inherited from
a pre-migration snapshot, handled by deep copy.

Mark all post-migration snapshots EXISTS_CLEAN in this case.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-04-25 17:15:12 +02:00
Ilya Dryomov
4456dc3939 librbd: don't update snapshot object maps if copyup data is all zeros
If the data read from the parent is all zeros, deep copyup isn't
performed.  However snapshot object maps are updated unconditionally,
causing inconsistencies where nonexistent objects are marked
OBJECT_EXISTS or OBJECT_EXISTS_CLEAN.

Fixes: http://tracker.ceph.com/issues/39435
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-04-25 17:15:12 +02:00
Sage Weil
0fe0ec3343 qa/suites/upgrade/*-x-singleton: enable bluestore debugging settings
We default to bluestore already; use the yaml explicitly so that we get
debugging turned up.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 08:33:56 -05:00
Abhishek Lekshmanan
90999a4de8 rgw: admin: handle delete_at attr in object stat output
Fixes: https://tracker.ceph.com/issues/39481

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-04-25 14:13:33 +02:00
Kefu Chai
878e488be3
Merge pull request #27574 from iotcg/iwarp_rocev2
msg/async/rdma: fix memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-25 16:35:07 +08:00
Changcheng Liu
d775a46959 msg/async/rdma: use ibv_device_attr object type in Device class
1. Avoid to do memory management without using pointer to operate the
allocated space.
2. Since member type has been changed in class Device, it use member
domain operator "." to access to the sub-member in object.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-25 14:08:00 +08:00
Changcheng Liu
ad8a9d542f msg/async/rdma: free memory when Infiniband obj is freed
device_list is allocated in Infiniband::init.
When Infiniband obj is destructed, free the allocated
space in case of memory leak.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-25 14:08:00 +08:00
Changcheng Liu
09c9fa11ac msg/async/rdma: check allocated memory immediately before using it
The new "allocation functions" will through exception if it failes to
allocate space. There's no need to assert(devices) here for every
get_device operation.
Refer to: https://en.cppreference.com/w/cpp/memory/new/bad_alloc

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-25 14:08:00 +08:00
Changcheng Liu
7a35128e39 msg/async/rdma: free member hold memory when destructing obj
device_context_list hold the memory allocated through
ibv_get_device_list. release the memory through calling
ibv_free_device_list.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-25 14:08:00 +08:00
Changcheng Liu
8328312a0d msg/sync/rdma: abort rdma operation when ibv_query_device failed
ibv_query_device return none zero when failing to query rdma
device. run ceph_abort in any case when ibv_query_device return
none zero.

Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-25 14:07:50 +08:00
Kefu Chai
49117a0414
Merge pull request #27741 from tchaikov/doc/dev/wip-corpus.rst
doc/dev/corpus.rst: correct instructions

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-25 13:30:10 +08:00
Sage Weil
30a8d911eb Merge PR #27626 into master
* refs/pull/27626/head:
	mgr/dashboard: Do not default to 'admin' as Admin Resource

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Gabriel Brascher <gabriel@apache.org>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
2019-04-24 20:40:17 -05:00
Sage Weil
5ef26e08ca Merge PR #26184 into master
* refs/pull/26184/head:
	mgr/dashboard: Replace IP address validation with Python standard library functions

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-04-24 20:38:19 -05:00
Sage Weil
d64d14a577 Merge PR #26730 into master
* refs/pull/26730/head:
	src/test: remove unused objects to calculate accurate dedup ratio.
	src/tools: fix counting total objects
	src/tools: fix printing wrong offset.
	src/tools: fix dedup tool's name
	qa/workunit: add rabin chunk test
	src/tool: add rabin fingerprint option to cephdeduptool
	src/common: add rabin fingerprint class
	Initail work for rabin fingerprint

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-24 20:38:02 -05:00
David Zafman
f0e7202cff osd: For recovery track OSDs that have 0 missing to know how degraded we are
Add backfill priority log message and remove redundanacy to recovery priority

Fixes: https://tracker.ceph.com/issues/39099

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-24 15:09:23 -07:00
Sage Weil
66dac9d28a Merge PR #27566 into master
* refs/pull/27566/head:
	auth: kill AuthUnknownSessionHandler
	auth: kill AuthUnknown* (except AuthUnknownSessionHandler)
	msg: remove old ms_* auth methods from Dispatcher interface
	mon/MonClient: discard old challenge if authorizer is bad
	msg/async/ProtocolV1: use AuthServer and AuthClient
	mon/Monitor: handle v1 call into handle_auth_request
	msg/Connection: add is_msgr2()
	mon/MonClient: tolerate lack of authorizer for some dispatchers

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-24 17:00:57 -05:00
Casey Bodley
7e424e7208
Merge pull request #27579 from cbodley/wip-rgw-split-log-trim
rgw: split mdlog/datalog trimming into separate files

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-04-24 16:07:33 -04:00
Neha Ojha
5f1702e87c qa/suites/upgrade: use correct branch names
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-04-24 13:01:20 -07:00
Casey Bodley
7b27e627a7
Merge pull request #26173 from cbodley/wip-rgw-async-putobjprocessor
rgw: add YieldingAioThrottle for async PutObj/GetObj

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-04-24 15:47:59 -04:00
Sage Weil
a452d0707b auth: kill AuthUnknownSessionHandler
The only user is ProtocolV1, which passes in an in-use auth_method...
which must be either AUTH_NONE or AUTH_CEPHX since we successfully
authenticated at this point.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-24 13:46:04 -05:00
Sage Weil
31d71e8597 auth: kill AuthUnknown* (except AuthUnknownSessionHandler)
These are unreferenced by any other code.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-24 13:46:04 -05:00
Sage Weil
91c6016dd9 msg: remove old ms_* auth methods from Dispatcher interface
In the mon there is an opportunity to further clean up the code around
get_authorizer().

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-24 13:46:04 -05:00
Sage Weil
fc8d9c1135 mon/MonClient: discard old challenge if authorizer is bad
If we are a server/accepter, and we get a bad authorizer, discard our old
challenge, since the previous exchange has failed.  This way if the client
retries, their new authorize attempt won't conflict with our old challenge.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-24 13:46:04 -05:00
Sage Weil
c58c5754df msg/async/ProtocolV1: use AuthServer and AuthClient
Stop using the old ms_* auth methods and instead use the new interfaces
(like V2).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-24 13:46:04 -05:00
David Zafman
3a234164d0
Merge pull request #27279 from dzafman/wip-divergent
Improvements to standalone tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-24 10:58:11 -07:00
Kefu Chai
67a34f2cf8
Merge pull request #27750 from tchaikov/wip-silence-gcc-warning
test/objectstore: silence -Wsign-compare warning

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-04-25 00:18:48 +08:00
Sage Weil
dd93973894 Merge PR #27721 into master
* refs/pull/27721/head:
	tools: pin the version of breathe that works with Python2

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-24 10:48:17 -05:00