Commit Graph

94764 Commits

Author SHA1 Message Date
Radoslaw Zarzynski
c7a53ecfbc common: bufferlist can contain 0-length bptrs.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-02-01 22:54:50 +01:00
Radoslaw Zarzynski
0326cf33da common: contiguous_appender doesn't use append_buffer anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-02-01 22:54:50 +01:00
Radoslaw Zarzynski
a94e54af68 common: bl::appends don't use the append_buffer anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-02-01 22:54:50 +01:00
Sage Weil
54fd6dce26 Merge PR #25703 into master
* refs/pull/25703/head:
	os/bluestore: option to disable bluestore warns on spillover.
	mon: expose object store health warnings
	osd: expose object store alerts
	os/bluestore: indicate BlueFS spillover and lacking compressor alerts
	os:enable ObjectStore::statfs() api to return OS alert list.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-01 12:43:03 -06:00
Sage Weil
c4ad6044c1 Merge PR #25773 into master
* refs/pull/25773/head:
	common: Use OpenSSL for cryptographic hashing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-02-01 12:42:38 -06:00
Sage Weil
2129cfc431 Merge PR #26067 into master
* refs/pull/26067/head:
	pybind/rados: fix application metadata list

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-01 12:41:25 -06:00
Sage Weil
3b5b96b90a Merge PR #26112 into master
* refs/pull/26112/head:
	mgr: load modules in finisher to avoid potential lock cycles

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-02-01 12:40:56 -06:00
Sage Weil
5481532554 Merge PR #26164 into master
* refs/pull/26164/head:
	mon/LogMonitor: lower logging level for check_sub

Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-01 12:40:12 -06:00
Sage Weil
632a1033aa os/bluestore/KernelDevice: use flock(2) for block device lock
The fcntl locks fail due to the classic posix lock gotcha: if you close
*any* fd to the same inode from the process, the lock(s) go away.

Use flock(2) instead.  We have to be careful because we open the main
bluestore device via two KernelDevice instances: one for bluestore and
one for bluefs.  Add a no-lock flag so that the bluefs instance does not
try to lock and does not conflict with bluestore's.

Fixes: http://tracker.ceph.com/issues/38150
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-01 11:39:41 -06:00
Ilya Dryomov
fa32d97a9f rbd: recognize alloc_size map option
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-01 17:59:38 +01:00
Kefu Chai
e124ad72cf
Merge pull request #26224 from tchaikov/radek-lru-cleanup
common: drop the unused methods from SharedLRU.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-02 00:01:58 +08:00
Sebastian Wagner
d361f77a78
Merge pull request #26171 from sebastian-philipp/orchestrator-merge-create-add
mgr/orchestrator: Unify `osd create` and `osd add`

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2019-02-01 16:42:36 +01:00
Patrick Nawracay
c870477461
Merge pull request #26151 from nathan-weinberg/cluster-tests
mgr/dashboard: Added additional breadcrumb and tab tests to Cluster menu

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-02-01 16:09:11 +01:00
J. Eric Ivancich
fdae3a71d7
Merge pull request #26231 from ivancich/wip-infinite-bucket-rm
rgw: `radosgw-admin bucket rm ... --purge-objects` can hang...

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-02-01 09:08:06 -05:00
Nathan Weinberg
2b8ed2cf18 mgr/dashboard: Added additional breadcrumb and tab tests to Cluster menu
Fixes: https://tracker.ceph.com/issues/38050
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
2019-02-01 08:55:37 -05:00
Lenz Grimmer
eba6be512b
Merge pull request #26203 from rhcs-dashboard/updated-pg-status-chart-colors
mgr/dashboard: Updated colors in PG Status chart

Reviewed-by: Volker Theile <vtheile@suse.com>
2019-02-01 12:29:21 +01:00
Volker Theile
8210be29ad mgr: Use Py_BuildValue to create the argument tuple
Reduce the complexity of the code by using the generic function, Py_BuildValue(), that can create most common objects from C values, directed by a format string.

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-02-01 11:48:29 +01:00
Kefu Chai
5f030b0aa8
Merge pull request #25929 from falcon78921/wip-doc-37880
doc: updated rgw multitenancy link

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-01 17:21:54 +08:00
Sebastian Wagner
9e926e9927 mgr/orchestrator: Unify osd create and osd add
Also:

* Added some more tests
* Better validation of drive Groups
* Simplified `TestWriteCompletion`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-02-01 10:10:59 +01:00
Patrick Nawracay
ac8efbbed0 mgr/dashboard: Fix modified files only (frontend)
Introduces a new npm command: `npm run fixmod` which, unlike `npm run
fix`, only fixes modified files in the current git repository.  This is
faster than the comprehensive `npm run fix` solution and can hence be
called more often with less disturbance.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-02-01 09:02:59 +01:00
Abhishek L
d28ace9bef
Merge pull request #26228 from theanalyst/doc/releases/12.2.11
doc: add release notes for 12.2.11 luminous

Reviewed-By: Sage Weil <sage@redhat.com>
Reviewed-By: Neha Ojha <nojha@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-By: Eric Ivanvich <ivanvich@redhat.com>
2019-02-01 08:46:53 +01:00
Pritha Srivastava
512b6d8bd9 rgw: Adding documentation for controlling STS auth via policy.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2019-02-01 11:26:29 +05:30
Sage Weil
b7b78809f1 os/bluestore: add bluestore_ignore_data_csum option
This option just suppresses the EIO errors.  We still get the messages
in the logs, though.  This can be useful when you are in dire straights
and prefer some data to no data.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-31 17:33:57 -06:00
J. Eric Ivancich
5b3dc38c62 rgw: radosgw-admin bucket rm ... --purge-objects can hang...
This command can hang (i.e., enter an infinite loop) due to
problematic bucket index entries left as a result of bug
https://tracker.ceph.com/issues/38007 .

The fix is to ignore the false bucket index entries -- since they do
not represent actual objects -- and remove all actual objects in the
bucket, so that bucket itself can be removed.

This fixes the both code paths whether `--bypass-gc` is specified or
not.

Furthermore, to made these operations more efficient, the internal
listing of the bucket is done unordered. This would improve behavior
when removing buckets with a large number of objects.

Fixes: http://tracker.ceph.com/issues/38134
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-01-31 17:43:25 -05:00
Abhishek Lekshmanan
7c677047a6 doc: add release notes for 12.2.11 luminous
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 23:32:24 +01:00
Patrick Donnelly
359e091c80
Merge PR #26193 into master
* refs/pull/26193/head:
	mds: move session setup to ms_handle_accept
	common: add method to get StackStringStream ptr

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-31 12:35:36 -08:00
Patrick Donnelly
0d26266ccb
Merge PR #26038 into master
* refs/pull/26038/head:
	mds: simplify recall warnings
	mds: add extra details for cache drop output
	qa: test mds_max_caps_per_client conf
	mds: limit maximum number of caps held by session
	mds: adapt drop cache for incremental recall
	mds: recall caps incrementally
	mds: adapt drop cache for incremental trim
	mds: add throttle for trimming MDCache
	mds: cleanup SessionMap init
	mds: cleanup Session init

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-31 12:08:26 -08:00
Patrick Donnelly
c0b3a11484
mds: simplify recall warnings
Instead of a timeout and complicated decisions about whether the client is
releasing caps in an expeditious fashion, just use a DecayCounter that tracks
the number of caps we've recalled. This counter is decremented whenever the
client releases caps. If the counter passes a threshold, then we raise the
warning.

Similar reworking is done for the steady-state recall of client caps. Another
release DecayCounter is added so we can tell when the client is not releasing
any more caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-31 12:07:54 -08:00
Patrick Donnelly
c2405957ed
mds: move session setup to ms_handle_accept
Session setup in ms_handle_authentication is (historically) racy where multiple
connections from the same client can come in before one is finally accepted.  A
session should only be created after ms_handle_accept. The MDS did some
backflips before this commit to ensure this.

Moreover, with the msgr2 changes, it is even more necessary since the address
nonce is not set until before ms_handle_accept is called.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-31 11:29:34 -08:00
Abhishek Lekshmanan
06ffab011f drop yield ctx from real_client
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:52:35 +01:00
Abhishek Lekshmanan
d6dc57e78c rgw: make dmclock scheduler experimental
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:42 +01:00
Abhishek Lekshmanan
82ea44bb35 rgw: rgw_rest add a http status code for 503/498 errors
The swift 498 status line is from
https://github.com/openstack/swift/blob/master/swift/common/middleware/ratelimit.py#L286

S3 responses codes are well documented at
https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:41 +01:00
Abhishek Lekshmanan
5dfcb05b90 rgw: set throttler as the default scheduler
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:41 +01:00
Abhishek Lekshmanan
9fa0c8aa15 rgw: remove throttle counter from common
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:41 +01:00
Abhishek Lekshmanan
8c4f3a2756 rgw: complete the request once scheduler errors
We were missing a `goto done` which caused the request handling to be erratic
once limits were reached.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:41 +01:00
Abhishek Lekshmanan
5fd54c4d48 rgw: add rgw_dmclock.h
For the future, we could fwd declare these types in rgw_dmclock so that whole of
dmclock_server.h needn't be included

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:41 +01:00
Abhishek Lekshmanan
ca3fd8824c rgw: bring back scheduler ctx
Using unified perf counters across frontends

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:41 +01:00
Abhishek Lekshmanan
07cb9673b1 rgw: async scheduler uses a global throttle counter
Using the last array element, global blocked requests will be tracked. This can
be used for other global counters

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:40 +01:00
Abhishek Lekshmanan
1f85882016 rgw: SimpleThrottler uses a counter for throttled requests
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:40 +01:00
Abhishek Lekshmanan
e5a3715122 rgw: dmclock: implement perf counters for throttle
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:40 +01:00
Abhishek Lekshmanan
4ff3b843e4 rgw: civetweb: drop unused includes
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:39 +01:00
Abhishek Lekshmanan
8c97293249 rgw: main: avoid passing cct to civetweb
Since we already have a copy from the env

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:39 +01:00
Abhishek Lekshmanan
1878047a14 rgw: sync scheduler directly uses ClientCounters
instead of a function object

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:39 +01:00
Abhishek Lekshmanan
378b4ec671 common: add rgw_scheduler_type as an option
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:39 +01:00
Abhishek Lekshmanan
b663bc9ece rgw: kill optional scheduler ctx
Each frontend maintains its own counters and client config

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:39 +01:00
Abhishek Lekshmanan
aba3d5ee4e rgw: dmclock: move client config and counters to its own header
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:38 +01:00
Abhishek Lekshmanan
5fce502e25 cmake: dmclock fixes
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:38 +01:00
Abhishek Lekshmanan
95cc4224d1 rgw: StreamIO takes a yield
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:38 +01:00
Abhishek Lekshmanan
089190435c rgw: optional_yield_context -> optional_yield
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:38 +01:00
Abhishek Lekshmanan
cd436113c1 rgw:make base scheduler class return a completer
the base class now returns a pair containing error code and a completer object
that will do raii style cleanup to call request_complete on destruction.
schedule_request is a non virtual function that'll call the impl. which is a
virtual function to be implemented by the subclasses

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-31 19:39:38 +01:00