Commit Graph

115838 Commits

Author SHA1 Message Date
Jan Fajerski
98c991fc6f ceph-volume lvm batch: use namedtuple instead of tuple
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
d0735ce1c9 ceph-volume: address review comments, mostly tidying, clarification
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
fcacd0b96a idempotency must result in the same outcome
...not should

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
55ba863017 ceph-volume: batch test should pass --journal-devices with filestore
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
7f76684632 ceph-volume: make --journal optional, add --journal-slots
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
7695d1ec53 doc: update ceph-volume lvm batch docs
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
0f6f2b46a3 py-common/deployment: drop workarounds for old batch
This commit drops two workarounds in drive groups for the old batch
behaviour. OSDs on LVs no longer need to be created individually by
create (related to functional testing) and ceph-volume batch now filters
unavailable drives itself.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
1239f77c88 ceph-volume batch: add deprecation warning for auto behaviour
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
eef9dc7a1d ceph-volume batch: add ceph.conf mocking to pass tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
0bc7f7424c ceph-volume batch: use disk.Size for size args
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
2124aa57b3 ceph-volume batch: Fix osd_ids passing and improve plan formatting
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
2327e92aba ceph-volume batch: track rel_size in percent, more tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:23 +02:00
Jan Fajerski
a23a02df02 ceph-volume batch: improve backwards compatibility
This restores legacy batch behavior and also adds some initial test and
adjusts existing tests to changes.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:50:20 +02:00
Jan Fajerski
d32e0e4320 ceph-volume: batch - enable legacy auto behaviour
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:49:48 +02:00
Jan Fajerski
b0b797363f ceph-volume: batch - major refactor
This completely refactors the batch code in order to make use of the
create/prepare code path for creating OSDs instead of having a second
code path doing this. This not only eases the maintenance burden but
also adds various features and fixes bugs. This subcommand can now
handle LVs, replace OSDs, reuse VGs and has a better notion of
idempotency.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:49:44 +02:00
Jan Fajerski
6cb0841658 ceph-volume: Device - available_lvm if 10 extents are free.
This changes the available_lvm notion to only require 10 free extents
instead of 5GB.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
b34f130f30 ceph-volume: Device - add vg_free property
This new property returns the free space in any VGs present. If no VGs
are on the device we project how much space a VG will have.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
2ff3732164 ceph-volume: prepare/create - size args as Size class
This add the disk.Size class as all size related argument types. We
often create this class form args like this anyway and it enables users
to pass not only bytes but also strings like 50G.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
e7cdeab9de ceph-volume: disk.Size - add cast to bool
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
bad54e9781 ceph-volume: api/lvm - add VolumeGroup.free_percent property
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
f48d225454 ceph-volume: util.device - add vg_free_percent property
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
de7d67603a ceph-volume: api/lvm - query LV units in bytes
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
fa62a7bf5f ceph-volume: lvm/common - refactor common arg specification
This makes it easier to create valid Namespace objects/arg lists when
programmatically calling create/prepare.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
51ca694bbe ceph-volume: batch: fix argument help message
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
c514214e0f
Merge pull request #37311 from mgariepy/fix-47502
[ceph-volume]: remove unneeded call to get_devices()
2020-09-25 11:45:16 +02:00
Xiubo Li
74c0890c69 client: switch to unique_lock to avoid use the client_lock directly
Fixes: https://tracker.ceph.com/issues/47039
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:52:56 +08:00
Xiubo Li
9d0e88c3ff client: unlock client_lock when copying data to bufferlist
It's no need to hold the lock when copying the data, which may
take a long time.

Fixes: https://tracker.ceph.com/issues/47047
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:52:49 +08:00
Xiubo Li
f483407b87 client: do more check for the client_lock
Fixes: https://tracker.ceph.com/issues/47039
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:46:03 +08:00
Xiubo Li
1a902c4631 client: simplify the iterating related code to make it more readable
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:46:03 +08:00
Patrick Donnelly
bbdc357a60
Merge PR #36685 into master
* refs/pull/36685/head:
	mds: allow quota for root directory

Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-09-24 17:22:32 -07:00
Patrick Donnelly
389ff2f302
Merge PR #37009 into master
* refs/pull/37009/head:
	client: remove the unecessary client_lock for objector->read/write()
	client: fix the comment for RWRef

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-24 17:21:36 -07:00
Patrick Donnelly
cc2c25b9fe
Merge PR #37266 into master
* refs/pull/37266/head:
	qa: ignore logrotate state rename error

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-24 14:37:08 -07:00
Marc Gariepy
5c6f66166a ceph-volume: remove unneeded call to get_devices()
there is no need to probe the device to generate the argparse help
message.

also removing the test on the function as it's not there anymore.

Signed-off-by: Marc Gariepy <gariepy.marc@gmail.com>
Fixes: https://tracker.ceph.com/issues/47502
2020-09-24 17:13:30 -04:00
Jason Dillaman
1cfacd11bd
Merge pull request #37374 from wjwithagen/wjw-fix-crypto-alloca
librbd: Use stdlib.h instead of alloca.h

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-24 14:45:55 -04:00
Nathan Cutler
f9b2940598 SubmittingPatches-backports.rst: clarify where to explain
The prevailing consensus is that rationales of direct fixes need
not be included in the commit messages. The matter must be
made clear to the reviewers, but it's not important how this is
accomplished.

(Perusers of the git history interested in finding out this information
are free to search for it by linking the commit in question to the PR it
originated from, and possibly from there to the related tracker issues.)

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-09-24 20:04:47 +02:00
Casey Bodley
ebd3fe4dea rgw: raise debug level of DatalogTrimImplCR message
the message was printed at level 0 even on success

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-24 14:02:32 -04:00
Willem Jan Withagen
32dd5c707e librbd: Use stdlib.h instead of alloca.h
src/librbd/crypto/BlockCrypto.cc:5:10: fatal error: 'alloca.h' file not found
         ^~~~~~~~~~
1 error generated.

Including <stdlib.h> is enough

fixes: https://tracker.ceph.com/issues/47614
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2020-09-24 18:23:23 +02:00
Ilya Dryomov
fa83dbd76e
Merge pull request #37385 from idryomov/wip-krbd-read-only-override
rbd: make common options override krbd-specific options

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-24 18:20:48 +02:00
Kefu Chai
5bb5f5a79e
Merge pull request #37384 from tchaikov/wip-crimson-doc
doc/dev/crimson: add instructions on how to build crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-09-24 22:30:12 +08:00
Jason Dillaman
ad1016b014 librbd: skip cache initialization if data IoCtx is invalid
This resolves the potential for an assertion failure in the cache
constructors that occurs when the data IoCtx is not valid. Errors
are deferred until actual IOs are issued against the invalid data
pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-09-24 09:13:49 -04:00
Kefu Chai
bc00a8feb0 admin: pass --use-feature=2020-resolver to pip
to silence the warning like

follow the suggestion from pip

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

Fixes: https://tracker.ceph.com/issues/47636
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-24 19:59:58 +08:00
Johannes M. Scheuermann
868e708c9e debian: Add missing Python dependency for ceph-mgr
The file mgr/volumes/fs/operations/pin_util.py imports distutils.util for using
strtobool and thus the python package is required.

Signed-off-by: Johannes M. Scheuermann <joh.scheuer@gmail.com>
2020-09-24 12:10:53 +02:00
Avan Thakkar
30b41fba49 mgr/dashboard: cluster > manager modules
Fixes: https://tracker.ceph.com/issues/47484

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2020-09-24 14:45:19 +05:30
Xiubo Li
f5a457b4f2 mds: defer encoding and storing the inode backtrace
When encoding the inode backtrace, it may take a bit longer time.
If we hold the mds_lock or other locks, it may cause other threads
to idle wait.

This will queue the encoding and storing work in the mds finisher
to get rid of the locks holded.

Fixes: https://tracker.ceph.com/issues/47148
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-24 16:53:18 +08:00
Xiubo Li
0ec5f3a84a mds: defer encoding and storing the CDir dentries
When encoding the dentries infomation to remove or update, it may
take a bit longer time. If we hold the mds_lock or other locks, it
may cause other threads to idle wait.

This will queue the encoding and storing work in the mds finisher
to get rid of the locks holded.

Fixes: https://tracker.ceph.com/issues/47148
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-24 16:53:18 +08:00
Ilya Dryomov
a107c47360 rbd: make common options override krbd-specific options
ceph-csi has added support for passing custom map and unmap options via
mapOptions and unmapOptions storage class parameters.  However, it also
uses --read-only for implementing ROX (ReadOnlyMany) PVs.  If the user
supplies "mapOptions: rw", they will get around the intended read-only
restriction (at least on the block device).

ceph-csi could be patched to use "-o ro", but it actually makes sense
for common options to win over device type-specific equivalents.

Fixes: https://tracker.ceph.com/issues/47625
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-09-24 10:50:24 +02:00
Kefu Chai
e492447fde
Merge pull request #37236 from jschmid1/osd_rm_doc_fix
docs: remove docs should say osd_id instead of svc_id

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-24 15:51:05 +08:00
Kefu Chai
17d35b367d doc/dev/crimson: add instructions on how to build crimson
and add a link to Scylla's document on debugging with GDB

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-24 15:32:09 +08:00
Xiubo Li
9eba6f4bd0 mds: add error handler with lock support
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-24 15:31:09 +08:00
Lenz Grimmer
e270e8e864
Merge pull request #37275 from bk201/wip-47494
mgr/dashboard: display devices' health information within a tabset

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-24 09:27:18 +02:00