Commit Graph

115765 Commits

Author SHA1 Message Date
Jan Fajerski
51ca694bbe ceph-volume: batch: fix argument help message
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-25 11:48:34 +02:00
Jan Fajerski
c514214e0f
Merge pull request #37311 from mgariepy/fix-47502
[ceph-volume]: remove unneeded call to get_devices()
2020-09-25 11:45:16 +02:00
Xiubo Li
74c0890c69 client: switch to unique_lock to avoid use the client_lock directly
Fixes: https://tracker.ceph.com/issues/47039
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:52:56 +08:00
Xiubo Li
9d0e88c3ff client: unlock client_lock when copying data to bufferlist
It's no need to hold the lock when copying the data, which may
take a long time.

Fixes: https://tracker.ceph.com/issues/47047
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:52:49 +08:00
Xiubo Li
f483407b87 client: do more check for the client_lock
Fixes: https://tracker.ceph.com/issues/47039
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:46:03 +08:00
Xiubo Li
1a902c4631 client: simplify the iterating related code to make it more readable
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-25 08:46:03 +08:00
Patrick Donnelly
bbdc357a60
Merge PR #36685 into master
* refs/pull/36685/head:
	mds: allow quota for root directory

Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-09-24 17:22:32 -07:00
Patrick Donnelly
389ff2f302
Merge PR #37009 into master
* refs/pull/37009/head:
	client: remove the unecessary client_lock for objector->read/write()
	client: fix the comment for RWRef

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-24 17:21:36 -07:00
Patrick Donnelly
cc2c25b9fe
Merge PR #37266 into master
* refs/pull/37266/head:
	qa: ignore logrotate state rename error

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-09-24 14:37:08 -07:00
Marc Gariepy
5c6f66166a ceph-volume: remove unneeded call to get_devices()
there is no need to probe the device to generate the argparse help
message.

also removing the test on the function as it's not there anymore.

Signed-off-by: Marc Gariepy <gariepy.marc@gmail.com>
Fixes: https://tracker.ceph.com/issues/47502
2020-09-24 17:13:30 -04:00
Jason Dillaman
1cfacd11bd
Merge pull request #37374 from wjwithagen/wjw-fix-crypto-alloca
librbd: Use stdlib.h instead of alloca.h

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-24 14:45:55 -04:00
Nathan Cutler
f9b2940598 SubmittingPatches-backports.rst: clarify where to explain
The prevailing consensus is that rationales of direct fixes need
not be included in the commit messages. The matter must be
made clear to the reviewers, but it's not important how this is
accomplished.

(Perusers of the git history interested in finding out this information
are free to search for it by linking the commit in question to the PR it
originated from, and possibly from there to the related tracker issues.)

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-09-24 20:04:47 +02:00
Casey Bodley
ebd3fe4dea rgw: raise debug level of DatalogTrimImplCR message
the message was printed at level 0 even on success

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-24 14:02:32 -04:00
Willem Jan Withagen
32dd5c707e librbd: Use stdlib.h instead of alloca.h
src/librbd/crypto/BlockCrypto.cc:5:10: fatal error: 'alloca.h' file not found
         ^~~~~~~~~~
1 error generated.

Including <stdlib.h> is enough

fixes: https://tracker.ceph.com/issues/47614
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2020-09-24 18:23:23 +02:00
Ilya Dryomov
fa83dbd76e
Merge pull request #37385 from idryomov/wip-krbd-read-only-override
rbd: make common options override krbd-specific options

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-24 18:20:48 +02:00
Kefu Chai
5bb5f5a79e
Merge pull request #37384 from tchaikov/wip-crimson-doc
doc/dev/crimson: add instructions on how to build crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-09-24 22:30:12 +08:00
Jason Dillaman
ad1016b014 librbd: skip cache initialization if data IoCtx is invalid
This resolves the potential for an assertion failure in the cache
constructors that occurs when the data IoCtx is not valid. Errors
are deferred until actual IOs are issued against the invalid data
pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-09-24 09:13:49 -04:00
Kefu Chai
bc00a8feb0 admin: pass --use-feature=2020-resolver to pip
to silence the warning like

follow the suggestion from pip

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

Fixes: https://tracker.ceph.com/issues/47636
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-24 19:59:58 +08:00
Johannes M. Scheuermann
868e708c9e debian: Add missing Python dependency for ceph-mgr
The file mgr/volumes/fs/operations/pin_util.py imports distutils.util for using
strtobool and thus the python package is required.

Signed-off-by: Johannes M. Scheuermann <joh.scheuer@gmail.com>
2020-09-24 12:10:53 +02:00
Avan Thakkar
30b41fba49 mgr/dashboard: cluster > manager modules
Fixes: https://tracker.ceph.com/issues/47484

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2020-09-24 14:45:19 +05:30
Xiubo Li
f5a457b4f2 mds: defer encoding and storing the inode backtrace
When encoding the inode backtrace, it may take a bit longer time.
If we hold the mds_lock or other locks, it may cause other threads
to idle wait.

This will queue the encoding and storing work in the mds finisher
to get rid of the locks holded.

Fixes: https://tracker.ceph.com/issues/47148
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-24 16:53:18 +08:00
Xiubo Li
0ec5f3a84a mds: defer encoding and storing the CDir dentries
When encoding the dentries infomation to remove or update, it may
take a bit longer time. If we hold the mds_lock or other locks, it
may cause other threads to idle wait.

This will queue the encoding and storing work in the mds finisher
to get rid of the locks holded.

Fixes: https://tracker.ceph.com/issues/47148
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-24 16:53:18 +08:00
Ilya Dryomov
a107c47360 rbd: make common options override krbd-specific options
ceph-csi has added support for passing custom map and unmap options via
mapOptions and unmapOptions storage class parameters.  However, it also
uses --read-only for implementing ROX (ReadOnlyMany) PVs.  If the user
supplies "mapOptions: rw", they will get around the intended read-only
restriction (at least on the block device).

ceph-csi could be patched to use "-o ro", but it actually makes sense
for common options to win over device type-specific equivalents.

Fixes: https://tracker.ceph.com/issues/47625
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-09-24 10:50:24 +02:00
Kefu Chai
e492447fde
Merge pull request #37236 from jschmid1/osd_rm_doc_fix
docs: remove docs should say osd_id instead of svc_id

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-24 15:51:05 +08:00
Kefu Chai
17d35b367d doc/dev/crimson: add instructions on how to build crimson
and add a link to Scylla's document on debugging with GDB

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-24 15:32:09 +08:00
Xiubo Li
9eba6f4bd0 mds: add error handler with lock support
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-09-24 15:31:09 +08:00
Lenz Grimmer
e270e8e864
Merge pull request #37275 from bk201/wip-47494
mgr/dashboard: display devices' health information within a tabset

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-24 09:27:18 +02:00
zdover23
4d4fae3672
Merge pull request #37305 from zdover23/wip-doc-3-dev-guide-basic-workflow-date-update-sep-2020
doc/dev: updated date from 2016

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-24 15:11:50 +10:00
Changcheng Liu
4c338fa9f1 msgr: remove the macro not referenced anywhere
The default port range should be between:
  [ms_bind_port_min, ms_bind_port_max]

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-09-24 12:47:33 +08:00
Jos Collin
af0c0cd6b7
mon: test before adding/removing required_client_features
Fixes: https://tracker.ceph.com/issues/47006
Signed-off-by: Jos Collin <jcollin@redhat.com>
2020-09-24 06:43:49 +05:30
Zac Dover
5985eb9413 doc/dev: added the other four tests
This PR adds a list of tests that are performed
when a PR hits Github.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2020-09-24 09:49:36 +10:00
Samuel Just
da17def877 crimson/os/seastore/journal: return addr and seq from submit_record
Reworks journal_seq_t to be a monotonically increasing, but non-dense
value from the current segment_seq_t and segment offset.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:15:59 -07:00
Samuel Just
272fd5ef8f crimson/os/seastore/journal: add interface for trimming the journal
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:52 -07:00
Samuel Just
001bed8659 crimson/os/seastore: rework pending extents
Previously, we simply added the old extent to the retired set and
treated the new instance seperately.  Instead, let the new pending
instance reference the old instance via a prior_instance reference.
This has a few advantages:
- During commit, we swap cache hooks from the prior instance to
  avoid traversal
- Extents in the retired set are only those extents that are no
  longer mapped/valid.
- btree range pins can be swapped without a traversal
- position in the dirty list can be maintained

That last will be particularly important as the dirty list will
therefore be ordered from the extent with the oldest initial
delta forward.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
053f4017d1 crimson/os/seastore: factor out LogicalCachedExtent::on_delta_write
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
ed6abb884b crimson/os/seastore/journal: rename journal_replay_lb to journal_tail
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
47dc350625 crimson/os/seastore/lba_manager/btree: factor out common LBANode helpers
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
7670fccc42 crimson/os/seastore: remove CachedExtent move constructor
I'm not sure why I added this in the first place, there aren't really
any scenarios where invoking it wouldn't be a bug.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
5f2bd86f21 crimson/os/seastore/.../btree_lba_node_impl: mutate_mapping adjust addr for extent state
Addresses must always be adjusted depending on whether the extent will
be written as a block or as a delta.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
ca28f2dc56 crimson/os/seastore: improve misc debugging
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
c1a4471192 crimson/os/seastore/seastore_types: permit paddr_t::operator- to operate on block relative addrs
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
534f014819 crimson/os/seastore/segment_manager: use smaller segments for tests
This way, we'll roll segments more frequently.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
52c03750a4 crimson/os/seastore/lba_manager/btree/btree_lba_manager: manage pin in init
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
5e840a20b8 crimson/os/seastore/lba_manager: drop from cache directly during replay if dead
Dead extents can't go through the normal retire_extent path as they don't
have valid pins or other elements of a real extent.  Instead, simply drop
them on the floor directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
5539b54547 crimson/os/seastore/.../lba_btree_node_impl: fix find_hole
find_hole shouldn't check entries prior to min.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
05c62821d9 crimson/os/seastore: initialize crc *after* read
Previously, all extents ended up initialized to the same crc value
upon read -- the one for an empty buffer.  This bug wasn't detected
because an extent was always evicted and reread following the first
write and subsequently kept in cache once initially written.

Also, add crc to operator<< for extents.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
9016932fef crimson/os/seastore/transaction_manager: only add pin if not already pending
If it's already pending, cache.duplicate_for_write is a noop and the extent
will already have a pin.  Also, add some debugging.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
31d3057a35 crimson/os/seastore/cache: let root exist in the dirty list
Most handling remains special, but this way the dirty list we
work on will include root directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
d5932ba049 crimson/os/seastore: clarify crc initial state
Otherwise, different users may use different values complicating
debugging.  We can re-add this in the future if it becomes
useful.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
f9d0aae639 crimson/os/seastore/transaction: let get_extent signal retired extents
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00