Commit Graph

115520 Commits

Author SHA1 Message Date
Samuel Just
001bed8659 crimson/os/seastore: rework pending extents
Previously, we simply added the old extent to the retired set and
treated the new instance seperately.  Instead, let the new pending
instance reference the old instance via a prior_instance reference.
This has a few advantages:
- During commit, we swap cache hooks from the prior instance to
  avoid traversal
- Extents in the retired set are only those extents that are no
  longer mapped/valid.
- btree range pins can be swapped without a traversal
- position in the dirty list can be maintained

That last will be particularly important as the dirty list will
therefore be ordered from the extent with the oldest initial
delta forward.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
053f4017d1 crimson/os/seastore: factor out LogicalCachedExtent::on_delta_write
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
ed6abb884b crimson/os/seastore/journal: rename journal_replay_lb to journal_tail
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
47dc350625 crimson/os/seastore/lba_manager/btree: factor out common LBANode helpers
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
7670fccc42 crimson/os/seastore: remove CachedExtent move constructor
I'm not sure why I added this in the first place, there aren't really
any scenarios where invoking it wouldn't be a bug.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
5f2bd86f21 crimson/os/seastore/.../btree_lba_node_impl: mutate_mapping adjust addr for extent state
Addresses must always be adjusted depending on whether the extent will
be written as a block or as a delta.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
ca28f2dc56 crimson/os/seastore: improve misc debugging
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
c1a4471192 crimson/os/seastore/seastore_types: permit paddr_t::operator- to operate on block relative addrs
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
534f014819 crimson/os/seastore/segment_manager: use smaller segments for tests
This way, we'll roll segments more frequently.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
52c03750a4 crimson/os/seastore/lba_manager/btree/btree_lba_manager: manage pin in init
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
5e840a20b8 crimson/os/seastore/lba_manager: drop from cache directly during replay if dead
Dead extents can't go through the normal retire_extent path as they don't
have valid pins or other elements of a real extent.  Instead, simply drop
them on the floor directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
5539b54547 crimson/os/seastore/.../lba_btree_node_impl: fix find_hole
find_hole shouldn't check entries prior to min.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
05c62821d9 crimson/os/seastore: initialize crc *after* read
Previously, all extents ended up initialized to the same crc value
upon read -- the one for an empty buffer.  This bug wasn't detected
because an extent was always evicted and reread following the first
write and subsequently kept in cache once initially written.

Also, add crc to operator<< for extents.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
9016932fef crimson/os/seastore/transaction_manager: only add pin if not already pending
If it's already pending, cache.duplicate_for_write is a noop and the extent
will already have a pin.  Also, add some debugging.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
31d3057a35 crimson/os/seastore/cache: let root exist in the dirty list
Most handling remains special, but this way the dirty list we
work on will include root directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
d5932ba049 crimson/os/seastore: clarify crc initial state
Otherwise, different users may use different values complicating
debugging.  We can re-add this in the future if it becomes
useful.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
f9d0aae639 crimson/os/seastore/transaction: let get_extent signal retired extents
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Samuel Just
39dee50f75 crimson/common/fixed_kv_node_layout: fix lower_bound
This went undetected because all existing users ensure that
the key does not already exist.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-09-23 15:13:51 -07:00
Michael Fritch
3aa3484425
Merge pull request #37186 from guits/guits-cephadm-filter-dangling-images
cephadm: filter dangling images in get_last_local_ceph_image()
2020-09-23 15:30:43 -06:00
Michael Fritch
6a669255a7
Merge pull request #37133 from jschmid1/ptrace_priv
cephadm: capadd and privileged are mutex with podman >= 2.0

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-09-23 15:29:30 -06:00
Patrick Donnelly
e089ff759f
Merge PR #37310 into master
* refs/pull/37310/head:
	client: refresh rstat for quota check of rename

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-09-23 08:05:47 -07:00
Lenz Grimmer
45d280c9cc
Merge pull request #37041 from bk201/wip-47356
mgr/dashboard: fix security scopes of some NFS-Ganesha endpoints

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-09-23 17:01:00 +02:00
Lenz Grimmer
a433ed1282
Merge pull request #37244 from ceph/47543-bucket-list-perf
mgr/dashboard: fix perf. issue when listing large amounts of buckets

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-09-23 16:26:09 +02:00
Daniel Gryniewicz
16a3103f22
Merge pull request #37242 from dang/wip-dang-47520
Fix missing req_state for librgw
2020-09-23 07:57:27 -04:00
Ilya Dryomov
7270360809
Merge pull request #37330 from idryomov/wip-backport-create-issue-key
doc: SubmittingPatches-backports.rst: backport scripts read key from file

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-09-23 13:39:59 +02:00
Lenz Grimmer
e066050636
Merge pull request #37136 from p-se/cd-add-per-rbd-graphs
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-09-23 12:59:06 +02:00
Ilya Dryomov
321bcd4559 SubmittingPatches-backports.rst: backport scripts read key from file
The --key option was removed in commit 1b5f37bbcd
("backport-create-issue: read redmine key from file").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-09-23 11:37:17 +02:00
Lenz Grimmer
fac25773b6
Merge pull request #37192 from rhcs-dashboard/fix-46898-master
mgr/dashboard/api: move/create OSD histogram in separate endpoint

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-09-23 11:01:06 +02:00
Kefu Chai
8262c5571a
Merge pull request #37303 from tchaikov/wip-crimson-osd
crimson: do not set_value() twice and fix Message leak

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-09-23 16:49:12 +08:00
Kefu Chai
db2779eb68 crimson/mon: move handle_auth_done() to "v1 and v2" section
crimson::mon::Connection::handle_auth_done() is shared by both v1 and v2
authentication implementation, so move it to "v1 and v2" section

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-23 15:56:26 +08:00
Kefu Chai
fdbd144356
Merge pull request #37326 from tchaikov/wip-47324
test/crimson: better error message when bind fails

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-09-23 15:11:04 +08:00
Kefu Chai
2778514039
Merge pull request #37323 from tchaikov/wip-47593
crimson/osd: add_recovery(oid) before recover_object(oid)

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2020-09-23 14:24:48 +08:00
Kefu Chai
d9ee3a1818 test/crimson: better error message when bind fails
See also: https://tracker.ceph.com/issues/47324

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-23 14:19:34 +08:00
Kefu Chai
33547674da crimson/osd: more constness
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-23 12:24:52 +08:00
Kefu Chai
35d37e2bff crimson/osd: add_recovery(oid) before recover_object(oid)
in 7736bddc53, we assumed that the object
to be recovered did not exist in `recovering` before
`recover_object(oid)` was called. but this turns out not true. so, in
this change, `add_object(oid)` is called before `recover_object(oid)`
gets called.

Fixes: https://tracker.ceph.com/issues/47593
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-23 12:20:16 +08:00
Kefu Chai
6cbc51db4e crimson/osd: use recoverying.at() when appropriate
this helps to avoid the confusion of "where the recovery is added" and
"are we adding a new instance of recovery here".

should call add_recovery() explicitly when we need to add a new recovery
instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-23 12:20:16 +08:00
Kefu Chai
3f0b358cc3 crimson/osd: add RecoveryBackend:add_recovering()
before this change, get_recovery() can also be used for adding a
recovery instance to `recovering`. this behavior is error-prone and
confusing.

after this change, add_recovery() is used in the place where we
want to add a new instance of recovery instance.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-23 12:20:16 +08:00
Kefu Chai
8fd1bd4a29
Merge pull request #37290 from tchaikov/wip-run-make-check
run-make-check.sh: fix syntax error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-23 09:36:45 +08:00
Sebastian Wagner
9899c6c987
Merge pull request #37106 from sebastian-philipp/cephadm-overwritable-templates
mgr/cephadm: make j2 teamplates overwritable

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-09-23 01:34:57 +02:00
Jason Dillaman
1f31e8bdb3
Merge pull request #37121 from Songweibin/wip-rbd-metadata
librbd: async version of metadata_set/remove

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-22 19:14:24 -04:00
Jason Dillaman
3baecb8614
Merge pull request #37083 from orozery/librbd-data-cryptor
librbd: add data encryptor/decryptor using openssl

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-22 19:14:07 -04:00
Jason Dillaman
78d630d4a5
Merge pull request #37110 from CongMinYin/qa_immutable_object_cache_check
qa: check immutable object cache content

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-22 19:12:42 -04:00
Jason Dillaman
d6dc783e83
Merge pull request #37237 from MahatiC/wip-fix-rwl-path
librbd/cache: Fix pwl_path read issue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-09-22 19:11:47 -04:00
Michael Fritch
72b10c43c5
Merge pull request #37206 from bk201/wip-47387
mgr/rook: fix listing daemons

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-09-22 16:37:09 -06:00
Yan, Zheng
2e536cdd4a client: refresh rstat for quota check of rename
Fixes: https://tracker.ceph.com/issues/47224
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-09-22 20:31:08 +08:00
Lenz Grimmer
75763e67ed
Merge pull request #37043 from s0nea/wip-dashboard-47331
mgr/dashboard: log in non-admin users successfully if the telemetry notification is shown

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-09-22 13:55:47 +02:00
Kefu Chai
b33e414000
Merge pull request #37293 from changchengx/osd_comment
osd: fix wrong comment about setting auth_client for messenger

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-09-22 17:46:09 +08:00
Kefu Chai
70b16d5b01 crimson/osd: do not add ref when constructing MessageRef
crimson::osd::PG::send_cluster_message() accepts a `Message*`
pointer, and then hand it over to `shard_services.send_to_osd()`,
which expects a `Ref<Message>`. so the raw pointer is used to
construct an `intrusive_ptr<Message>`, which increment the
refcount of that Message instance by one. but that Message
was owned by nobody before that, so we end up with an
`intrusive_ptr<Message>` of 2 refcount, and only a single
owner. hence the memory leak.

in this change, instructs the constructor to not add the refcount.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-22 17:13:21 +08:00
Kefu Chai
a631510d4f crimson/mon: do not set_value() again
once the continuation consuming the stored value of the associated
future, we cannot set_value() again. otherwise, ASan complains that we
are accessing the memory on heap after it is freed.

in this change, std::optional<> is used for holding
promise<auth_result_t>, once the promise is fulfilled, `auth_done` is
reset to prevent another call of `set_value()` or `set_exception()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-22 17:13:21 +08:00
Kefu Chai
182f6db9b1 crimson/mon: only finish auth when auth is completed
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-22 17:07:47 +08:00