Commit Graph

135306 Commits

Author SHA1 Message Date
Ilya Dryomov
c97b7c4e8b librbd: introduce reduce_parent_overlap() and switch overlap API
When encryption is loaded, rbd_get_overlap() and Image::overlap() now
return "effective" overlap, similar to rbd_get_size() and Image::size().
Previously, returned overlap could have been bigger than "effective"
size.

Note that get_effective_image_size() successor doesn't take snap_id
because passing anything but ictx->snap_id was broken.  Since the size
of the crypto header is stored in the crypto header itself, image areas
are defined only for the "opened at" snap_id.  Getting "effective" size
for an arbitrary snapshot requires actually opening it and loading
encryption on it.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
50070a78a8 librbd: tweak get_parent_overlap() signature
Make it clear that get_parent_overlap() returns the raw parent overlap
value (i.e. physical offset into the parent image).  Also drop redundant
ceph_mutex_is_locked assert -- get_parent_info() already has one.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
2279ae08f1 librbd: remap resize target size if encryption is loaded
When encryption is loaded, rbd_get_size() and Image::size() return
"effective" size, but rbd_resize() and Image::resize() continue to take
raw size.  The user has to constantly keep these domains in mind.

Saying that resize must be done without encryption loaded is not an
answer because shrinking a clone that has snapshots involves copying up
objects in the affected range (identical to flattening).  In addition,
even if a clone doesn't have snapshots, shrinking it to a size that
isn't an object boundary is going to involve a copyup for the victim
object as well.

To avoid subtle data corruption on shrink, treat resize operation the
same as flatten operation (including on the CLI).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
b3deeb152a librbd: check stripe pattern when loading encryption
Currently it's done in FormatRequest but not in LoadRequest.  However
an image can be deep copied or exported and imported with a different
stripe pattern such that an area boundary would fall in the middle of
an object.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
36c2d589af librbd: check image size when loading encryption
Currently it's done in FormatRequest but not in LoadRequest.  However
an image can be shrunk to a size such that encryption can loaded (i.e.
enough of the header is still present) but nothing else can, breaking
implicit assumptions all around.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
203560999d librbd: relax image size check in luks::FormatRequest
Proceed with formatting an image even if all space would be consumed by
the crypto header.  There is no reason to be strict here since we allow
creating zero-sized images as well as shrinking any image to 0.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
4f5b735c57 librbd: no need to return format from ShutDownCryptoRequest
luks::FlattenRequest was the only user of this interface.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
abded6eeda librbd: don't temporarily shut down crypto when flattening
(Temporarily) shutting down crypto can lead to data corruption in the
face of concurrent I/O, especially when flatten operation is proxied to
the remote lock owner.  This was added to be able to read, optionally
modify and write crypto header without it being subjected to remapping
and encryption itself.  read_header() and write_header() now achieve
that by specifying CRYPTO_HEADER area explicitly.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
744379b8c6 librbd: move get_file_offset() into CryptoObjectDispatch
This method doesn't propagate area.  Since its only user is
CryptoObjectDispatch which is now applied only to DATA area,
move get_file_offset() there to avoid misuse in the future.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
a54f1a25fa librbd: apply CryptoObjectDispatch layer only to DATA area
Objects in CRYPTO_HEADER area should not be subjected to encryption.
Unit tests needed adjustment because MockCryptoInterface is configured
with DATA_OFFSET = 4 * 1024 * 1024, thus disqualifying object 0.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
b9a2384cdc librbd: propagate area down to file_to_extents()
- readahead and PWL cache are limited to DATA area as explained in
  the previous commit
- DATA area is assumed for the journal as encryption can't be used
  with journaling anyway

To postpone the churn associated with passing area through
ImageDispatchInterface (where only WriteLogImageDispatch and
ImageDispatch care), add a new image dispatch flag.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
be286db1f1 librbd: pass area to ImageDispatchSpec::create_*()
- DATA area is assumed at the API layer as there is no way to pass
  an area
- DATA area is assumed by ImageWriteback because PWL cache persists
  image extents as provided by the user without any kind of designator
  and therefore can be active only in either area
- luks::FlattenRequest operates on CRYPTO_HEADER area

The passed area is acted upon in ImageDispatchSpec constructor in the
next commit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
b2c8882092 librbd: return area from extents_to_file()
Note that, as suggested by extents_to_file() signature, all returned
image extents would pertain to the same area.  This means that an area
boundary must coincide with an object boundary.

luks::FormatRequest is actually more strict: crypto header area size is
set to a multiple of stripe period (i.e. one or more whole objects).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
830b919840 librbd: introduce ImageArea and split remap_extents() into two methods
Since remap in either direction can really be done only once, iterating
through image dispatch layers in ImageDispatcher::remap_extents() makes
no sense.  For now, just replace the iteration with CryptoImageDispatch
lookup.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
37057ba2f6 librbd: pass image_extents to create_{discard,write_same}()
These are still taking off and len separately which is inconsistent
with the rest of ImageDispatchSpec and also ImageDiscardRequest and
ImageWriteSameRequest.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Ilya Dryomov
c67e254139 librbd: CopyupRequest can move image_extents instead of copying
image_extents is already taken by rvalue reference.
CopyupRequest::create() callers are prepared for the move.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 18:19:19 +01:00
Matan Breizman
9ff1255d92 crimson/osd/osd_operations/client_request: Remove seq_mode_t
No longer in use.
See 5aa342c9f0

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-12-04 16:13:57 +00:00
yuval Lifshitz
1418535a56 rgw/notification: initialize member in reservation ctor
coverity ID: CID 1517212
fix to commit: e4e05dde60

Signed-off-by: yuval Lifshitz <ylifshit@redhat.com>
2022-12-04 17:35:42 +02:00
Ronen Friedman
c43e981de0
Merge pull request #48551 from ronen-fr/wip-rf-rep-machine
osd/scrub: no 'ScrubFinish' transition in ActiveReplica state

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-12-04 16:49:33 +02:00
Ronen Friedman
8557cb320e osd/scrub: no 'ScrubFinish' transition in ActiveReplica state
That event is not generated for a replica.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-04 15:21:31 +02:00
Ilya Dryomov
8f712733af qa: rbd_groups.sh: change interpreter to bash
Commit e0da2a4e8c ("qa/workunits/rbd: Add test to list snapshots of
consistency group") added bash-specific syntax.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-04 13:20:44 +01:00
zdover23
33a021e5fb
Merge pull request #49217 from zdover23/wip-doc-2022-12-04-rados-ops-health-check-5-of-5-prompts
doc/rados: add prompts to health-checks (5 of 5)

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:30:27 +10:00
zdover23
e184a1b839
Merge pull request #49216 from zdover23/wip-doc-2022-12-04-rados-ops-health-check-4-of-5-prompts
doc/rados: add prompts to health-checks (4 of 5)

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:27:11 +10:00
zdover23
c2e1677e40
Merge pull request #49215 from zdover23/wip-doc-2022-12-04-rados-ops-health-check-3-of-5-prompts
doc/rados: add prompts to health-checks (3 of 5)

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:22:18 +10:00
zdover23
6bd6d17a24
Merge pull request #49214 from zdover23/wip-doc-2022-12-04-rados-ops-health-chech-2-of-5-prompts
doc/rados: add prompts to health-checks (2 of 5)

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:21:16 +10:00
zdover23
8739a6ab9e
Merge pull request #49213 from zdover23/wip-doc-2022-12-04-rados-ops-health-chech-1-of-4-prompts
doc/rados: add prompts to health-checks (1 of 5)

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:15:06 +10:00
zdover23
e81f59dd90
Merge pull request #49212 from zdover23/wip-doc-2022-12-04-rados-ops-erasure-code-shec-prompts
doc/rados: add prompts to erasure-code-shec.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:09:25 +10:00
zdover23
9faf047bf1
Merge pull request #49211 from zdover23/wip-doc-2022-12-04-rados-ops-erasure-code-erasure-code-lrc-prompts
doc/rados: add prompts to erasure-code-lrc.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-04 04:00:52 +10:00
Zac Dover
b0851995af doc/rados: add prompts to health-checks (5 of 5)
Add unselectable prompts to doc/rados/operations/health-checks.rst,
fifth 300 lines.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 03:03:53 +10:00
Zac Dover
df16a23bfc doc/rados: add prompts to health-checks (4 of 5)
Add unselectable prompts to doc/rados/operations/health-checks.rst,
fourth 300 lines.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 02:56:31 +10:00
Zac Dover
73e1a29525 doc/rados: add prompts to health-checks (3 of 5)
Add unselectable prompts to doc/rados/operations/health-checks.rst,
third 300 lines.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 02:49:02 +10:00
Zac Dover
c850569e52 doc/rados: add prompts to health-checks (2 of 5)
Add unselectable prompts to doc/rados/operations/health-checks.rst,
second 300 lines.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 02:36:07 +10:00
Zac Dover
cbc334e1dd doc/rados: add prompts to health-checks (1 of 5)
Add unselectable prompts to doc/rados/operations/health-checks.rst,
first 300 lines.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 02:13:08 +10:00
Zac Dover
d6cfb65999 doc/rados: add prompts to erasure-code-shec.rst
Add unselectable prompts to doc/rados/operations/erasure-code-shec.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 01:57:13 +10:00
Zac Dover
fca696cc71 doc/rados: add prompts to erasure-code-lrc.rst
Add unselectable prompts to doc/rados/operations/erasure-code-lrc.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-04 01:28:11 +10:00
zdover23
c2690ec297
Merge pull request #49202 from zdover23/wip-doc-2022-12-01-rados-ops-erasure-code-jerasure-prompts
doc/rados: add prompts to erasure-code-jerasure.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-03 13:10:05 +10:00
zdover23
67f74ecaaa
Merge pull request #49201 from zdover23/wip-doc-2022-12-01-rados-ops-erasure-code-isa-prompts
doc/rados: add prompts to erasure-code-isa

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-03 13:07:32 +10:00
zdover23
2b28f25347
Merge pull request #49200 from zdover23/wip-doc-2022-12-01-rados-ops-erasure-code-clay-prompts
doc/rados: add prompts to erasure-code-clay.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-03 13:02:27 +10:00
Uli Fahrer
974fec8497 containers: use quay.io/ceph/keepalived in docs
the documentation should reflect the keepalived
container image fork rather than arcts/keepalived

Signed-off-by: Uli Fahrer <github@uli-fahrer.de>
2022-12-02 22:29:21 +01:00
Daniel Gryniewicz
8dae4aab39
Merge pull request #49162 from Gaoweinan/main-rgw-rgwerror-StatementRepetition-gaoweinan
radosgw:Fix repeated statement

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-02 11:47:26 -05:00
Zac Dover
a35d344f0a doc/rados: add prompts to erasure-code-jerasure.rst
Add unselectable prompts to
doc/rados/operations/erasure-code-jerasure.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-03 02:27:59 +10:00
Zac Dover
7de786ca12 doc/rados: add prompts to erasure-code-isa
Add unselectable prompts to doc/rados/operations/erasure-code-isa.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-03 02:16:43 +10:00
Zac Dover
fe2a38436f doc/rados: add prompts to erasure-code-clay.rst
Add unselectable prompts to doc/rados/operations/erasure-code-clay.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-03 02:11:49 +10:00
Adam King
55661cc859 mgr/cephadm: fix check for if devices have changed
Directly comparing the dicts doesn't work since the
"created" field was added in. We should instead make use
of the existing Devices equality function.

Fixes: https://tracker.ceph.com/issues/57999

Signed-off-by: Adam King <adking@redhat.com>
2022-12-02 09:24:11 -05:00
Ilya Dryomov
9ca2ec704e
Merge pull request #48549 from pkalever/snap-list
cls/rbd: update last_read in group::snap_list

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-12-02 13:18:08 +01:00
Yuval Lifshitz
9965e57cb7
Merge pull request #49177 from yuvalif/remove-pubsub-release-notes
rgw: update release notes on the removal of pubsub
2022-12-02 08:00:03 +02:00
Kefu Chai
279fcd0071
Merge pull request #49052 from tchaikov/wip-fmt-v9
*: include fmt/ostream.h if FMT_VERSION >= 9

Reviewed-by: Tim Serong <tserong@suse.com>
2022-12-02 13:03:50 +08:00
zdover23
f4a94fed5e
Merge pull request #49185 from zdover23/wip-doc-2022-12-01-rados-ops-devices-prompts
doc/rados: add prompts to devices.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-02 12:45:41 +10:00
Zac Dover
66ddc41be8 doc/rados: add prompts to devices.rst
Add unselectable prompts to doc/rados/operations/devices.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-02 12:28:58 +10:00
zdover23
9b3c3db673
Merge pull request #49182 from zdover23/wip-doc-2022-12-01-rados-ops-crush-map-prompts
doc/rados: add prompts to crush-map.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-02 12:28:06 +10:00