Commit Graph

126754 Commits

Author SHA1 Message Date
Kefu Chai
fd9d563a51 debian/control: make cryptsetup-bin a ceph-volume runtime dep
was originally added by 6a45b8ed04 and
cfcaceac44 as a dependency of ceph-disk.
although, ceph-disk was replaced by ceph-volume, cryptsetup-bin is still
used.

but it should be a runtime dependency of ceph-volume. and it is not a
build dependency of ceph, so it's removed from the build dependency list
as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
a4d9280c0c ceph.spec.in: make cryptsetup a ceph-volume runtime dep.
cryptsetup is solely used by ceph-volume, so it should be a runtime
dependency of ceph-volume, not ceph-base.

and remove it fomr build-dependency list, as we don't use this tool when
building ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
8e0e9ef382 ceph.spec.in: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:17 +08:00
Kefu Chai
c6c260f15b debian: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.

in this change

* ceph-volume is extracted out into a separated package
* ceph-volume depends on ceph-osd, as it deploys it and relies on
  an already-installed ceph-osd in the system.
* ceph-osd recommends ceph-volume. as ceph-osd can be used as a
  standalone package. but ceph-volume enhances it. also, to ensure
  the existing users to get ceph-volume installed along with
  ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:19:54 +08:00
Kefu Chai
c1d60c4d00 rpm,deb: make parted runtime dependency of ceph-osd
we don't run parted when building ceph, instead, it is a runtime
dependency of ceph-volume, which is packaged in ceph-osd.

so in this change

* parted is removed from build dependency list
* parted is added as a runtime depedency of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:18:05 +08:00
Radoslaw Zarzynski
92974b300c crimson/os: add the preamble to futurized_store.cc.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 15:05:32 +00:00
Radoslaw Zarzynski
f01add8c34 crimson/os: allow to build crimson when WITH_BLUESTORE=OFF.
This lets to ensure nobody is accidentally linking with the alienized
version of `common`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 15:04:35 +00:00
Sebastian Wagner
b68c99cd73
doc/cephadm/service-management: Move Deployment of Daemons
.. into "Daemon Placement"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-14 16:49:25 +02:00
Sebastian Wagner
80f33a4e7d
doc/cephadm/service-management: Move "Updating Service Specifications"
into "Service Specification"

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-14 16:47:56 +02:00
Radoslaw Zarzynski
d5f96ce947 crimson: switch try_bind() to bind().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 10:01:35 +00:00
Radoslaw Zarzynski
4a00a14525 crimson/net: bind() respects now ms_bind_retry_{delay,count}.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 10:01:35 +00:00
Radoslaw Zarzynski
e90c6a9444 crimson/net: make try_bind() aware about address_not_available.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 10:01:35 +00:00
Radoslaw Zarzynski
0f3779e2ac crimson/net: decouple listen_ertr from bind_ertr.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 10:01:35 +00:00
Radoslaw Zarzynski
7f85b51161 crimson/net: decouple listen_ertr from FixedCPUServerSocket.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-14 10:01:35 +00:00
Guillaume Abrioux
569368a2fc
Merge pull request #43109 from guits/guits-cv-24044
ceph-volume: util/prepare fix osd_id_available()
2021-09-14 10:00:14 +02:00
Pritha Srivastava
cdb716252e rgw/sts: add check for s->auth.identity pointer
which can be null for some calls like
AssumeRoleWithWebIdentity in rgw_log.cc

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2021-09-14 11:40:29 +05:30
Samuel Just
6a1ed1fda9 crimson/os/seastore: SeaStore::get_omap_iterator should seek_to_first
Add debugging as well.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 21:37:18 -07:00
Samuel Just
a87bf7b055 vstart.sh: default to msgr2 for crimson
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 21:36:37 -07:00
Samuel Just
771f5a8f8f
Merge pull request #43123 from liu-chunmei/lba_metrics_iter
crimson/seastore: add lba tree metrics for lba tree profiling

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-09-13 20:33:31 -07:00
chunmei-liu
47389d5a73 crimson/seastore: add lba tree metrics for lba tree profiling
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-09-13 18:52:14 -07:00
Samuel Just
3e3ec79de0
Merge pull request #43152 from athanatos/sjust/wip-simplify-delayed
crimson/sesatore: simplify epm related transaction structures, fix some bugs

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-09-13 18:31:47 -07:00
Samuel Just
86898d1b2e crimson/os/seastore/transaction: add trace debugging for write_set
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
e9e34d7f93 crimson/os/seastore/cached_extent: fix unused variable warning
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
f9e51c2f0d crimson/os/seastore/transaction_manager: fix delayed alloc addr in rewrite_logical_extent
Setting nlextent's to the addr of the original extent violates invariants
in the intrusive_set we're using for ExtentIndex and generally breaks the
assumption in cache that paddrs are unique until reuse.  This means we'll
do an extra update_mapping for now -- we can refactor SegmentCleaner to
avoid it later.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
d8326344c9 crimson/os/seastore/scanner: fix budget_used ref in scan_valid_records
Fixes: https://tracker.ceph.com/issues/52582
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
984145d0e3 crimson/os/seastore/cache: more get_extent debugging
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
2fe252cb00 crimson/os/seastore/extent_placement_manager: clarify else branch delayed_alloc_or_ool_write
Also shorten long line.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
77af47a838 crimson/os/seastore: replace fresh_block_list with inline/ool_block_list
With this change, Transaction::write_set is the union of mutually exclusive
lists
- delayed_alloc_list, inline_block_list, and mutated_block_list prior to
  calling ExtentPlacementManager::delayed_alloc_or_ool_write

and

- inline_block_list, ool_block_list, and mutated_block_list after
  calling ExtentPlacementManager::delayed_alloc_or_ool_write

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:14 -07:00
Sage Weil
688f963da7 qa/suites/orch/rook/smoke: add flannel back in
Signed-off-by: Sage Weil <sage@newdream.net>
2021-09-13 17:28:52 -05:00
Sage Weil
13238ade13 Merge PR #43136 into master
* refs/pull/43136/head:
	qa/tasks/kubeadm: change calico encap to IPIPCrossSubnet
	qa/suites/orch/rook/smoke: add host networking to matrix
	qa/tasks/rook: fix shadowing of config arg in rook_cluster()

Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
2021-09-13 18:28:43 -04:00
Ilya Dryomov
6df3293893
Merge pull request #42417 from rzarzynski/wip-bl-optimize-emptybptrs
common/bl, tests: optimize carriage handling in bufferlist::c_str()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-13 23:00:17 +02:00
Sage Weil
528880d3bb qa/tasks/kubeadm: change calico encap to IPIPCrossSubnet
Signed-off-by: Sage Weil <sage@newdream.net>
2021-09-13 15:26:54 -05:00
Samuel Just
2a2fd3d539 crimson/os/seastore/cached_extent: assert !parent_index in ExtentIndex::insert
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 13:02:36 -07:00
Samuel Just
8fbbb1a928 crimson/so/seastore: clarify ExtentIndex clear()
ExtentIndex::clear() needs to handle clearing parent_index.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 13:02:36 -07:00
Samuel Just
fa96a687a8 crimson/os/seastore/transaction: remove delayed_set
delayed_set isn't necessary.  Delayed extents have distinct paddr segments
already, so it's simpler to have a single index for all pending extents.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 13:02:36 -07:00
Radoslaw Zarzynski
304188082a tests/bl: verify the carriage handling in bufferlist::c_str().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-13 21:16:32 +02:00
Radoslaw Zarzynski
9d1a84a592 common/bl: abstract from _carriage checking in c_str().
This is intended to enchance readability and sligthly
generalize the previous, `_carriage`-aware optimization.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-13 21:16:32 +02:00
Radoslaw Zarzynski
578505c0c9 common/bl: c_str() considers _carriage now.
It could be that `_carriage` points on empty bptr which is
also stored in the `_buffers` container -- this happens
when we'are trying to preserve the appendable space to not
waste memory and avoid new allocation.
Before the change this approach was imposing unncessary
rebuilding on `c_str()`. Now we take into consideration
the special case when a bufferlist has the extra, empty
bptr at its end.

Fixes: https://tracker.ceph.com/issues/51725
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-13 21:16:14 +02:00
Guillaume Abrioux
73bfa5d2b0 ceph-volume: util/prepare fix osd_id_available()
The current check only allows to request an OSD id that exists but
marked as 'destroyed'.
With this small fix, we can now use `--osd-id` with an id that doesn't
exist at all.

Fixes: https://tracker.ceph.com/issues/50880

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-13 17:00:57 +02:00
Joseph Sawaya
5d73421079 mgr/rook: add unit tests to Rook module
This commit creates a unit test folder for rook, configurates tox
and creates a unit test to test the translation of placement specs
to node selectors and vice versa.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-09-13 11:00:09 -04:00
Joseph Sawaya
58d13a5dc4 mgr/rook: translate placement spec to node selector and vice versa
This commit creates methods for translating PlacementSpecs to NodeSelectors
and vice versa, this is a general method that would be used by orch commands that
have a placement option. It's able to translate specs that specify a label, hosts
and the host_pattern '*'.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-09-13 11:00:07 -04:00
Sebastian Wagner
06c0a42bf7
Merge pull request #43141 from sebastian-philipp/doc-orch-rm
doc/cephadm: Removing a service

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-09-13 16:38:01 +02:00
Sage Weil
20ed7c9b57 Merge PR #42998 into master
* refs/pull/42998/head:
	qa/tasks/rook: add testing for host label add/rm
	mgr/rook: host add/rm label in rook orchestrator

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-09-13 10:21:32 -04:00
Kefu Chai
3a95b8e194
Merge pull request #43147 from krunerge/fix_typo
doc/dev/crimson/poseidonstore: fix typo

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-09-13 21:45:11 +08:00
Dimitri Savineau
df1b0a039a
Merge pull request #43063 from guits/guits-trailing-spaces
ceph-volume: remove trailing spaces in code
2021-09-13 09:13:07 -04:00
Ronen Friedman
4ce3989f0a
Merge pull request #42951 from ronen-fr/wip-ronenf-pg-dtor
osd/scrub: destruct the scrubber shortly before the PG is destructed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>-
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-09-13 14:23:52 +03:00
krunerge
0c095374bd doc/dev/crimson/poseidonstore: fix typo
Signed-off-by: krunerge <krunerge@tencent.com>
2021-09-13 19:12:36 +08:00
Jianwei Zhang
dcdb188b6f osd: fix partial recovery become whole object recovery after restart osd
support SERVER_OCTOPUS feature for pg_missing_item::encode()

Fixes: https://tracker.ceph.com/issues/52583

Signed-off-by: Jianwei Zhang <jianwei1216@qq.com>
2021-09-13 18:16:07 +08:00
Ilya Dryomov
6d4535f221
Merge pull request #42555 from hualongfeng/retire_error_change
librbd/cache/pwl/ssd: remove correct m_blocks_to_log_entries entry

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-13 12:04:34 +02:00
Ilya Dryomov
254142a04b
Merge pull request #43038 from majianpeng/librbd-pwl-exclusive-lock
librbd: require exclusive lock for reads if pwl cache is enabled

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-13 11:44:09 +02:00