Commit Graph

126547 Commits

Author SHA1 Message Date
Kefu Chai
a4d9280c0c ceph.spec.in: make cryptsetup a ceph-volume runtime dep.
cryptsetup is solely used by ceph-volume, so it should be a runtime
dependency of ceph-volume, not ceph-base.

and remove it fomr build-dependency list, as we don't use this tool when
building ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:19 +08:00
Kefu Chai
8e0e9ef382 ceph.spec.in: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:25:17 +08:00
Kefu Chai
c6c260f15b debian: split ceph-volume into a separated package
ceph-volume is a tool implemented in pure python, so it would be better
to make it a architecture independent package for better
maintainability.

in this change

* ceph-volume is extracted out into a separated package
* ceph-volume depends on ceph-osd, as it deploys it and relies on
  an already-installed ceph-osd in the system.
* ceph-osd recommends ceph-volume. as ceph-osd can be used as a
  standalone package. but ceph-volume enhances it. also, to ensure
  the existing users to get ceph-volume installed along with
  ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:19:54 +08:00
Kefu Chai
c1d60c4d00 rpm,deb: make parted runtime dependency of ceph-osd
we don't run parted when building ceph, instead, it is a runtime
dependency of ceph-volume, which is packaged in ceph-osd.

so in this change

* parted is removed from build dependency list
* parted is added as a runtime depedency of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-09-14 23:18:05 +08:00
Guillaume Abrioux
569368a2fc
Merge pull request #43109 from guits/guits-cv-24044
ceph-volume: util/prepare fix osd_id_available()
2021-09-14 10:00:14 +02:00
Samuel Just
771f5a8f8f
Merge pull request #43123 from liu-chunmei/lba_metrics_iter
crimson/seastore: add lba tree metrics for lba tree profiling

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-09-13 20:33:31 -07:00
chunmei-liu
47389d5a73 crimson/seastore: add lba tree metrics for lba tree profiling
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-09-13 18:52:14 -07:00
Samuel Just
3e3ec79de0
Merge pull request #43152 from athanatos/sjust/wip-simplify-delayed
crimson/sesatore: simplify epm related transaction structures, fix some bugs

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-09-13 18:31:47 -07:00
Samuel Just
86898d1b2e crimson/os/seastore/transaction: add trace debugging for write_set
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
e9e34d7f93 crimson/os/seastore/cached_extent: fix unused variable warning
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
f9e51c2f0d crimson/os/seastore/transaction_manager: fix delayed alloc addr in rewrite_logical_extent
Setting nlextent's to the addr of the original extent violates invariants
in the intrusive_set we're using for ExtentIndex and generally breaks the
assumption in cache that paddrs are unique until reuse.  This means we'll
do an extra update_mapping for now -- we can refactor SegmentCleaner to
avoid it later.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
d8326344c9 crimson/os/seastore/scanner: fix budget_used ref in scan_valid_records
Fixes: https://tracker.ceph.com/issues/52582
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
984145d0e3 crimson/os/seastore/cache: more get_extent debugging
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
2fe252cb00 crimson/os/seastore/extent_placement_manager: clarify else branch delayed_alloc_or_ool_write
Also shorten long line.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:17 -07:00
Samuel Just
77af47a838 crimson/os/seastore: replace fresh_block_list with inline/ool_block_list
With this change, Transaction::write_set is the union of mutually exclusive
lists
- delayed_alloc_list, inline_block_list, and mutated_block_list prior to
  calling ExtentPlacementManager::delayed_alloc_or_ool_write

and

- inline_block_list, ool_block_list, and mutated_block_list after
  calling ExtentPlacementManager::delayed_alloc_or_ool_write

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 16:37:14 -07:00
Sage Weil
13238ade13 Merge PR #43136 into master
* refs/pull/43136/head:
	qa/tasks/kubeadm: change calico encap to IPIPCrossSubnet
	qa/suites/orch/rook/smoke: add host networking to matrix
	qa/tasks/rook: fix shadowing of config arg in rook_cluster()

Reviewed-by: Joseph Sawaya <jsawaya@redhat.com>
2021-09-13 18:28:43 -04:00
Ilya Dryomov
6df3293893
Merge pull request #42417 from rzarzynski/wip-bl-optimize-emptybptrs
common/bl, tests: optimize carriage handling in bufferlist::c_str()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-13 23:00:17 +02:00
Sage Weil
528880d3bb qa/tasks/kubeadm: change calico encap to IPIPCrossSubnet
Signed-off-by: Sage Weil <sage@newdream.net>
2021-09-13 15:26:54 -05:00
Samuel Just
2a2fd3d539 crimson/os/seastore/cached_extent: assert !parent_index in ExtentIndex::insert
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 13:02:36 -07:00
Samuel Just
8fbbb1a928 crimson/so/seastore: clarify ExtentIndex clear()
ExtentIndex::clear() needs to handle clearing parent_index.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 13:02:36 -07:00
Samuel Just
fa96a687a8 crimson/os/seastore/transaction: remove delayed_set
delayed_set isn't necessary.  Delayed extents have distinct paddr segments
already, so it's simpler to have a single index for all pending extents.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-13 13:02:36 -07:00
Radoslaw Zarzynski
304188082a tests/bl: verify the carriage handling in bufferlist::c_str().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-13 21:16:32 +02:00
Radoslaw Zarzynski
9d1a84a592 common/bl: abstract from _carriage checking in c_str().
This is intended to enchance readability and sligthly
generalize the previous, `_carriage`-aware optimization.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-13 21:16:32 +02:00
Radoslaw Zarzynski
578505c0c9 common/bl: c_str() considers _carriage now.
It could be that `_carriage` points on empty bptr which is
also stored in the `_buffers` container -- this happens
when we'are trying to preserve the appendable space to not
waste memory and avoid new allocation.
Before the change this approach was imposing unncessary
rebuilding on `c_str()`. Now we take into consideration
the special case when a bufferlist has the extra, empty
bptr at its end.

Fixes: https://tracker.ceph.com/issues/51725
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-13 21:16:14 +02:00
Guillaume Abrioux
73bfa5d2b0 ceph-volume: util/prepare fix osd_id_available()
The current check only allows to request an OSD id that exists but
marked as 'destroyed'.
With this small fix, we can now use `--osd-id` with an id that doesn't
exist at all.

Fixes: https://tracker.ceph.com/issues/50880

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-13 17:00:57 +02:00
Sebastian Wagner
06c0a42bf7
Merge pull request #43141 from sebastian-philipp/doc-orch-rm
doc/cephadm: Removing a service

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-09-13 16:38:01 +02:00
Sage Weil
20ed7c9b57 Merge PR #42998 into master
* refs/pull/42998/head:
	qa/tasks/rook: add testing for host label add/rm
	mgr/rook: host add/rm label in rook orchestrator

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-09-13 10:21:32 -04:00
Kefu Chai
3a95b8e194
Merge pull request #43147 from krunerge/fix_typo
doc/dev/crimson/poseidonstore: fix typo

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-09-13 21:45:11 +08:00
Dimitri Savineau
df1b0a039a
Merge pull request #43063 from guits/guits-trailing-spaces
ceph-volume: remove trailing spaces in code
2021-09-13 09:13:07 -04:00
Ronen Friedman
4ce3989f0a
Merge pull request #42951 from ronen-fr/wip-ronenf-pg-dtor
osd/scrub: destruct the scrubber shortly before the PG is destructed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>-
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-09-13 14:23:52 +03:00
krunerge
0c095374bd doc/dev/crimson/poseidonstore: fix typo
Signed-off-by: krunerge <krunerge@tencent.com>
2021-09-13 19:12:36 +08:00
Ilya Dryomov
6d4535f221
Merge pull request #42555 from hualongfeng/retire_error_change
librbd/cache/pwl/ssd: remove correct m_blocks_to_log_entries entry

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-13 12:04:34 +02:00
Ilya Dryomov
254142a04b
Merge pull request #43038 from majianpeng/librbd-pwl-exclusive-lock
librbd: require exclusive lock for reads if pwl cache is enabled

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-13 11:44:09 +02:00
Feng Hualong
01bb75a105 librbd/cache/pwl/ssd: remove correct m_blocks_to_log_entries entry
When retiring, m_blocks_to_log_entries doesn't remove
corresponding write_entry (should be `*it` not `entry`)
that will be retired. It leads to read error. And
there should also consider discard entries.

Fixes: https://tracker.ceph.com/issues/52579
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-09-13 10:49:18 +02:00
Sebastian Wagner
bae0e903eb
Merge pull request #43143 from sebastian-philipp/further-reading
doc/cephadm: Add lots of links to other chapters

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2021-09-13 09:39:16 +02:00
Ronen Friedman
4acdc2321e
Merge pull request #41869 from ronen-fr/wip-ronenf-fmt-scrub
common, osd: add fmtlib formatting of some OSD types

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-12 16:32:44 +03:00
Ronen Friedman
93212e34f6
Merge pull request #43056 from Matan-B/wip-matanb-scrub-passkey
osd: Replacing friend declarations for passkey interface

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-09-12 15:47:52 +03:00
Ronen Friedman
0e4981bd1e
Merge pull request #42684 from ronen-fr/wip-ronenf-scrub-token
osd/scrub: tag replica scrub messages to identify stale events

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-12 15:40:22 +03:00
Sebastian Wagner
d9ec8eb7a8
doc/cephadm: Add lots of links to other chapters
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-11 20:04:14 +02:00
Sebastian Wagner
7af138e089
doc/cephadm: Removing a service
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-11 19:15:38 +02:00
Kefu Chai
c46bffccbc
Merge pull request #43134 from ceph/debhelpermerged
debian/control: dh-systemd is part of debhelper now

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-09-11 07:42:01 +08:00
David Galloway
4f21d41b73 debian/control: dh-systemd is part of debhelper now
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=822670

Signed-off-by: David Galloway <dgallowa@redhat.com>
2021-09-10 11:56:58 -04:00
Joseph Sawaya
7dfbf7c9f5 qa/tasks/rook: add testing for host label add/rm
This commit adds simple tests that check if the orchestrator
can add and remove labels from nodes.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-09-10 09:44:20 -05:00
Ernesto Puerta
77d703da2b
Merge pull request #43104 from ceph/e2e-tests-documentation
doc/dev: specify location of e2e-tests script

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-09-10 14:32:18 +02:00
Ernesto Puerta
b11439f7d0
Merge pull request #43053 from a16bitsysop/fsevents
mgr/dashboard: use -f for npm ci to skip fsevents error

Reviewed-by: a16bitsysop <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-09-10 14:25:31 +02:00
Ilya Dryomov
d54ece56cc
Merge pull request #42984 from majianpeng/pwl-ssd-race-bug
librbd/cache/pwl/ssd: fix a race between get_cache_bl() and remove_cache_bl()

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-10 11:36:45 +02:00
Ilya Dryomov
f455a7a6c3
Merge pull request #42883 from majianpeng/pwl-ssd-calc-allocated-bug
librbd/cache/pwl: fix m_bytes_{allocated,cached} calculation on reopen

Reviewed-by: Yin Congmin <congmin.yin@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-10 09:41:18 +02:00
Samuel Just
33d826e490
Merge pull request #43124 from xxhdx1985126/wip-om-log-fix
crimson/os/seastore: fix some debug outputs in omap manager

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-09-09 21:53:55 -07:00
Xuehan Xu
0b7547644b crimson/os/seastore: fix some debug outputs in omap manager
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-09-10 10:39:53 +08:00
Samuel Just
ac460ffdd5
Merge pull request #43108 from xxhdx1985126/wip-epm-roll-leak
crimson/os/seastore: fix epm rolling_segment leak when the transaction is interrupted

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-09 13:40:34 -07:00