Commit Graph

129450 Commits

Author SHA1 Message Date
Ronen Friedman
ab032e9ac5 scrub/osd: add a missing 'publish stats to osd'
to publish the last scrub status report.
The change is needed following the merge of
PR #42735.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-23 08:54:58 +02:00
Kefu Chai
0f448714c2
Merge pull request #44572 from 5cs/fix-spdk-cmake
cmake/modules/BuildSPDK.cmake: link whole-archive

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-01-22 19:51:58 +08:00
Ilya Dryomov
1f07820579 qa/run_xfstests_qemu.sh: fall back to ide disks if needed
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-22 12:32:02 +01:00
Ronen Friedman
a007b29859 osd/scrub: change missing_digest fix-list type
...allowing for efficient creation and, more important,
direct consumption by the function performing the
actual I/O.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-22 10:37:06 +00:00
Ronen Friedman
d3283fdf2c osd/scrub: scrub components embedded documentation
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-22 10:37:05 +00:00
Ronen Friedman
2a3483f7d3 osd/scrub: add to PgScrubber a local ref to next-scrub's details
The owner is still the PG.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: removing PgScrubber's ref to the primary scrub map

as caching this reference (to an object that is owned by the
scrubber backend) creates a dangling-pointer risk.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-22 10:37:05 +00:00
Ilya Dryomov
e1793ebbf0
Merge pull request #44669 from idryomov/wip-rbd-switch-arguments-fix
rbd: add missing switch arguments for recognition by get_command_spec()

Reviewed-by: Mykola Golub <mgolub@suse.com>
2022-01-22 10:37:20 +01:00
Ronen Friedman
7b1c239ccf osd/scrub: minor fixes split from main "scrub backend" commit
Reformatting, minor compiler warnings, etc.
Split from "osd/scrub: extract scrub backend code into separate classes"

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-22 08:59:59 +00:00
Ronen Friedman
1fdf23b4bc osd/scrub: extract scrub backend code into separate classes
A refactoring of the scrub backend code (all Scrub related code
that checks/manipulates objects' (meta)data).

Part of the refactoring was breaking long chunks of code into
separate functions. In order to avoid passing endless lists
of state parameters from/to these new methods, the scrub backend
state is now maintained within its own objects.

a change to note:
Instead of two scrub-map collections - one that holds all
incoming maps, and one that holds pointers to all of those incoming
plus the one we (the Primary) manage - just use one collection
for both incoming and our own.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Split from "osd/scrub: minor fixes split from main "scrub backend" commit"

Split from "osd/scrub: scrub components embedded documentation"

fix

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: fix the scrubber backend to include all PR #42735

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-22 08:59:59 +00:00
Kefu Chai
67ee1fba6c
Merge pull request #44736 from adamemerson/wip-no-using-namespace-std
src: Remove some 'using namespace std' lines that crept back into headers

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-01-22 14:30:38 +08:00
Adam C. Emerson
636ea2382e neorados: Remove using namespace std::literals from header
We don't actually need it.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-21 22:28:45 -05:00
Adam C. Emerson
0878150f8b mgr: Remove straggling using namespace std from headers
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-21 22:28:35 -05:00
Adam C. Emerson
ae07ccb152 test: Remove straggling using namespace std from headers
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-21 22:28:28 -05:00
Adam C. Emerson
6bb7dce6a5 {osd,crimson}: Remove straggling using namespace std from headers
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-21 22:28:28 -05:00
Adam C. Emerson
3c02841a56 rgw: Remove straggling using namespace std from headers
A few crept back in.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-21 22:28:18 -05:00
Adam C. Emerson
f8b9d3f52b cleanup: Change incorrect modelines
Change C modelines to C++ for C++ files.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-21 22:27:36 -05:00
zdover23
dd917a54bc
Merge pull request #44734 from anthonyeleven/anthonyeleven/cephfs-pool-recs
doc/cephfs: Clarify CephFS pool guidance

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2022-01-22 12:35:04 +10:00
Kefu Chai
4ff206fc1f
Merge pull request #44509 from rzarzynski/wip-crimson-alienstore-drop-constant
crimson/alienstore: drop unused constant from AlienStore.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-01-22 08:41:01 +08:00
Neha Ojha
f849f1554c qa/suites/rados: reduce the number of cephadm tests
Currently, every rados run of ~400 jobs is running ~150 cephadm tests,
which is unnecessary and redundant. With this change, we will run some
basic cephadm tests within the rados suite. The following seems to be
a good start.

qa/suites/rados/cephadm/osds
qa/suites/rados/cephadm/smoke
qa/suites/rados/cephadm/smoke-singlehost
qa/suites/rados/cephadm/workunits

Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-01-21 23:38:53 +00:00
Neha Ojha
427d9c0407
Merge pull request #44646 from neha-ojha/wip-backport-q
script: backport updates for quincy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-01-21 15:04:41 -08:00
Igor Fedotov
87a30bdb73
Merge pull request #44265 from KellenRenshaw/WIP-replay_alloc
test/allocator_replay_test: Add replay_alloc option

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2022-01-21 23:52:32 +03:00
Ilya Dryomov
4583c63dba
Merge pull request #44705 from ceph/wip-yuriw-build-script
script/build-integration-branch: add quincy to the list of releases

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-21 21:43:27 +01:00
Anthony D'Atri
822cc71b56 doc/cephfs: Clarify CephFS pool guidance
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-01-21 12:28:05 -08:00
Adam C. Emerson
485d85cd3e
Merge pull request #44674 from adamemerson/wip-53941
rgw: Report empty endpoints as error instead of crashing

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2022-01-21 15:14:42 -05:00
Neha Ojha
87a490d7b0
Merge pull request #44706 from rzarzynski/wip-tests-bl-fix-rebuild
test/bufferlist: ensure rebuild_aligned_size_and_memory() always rebuilds.

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-01-21 11:22:07 -08:00
Casey Bodley
a9b3e2f801
Merge pull request #44717 from zdover23/wip-doc-2021-01-22-man-page-radosgw-admin-missing-apostrophe
doc/man: radosgw-admin.rst: missing apostrophe

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-01-21 13:22:44 -05:00
Zac Dover
e9ab3e2ee0 doc/radosgw: replace double quotes with single
This changes the string
  "tenant$<user>"
to
  'tenant$<user>'
in order to avoid users running into confusing
shell expansion behavior.

Hat tip to IcePic.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-01-22 03:05:01 +10:00
Zac Dover
9f65f42d9e doc/man: radosgw-admin.rst: missing apostrophe
This commit adds two missing apostrophes to two
different strings.

Hat tip to IcePic.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-01-22 02:57:28 +10:00
Ali Maredia
262286776d
Merge pull request #44694 from alimaredia/wip-rgw-kmip-teuthology-test
qa: move certificates for kmip task into /etc/ceph

Reviewed-by: Al Maredia <amaredia@redhat.com>
2022-01-21 11:56:45 -05:00
Ilya Dryomov
303d3ede48 test/rbd_mirror: drop redundant MockJournaler instances
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-21 17:32:40 +01:00
Ilya Dryomov
d634a1df5b rbd-mirror: fix races in snapshot-based mirroring deletion propagation
When remote image is deleted, rbd-mirror can encounter three cases:

  1) no remote image id
  2) no remote mirror metadata
  3) MIRROR_IMAGE_STATE_DISABLING in remote mirror metadata

Commit d4c66ac5c6 ("rbd-mirror: fix issue with snapshot-based
mirroring deletion propagation") fixed case 1.  Cases 2 and 3 remained
broken because for both of them finalize_snapshot_state_builder() would
populate not only remote_mirror_peer_uuid but also remote_image_id,
thus disabling ENOLINK logic in handle_prepare_remote_image() and
handle_bootstrap().  Commit ff60aec2d9 ("rbd-mirror: fix bootstrap
sequence while the image is removed") touched on case 3, but it made
a difference only for journal-based mirroring.

Stop calling finalize_snapshot_state_builder() on errors.  Instead,
align with journal-based mirroring by filling remote_mirror_peer_uuid
together with remote_mirror_uuid.

Fixes: https://tracker.ceph.com/issues/53963
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-21 17:32:36 +01:00
Pete Zaitcev
06e1ef35a8
Merge pull request #44112 from zaitcev/wip-py_ssize_t_clean
mgr: Define PY_SSIZE_T_CLEAN ahead of every Python.h
2022-01-21 09:05:16 -06:00
Ilya Dryomov
ccfbf3e97e rbd-mirror: don't default replay_requires_remote_image() implementation
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-21 14:31:07 +01:00
Ilya Dryomov
f49fa483ec rbd-mirror: untangle StateBuilder::is_linked() overloads
Make it clear that the local image non-primariness is asserted
independent of the mode; avoid the default implementation being
overridden but still relied on by both modes.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-21 14:31:07 +01:00
Ilya Dryomov
baf57925ab rbd-mirror: drop redundant initialization of StateBuilder members
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-21 14:31:07 +01:00
Zac Dover
930086797b src/rados: rewrite bs cache autotune instructions
This commit rewrites one sentence in the "Bluestore
Config Reference". The sentence does not itself appear
in the directory structure under /docs/rados/, but is
instead in /src/common/options/global.yaml.in and is
included in the Bluestore Config Reference by means of
a confval directive.

The rewritten sentence has been rewritten from the passive
voice to the active voice, and a "how"-formation has been
removed. The removal of the "how"-formation, in my opinion,
makes the sentence go down much smoother for the first-time
reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-01-21 23:18:29 +10:00
Guillaume Abrioux
f23e1ba25f
Merge pull request #44604 from mgfritch/c-v-skip-rbd-devs
ceph-volume: show RBD devices as not available
2022-01-21 13:37:25 +01:00
Sebastian Wagner
f3006873dc
Merge pull request #44687 from sebastian-philipp/gov
doc/governance: Adam King

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-01-21 09:56:00 +01:00
Yingxin Cheng
7c23739c96 crimson/os/seastore/journal: add logs to flush operation
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:59:22 +08:00
Yingxin Cheng
abb0951eda crimson/os/seastore/cache: consolidate logs in levels
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:59:22 +08:00
Yingxin Cheng
9d6bf63bb1 crimson/os/seastore: introduce seastore_t logs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:58:17 +08:00
Yingxin Cheng
04ff56ebba crimson/os/seastore: move transaction-id forward in log
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:58:17 +08:00
Yingxin Cheng
6651dad143 crimson/os/seastore/cache: misc cleanup
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:58:17 +08:00
Yingxin Cheng
faf1bc5c47 crimson/os/seastore: cleanup effort_t and reuse Transaction::io_stat_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:58:17 +08:00
Yingxin Cheng
96ad6104d7 crimson/os/seastore: consolidate Cache::init_cached_extent with drop_from_cache
So Cache::drop_from_cache() can be removed, and make it possible to add
related logs within the Cache component.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-21 14:58:17 +08:00
Radoslaw Zarzynski
88176acd27 test/bufferlist: ensure rebuild_aligned_size_and_memory() always rebuilds.
Before the patch the test case was showing an unreliable behaviour
dependent on the underlying memory allocator. It was because
the bufferlist rebuild can be skipped, resulting in unchanged number
of buffers, if all of them begin at aligned addresses.

The commit fixes that by allocating a 4 KiB-aligned buffer and
offsetting it by a small constant (42) to ensure the memory added
to the bufferlist begins at non-4 KiB address.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-01-21 02:13:52 +00:00
Yuri Weinstein
785b921d09 qa/tests: add quincy name to teh wip name
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2022-01-20 16:17:19 -08:00
Ali Maredia
99f0e82a95 qa: move certificates for kmip task into /etc/ceph
On rhel/centos the ceph user does not have permission
to access these certs which leads to s3-test failures
in teuthology.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2022-01-20 17:43:55 -05:00
John Mulligan
c0b1a910f1 cephadm: add tests for Packager validate function
With the validate function split from the add_repo function we can
independently test the behavior of the validate function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-01-20 16:31:23 -05:00
John Mulligan
ee71ff79a3 cephadm: validate that the constructed YumDnf baseurl is usable
If the inputs to the `cephadm add-repo` command would result in an
invalid URL for repo metadata fail the command early with a (somewhat)
helpful error.

Fixes: https://tracker.ceph.com/issues/46773

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-01-20 16:30:58 -05:00