Commit Graph

139223 Commits

Author SHA1 Message Date
Guillaume Abrioux
e5e429617c ceph-volume: fix raw list for lvm devices
raw list shouldn't list lvm OSDs.
This confuses the generic activate process given that it relies
on the raw list to detect which kind of OSD is being activated.

Fixes: https://tracker.ceph.com/issues/62002

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-07-19 10:17:00 +02:00
Guillaume Abrioux
b3fd5b5131 ceph-volume: fix broken workaround for atari partitions
broken by bea9f4b643
This commits fixes that regression.

Fixes: https://tracker.ceph.com/issues/62001

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-07-19 10:17:00 +02:00
Matan
460e4bf672
Merge pull request #52266 from amathuria/wip-crimson-amat-fix-config-show
crimson/mgr: Fix config show command

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-19 10:45:15 +03:00
tridao
db95414abd Add allocator op metrics
Specifically total latency, average latency and total ops

Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-18 18:03:18 -03:00
Patrick Donnelly
23e8ddb09c
mds: remove dead method
Fixes: cf5fdcd6e2
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-07-18 10:52:19 -04:00
Venky Shankar
4e5d800406 Merge PR #51539 into main
* refs/pull/51539/head:
	doc: users now need to provide scrub_mdsdir and recursive flags
	qa: add recursive flag to test_flag_scrub_mdsdir
	mds: remove code to bypass dumping empty header scrub info
	mds: dump_values no more needed
	mds: enqueue ~mdsdir at the time of enqueing root

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-07-18 20:15:24 +05:30
Casey Bodley
21ad52551d
Merge pull request #51700 from cfsnyder/wip-cfsnyder-put-404
rgw: fix consistency bug with OLH objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-07-18 09:28:48 -04:00
Lucian Petrut
80d2d01d61 test/common: skip unittest_back_trace with mingw-gcc
We're getting an ICE when trying to compile this test using
mingw-gcc and recent Boost versions. Note that mingw-llvm works fine.

    during IPA pass: inline
    /mnt/data/workspace/ceph.pr/src/test/common/test_back_trace.cc:44:1:
    internal compiler error: Segmentation fault
       44 | }
          | ^
    0x7f9c4a86c51f ???
            ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
    0x7f9c4a853d8f __libc_start_call_main
            ../sysdeps/x86/libc-start.c:58
    0x7f9c4a853e3f __libc_start_main_impl
            ../sysdeps/nptl/libc_start_call_main.h:392

For now, we'll just skip the test when using mingw-gcc. Note
that we're planing to switch to mingw-llvm anyway:
https://github.com/ceph/ceph/pull/51197

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-07-18 12:28:12 +00:00
Lucian Petrut
9d92c2d74e win32_deps_build: skip patching removed boost files
We're attempting to patch some Python files that have been removed
from recent Boost versions.

Now that the Boost version has been bumped, we'll need to address
this in order to unblock the Windows build.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-07-18 12:28:11 +00:00
Adam Kupczyk
236f8b632f
Merge pull request #52193 from aclamk/wip-aclamk-fix-55260
os/bluestore: Fixes ceph-bluestore-tool bluefs-bdev-new-db
2023-07-18 14:27:24 +02:00
Venky Shankar
09fb085cd3 Merge PR #52173 into main
* refs/pull/52173/head:
	mds: MDLog::_recovery_thread: handle the errors gracefully

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-07-18 16:10:24 +05:30
Xuehan Xu
5eb473b99c crimson/os/seastore/collection_manager: fill CollectionNode::decoded on
clean reads

Otherwise, deltas of collection nodes can't be replayed correctly,
because the data of nodes is not loaded to CollectionNode::decoded

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-07-18 17:09:09 +08:00
Casey Bodley
cbdd520995 qa/suites: install pytest for pybind tasks
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-07-17 16:31:08 -04:00
Ali Masarwa
10addc6485 RGW: fix issue in observability over the persistent topics queue
when releasing entires, we don't decrease them in the urgent data

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-07-17 18:13:50 +03:00
Igor Fedotov
5cdd29f78b os/bluestore: do not include single AU allocations to allocation
fragmentation stats.

The rationale is that such allocations always get single framgent.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2023-07-17 16:25:27 +03:00
Igor Fedotov
600fc5b0f1 test/allocator_replay: introduce allocator fragmentation histogram.
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2023-07-17 16:25:27 +03:00
Igor Fedotov
625e3461fb os/bluestore: implement allocator fragmentation histogram
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2023-07-17 16:25:27 +03:00
zdover23
82ddbb6cd1
Merge pull request #52483 from bluikko/patch-15
doc/mgr/rgw.rst: add missing "ceph" command in cli specification

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-07-17 23:19:46 +10:00
Ville Ojamo
37ea3e50f0
doc/mgr/rgw.rst: add missing "ceph" command in cli specification
The line begins with a prompt so I would
expect it to be a "complete" command,
including the `ceph` binary to run with all
the various listed arguments.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2023-07-17 20:05:39 +07:00
Venky Shankar
923e7bacc9 Merge PR #51383 into main
* refs/pull/51383/head:
	client: clean up the queue_cap_snap() code to remove unnecessary check
	client: always add one new capsnap if Fb is used and Fw is not used

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-07-17 13:59:40 +05:30
Venky Shankar
f106724ae5 Merge PR #51850 into main
* refs/pull/51850/head:
	client: do not send metrics until the MDS rank is ready

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-07-17 13:57:41 +05:30
Guillaume Abrioux
6d3dc771e0
Merge pull request #52422 from guits/cv-add-raw-osd-id-param
ceph-volume: add --osd-id option to raw prepare
2023-07-17 09:21:34 +02:00
zdover23
0334253a75
Merge pull request #52476 from bluikko/patch-14
doc/mgr/rgw.rst: fix multisite typed wrong

Reviewed-by: Rongqi Sun (Svelar) <sunrongqi@huawei.co>
2023-07-17 16:50:14 +10:00
Ville Ojamo
33b8fdb203
doc/mgr/rgw.rst: multisite typed wrong
Multisite is typed wrong as mutlisite so fix the spelling.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2023-07-17 10:35:46 +07:00
tridao
17bf50c320 First iteration of OpsReplayer
Signed-off-by: tridao <daominhtri0503@gmail.com>
2023-07-16 23:23:46 -03:00
tridao
a84364182a Validate all collections are in the same pool
Signed-off-by: tridao <daominhtri0503@gmail.com>
2023-07-16 23:23:24 -03:00
tridao
9e95ffb924 Minor cleanup
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:16:44 -03:00
tridao
7a028fdfb4 Add simple multi-threaded test
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:16:24 -03:00
tridao
d83c5a73eb Tweaking per-obj fragmentation score
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:16:20 -03:00
tridao
0956ff67ea Create per-access fragmentation metrics
- Keeps track of read ops in the imitator
- The metrics is defined as jumps per blocks read with jumps as the
  number of times having to stop reading continuous extents
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:16:08 -03:00
tridao
8dd44ea9f5 Add another object to RandomCW
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:14:27 -03:00
tridao
61e27564b1 Move Object's method definitions out of header
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:13:56 -03:00
tridao
f4619147a5 First iteration of per-object fragmentaion metrics
Added a RandomCW generator and use an map to map logical offset to physical
extents (this allows for sparse objects)
Signed-off-by: Tri Dao <daominhtri0503@gmail.com>
2023-07-16 23:13:01 -03:00
Xuehan Xu
c1fdb2a8a4 crimson/os/seastore/object_data_handler: add ObjectDataHandler(extent_len_t)
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-07-17 09:47:11 +08:00
Yuval Lifshitz
563c501ff4 rgw/test: allow passing vstart parameters to multisite tests script
via the VSTART_PARAMETERS env variable

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2023-07-16 16:52:47 +00:00
Vedansh Bhartia
ca3a1dbbca rgw: add coverity annotations and asserts for null dereferences
Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-07-16 21:35:46 +05:30
Vedansh Bhartia
b0e6fc0dda rgw: refactor D3nDataCache::d3n_io_write to avoid resource leaks
1. D3nChunkDataInfo was allocated at the beginning of the function on
   the heap, and was not freed in case of errors. This memory allocation
   has been moved to where it is being utilised.
2. cache_file file descriptor was not closed if the write to the file
   failed. To avoid such issues, cache_file is declared as a unique_ptr
   with a custom deleter that closes the descriptor when the ptr goes
   out of scope.

Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-07-16 21:19:17 +05:30
zdover23
652434f8f7
Merge pull request #52465 from zdover23/wip-doc-2023-07-15-rados-configuration-ceph-conf-2-of-x
doc/rados: edit ceph-conf.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-07-16 19:16:32 +10:00
Zac Dover
69b9b20be1 doc/rados: edit ceph-conf.rst (2 of x)
Edit doc/rados/configuration/ceph-conf.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-07-16 19:08:45 +10:00
Vedansh Bhartia
dc00ae6a66 rgw: Remove logically dead code
Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-07-16 12:32:20 +05:30
Vedansh Bhartia
a590feb28a rgw: Add coverity annotation for missing break statement in switch
Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-07-16 10:41:24 +05:30
Vedansh Bhartia
9ce3bcc12a rgw: When finding bucket owner info, handle account name being empty
Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-07-16 10:23:55 +05:30
Rishabh Dave
5f1dece001
Merge pull request #51317 from rishabh-d-dave/mdscaps-combinations
mds: allow all types of mds capsq

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-07-15 08:05:56 +05:30
Yuri Weinstein
0c22b59631
Merge pull request #52185 from YiteGu/raise-prio-before-queue-op
osd/perf_counters: raise prio of before queue op perfcounter

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-14 16:31:56 -04:00
Yuri Weinstein
108f485bae
Merge pull request #52088 from gazdi31/fix_1
mon/ConfigMonitor: update crush_location from osd entity

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-14 16:30:39 -04:00
Yuri Weinstein
06d93ef558
Merge pull request #50393 from rhcs-dashboard/mirror-con
msgr: AsyncMessenger faulted connections metrics

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2023-07-14 16:30:12 -04:00
Yuri Weinstein
deae3a6f18
Merge pull request #50291 from batrick/i55606
pybind/mgr: reopen database handle on blocklist

Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-07-14 16:28:40 -04:00
Yuri Weinstein
c57faa651d
Merge pull request #51947 from alimaredia/wip-counter-dump-array-add
common: `counter dump` command revision

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2023-07-14 16:26:00 -04:00
Shilpa Jagannath
b51bafd88f
Merge pull request #51493 from smanjara/wip-data-sync-fairness
rgw: data sync fairness
2023-07-14 16:11:53 -04:00
Casey Bodley
3ca0f45de9
Merge pull request #52431 from ronen-fr/wip-rf-fix-opsfmt
OSD, common: fix fmtlib v8 handling of OpSchedItem

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-14 10:34:30 -04:00