Commit Graph

138308 Commits

Author SHA1 Message Date
Laura Flores
96dac3b18c
Merge pull request #51925 from ljflores/wip-tracker-59192
qa/suites/rados: whitelist POOL_APP_NOT_ENABLED for cls tests
2023-06-06 11:14:32 -05:00
Samuel Just
03a63b94a1
Merge pull request #51875 from Matan-B/wip-matanb-crimson-supress-tcmalloc
crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-06-06 07:48:30 -07:00
zdover23
cf1446c667
Merge pull request #51914 from zdover23/wip-doc-2023-06-05-rados-operations-pools-2-of-x
doc/rados: edit pools.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-06 22:23:12 +10:00
zdover23
f582a1ab12
Merge pull request #51932 from stackhpc/typo-fix
doc/rados/operations: fix typo in balancer.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-06 22:15:45 +10:00
Zac Dover
fe9a4d2567 doc/rados: edit pools.rst (2 of x)
Edit doc/operations/rados/pools.rst.

There remains confusion in this part of the document regarding pg_num
and pgp_num. pg_num and pgp_num are not explained with sufficient
clarity. A future commit will clear up this confusion. There is also
some potential confusion between on the one hand the strings "pg-num"
and "pgp-num" and on the other hand "pg_num" and "pgp_num". The strings
with the hyphens are used in dummy commands, and the strings with the
underscores are used as key names. I think it possible that this could
confuse a reader, but I am open to discussion on the matter.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-06 22:07:51 +10:00
Pierre Riteau
e4083b3a68 doc/rados/operations: fix typo in balancer.rst
Signed-off-by: Pierre Riteau <pierre@stackhpc.com>
2023-06-06 08:31:07 +02:00
Ilya Dryomov
1caadd7f2a
Merge pull request #50341 from yangdongsheng/ubbd
rbd: add support for new device type of ubbd

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-06-05 23:12:28 +02:00
Laura Flores
66a6e7fdeb qa/suites/rados: whitelist POOL_APP_NOT_ENABLED for rados cls tests
Fixes: https://tracker.ceph.com/issues/59192
Signed-off-by: Laura Flores <lflores@redhat.com>
2023-06-05 15:35:54 -05:00
Pere Diaz Bou
5e58d00957
Merge pull request #51294 from cfsnyder/wip-59588
mgr/dashboard: add 'omit_usage' query param to dashboard api 'get rbd' endpoint

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-06-05 22:15:35 +02:00
Yuri Weinstein
d146d2b56c
Merge pull request #51729 from NitzanMordhai/wip-nitzan-striper-api-aio-futex-wait
test: futex fail if more notification sent after destroy

Reviewed-by: Laura Flores <lflores@redhat.com>
2023-06-05 11:35:35 -04:00
Yuri Weinstein
cd6e5c5924
Merge pull request #51706 from Wangwenjuan1105/wwj_clist_lat
Bluestore: fix bluestore collection_list latency perf counter

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2023-06-05 11:34:55 -04:00
Yuri Weinstein
ca59b0fc53
Merge pull request #51668 from YiteGu/fix-TrackedOp-event-order
osd/TrackedOp: Fix TrackedOp event order

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-06-05 11:34:31 -04:00
Yuri Weinstein
f9df08c831
Merge pull request #51424 from NitzanMordhai/wip-nitzan-monclient-reconnect-finish-auth-with-error
mon/MonClient: before complete auth with error, reopen session

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2023-06-05 11:33:45 -04:00
Yuri Weinstein
b2ec2aff80
Merge pull request #50651 from rosinL/cleanup
Cleanup the LevelDB residue


Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-06-05 11:32:51 -04:00
Mark Nelson
380bc6d2e4 [CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar
classic-osds have always caused significant memory fragmentation
when using the libc memory allocator due to the way that Ceph
tends to utilize memory.  In recent testing, crimson-osd was found
to use 25-27GB of RAM with the stock 3GB bluestore cache settings
(osd_memory_target is only used when tcmalloc is available).  Upon
further testing, it was found that the classic OSD is even worse,
using between 32-33GB of RAM after a 5 minute 4K sequential
write test when using libc malloc.

The good news is that it appears that crimson-osd is able to use
tcmalloc for alienstore without significant modification. Better
still, it drastically reduces memory usage.  In the same test that
resulted in 25GB RSS memory usage for crimson-osd with libc malloc,
a tcmalloc linked version took around 9GB (with an 8GB
osd_memory_target).  Since we do not yet (afaik) expose classic OSD
debugging in crimson it is tough to tell why we are still a little
over, but it's clear that for alienstore we are going to need to
use tcmalloc as we do in classic.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9f31ad7b7388996307808e26722260aed29d4ea1)
2023-06-05 12:03:41 +00:00
Matan Breizman
814a3c23c7 test/crimson: embed lsan_suppressions into tests executables
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-06-05 11:58:28 +00:00
Anthony D'Atri
b554518133
Merge pull request #51915 from zdover23/wip-doc-2023-06-05-rados-operations-pools-typo
doc/rados: pools.rst: "decreaesed"
2023-06-05 06:53:03 -04:00
Matan
46ba1b07ff
Merge pull request #51746 from rosinL/fix-61227
crimson/osd/ops_executer: fix double apply_stats() when writing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-06-05 11:10:50 +02:00
Zac Dover
a003231817 doc/rados: pools.rst: "decreaesed"
s/decreased/decreased/

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-05 13:26:20 +10:00
zdover23
a7be3649f2
Merge pull request #51908 from zdover23/wip-doc-2023-06-04-rados-operations-pools-1-of-x
doc/rados: edit pools.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-05 10:16:12 +10:00
Zac Dover
ebaebffa03 doc/rados: edit pools.rst (1 of x)
Edit doc/rados/operations/pools.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-05 10:04:15 +10:00
zdover23
aef7f67874
Merge pull request #51901 from zdover23/wip-doc-2023-06-03-rados-troubleshooting-log-and-debug-2-of-x
doc/rados: edit log-and-debug.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-04 09:18:27 +10:00
Zac Dover
dc64bcd9d9 doc/rados: edit log-and-debug.rst (2 of x)
Edit roughly the second half of
doc/rados/troubleshooting/log-and-debug.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-04 08:37:01 +10:00
Anthony D'Atri
0da6f95bc4
Merge pull request #51895 from zdover23/wip-doc-2023-06-03-rados-troubleshooting-troubleshooting-mon-1-of-x
doc/rados: edit troubleshooting-mon.rst (1 of x)
2023-06-03 18:31:06 -04:00
zdover23
40a0ac1e21
Merge pull request #51885 from zdover23/wip-doc-2023-06-02-rados-troubleshooting-log-and-debug-1-of-x
doc/rados: edit log-and-debug.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-04 08:04:32 +10:00
Zac Dover
0d74e6a0dd doc/rados: edit log-and-debug.rst (1 of x)
Edit roughly the first half of
doc/rados/troubleshooting/log-and-debug.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-04 07:39:17 +10:00
Zac Dover
0f0896d30a doc/rados: edit troubleshooting-mon.rst (1 of x)
Edit the first 150 lines of
doc/rados/troubleshooting/troubleshooting-mon.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-04 06:55:31 +10:00
Radoslaw Zarzynski
4d3e9642f7
Merge pull request #50576 from rzarzynski/wip-script-unmerged-backport
script: ceph-backport.sh respects --force when  PR is not merged in main

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2023-06-03 21:54:59 +02:00
Anthony D'Atri
078bfe8714
Merge pull request #51896 from zdover23/wip-doc-2023-06-03-rados-operations-pg-repair
doc/rados: fix typos in pg-repair.rst
2023-06-03 04:36:41 -04:00
Zac Dover
fbee2fbba4 doc/rados: fix typos in pg-repair.rst
Fix one RST formatting error and one mistakenly-left-in plural noun in
doc/rados/operations/pg-repair.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-03 15:17:57 +10:00
Yuri Weinstein
a72886263a
Merge pull request #51821 from markhpc/wip-bs-rocksdb-cf-tuning
common/options: Update RocksDB CF Tuning

Reviewed-by: Laura Flores <lflores@redhat.com>
2023-06-02 12:37:04 -04:00
zdover23
3fbfd18fe9
Merge pull request #51872 from zdover23/wip-doc-2023-06-01-cephadm-48180-cleanup-3-of-x
doc: update the cephadm download instructions

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-02 13:51:12 +08:00
Samuel Just
f76c446f75
Merge pull request #51851 from Matan-B/wip-matanb-cmake-allocator-status
CMakeLists.txt: increase verbosity for selected allocator

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2023-06-01 12:56:18 -07:00
Matan Breizman
15e6ca9ec1 crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()
In similarity to lsan.supp:
```
leak:^MallocExtension::Register
```

Shouls suppress this:
```
==2692420==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x5641d4ffaedd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest-object-data-handler+0x7c01edd) (BuildId: 7aeb00a388469f4de54fedaaceb47fc006ce2906)
..
    #7 0x7fb22a6426f0 in MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId: eeef3d1257388a806e122398dbce3157ee568ef4)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-06-01 15:26:04 +00:00
Casey Bodley
bdca2dcefa
Merge pull request #51176 from linuxbox2/wip-fix-rgwfile-ver
rgw/file: advance rgw_file interface version

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-06-01 10:25:32 -04:00
John Mulligan
7c6a2cad96 doc: update the cephadm download instructions
Starting with reef, cephadm is a compiled (zipapp) python application.
The cephadm script has been renamed and thus the old curl-based
download instructions will no loner work. While cephadm still has
no dependencies outside the Python stdlib, this will be changed in
future versions so it is no longer appropriate to just download the
source file of cephadm and run it either.

This change updates the `Install cephadm` section of the doc to explain
how to acquire a "compiled" version of cephadm as well as:
* moving and tweaking the note that the two installation methods are
  distinct
* adding a new note linking to instructions on building cephadm
* moving the distribution-specific installations before the curl-based
  installation to subtly hint that we prefer you to get it using
  packages if you can
* Noting cephadm's minimal required python verision and how to run it
  with a particular python version.

Note from Zac Dover, June 1, 2023: Note: This commit is a cherry-pick of
d11cf0e, which was introduced by John Mulligan in #48180. This is one of
three commits introduced in that PR, and this cherry-pick cleans up
omissions I (Zac Dover) inadvertently introduced while attempting to
rectify the merge conflicts in #51843. This should be the final
main-branch-targeting commit that cleans up PR#51483.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d11cf0e82a)
2023-06-01 22:04:58 +08:00
zdover23
7fa418a02d
Merge pull request #51857 from zdover23/wip-doc-2023-05-31-cephadm-48180-cleanup-1-of-x
doc: make instructions to get an updated cephadm common

Reviewed-by : Adam King <adking@redhat.com>
2023-06-01 19:24:10 +08:00
Nizamudeen A
553a0c9ad1
Merge pull request #51844 from rhcs-dashboard/fix-qa-failure-orch
mgr/dashboard: fix test_dashboard_e2e.sh failure

Reviewed-by: Pegonzal <NOT@FOUND>
2023-06-01 11:59:23 +05:30
Adam King
9d4a73c70f
Merge pull request #51347 from adk3798/maint-host-no-write
mgr/cephadm: also don't write client files/tuned profiles to maintenance hosts

Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-05-31 19:52:20 -04:00
Adam King
93b6f716e6
Merge pull request #51701 from rkachach/fix_issue_61361
cephadm: delete /tmp/cephadm-<fsid> when removing the cluster

Reviewed-by: Adam King <adking@redhat.com>
2023-05-31 14:25:49 -04:00
Adam King
a0ae7c8224
Merge pull request #51660 from rkachach/fix_issue_61330
cephadm: Adding support to configure public_network cfg section

Reviewed-by: Adam King <adking@redhat.com>
2023-05-31 14:18:31 -04:00
Adam King
04ea7bdf64
Merge pull request #50805 from rkachach/fix_issue_59254
mgr/cephadm: show meaningful messages when failing to execute cmds

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-05-31 14:06:06 -04:00
Yuval Lifshitz
b39f27bec7
Merge pull request #51816 from AliMasarweh/wip-alimasa-admin-delete-persistent-topic
RGW:notifications: persistent topics are not deleted via radosgw-admin
2023-05-31 21:01:02 +03:00
Adam King
6c5e5dcabf
Merge pull request #50614 from phlogistonjohn/jjm-nfs-haproxy-cfg
mgr/cephadm: add support for nfs with haproxy protocol

Reviewed-by: Adam King <adking@redhat.com>
2023-05-31 14:00:13 -04:00
Anthony D'Atri
53a3ae7214
Merge pull request #51859 from zdover23/wip-doc-2023-05-31-cephadm-48180-cleanup-2-of-x
doc: add instructions for compiling cephadm
2023-05-31 12:48:27 -04:00
John Mulligan
421147f97c doc: add instructions for compiling cephadm
Now that cephadm is based on zipapp, add a short section to the
developer docs explaining how to build cephadm yourself.

Note: This commit is a cherry-pick of
9ad38033cc, which was introduced by John
Mulligan in #48180. This is one of three commits introduced in that PR,
and this cherry-pick cleans up omissions I (Zac Dover) inadvertently
introduced while attempting to rectify the merge conflicts in #51843. I
expect that one more cherry-picked commit (specifically,
d11cf0e82a, which cannot be merged easily
until d7921e88d6 has been merged into
main, for reasons that are too
Rick-and-Morty-there-should-never-be-more-than-one-dot to go into here)
will follow this one.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 9ad38033cc)
2023-05-31 21:44:21 +08:00
John Mulligan
d9e8a1de0b doc: make instructions to get an updated cephadm common
As discussed in person and over the ceph orch weekly, we want all users
to use a recent supported version of cephadm. Previously, the
instructions only had those downloading cephadm with curl using the
"add-repo" and "install" commands to get a up-to-date cephadm build.
According to ADK we've seen cases of users get "old" distro packages
in the past. Change the instructions so that the "update cephadm" steps
are common after acquiring a "bootstrap copy" of cephadm.

Note: This commit is a cherry-pick of
d7921e88d6, which was introduced by John
Mulligan in https://github.com/ceph/ceph/pull/48180. This is one of
three commits introduced in that PR, and this cherry-pick cleans up
omissions I (Zac Dover) inadvertently introduced while attempting to
rectify the merge conflicts in https://github.com/ceph/ceph/pull/51843.
I expect that two more cherry-picked commits will follow this one.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d7921e88d6)
2023-05-31 21:26:26 +08:00
Ilya Dryomov
a5483da7c7
Merge pull request #51828 from petrutlucian94/it_decrement_issue
librbd: avoid decrementing iterator before first element

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-05-31 15:07:12 +02:00
Ilya Dryomov
0e909583a0
Merge pull request #51814 from idryomov/wip-61382
osd/OSDCap: allow rbd.metadata_list method under rbd-read-only profile

Reviewed-by: Mykola Golub <mgolub@suse.com>
2023-05-31 15:05:16 +02:00
Ali Masarwa
8374da4dde RGW:notifications: persistent topics are not deleted via radosgw-admin
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-05-31 14:15:39 +03:00