Commit Graph

144191 Commits

Author SHA1 Message Date
Kefu Chai
a837f494b5 cmake: prevent ASAN_OPTIONS from detect_odr_violation
turns out we have multiple copies of following symbol defined by rbd executable:
```
AddressSanitizer: odr-violation: global 'ceph::buffer::list::always_empty_bptr' at /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/buffer.cc:1267:34
```
before addressing it. let's disable this warning.

Refs https://tracker.ceph.com/issues/65098

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-27 07:35:28 +08:00
Kefu Chai
f682489448 pybind: use LDFLAGS in env variable when check for building env
when building python bindings extensions using Cython, we first perform
sanity checks by building an executable and linking it against the
shared library of the C language binding. if the executable builds,
we consider the sanity test passes.

before this change, we don't add `LDFLAGS` specified in environmental
variable to the ldflags when building the executable. but we need to
link against libasan, so that the symbols used by, for instance,
librados.so, can be found by the executable.

in this change, we always check `LDFLAGS`, and add it to the ldflags
when building the executable for testing, if it's set.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-27 07:35:28 +08:00
Kefu Chai
d22734f6cb cmake: add ${CMAKE_SHARED_LINKER_FLAGS} to LDFLAGS when building python extensions
if sanitizers are enabled, we have to populate the required link flags
to python extensions's building workflow. otherwise ld would fail to
link like:
```
/usr/bin/ld: /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libceph-common.so.2: undefined reference to `__asan_stack_free_10'
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-27 07:35:28 +08:00
Kefu Chai
0dc61a4be3 cmake: do not override CMAKE_EXE_LINKER_FLAGS
instead of overriding CMAKE_EXE_LINKER_FLAGS, let's append to it.
so that the existing `CMAKE_EXE_LINKER_FLAGS` is not overriden.

this should enable us to build with Clang and with sanitizer(s) enabled.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-27 07:35:28 +08:00
Kefu Chai
a13a61d63d cmake: error out on UBSan error
so we can be alerted if UBSan identify something wrong.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-27 07:35:28 +08:00
Kefu Chai
ce444d473c cmake: quote a list using quotes
otherwised we'd have

```
CMake Error at cmake/modules/FindSanitizers.cmake:17 (if):
  if given arguments:

    "address" "IN_LIST" "address" "thread" "undefined_behavior" "OR" "leak" "IN_LIST" "address" "thread" "undefined_behavior"

  Unknown arguments specified
```

when enabling TSan with WITH_TSAN=ON.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-27 07:35:28 +08:00
Kefu Chai
996de1b12f
Merge pull request #56461 from tchaikov/wip-test-test_config_map-fix-leaks
test/mon/test_config_map: fix leaks identified by LeakSanitizer

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-03-27 07:31:34 +08:00
Zac Dover
3c04e43e10
Merge pull request #56450 from tahajahangir/patch-2
docs/rados: remove incorrect ceph command

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2024-03-27 06:11:54 +10:00
Adam King
ae6d67eebe
Merge pull request #56458 from adk3798/rgw-bootstrap-hostnames
mgr/cephadm: update rgw zonegroup w/ zonegroup_hostnames from rgw spec

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2024-03-26 15:31:06 -04:00
Patrick Donnelly
17a4fdbadc
Merge PR #56454 into main
* refs/pull/56454/head:
	vstart: require debugging switch for all mds debug configs

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-03-26 11:08:50 -04:00
Nizamudeen A
00a5a3dae9
Merge pull request #56351 from afreen23/wip-advanced-sec
mgr/dashboard: Add advanced fieldset component

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-26 19:21:29 +05:30
Yingxin
bcb2e08551
Merge pull request #56462 from zhscn/fix-uaf
crimson/os/seastore: fix use after free error

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2024-03-26 17:08:30 +08:00
Zac Dover
a1764c6a72
Merge pull request #56436 from zdover23/wip-doc-2024-03-25-start-intro
doc/start: improve MDS explanation

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-03-26 18:29:26 +10:00
Aashish Sharma
3472eeb01d
Merge pull request #56328 from rhcs-dashboard/cors-through-cli
mgr/dashboard: introduce cli commands to set cors urls

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2024-03-26 13:44:09 +05:30
Aashish Sharma
50d48d9e39
Merge pull request #55783 from rhcs-dashboard/multi-cluster-imrovements
mgr/dashboard: Multi cluster improvements


Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-26 13:02:28 +05:30
Nizamudeen A
4a987d7af9
Merge pull request #56390 from rhcs-dashboard/rgw-add-system-user
mgr/dashboard: add system users to rgw user form

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-26 12:33:02 +05:30
Afreen
d0979e9c52 mgr/dashboard: Add advanced fieldset component
Fixes https://tracker.ceph.com/issues/65024

- adds a new shared component for displaying advanced fields in /shared
- utilizes that component in bucket form
- placement targets are under advanced fieldset
- minor help text changes in object locking help text
- updated e2e tests

Signed-off-by: Afreen <afreen23.git@gmail.com>
2024-03-26 12:04:43 +05:30
Zhang Song
1b5ca67ef8 crimson/os/seastore: fix use after free error
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
2024-03-26 14:17:02 +08:00
Nizamudeen A
bf78b9df6b
Merge pull request #56055 from rhcs-dashboard/add-nfsv3-support
mgr/dashboard: add support for NFSv3 exports

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2024-03-26 11:28:15 +05:30
Aashish Sharma
7a08bdd264 mgr/dashboards: multi-cluster improvements and bug fixes
Fixes: https://tracker.ceph.com/issues/64880

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2024-03-26 10:27:15 +05:30
Yingxin
9555086eaf
Merge pull request #56332 from cyx1231st/wip-fix-racing-get-or-create-pg-mapping
crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-03-26 11:41:42 +08:00
Kefu Chai
ec64c3e61e test/mon/test_config_map: free CrushWrapper created with new
before this change, we create a new CrushWrapper instance with `new`, but
we never free this instance after done with it. and LeakSanitizer
points this out:

```
Direct leak of 544 byte(s) in 1 object(s) allocated from:
    #0 0x561afe148fed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_config_map+0x1c2fed) (BuildId: 3ce9eeed38cee335628fa74fdd08cd215b15019e)
    #1 0x561afe151cbd in ConfigMap_result_sections_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/test_config_map.cc:93:16
    #2 0x561afe2689b6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x561afe221262 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x561afe1d1f7c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x561afe1d3fb2 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x561afe1d55eb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x561afe1f2a78 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x561afe2711e6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x561afe227bd2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x561afe1f1e02 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x561afe176ec0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x561afe176e51 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #13 0x7f37d9397d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so in this change, we manage the `CrushWrapper` pointer with a smart
pointer. because the size of `CrushWrapper` is relatively large, we
don't create it on stack.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-26 11:06:54 +08:00
Kefu Chai
ffece5ff61 test/mon/test_config_map: free CephContext created with new
before this change, we create a new cct instance with `new`, but
we never free this instance after done with it. and LeakSanitizer
points this out:

```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x561afe148fed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_config_map+0x1c2fed) (BuildId: 3ce9eeed38cee335628fa74fdd08cd215b15019e)
    #1 0x7f37dc9ac189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7f37dc9ab563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7f37dc9ab563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7f37dc9ab2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7f37dc9a5192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    #6 0x7f37dc9981e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7f37dbc6e96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7f37dbc6d93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x561afe14e983 in ConfigMap_add_option_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/test_config_map.cc:58:18
    #10 0x561afe2689b6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x561afe221262 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x561afe1d1f7c in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x561afe1d3fb2 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x561afe1d55eb in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x561afe1f2a78 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x561afe2711e6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x561afe227bd2 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x561afe1f1e02 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x561afe176ec0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x561afe176e51 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #21 0x7f37d9397d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

so in this change, we manage the `CephContext` pointer with a smart
pointer. because the size of CephContext could be large, we don't create
it on stack.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-26 11:06:54 +08:00
Kefu Chai
742d5fa4ab
Merge pull request #56455 from tchaikov/wip-test-test_rgw_iam_policy-fix-leak
test/rgw/test_rgw_iam_policy: do not increase ref when creating intru…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-03-26 10:09:56 +08:00
Adam King
c743075db4
Merge pull request #55068 from phlogistonjohn/jjm-cephadm-smb-svc-spec
cephadm: add a new SMB service to the mgr module

Reviewed-by: Adam King <adking@redhat.com>
2024-03-25 16:24:54 -04:00
Yuri Weinstein
3b8cc0f3eb
Merge pull request #56142 from Matan-B/wip-matanb-fix_hobject_to_str
common/hobject: fix hobject_t::to_str()

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-03-25 11:30:45 -07:00
Yuri Weinstein
d4caf2ce70
Merge pull request #55676 from baergj/cod-for-all-upstream
kv/RocksDBStore: Configure compact-on-deletion for all CFs

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2024-03-25 11:29:46 -07:00
Yuri Weinstein
a3e993114c
Merge pull request #55599 from ifed01/wip-ifed-kill-build-warns
os/bluestore: eliminate build warnings

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2024-03-25 11:28:58 -07:00
Yuri Weinstein
58b2460876
Merge pull request #55562 from Matan-B/wip-matanb-fix-lookup-purged-snaps
osd/SnapMapper: fix _lookup_purged_snap

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2024-03-25 11:27:16 -07:00
Adam King
3ba87243ed mgr/cephadm: update zonegroup w/ zonegroup_hostnames from rgw spec
If users provide this field to the rgw spec, we need to modify
the zonegroup to includes these hostnames. This most just makes
use of another commit that added the `ceph rgw zonegroup modify`
command to the rgw module for this specific purpose.

Signed-off-by: Adam King <adking@redhat.com>
2024-03-25 13:51:00 -04:00
Laura Flores
be90c592d7
Merge pull request #55609 from ljflores/wip-telemetry-opt-in-to-new-collection
qa/workunits: opt in to new collection after upgrade from reef to main
2024-03-25 12:13:43 -05:00
Adam King
5f7c6d58cc mgr/rgw: add support to modify zonegroup parameters
This is being done with `radosgw-admin zonegroup set`
rather than `radosgw-admin zonegroup modify` as I don't
think the hostnames parameter (which is the primary focus
for this specific change) can be set using the modify
command. The nice bit about that is it should in theory
make it easy to extend this to allow setting other parameters
to be modified in the zonegroup in the future.

Signed-off-by: Adam King <adking@redhat.com>
2024-03-25 12:13:32 -04:00
Kefu Chai
2ee77b8bd6 test/rgw/test_rgw_iam_policy: do not increase ref when creating intrusive_ptr<CephContext>
before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance. and LeakSanitizer points this out:

```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0xaaaac359c7c8 in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/bin/unittest_rgw_iam_policy+0x211c7c8) (BuildId: 060fadb10da261b52fd5757c7b1e9812d34542f1)
    #1 0xffff96f764e4 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0xffff96f757cc in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0xffff96f757cc in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:2396:39
    #4 0xffff96f75500 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:2494:18
    #5 0xffff96f6ec4c in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/boost/include/boost/circular_buffer/base.hpp:1039:9
    #6 0xffff96f63528 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/log/Log.cc:53:5
    #7 0xffff96045300 in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/common/ceph_context.cc:729:16
    #8 0xffff960446ec in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/common/ceph_context.cc:697:5
    #9 0xaaaac3629238 in IPPolicyTest::IPPolicyTest() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/rgw/test_rgw_iam_policy.cc:864:15
    #10 0xaaaac3628da0 in IPPolicyTest_MaskedIPOperations_Test::IPPolicyTest_MaskedIPOperations_Test() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/rgw/test_rgw_iam_policy.cc:869:1
    #11 0xaaaac3628d3c in testing::internal::TestFactoryImpl<IPPolicyTest_MaskedIPOperations_Test>::CreateTest() /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/googletest/googletest/include/gtest/internal/gtest-internal.h:472:44
```

so, in this change, we do not increase the refcount when creating cct.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-25 23:00:27 +08:00
Kefu Chai
ac09ac8c56
Merge pull request #56437 from tchaikov/wip-cephcontext-leak-fixes
test: fix the leakages of `CephContext` identified by LeakSanitizer

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-03-25 22:18:26 +08:00
Patrick Donnelly
012b701d1d
vstart: require debugging switch for all mds debug configs
mds_debug_subtrees wrecks vstart cluster performance particularly egregiously.
Do not turn it on by default.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-25 10:15:19 -04:00
Matan Breizman
f19765ec08
Merge pull request #56173 from Matan-B/wip-matanb-clone-overlap-doc
doc/dev: osd_internals/snaps.rst: add clone_overlap doc

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>
2024-03-25 12:15:22 +02:00
Taha Jahangir
3cd39e3582
docs/rados: remove incorrect ceph command
The removed line was (incorrectly changed) output of the previous command.

Signed-off-by: Taha Jahangir <mtjahangir@gmail.com>
2024-03-25 13:32:12 +03:30
Nizamudeen A
d3ec86a124
Merge pull request #56357 from rhcs-dashboard/sanitize-users-form
mgr/dashboard: sanitize dashboard user creation

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-25 15:31:15 +05:30
Kefu Chai
9cd0f817b9
Merge pull request #56438 from tchaikov/wip-test-unittest-mem-fix-leak
test/common: do not leak in MemoryIsZeroSmallTest

Reviewed-by: Rongqi Sun <sunrongqi@huawei.com>
2024-03-25 15:38:17 +08:00
Yingxin Cheng
2422817b75 crimson/os/pg_map: allow multiple shards to create new pg mappings at the same time
Also:
* Better detections in case of inconsistent racings, such as:
  * The new mapping is creating towards different cores.
  * Mapping creation is racing with its eracing.
  * Multiple shards are erasing the same mapping at the same time.
* Add more logs to debug in case of unexpected issues.

Fixes: https://tracker.ceph.com/issues/64934
Fixes: https://tracker.ceph.com/issues/64009

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2024-03-25 14:41:31 +08:00
Pedro Gonzalez Gomez
b658bb9c32 mgr/dashboard: add system users to rgw user form
Fixes: https://tracker.ceph.com/issues/65074
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2024-03-25 11:32:01 +05:30
Venky Shankar
bc2aba4272 Merge PR #56276 into main
* refs/pull/56276/head:
	MDS metrics typo fix

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-03-25 10:27:42 +05:30
Nizamudeen A
5b731beb2a mgr/dashboard: introduce cli commands to set cors urls
The benefit is that this will take care of cherrypy reconfiguration

Fixes: https://tracker.ceph.com/issues/65104
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-03-25 10:25:05 +05:30
Venky Shankar
69e62a9b6c Merge PR #56293 into main
* refs/pull/56293/head:
	qa/suites/fs: link in pg_health ignorelist
	qa/cephfs: add probabilistic ignorelist for pg_health

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-03-25 10:21:58 +05:30
Nizamudeen A
a8b1b0ed7a
Merge pull request #56164 from rhcs-dashboard/wip-64890-main
mgr/dashboard: fix NVMeoF API

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-25 10:19:40 +05:30
Nizamudeen A
ad40d6f1bb
Merge pull request #56295 from rhcs-dashboard/cpu-usage-rm-danger-indicator
mgr/dashboard: rm warning/error threshold for cpu usage

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: ivoalmeida <NOT@FOUND>
2024-03-25 10:16:09 +05:30
Nizamudeen A
70cd1bc8f4
Merge pull request #56297 from rhcs-dashboard/grafana-issue-multi-cluster
mgr/dashboard: update grafana api url when switching cluster

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2024-03-25 10:14:44 +05:30
Kefu Chai
4deebd6294 test/common: do not leak in MemoryIsZeroSmallTest
before this change, we allocate memory chunks with specified
size using `new []`, but we never free them. when testing with
LeakSanitizer enabled, it rightly points identifies the leakage:

```
Direct leak of 8754 byte(s) in 184 object(s) allocated from:
    #0 0x55c0b2470f0d in operator new[](unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_memory+0x196f0d) (BuildId: d3267dd8819427b804c4729e0467dbe7601fb321)
    #1 0x55c0b247456c in MemoryIsZeroSmallTest_MemoryIsZeroTestSmall_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/common/test_memory.cc:33:18
    #2 0x55c0b2598ee6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #3 0x55c0b2553b92 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #4 0x55c0b25049dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #5 0x55c0b2506a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #6 0x55c0b250804b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #7 0x55c0b25254d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #8 0x55c0b25a16f6 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #9 0x55c0b255a502 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #10 0x55c0b2524862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #11 0x55c0b24ab4c0 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #12 0x55c0b24ab451 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
    #13 0x7f45e065ad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

in this change, we free the allocate memory.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-25 11:25:25 +08:00
Kefu Chai
e316c2e481 test/common: avoid leakage of CephContext
before this change, in test_util.cc, we increment the refcount of
when constructing it. but at that moment, nobody really owns it.
also, `CephContext` 's refcount is set to 1 in its constructor.
so, we should not do this. otherwise, the created `CephContext`
is leaked as LeakSanitizer rightly points out:
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x5632320d27ed in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_util+0x1917ed) (BuildId: ff1df1455bd07b651ad580584a17ea204afeb36e)
    #1 0x7ff9d535b189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7ff9d535a563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7ff9d535a563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7ff9d535a2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7ff9d5354192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    #6 0x7ff9d53471e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7ff9d461d96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7ff9d461c93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x5632320d52e0 in util_collect_sys_info_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/common/test_util.cc:34:27
    #10 0x563232205c16 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x5632321c2742 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x5632321736dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
```
in this change, instead of using a raw pointer, let's
use `boost::intrusive_ptr<CephContext>` to manage the lifecyle
of `CephContext`, this also address the leakage reported by
LeakSanitizer.

the same applies to common/test_context.cc

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-25 08:25:34 +08:00
Kefu Chai
dc53b767e3 test: do not increase ref when creating intrusive_ptr<CephContext>
before this change, we increment the refcount when constructing
`cct` instrusive_ptr, but nobody owns this smart pointer. also,
`CephContext` 's constructor set its refcount to 1. so, when the
test finishes, the refcount is 1, and this leads to a leakage of
the `CephContext` instance, this not only annoys ASan, and defeats
the purpose of 14d878c8.
```
Indirect leak of 10880000 byte(s) in 1 object(s) allocated from:
    #0 0x5564d173537d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest_ipaddr+0x19b37d) (BuildId: 45c0c7f28b253c04fcb7bb1a43aed52a5526d734)
    #1 0x7fe7f2ccd189 in __gnu_cxx::new_allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27
    #2 0x7fe7f2ccc563 in std::allocator<ceph::logging::ConcreteEntry>::allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/allocator.h:185:32
    #3 0x7fe7f2ccc563 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::allocate(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2396:39
    #4 0x7fe7f2ccc2c0 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::initialize_buffer(unsigned long) /opt/ceph/include/boost/circular_buffer/base.hpp:2494:18
    #5 0x7fe7f2cc6192 in boost::circular_buffer<ceph::logging::ConcreteEntry, std::allocator<ceph::logging::ConcreteEntry> >::circular_buffer(unsigned long, std::allocator<ceph::logging::ConcreteEntry> const&) /opt/ceph/include/boost/circular_buffer/base.hpp:1039:9
    #6 0x7fe7f2cb91e4 in ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/log/Log.cc:53:5
    #7 0x7fe7f1f8f96d in ceph::common::CephContext::CephContext(unsigned int, ceph::common::CephContext::create_options const&) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:729:16
    #8 0x7fe7f1f8e93b in ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/ceph_context.cc:697:5
    #9 0x5564d1752eb9 in pick_address_find_ip_in_subnet_list_Test::TestBody() /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_ipaddr.cc:706:47
    #10 0x5564d18694d6 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #11 0x5564d1820fc2 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #12 0x5564d17d19dc in testing::Test::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2680:5
    #13 0x5564d17d3a12 in testing::TestInfo::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2858:11
    #14 0x5564d17d504b in testing::TestSuite::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:3012:28
    #15 0x5564d17f24d8 in testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5723:44
    #16 0x5564d1871d06 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2605:10
    #17 0x5564d1827932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:2641:14
    #18 0x5564d17f1862 in testing::UnitTest::Run() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/src/gtest.cc:5306:10
    #19 0x5564d1775d80 in RUN_ALL_TESTS() /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2486:46
    #20 0x5564d1775d11 in main /home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googlemock/src/gmock_main.cc:70:10
```

so, in this change, we do not increase the refcount when
creating cct.

the same applies to `test/common/test_fault_injector.cc`.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-03-25 08:25:34 +08:00