Commit Graph

125269 Commits

Author SHA1 Message Date
Kefu Chai
49bd620f75 cmake: add unittest_intarith back
this unit test was missed when we migrated from autotools to CMake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
5d68abe194 common/bloom_filter: use vector for holding bits
to be more consistent with the upstream at https://github.com/ArashPartow/bloom/blob/master/bloom_filter.hpp
and for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
5079e56b0c common/bloom_filter: use "using" for better readability
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
89770fe691 common/bloom_filter: return by return values not by input params
it'd be easier for the static analyzer (like GCC), to reason about if
a variable is initialized before being used.

this change also helps to improve the readability, and to silence the
false alarm like:

In file included from ../src/os/bluestore/BlueStore.h:42,
                 from ../src/os/bluestore/BlueStore.cc:26:
../src/common/bloom_filter.hpp: In member function 'void std::vector<_Tp, _Alloc>::_M_fill_insert(std::vector<_Tp, _Alloc>::iterator, std::vector<_Tp, _Alloc>::size_type, const value_type&) [with _Tp = bloom_filter; _Alloc = mempool::pool_allocator<mempool::mempool_bluestore_fsck, bloom_filter>]':
../src/common/bloom_filter.hpp:118:46: warning: '*((void*)(& __tmp)+8).bloom_filter::table_size_' may be used uninitialized in this function [-Wmaybe-uninitialized]
  118 |  mempool::bloom_filter::alloc_byte.deallocate(bit_table_, table_size_);
      |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
0edc573fca include/denc: always initialize local variable
this change silences following false positive warning:

In file included from ../src/include/encoding.h:41,
                  from ../src/kv/KeyValueDB.h:12,
                  from ../src/os/bluestore/bluestore_common.h:20,
                  from ../src/os/bluestore/BlueFS.cc:5:
 ../src/include/denc.h: In function ‘std::enable_if_t<(is_same_v<T, bluefs_extent_t> || is_same_v<T, const bluefs_extent_t>)> _denc_friend(T&, P&) [with T = bluefs_extent_t; P = ceph::buffer::v15_2_0::p$
 ../src/include/denc.h:639:11: warning: ‘shift’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   639 |     shift += 7;
       |     ~~~~~~^~~~
 ../src/include/denc.h:613:7: note: ‘shift’ was declared here
   613 |   int shift;
       |       ^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 11:43:30 +08:00
Kefu Chai
d7862cb607
Merge pull request #42475 from tchaikov/wip-crimson-cleanup
crimson/osd: cleanups

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-07-27 10:37:07 +08:00
Kefu Chai
03ac43a4b4
Merge pull request #42483 from Astroakanksha24/master
mgr/volumes/fs/operations/group.py: add extra blank lines

Reviewed-by: Varsha Rao <varao@redhat.com>
2021-07-27 10:32:29 +08:00
Kefu Chai
d637dcda22
Merge pull request #42485 from tchaikov/wip-strtoll
common/strtol: replace `const char*` with `std::string_view`

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-27 10:31:41 +08:00
Sage Weil
f8f7b86571 Merge PR #42292 into master
* refs/pull/42292/head:
	qa/suites/upgrade: log_to_journald=false

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-26 19:23:26 -04:00
Sage Weil
ac63ab6125 Merge PR #42489 into master
* refs/pull/42489/head:
	qa/suites/upgrade/pacific-x/stress-split: do not avoid_pacific_features

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 19:23:03 -04:00
Guillaume Abrioux
4bfa87ee55
Merge pull request #42181 from lyind/51526-fix-ceph-volume-zero-division-error
ceph-volume: lvm batch: fast_allocations(): avoid ZeroDivisionError
2021-07-26 21:58:07 +02:00
Guillaume Abrioux
d69ba6ce58
Merge pull request #42443 from dsavineau/ceph_volume_tests_functional_ssh_args
ceph-volume/tests: update ansible environment variables in tox
2021-07-26 17:08:10 +02:00
Kefu Chai
54c3e1b688 *: drop strict_iecstrtoll(const char *str,..)
replace strict_iecstrtoll(const char *str,..) with
strict_iecstrtoll(std::string_view, ..). which is more convenient.
and both of them share the same implementation:

strict_iec_cast<uint64_t>(str, err);

so they are interchangeable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
5d22af9316 common,mon: drop strict_sistrtoll()
strict_sistrtoll() is but an alias of strict_si_cast<uint64_t>(..),
let's just drop the former. there are way too many thin wrappers in
strtol.{h,cc}. they don't offer lots of benefit to us.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
650a0c9363 common/strtol: drop strict_strto*(const char*,..)
they are but proxies to strict_strto*(string_view, ...). and their
callers can just call the latter without any change. so drop these
proxies.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
55ed86bb98 common, mon: use strict_si_cast<>(std::string_view,..)
simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
b6f29f4c1b common/strtol: expose strict_si_cast(std::string_view, ..)
this variant is better than strict_si_cast(const char*), because:

* we can just pass std::string to it, as std::string_view can be
  constructed from a std::string implicitly
* strict_si_cast(std::string_view, ..) is the underlying
  implementation of strict_si_cast(const char*,..), so less
  indirection helps with the readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
58c4ece8f3 rgw,mon,common/strtol: use strict_iec_cast(std::string_view, ..)
this variant is better than strict_iec_cast(const char*), because:

* we can just pass std::string to it, as std::string_view can be
  constructed from a std::string implicitly
* strict_iec_cast(std::string_view, ..) is the underlying
  implementation of strict_iec_cast(const char*,..), so less
  indirection helps with the readability.
* rgw,mon: use strict_iec_cast(std::string_view, ..) instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-26 20:12:17 +08:00
Kefu Chai
f608782797
Merge pull request #42403 from BlaineEXE/ceph-volume-use-safer-check-for-has-bluestore-label
ceph-volume: use safer check for bluestore label

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-26 18:31:13 +08:00
Kefu Chai
0dfab38c01
Merge pull request #42474 from tchaikov/wip-kv-cleanup
kv/RocksDBStore: always initialize local variable

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-26 18:29:01 +08:00
Igor Fedotov
8c49d56b93
Merge pull request #42456 from ifed01/wip-ifed-alloc-histogram2
os/bluestore: implement allocation perf histogram.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 13:21:28 +03:00
Akanksha Chaudhari
2f785b1004 mgr/volumes/fs/operations/group.py: add extra blank line
Fixes:https://tracker.ceph.com/issues/51393
Signed-off-by: Akanksha Chaudhari <astroakanksha24@gmail.com>
2021-07-26 12:23:17 +05:30
Kefu Chai
da35cbe6bc
Merge pull request #42466 from adk3798/mgr-stop
mgr/cephadm: Don't allow stopping full mgr, mon or osd services

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-07-25 21:42:29 +08:00
Kefu Chai
c4e1b4d7ef crimson/osd: print logging using logger() in main()
more consistent and easier to parse this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-25 11:07:52 +08:00
Kefu Chai
2e3f4689a1 kv/RocksDBStore: always initialize local variable
silences following warning from GCC:

 ../src/kv/RocksDBStore.cc:386:7: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   386 |       if (r < 0) {
       |       ^~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:31:15 +08:00
Kefu Chai
d6c931dc78 crimson/osd: cancel timer when stopping
so that reactor does not complain when being stopped, like:

ERROR 2021-07-24 20:50:23,711 [shard 0] seastar - Timer callback failed: seastar::gate_closed_exception (gate closed)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
9816cb9940 crimson/osd: disable var-tracking-assignments
to disable following warning:

In file included from ../src/common/config.h:27,
                 from ../src/crimson/common/config_proxy.h:8,
                 from ../src/crimson/osd/main.cc:23:
../src/common/config_values.h: In copy constructor ‘ConfigValues::ConfigValues(const ConfigValues&)’:
../src/common/config_values.h:19:7: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
   19 | class ConfigValues {
      |       ^~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
1c45d3ae02 crimson: use seastar::try_with_gate to guard background task
for less indent, and for more consistency with how tools/store_nbd
runs background task.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
d686621fe7
Merge pull request #42467 from dsavineau/issue_51796
mgr/cephadm/grafana: check if dashboard is enabled

Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:51:25 +08:00
Kefu Chai
7520a391fd
Merge pull request #42468 from dsavineau/refact_mgr_cephadm_iscsi_dashboard_check
mgr/cephadm/iscsi: simplify the dashboard check

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:41:19 +08:00
Kefu Chai
6144acfb32
Merge pull request #41168 from aclamk/wip-bluefs-more-sync-metadata
os/bluestore/bluefs: Force metadata sync after file rename and file deletion

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-07-24 19:39:51 +08:00
Kefu Chai
65c65f7cf4
Merge pull request #42460 from MrFreezeex/fix-snaptrim-log
osd: log snaptrim message to dout

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:13:54 +08:00
Kefu Chai
aecb0bf250
Merge pull request #42448 from tchaikov/wip-rgw-lua-fs
rgw/rgw_lua: s/boost::filesystem/std::filesystem/

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-07-24 19:12:26 +08:00
Kefu Chai
fa8f07561c
Merge pull request #42286 from dvanders/dvanders_selinux
ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Wido den Hollander <wido@widodh.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:57:19 +08:00
Kefu Chai
d25cd60776
Merge pull request #42451 from ideepika/wip-jaeger-buildthrift
cmake: fix failure due missing thrift build scripts if building with jaeger

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:54:40 +08:00
Kefu Chai
ab7e753ab5
Merge pull request #42452 from smithfarm/wip-first-arg-immutable
rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:53:33 +08:00
Kefu Chai
b278f07861
Merge pull request #42464 from tchaikov/wip-header-only-fmt
cmake, ceph.spec.in: build with header only fmt on RHEL

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-24 13:45:54 +08:00
Neha Ojha
c9ad86e9c5
Merge pull request #42438 from tchaikov/wip-qa-test_module_selftest
qa/tasks/mgr: clean crash reports before waiting for clean

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-23 15:39:14 -07:00
Adam King
6402c587d0 mgr/cephadm: Don't allow stopping full mgr, mon or osd services
I can't think of any case where we would want to allow this

Fixes: https://tracker.ceph.com/issues/51298

Signed-off-by: Adam King <adking@redhat.com>
2021-07-23 17:04:35 -04:00
Kefu Chai
5419131499 ceph.spec.in: build with header only fmt on RHEL
because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue, we should compile librados either with static library
or with header-only library of fmt. but because the fedora packaging
guideline does no encourage us to package static libraries, and it would
be complicated to package both static and dynamic library for fmt.

the simpler solution would be to compile Ceph with the header-only
version of fmt.

in this change, we compile ceph with the header-only version of fmt
on RHEL to address the runtime dependency issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Kefu Chai
d81478b569 cmake: add an option "WITH_FMT_HEADER_ONLY"
in this change:

* an interface library named "fmt-header-only" is introduced. it brings
  the support to the header only fmt library.
* fmt::fmt is renamed to fmt
* an option named "WITH_FMT_HEADER_ONLY" is introduced
* fmt::fmt is an alias of "fmt-header-only" if "WITH_FMT_HEADER_ONLY"
  is "ON", and an alias of "fmt" otherwise.

because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue an option "WITH_FMT_HEADER_ONLY" is introduced, so
that we can enable it when building Ceph with the header version of fmt.
and the built packages won't have runtime dependency of fmt.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Patrick Donnelly
a54e2a66c1
Merge PR #42256 into master
* refs/pull/42256/head:
	mds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:18:13 -07:00
Patrick Donnelly
bab1d1fa0c
Merge PR #42173 into master
* refs/pull/42173/head:
	client: flush the mdlog in unsafe requests' relevant and auth MDSes only

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:17:03 -07:00
Patrick Donnelly
0e71ea4a13
Merge PR #42106 into master
* refs/pull/42106/head:
	mds: create file system with specific ID

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:15:33 -07:00
Kefu Chai
0e0d320525 osdc/Objecter: move LingerOp's ctor to .cc
so the linkage of fmt::fmt does not spill out to other compilation
units.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 01:01:57 +08:00
Josh Durgin
ce483a14ed
Merge pull request #42131 from tchaikov/wip-doc-dev-ci
doc/dev/continuous-integration: use ceph repos

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-23 09:35:58 -07:00
Dimitri Savineau
a6808efca4 mgr/cephadm/iscsi: simplify the dashboard check
We don't need to run an extra command (mgr module ls) to obtain the mgr
modules list since we already have this information in the mgr_map.
This workflow is already done for the monitoring stack or for configuring
the iscsi integration within the dashboard (during creation) via the
config_dashboard method.

The mgr_map is mocked in the tests with the dashboard module enabled so we
don't need _mon_command_mock_mgr_module_ls anymore.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:21:02 -04:00
Dimitri Savineau
16bb5b8076 mgr/cephadm/grafana: check if dashboard is enabled
When deploying the grafana service but the mgr dashboard isn't enabled then
dashboard set-grafana-api-ssl-verify command fails.

Closes: https://tracker.ceph.com/issues/51796

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:03:42 -04:00
Kefu Chai
9054b1a7f7
Merge pull request #42157 from markhpc/wip-osd_client_message_cap
common/options: Set osd_client_message_cap to 256.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-23 20:58:53 +08:00
Arthur Outhenin-Chalandre
e2b2faef11
osd: log snaptrim message to dout
This log message is not an error and is done on every tick of the
snaptrim process. Replace the derr logging to dout(10) to not log it
by default.

Fixes: https://tracker.ceph.com/issues/51799
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2021-07-23 09:32:21 +02:00