Commit Graph

125414 Commits

Author SHA1 Message Date
Kefu Chai
d6c931dc78 crimson/osd: cancel timer when stopping
so that reactor does not complain when being stopped, like:

ERROR 2021-07-24 20:50:23,711 [shard 0] seastar - Timer callback failed: seastar::gate_closed_exception (gate closed)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
9816cb9940 crimson/osd: disable var-tracking-assignments
to disable following warning:

In file included from ../src/common/config.h:27,
                 from ../src/crimson/common/config_proxy.h:8,
                 from ../src/crimson/osd/main.cc:23:
../src/common/config_values.h: In copy constructor ‘ConfigValues::ConfigValues(const ConfigValues&)’:
../src/common/config_values.h:19:7: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
   19 | class ConfigValues {
      |       ^~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
1c45d3ae02 crimson: use seastar::try_with_gate to guard background task
for less indent, and for more consistency with how tools/store_nbd
runs background task.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:30:54 +08:00
Kefu Chai
d686621fe7
Merge pull request #42467 from dsavineau/issue_51796
mgr/cephadm/grafana: check if dashboard is enabled

Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:51:25 +08:00
Kefu Chai
7520a391fd
Merge pull request #42468 from dsavineau/refact_mgr_cephadm_iscsi_dashboard_check
mgr/cephadm/iscsi: simplify the dashboard check

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:41:19 +08:00
Kefu Chai
6144acfb32
Merge pull request #41168 from aclamk/wip-bluefs-more-sync-metadata
os/bluestore/bluefs: Force metadata sync after file rename and file deletion

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-07-24 19:39:51 +08:00
Kefu Chai
1af0c6195c cmake: define OpenLDAP::OpenLDAP library
* define OpenLDAP::OpenLDAP, so this library can be consumed in a simpler way.
* use OpenLDAP::OpenLDAP instead of OpenLDAP_LIBRARIES when appropriate
* do not link against unused ${OpenLDAP_LIBRARIES}

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:56 +08:00
Kefu Chai
1c3c42fe52 cmake: s/OPENLDAP/OpenLDAP/
the upstream project of OpenLDAP is named "OpenLDAP", so rename OPENLDAP to
OpenLDAP for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:56 +08:00
Kefu Chai
99ba98cef0 cmake: use generator exp. for passing fmt::fmt's COMPILE_DEFINITIONS
more concise this way, without yet another redirection.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:56 +08:00
Kefu Chai
43d10b9e44 cmake: make rgw_common a static library
it was changed from a shared library to object library back in
3c0bba40b2, because we wanted to link both
rgw_a and rgw_admin_user against it.

but now that rgw_admin_user was dropped in
c89b59428e, let's compile rgw_common
as a static library. it'd be much easier for us to apply various compile
options and linkages to it this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:54 +08:00
Kefu Chai
2d955ebb5f cmake: build rgw_kmip_client_impl as part of radosgw
instead compiling rgw_kmip as an object library, just include
rgw_kmip_client_impl.cc in radosgw. simpler this way, as
rgw_kmip_client_impl.cc includes rgw/rgw_common.h. while rgw_common.cc
is indirectly included by ${rgw_libs} as a part of rgw_a, so
it would be simpler if we just compile rgw_kmip_client_impl.cc
as a part of radosgw, and let radosgw link against ${rgw_libs},
which are able to take care of the include directories of 3rd party
libraries, like spawn/spawn.hpp, which is pulled in by rgw/rgw_common.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:06 +08:00
Kefu Chai
832c10dfe8 rgw: drop unused rgw_admin_user.{h,cc}
their only consumer was dropped in c89b59428e

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:28:06 +08:00
Kefu Chai
65c65f7cf4
Merge pull request #42460 from MrFreezeex/fix-snaptrim-log
osd: log snaptrim message to dout

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:13:54 +08:00
Kefu Chai
aecb0bf250
Merge pull request #42448 from tchaikov/wip-rgw-lua-fs
rgw/rgw_lua: s/boost::filesystem/std::filesystem/

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-07-24 19:12:26 +08:00
Kefu Chai
fa8f07561c
Merge pull request #42286 from dvanders/dvanders_selinux
ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Wido den Hollander <wido@widodh.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:57:19 +08:00
Kefu Chai
d25cd60776
Merge pull request #42451 from ideepika/wip-jaeger-buildthrift
cmake: fix failure due missing thrift build scripts if building with jaeger

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:54:40 +08:00
Kefu Chai
ab7e753ab5
Merge pull request #42452 from smithfarm/wip-first-arg-immutable
rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:53:33 +08:00
Kefu Chai
b278f07861
Merge pull request #42464 from tchaikov/wip-header-only-fmt
cmake, ceph.spec.in: build with header only fmt on RHEL

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-24 13:45:54 +08:00
Neha Ojha
c9ad86e9c5
Merge pull request #42438 from tchaikov/wip-qa-test_module_selftest
qa/tasks/mgr: clean crash reports before waiting for clean

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-23 15:39:14 -07:00
Adam King
6402c587d0 mgr/cephadm: Don't allow stopping full mgr, mon or osd services
I can't think of any case where we would want to allow this

Fixes: https://tracker.ceph.com/issues/51298

Signed-off-by: Adam King <adking@redhat.com>
2021-07-23 17:04:35 -04:00
Kefu Chai
5419131499 ceph.spec.in: build with header only fmt on RHEL
because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue, we should compile librados either with static library
or with header-only library of fmt. but because the fedora packaging
guideline does no encourage us to package static libraries, and it would
be complicated to package both static and dynamic library for fmt.

the simpler solution would be to compile Ceph with the header-only
version of fmt.

in this change, we compile ceph with the header-only version of fmt
on RHEL to address the runtime dependency issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Kefu Chai
d81478b569 cmake: add an option "WITH_FMT_HEADER_ONLY"
in this change:

* an interface library named "fmt-header-only" is introduced. it brings
  the support to the header only fmt library.
* fmt::fmt is renamed to fmt
* an option named "WITH_FMT_HEADER_ONLY" is introduced
* fmt::fmt is an alias of "fmt-header-only" if "WITH_FMT_HEADER_ONLY"
  is "ON", and an alias of "fmt" otherwise.

because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue an option "WITH_FMT_HEADER_ONLY" is introduced, so
that we can enable it when building Ceph with the header version of fmt.
and the built packages won't have runtime dependency of fmt.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Patrick Donnelly
a54e2a66c1
Merge PR #42256 into master
* refs/pull/42256/head:
	mds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:18:13 -07:00
Patrick Donnelly
bab1d1fa0c
Merge PR #42173 into master
* refs/pull/42173/head:
	client: flush the mdlog in unsafe requests' relevant and auth MDSes only

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:17:03 -07:00
Patrick Donnelly
0e71ea4a13
Merge PR #42106 into master
* refs/pull/42106/head:
	mds: create file system with specific ID

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:15:33 -07:00
Kefu Chai
0e0d320525 osdc/Objecter: move LingerOp's ctor to .cc
so the linkage of fmt::fmt does not spill out to other compilation
units.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 01:01:57 +08:00
Josh Durgin
ce483a14ed
Merge pull request #42131 from tchaikov/wip-doc-dev-ci
doc/dev/continuous-integration: use ceph repos

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-23 09:35:58 -07:00
Dimitri Savineau
a6808efca4 mgr/cephadm/iscsi: simplify the dashboard check
We don't need to run an extra command (mgr module ls) to obtain the mgr
modules list since we already have this information in the mgr_map.
This workflow is already done for the monitoring stack or for configuring
the iscsi integration within the dashboard (during creation) via the
config_dashboard method.

The mgr_map is mocked in the tests with the dashboard module enabled so we
don't need _mon_command_mock_mgr_module_ls anymore.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:21:02 -04:00
Dimitri Savineau
16bb5b8076 mgr/cephadm/grafana: check if dashboard is enabled
When deploying the grafana service but the mgr dashboard isn't enabled then
dashboard set-grafana-api-ssl-verify command fails.

Closes: https://tracker.ceph.com/issues/51796

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:03:42 -04:00
Kefu Chai
9054b1a7f7
Merge pull request #42157 from markhpc/wip-osd_client_message_cap
common/options: Set osd_client_message_cap to 256.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-23 20:58:53 +08:00
Arthur Outhenin-Chalandre
e2b2faef11
osd: log snaptrim message to dout
This log message is not an error and is done on every tick of the
snaptrim process. Replace the derr logging to dout(10) to not log it
by default.

Fixes: https://tracker.ceph.com/issues/51799
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2021-07-23 09:32:21 +02:00
Kefu Chai
ade690b549
Merge pull request #42449 from kevinzs2048/wip-test-sanitize
qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password…

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-23 14:18:11 +08:00
Kefu Chai
042aea0ad2
Merge pull request #41794 from ceph/dependabot/pip/src/ceph-volume/plugin/zfs/pip-19.2
ceph-volume: bump pip in /src/ceph-volume/plugin/zfs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-23 10:55:29 +08:00
Kefu Chai
8706aa15d0 ceph-volume: bump pip in /src/ceph-volume/plugin/zfs
Bumps [pip](https://github.com/pypa/pip) from 9.0.1 to 19.2.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst)
- [Commits](pypa/pip@9.0.1...19.2)

---
updated-dependencies:
- dependency-name: pip
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-23 09:57:35 +08:00
Kefu Chai
af79875460
Merge pull request #42255 from tchaikov/wip-cmake-mgr-cleanup
cmake: install mgr module separately

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-23 09:51:52 +08:00
Kefu Chai
b82b5f963d
Merge pull request #42437 from rzarzynski/wip-crimson-generic-alienstore
crimson/os: generalize AlienStore beyond BlueStore.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2021-07-23 09:47:53 +08:00
Nathan Cutler
1cb84a1160 rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.

Thanks to Franck Bui for providing the original patch
926433f5d4 that this patch is modeled after.

NOTE: the use of FIRST_ARG had already been eliminated by
926433f5d4 but was re-introduced later by
9466d70985

Fixes: 9466d70985
Fixes: https://tracker.ceph.com/issues/51797
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-07-22 22:52:39 +02:00
Casey Bodley
dc6e6aee83
Merge pull request #42455 from dang/wip-dang-zipper-get_obj_state
RGW - Zipper - Make default args match in get_obj_state

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-22 13:23:56 -04:00
Igor Fedotov
0a5ad09370 os/bluestore: implement allocation perf histogram.
This allows to monitor how fragmented resulting allocations are
depending on the requested block size.

Could be coupled with https://github.com/ceph/ceph/pull/41600/ for easy
live monitoring for either an individual OSD or full set of them.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2021-07-22 19:45:14 +03:00
Deepika Upadhyay
c33df0136c cmake: install libopentracing.so.1(dependency of libjaeger)
fixes:
dpkg-shlibdeps: error: cannot find library libopentracing.so.1 needed by debian/libjaeger/usr/lib/libjaegertracing.so.0.6.1 (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '/build/ceph-17.0.0-6279-g97a36368/obj-x86_64-linux-gnu/external')

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:38:06 +05:30
Deepika Upadhyay
c140147af9 cmake: always require thrift v0.13.0 if WITH_JAEGER
* without this the build shall continue and eventually fail
as thrift is a required dependency for jaeger

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:37:33 +05:30
Sebastian Wagner
8c5edecc13
.githubmap: Update myself
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-22 18:07:10 +02:00
Deepika Upadhyay
4df43cff61 cmake: reorder find_package_handle_standard_arg for thrift
* Findthrift: thrift_FOUND fails to set if called before
  find_package_handle_standard_arg, hence reorder adding
  thrift::libthrift target after it

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:35:29 +05:30
Adam C. Emerson
9c00a343e0
Merge pull request #42247 from pritha-srivastava/wip-rgw-sts-session-policy-fixes
rgw/sts: createbucket op should take session_policies into account

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-07-22 11:59:29 -04:00
Deepika Upadhyay
fcb1eb91c4 cmake: remove buildthrift methods
we use Findthrift.cmake method for adding thrift dependencies, the
cleanup 80e82686eb missed removing these
methods intended to buildthrift from source.

this address the missing failure due to ,
```
CMake Error at cmake/modules/BuildJaeger.cmake:61 (include):
  include could not find load file:

    Buildthrift
```
this fixes regression introduced by: 80e82686eb

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:25:56 +05:30
Casey Bodley
fb60094c97
Merge pull request #42249 from kalebskeithley/rgw-51599
rgw: cleanup virtual dtor decls and defns

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-07-22 11:52:02 -04:00
Daniel Gryniewicz
664fe75c2a RGW - Zipper - Make default args match in get_obj_state
The default arg for follow_olh was different between the base class and
the Rados class.  Fix it so they're the same.

Fixes: https://tracker.ceph.com/issues/51677
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2021-07-22 11:48:08 -04:00
Neha Ojha
54b41f71d9
Merge pull request #42365 from PracticalMetal/typo
doc: Fix typos and grammar in ecbackend.rst

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-22 08:46:33 -07:00
Patrick Donnelly
4ee63174e6
Merge PR #42431 into master
* refs/pull/42431/head:
	cmake: add "mypy" back to tox envlist of "qa""
	qa/tasks/vstart_runner: add optional "sudo" param to _run_python()

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-22 07:56:40 -07:00
Patrick Donnelly
e30663a18c
Merge PR #42444 into master
* refs/pull/42444/head:
	doc/cephfs/journaler: document options using confval directive

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-22 07:54:52 -07:00