Commit Graph

125257 Commits

Author SHA1 Message Date
Xiubo Li
3456ff2628 qa: ignore PG_RECOVERY_FULL and PG_DEGRADED for mds-full
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Xiubo Li
999c787ac6 qa: wait more time since there have many more pgs than before
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Xiubo Li
ba3833a622 qa: do not multiple the full ratio twice
The cluster has already multiple the full ratio before returning
the "max_avail".

Fixes: https://tracker.ceph.com/issues/50984
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Xiubo Li
a96ee41908 qa: do not raise for kclient for _fsync test
For kclient, the write() will return -ENOSPC instead of the fsync().

Fixes: https://tracker.ceph.com/issues/45434
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Xiubo Li
c1cea71299 qa: use the pg autoscale mode to calcuate the pg_num
Setting the pg_num to 8 is too small that some osds maybe not covered by the
pools, some osds maybe overloaded. Remove the hardcodeing pg_num here and let
the pg autoscale mode to calculate it as needed, and at the same time set the
pg_num_min to 64 to avoid the pg_num to small.

If ec pool is used, for the test cases most datas will go to the ec pool and
the primary replicated pool will store a small amount of metadata for all the
files only, so set the target size ratio to 0.05 should be enough.

Fixes: https://tracker.ceph.com/issues/45434
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Xiubo Li
c7837484d9 qa: set the object_size to 1M
Set the object_size to 1MB to make the objects destributed more even
among the OSDs.

Fixes: https://tracker.ceph.com/issues/45434
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Xiubo Li
f4288f2a9b qa: move the is_full() to parent class
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Sage Weil
f8f7b86571 Merge PR #42292 into master
* refs/pull/42292/head:
	qa/suites/upgrade: log_to_journald=false

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-26 19:23:26 -04:00
Sage Weil
ac63ab6125 Merge PR #42489 into master
* refs/pull/42489/head:
	qa/suites/upgrade/pacific-x/stress-split: do not avoid_pacific_features

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 19:23:03 -04:00
Guillaume Abrioux
4bfa87ee55
Merge pull request #42181 from lyind/51526-fix-ceph-volume-zero-division-error
ceph-volume: lvm batch: fast_allocations(): avoid ZeroDivisionError
2021-07-26 21:58:07 +02:00
Guillaume Abrioux
d69ba6ce58
Merge pull request #42443 from dsavineau/ceph_volume_tests_functional_ssh_args
ceph-volume/tests: update ansible environment variables in tox
2021-07-26 17:08:10 +02:00
Kefu Chai
f608782797
Merge pull request #42403 from BlaineEXE/ceph-volume-use-safer-check-for-has-bluestore-label
ceph-volume: use safer check for bluestore label

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-26 18:31:13 +08:00
Kefu Chai
0dfab38c01
Merge pull request #42474 from tchaikov/wip-kv-cleanup
kv/RocksDBStore: always initialize local variable

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-26 18:29:01 +08:00
Igor Fedotov
8c49d56b93
Merge pull request #42456 from ifed01/wip-ifed-alloc-histogram2
os/bluestore: implement allocation perf histogram.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 13:21:28 +03:00
Kefu Chai
da35cbe6bc
Merge pull request #42466 from adk3798/mgr-stop
mgr/cephadm: Don't allow stopping full mgr, mon or osd services

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-07-25 21:42:29 +08:00
Kefu Chai
2e3f4689a1 kv/RocksDBStore: always initialize local variable
silences following warning from GCC:

 ../src/kv/RocksDBStore.cc:386:7: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   386 |       if (r < 0) {
       |       ^~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 21:31:15 +08:00
Kefu Chai
d686621fe7
Merge pull request #42467 from dsavineau/issue_51796
mgr/cephadm/grafana: check if dashboard is enabled

Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:51:25 +08:00
Kefu Chai
7520a391fd
Merge pull request #42468 from dsavineau/refact_mgr_cephadm_iscsi_dashboard_check
mgr/cephadm/iscsi: simplify the dashboard check

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-07-24 19:41:19 +08:00
Kefu Chai
6144acfb32
Merge pull request #41168 from aclamk/wip-bluefs-more-sync-metadata
os/bluestore/bluefs: Force metadata sync after file rename and file deletion

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-07-24 19:39:51 +08:00
Kefu Chai
65c65f7cf4
Merge pull request #42460 from MrFreezeex/fix-snaptrim-log
osd: log snaptrim message to dout

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 19:13:54 +08:00
Kefu Chai
aecb0bf250
Merge pull request #42448 from tchaikov/wip-rgw-lua-fs
rgw/rgw_lua: s/boost::filesystem/std::filesystem/

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-07-24 19:12:26 +08:00
Kefu Chai
fa8f07561c
Merge pull request #42286 from dvanders/dvanders_selinux
ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Wido den Hollander <wido@widodh.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:57:19 +08:00
Kefu Chai
d25cd60776
Merge pull request #42451 from ideepika/wip-jaeger-buildthrift
cmake: fix failure due missing thrift build scripts if building with jaeger

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:54:40 +08:00
Kefu Chai
ab7e753ab5
Merge pull request #42452 from smithfarm/wip-first-arg-immutable
rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-24 13:53:33 +08:00
Kefu Chai
b278f07861
Merge pull request #42464 from tchaikov/wip-header-only-fmt
cmake, ceph.spec.in: build with header only fmt on RHEL

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-24 13:45:54 +08:00
Neha Ojha
c9ad86e9c5
Merge pull request #42438 from tchaikov/wip-qa-test_module_selftest
qa/tasks/mgr: clean crash reports before waiting for clean

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-23 15:39:14 -07:00
Adam King
6402c587d0 mgr/cephadm: Don't allow stopping full mgr, mon or osd services
I can't think of any case where we would want to allow this

Fixes: https://tracker.ceph.com/issues/51298

Signed-off-by: Adam King <adking@redhat.com>
2021-07-23 17:04:35 -04:00
Kefu Chai
5419131499 ceph.spec.in: build with header only fmt on RHEL
because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue, we should compile librados either with static library
or with header-only library of fmt. but because the fedora packaging
guideline does no encourage us to package static libraries, and it would
be complicated to package both static and dynamic library for fmt.

the simpler solution would be to compile Ceph with the header-only
version of fmt.

in this change, we compile ceph with the header-only version of fmt
on RHEL to address the runtime dependency issue.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Kefu Chai
d81478b569 cmake: add an option "WITH_FMT_HEADER_ONLY"
in this change:

* an interface library named "fmt-header-only" is introduced. it brings
  the support to the header only fmt library.
* fmt::fmt is renamed to fmt
* an option named "WITH_FMT_HEADER_ONLY" is introduced
* fmt::fmt is an alias of "fmt-header-only" if "WITH_FMT_HEADER_ONLY"
  is "ON", and an alias of "fmt" otherwise.

because fmt is packaged in EPEL, while librados is packaged
in RHEL, so we cannot have fmt as a runtime dependency of librados.
to address this issue an option "WITH_FMT_HEADER_ONLY" is introduced, so
that we can enable it when building Ceph with the header version of fmt.
and the built packages won't have runtime dependency of fmt.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 02:26:45 +08:00
Patrick Donnelly
a54e2a66c1
Merge PR #42256 into master
* refs/pull/42256/head:
	mds: update META_POP_READDIR/FETCH/STORE and cache_hit_rate for dump loads

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:18:13 -07:00
Patrick Donnelly
bab1d1fa0c
Merge PR #42173 into master
* refs/pull/42173/head:
	client: flush the mdlog in unsafe requests' relevant and auth MDSes only

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:17:03 -07:00
Patrick Donnelly
0e71ea4a13
Merge PR #42106 into master
* refs/pull/42106/head:
	mds: create file system with specific ID

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-23 11:15:33 -07:00
Kefu Chai
0e0d320525 osdc/Objecter: move LingerOp's ctor to .cc
so the linkage of fmt::fmt does not spill out to other compilation
units.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-24 01:01:57 +08:00
Josh Durgin
ce483a14ed
Merge pull request #42131 from tchaikov/wip-doc-dev-ci
doc/dev/continuous-integration: use ceph repos

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-23 09:35:58 -07:00
Dimitri Savineau
a6808efca4 mgr/cephadm/iscsi: simplify the dashboard check
We don't need to run an extra command (mgr module ls) to obtain the mgr
modules list since we already have this information in the mgr_map.
This workflow is already done for the monitoring stack or for configuring
the iscsi integration within the dashboard (during creation) via the
config_dashboard method.

The mgr_map is mocked in the tests with the dashboard module enabled so we
don't need _mon_command_mock_mgr_module_ls anymore.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:21:02 -04:00
Dimitri Savineau
16bb5b8076 mgr/cephadm/grafana: check if dashboard is enabled
When deploying the grafana service but the mgr dashboard isn't enabled then
dashboard set-grafana-api-ssl-verify command fails.

Closes: https://tracker.ceph.com/issues/51796

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-23 11:03:42 -04:00
Kefu Chai
9054b1a7f7
Merge pull request #42157 from markhpc/wip-osd_client_message_cap
common/options: Set osd_client_message_cap to 256.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-23 20:58:53 +08:00
Arthur Outhenin-Chalandre
e2b2faef11
osd: log snaptrim message to dout
This log message is not an error and is done on every tick of the
snaptrim process. Replace the derr logging to dout(10) to not log it
by default.

Fixes: https://tracker.ceph.com/issues/51799
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2021-07-23 09:32:21 +02:00
Kefu Chai
ade690b549
Merge pull request #42449 from kevinzs2048/wip-test-sanitize
qa/run-tox-mgr-dashboard: Do not write to /tmp/test_sanitize_password…

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-23 14:18:11 +08:00
Kefu Chai
042aea0ad2
Merge pull request #41794 from ceph/dependabot/pip/src/ceph-volume/plugin/zfs/pip-19.2
ceph-volume: bump pip in /src/ceph-volume/plugin/zfs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-23 10:55:29 +08:00
Kefu Chai
8706aa15d0 ceph-volume: bump pip in /src/ceph-volume/plugin/zfs
Bumps [pip](https://github.com/pypa/pip) from 9.0.1 to 19.2.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst)
- [Commits](pypa/pip@9.0.1...19.2)

---
updated-dependencies:
- dependency-name: pip
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-23 09:57:35 +08:00
Kefu Chai
af79875460
Merge pull request #42255 from tchaikov/wip-cmake-mgr-cleanup
cmake: install mgr module separately

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-07-23 09:51:52 +08:00
Kefu Chai
b82b5f963d
Merge pull request #42437 from rzarzynski/wip-crimson-generic-alienstore
crimson/os: generalize AlienStore beyond BlueStore.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2021-07-23 09:47:53 +08:00
Nathan Cutler
1cb84a1160 rpm: drop use of $FIRST_ARG in ceph-immutable-object-cache
The use of $FIRST_ARG was probably required because the SUSE-specific
%service_* rpm macros were playing tricks on the shell positional parameters.
This is bad practice and error-prone, so let's assume that no macros should do
that anymore and hence it's safe to assume that positional parameters remain
unchanged after any rpm macro call.

Thanks to Franck Bui for providing the original patch
926433f5d4 that this patch is modeled after.

NOTE: the use of FIRST_ARG had already been eliminated by
926433f5d4 but was re-introduced later by
9466d70985

Fixes: 9466d70985
Fixes: https://tracker.ceph.com/issues/51797
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-07-22 22:52:39 +02:00
Casey Bodley
dc6e6aee83
Merge pull request #42455 from dang/wip-dang-zipper-get_obj_state
RGW - Zipper - Make default args match in get_obj_state

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-22 13:23:56 -04:00
Igor Fedotov
0a5ad09370 os/bluestore: implement allocation perf histogram.
This allows to monitor how fragmented resulting allocations are
depending on the requested block size.

Could be coupled with https://github.com/ceph/ceph/pull/41600/ for easy
live monitoring for either an individual OSD or full set of them.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2021-07-22 19:45:14 +03:00
Deepika Upadhyay
c33df0136c cmake: install libopentracing.so.1(dependency of libjaeger)
fixes:
dpkg-shlibdeps: error: cannot find library libopentracing.so.1 needed by debian/libjaeger/usr/lib/libjaegertracing.so.0.6.1 (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '/build/ceph-17.0.0-6279-g97a36368/obj-x86_64-linux-gnu/external')

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:38:06 +05:30
Deepika Upadhyay
c140147af9 cmake: always require thrift v0.13.0 if WITH_JAEGER
* without this the build shall continue and eventually fail
as thrift is a required dependency for jaeger

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:37:33 +05:30
Deepika Upadhyay
4df43cff61 cmake: reorder find_package_handle_standard_arg for thrift
* Findthrift: thrift_FOUND fails to set if called before
  find_package_handle_standard_arg, hence reorder adding
  thrift::libthrift target after it

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-07-22 21:35:29 +05:30
Adam C. Emerson
9c00a343e0
Merge pull request #42247 from pritha-srivastava/wip-rgw-sts-session-policy-fixes
rgw/sts: createbucket op should take session_policies into account

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-07-22 11:59:29 -04:00