Commit Graph

122672 Commits

Author SHA1 Message Date
Sridhar Seshasayee
84fe087491 osd: Add missing tracked mclock config options and do some clean-up.
Add the missing tracked mclock config options related to the background
best effort clients of mclock scheduler within handle_conf_change().

The method mClockScheduler::set_global_recovery_options() doesn't serve
the purpose fully as in addition to overriding the "osd_max_backfills"
config option, the local and remote Async reserver objects have to be
updated accordingly which is not possible currently from within the
mClockScheduler. Therefore, rather than complicate the mClockScheduler
code to achieve this, a better option is to override the recovery options
during OSD init (and handle config changes) which the previous commit
introduced.  Therefore, this method is removed.

Fixes: https://tracker.ceph.com/issues/50501
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-04-27 11:11:58 +05:30
Sridhar Seshasayee
8876d27f38 osd: Override recovery/backfill/sleep options with mclock scheduler.
Make the osd_*_sleep options modifiable during runtime and add them
to the set of tracked conf keys. This is to ensure that the sleep
options can be disabled/overridden during OSD bring-up if mclock
scheduler is employed.

Introduce OSD::maybe_override_options_for_qos():
  This method does the following if the mclock scheduler is enabled:
  - overrides the "recovery_max_active" to a high limit of 1000,
  - overrides "osd_max_backfills" option to a high limit of 1000 and
    sets the corresponding Async local and remote reserver objects also
    to the same value (1000),
  - disables osd_*_sleep options so that appropriate QoS may be
    provided with the mclock scheduler.

The above method is called in the following scenarios:
- After all the op shards are brought up during OSD initialization.
- In OSD::handle_conf_change() to override any settings related to
  QoS that the user intended to change.

Modify the mclock config reference to accurately reflect what options
can be changed when using mclock's "custom" profile and clean up
some whitespaces.

Fixes: https://tracker.ceph.com/issues/50501
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-04-27 11:11:58 +05:30
Kefu Chai
e25c046f31 cmake: pass "CC" using configure when building liburing
since liburing's configure checks the CC and CXX env variables, let's
pass them when calling "configure". otherwise "make" still uses gcc and
g++ by default as they are specified in the "configure" script. this
also matches with the way how debian/rules calls "make".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-27 13:36:23 +08:00
Kefu Chai
f0e1a16898
Merge pull request #41013 from tchaikov/wip-doc-mds-confval
doc/cephfs/mds-config-ref: render options using confval directive 

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-27 13:20:44 +08:00
Kefu Chai
5b7128bdf2
Merge pull request #41022 from tchaikov/wip-cmake-findpmem
cmake/modules/Findpmem: always set pmem_VERSION_STRING

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-04-27 13:19:32 +08:00
Kefu Chai
98afc9e0a2 mgr/dashboard: upgrade package-lock.json
diminish "npm run fix:audit" divergences

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-27 05:04:07 +00:00
jianglong01
cc5b77ee87 mgr/cephadm: The command of 'ceph orch daemon restart mgr.xxx' may case mgr daemon loop to restart
Scene:
The mgr daemon is active. After execing restart command, it may be save "scheduled_daemon_actions": {"mgr.xxx": "restart"}}" to config-key.
So the mgr daemon will restart before call rm_scheduled_daemon_action which case mgr daemon will load restart forever.

Fix mgr infinite restart issue refering to the same solution as 'ceph orch daemon redeploy'.

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
2021-04-27 11:09:09 +08:00
Kefu Chai
7b5ab38ea4
Merge pull request #41010 from rzarzynski/wip-crimson-backtrace-fix-assert
crimson/common: fix broken assertion on FatalSignal setup.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-27 08:44:04 +08:00
Ken Dreyer
3041937fb5 librbd: fix spelling in immutable-object-cache conn error
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-04-26 15:01:41 -06:00
zdover23
c6d1d0e031
Merge pull request #41030 from zdover23/wip-doc-adding-one-word-to-cephadm-nfs-2021-Apr-26
doc/cephadm: add a single word

Reviewed-by: Sage Weil <sweil@redhat.com>
2021-04-27 03:41:03 +10:00
Sébastien Han
da1f9d215f ceph-volume: fix "device" output
During
ec0f5f3b22
I changed "device" by "dev", let's revert that!

Fixes: https://tracker.ceph.com/issues/50478
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-26 18:01:19 +02:00
Guillaume Abrioux
08223119b9
Merge pull request #38368 from rishabh-d-dave/cv-list-by-osd-id
ceph-volume: allow listing devices by OSD ID
2021-04-26 17:25:48 +02:00
Avan Thakkar
21318e8fa9 mgr/dashboard: fix HAProxy (now called ingress)
Fixes: https://tracker.ceph.com/issues/50319
Signed-off-by: Avan Thakkar <athakkar@redhat.com>

Support from Dashboard UI to create Ingress service type.
2021-04-26 20:21:26 +05:30
Igor Fedotov
d9e4d2d05c osd: admin cmd to dump pools statfs
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2021-04-26 17:36:32 +03:00
zdover23
d526c9c390
Merge pull request #40992 from zdover23/wip-doc-cephadm-osd-advanced-adding-a-word-2021-apr-22
doc/cephadm: adding "device" to a sentence

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-27 00:34:26 +10:00
Zac Dover
3af9fe3fd8 doc/cephadm: add a single word
s/If the nfs-ganesha doesn't/If the
nfs-ganesha pool doesn't/

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-04-27 00:32:21 +10:00
Dan van der Ster
6f04c4d6c3 doc/cephfs/cache-config: use config item macros
Correct the defaults following 8df2388b9f.

Related-to: https://tracker.ceph.com/issues/48403
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2021-04-26 16:27:01 +02:00
Kefu Chai
a1753f56bb
Merge pull request #40724 from rhcs-dashboard/upgrade-caniuse-lite
mgr/dashboard: upgrade Browserslist: caniuse-lite

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-26 22:26:58 +08:00
zdover23
284472939b
Merge pull request #41029 from zdover23/wip-doc-cephadm-nfs-service-corrections-2021-Apr-26
doc/cephadm: rewrite "nfs.rst"

Reviewed-by: Sage Weil <sweil@redhat.com>
2021-04-27 00:12:17 +10:00
Casey Bodley
a46f84d32e
Merge pull request #40594 from mkogan1/wip-rgw-multipart-complete
rgw: return OK on consecutive complete-multipart reqs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-04-26 10:03:44 -04:00
Venky Shankar
b88228e936 test: tests for *at() libcephfs APIs
Fixes: http://tracker.ceph.com/issues/50298
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-26 09:14:27 -04:00
Venky Shankar
3ad4521187 test: cleanup files/dirs before finishing test
Other tests might use the same file names and expect libcephfs
calls to succeed.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-26 09:14:27 -04:00
Venky Shankar
8e8c4041ef client: non-at APIs call at-APIs with CEPHFS_AT_FDCWD as file descriptor
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-26 09:14:27 -04:00
Venky Shankar
3831aa12f3 libcephfs: introduce basic *at() calls
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-26 09:14:27 -04:00
Venky Shankar
52c672350f client: remove redundant caps_issued_mask() call in Client::fstatx()
No need to include this check since _getattr() checks this anyway.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-26 09:14:27 -04:00
Zac Dover
36426d6df5 doc/cephadm: rewrite "nfs.rst"
This PR makes a few (very small) stylistic
corrections, mostly typographical, to the text.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-04-26 23:04:04 +10:00
Kefu Chai
5249e5fe03 doc/dev/osd_internals/mclock_wpq_cmp_study: fix a typo
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 21:01:44 +08:00
Kefu Chai
19ccf3655f doc/cephfs/mds-config-ref: render options using confval directive
less repeating, better maintanability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 21:01:44 +08:00
Kefu Chai
ecb8e5e9c7 doc/rados/operations/pools: define pool settings using "describe"
better readability and more structured this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 21:01:44 +08:00
Kefu Chai
4e27388ba7 doc/rados/configuration: reference options using :confval:
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 21:01:44 +08:00
Mark Kogan
ae00453515 rgw: fix segfault related to explicit object manifest handling
Fixes: https://tracker.ceph.com/issues/50467

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2021-04-26 15:36:08 +03:00
Kefu Chai
ad85af2ef8 cmake/modules/Findpmem: always set pmem_VERSION_STRING
before this change, `pmem_VERSION_STRING` is not set if it is not able
to fulfill the specified version requirement. the intention was to check
if the version is able to satisfy the requirement. but actually, passing
an empty `pmem_VERSION_STRING` to `find_package_handle_standard_args()`
as the option of `VERSION_VAR` does not fail this check. on the
contrary, it prints

-- Found pmem: pmem_pmemobj_INCLUDE_DIR;pmem_pmem_INCLUDE_DIR (Required
is at least version "1.17")

if we requires pmem 1.17, while the found version is, for instance,
1.10.

if the required version is 1.7, and the found version is 1.10, the
output from cmake is:

-- Found pmem: pmem_pmemobj_INCLUDE_DIR;pmem_pmem_INCLUDE_DIR (found
suitable version "1.10", minimum required is "1.7")

in this change, the version spec is not specified when calling
`pkg_check_modules()`. so, `PKG_${component}_VERSION` is always set.
and we can always delegate the version checking to
`find_package_handle_standard_args()`. please note, we use the lower
version returned by pkg-config if multiple components are required and
both pkg-config settings return their versions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 16:38:09 +08:00
Kefu Chai
ecb8d2cae2 ceph.spec.in: do not build with system pmdk by default
we need to use libpmem 1.10 in #40493.

without enabling the module stream offering libpmem 1.9.2, we can only
have access to libpmem 1.6.1. and fedora 33 only has libpmem 1.9
packaged. the same applies to openSUSE Tumbleweed and openSUSE Leap. so
let's stop using libpmem packaged by distro by default, until these
distros include libpmem 1.10.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 16:38:09 +08:00
Ernesto Puerta
813933f81e
Merge pull request #39929 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/elliptic-6.5.4
build(deps): bump elliptic from 6.5.3 to 6.5.4 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-26 10:08:45 +02:00
Sébastien Han
bf115b49b9 mgr/pybind/snap_schedule: return valid json for 'status' command
The status should return a valid list, prior to this path the code was
only appending json blob next to each other resulting in an invalid
json. Now the blobs will be enclosed properly so a valid list is
returned.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-26 08:57:31 +02:00
Sébastien Han
74a7cdda6b mgr/pybind/snap_schedule: do not fail when no fs snapshot schedules are available
When listing for available snapshot schedules, we should not an error in case
there is none. We should just return 0 with an empty dict.

Fixes: https://tracker.ceph.com/issues/49837
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-26 08:57:28 +02:00
Kefu Chai
55c02a81cf
Merge pull request #41015 from inspur-wyq/patch-doc
doc/radosgw/role.rst: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-25 21:25:38 +08:00
Kefu Chai
b5cae0d744
Merge pull request #39605 from inspur-wyq/patch-1
doc/radosgw/cloud-sync-module: fix typos

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-25 20:15:47 +08:00
wangyunqing
066c065508 doc/radosgw/role.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2021-04-25 09:15:12 +00:00
Kefu Chai
d5b5a7fb6c
Merge pull request #41011 from tchaikov/wip-cephadm-test
mgr/cephadm: s/_hosts_with_daemon_inventory/_schedulable_hosts/

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-25 10:06:21 +08:00
Kefu Chai
782fc0e7ca mgr/cephadm: s/_hosts_with_daemon_inventory/_schedulable_hosts/
`_hosts_with_daemon_inventory` was renamed to `_schedulable_hosts`
in f260c48f72 which was merged before
eebb842d04 got merged, hence the
regression.

this change just updates the test with the new method name.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-24 15:40:03 +08:00
Radoslaw Zarzynski
4ba84b85bb crimson/common: fix broken assertion on FatalSignal setup.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-24 07:14:28 +00:00
Kefu Chai
1721d11dbb osd/OSDMapMapping: mark OSDMapMapping::update() private
this method could be misused easily. and it is only used by test, so
mark it private.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-24 15:07:26 +08:00
Kefu Chai
06e479103c mon/OSDMonitor: let OSDMonitor::_get_pending_crush() return
let OSDMonitor::_get_pending_crush() return by return value not by a
input parameter, C++17 ensures that the copy ellision in this case, so
no overhead is introduced because of copying or even moving.

and this change improves the readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-24 15:07:11 +08:00
Kefu Chai
5dccba839b
Merge pull request #34109 from CongMinYin/optimizing-bluestore-lock
os/bluestore: Optimizing the lock of bluestore writing process

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-24 13:58:38 +08:00
Kefu Chai
9598175a58
Merge pull request #40863 from adk3798/maint-fix
mgr/cephadm: don't remove daemons on hosts in maintenance or offline mode

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-24 13:54:07 +08:00
Kefu Chai
9827ee5d5e
Merge pull request #38641 from agayev/zoned-cleaner
os/bluestore: Some more plumbing for zone cleaning (WIP)

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-04-24 13:53:50 +08:00
Kefu Chai
b30d0b8ab5
Merge pull request #40978 from idryomov/wip-reset-authenticate-err
mon/MonClient: reset authenticate_err in _reopen_session()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-24 13:51:17 +08:00
Yuri Weinstein
2771cb0887
Merge pull request #41006 from yuriw/wip-yuriw-simlinks-master
qa/tests: changed simlink to upgrade/parallel only

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-23 09:49:07 -07:00
Neha Ojha
edcf29b934
Merge pull request #40994 from neha-ojha/wip-50388
qa/suites/upgrade/pacific-x/rgw-multisite: fix install version

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-04-23 09:36:06 -07:00