Commit Graph

130291 Commits

Author SHA1 Message Date
Yin Congmin
769f3a06ec librbd/cache/pwl: add basic metrics to ImageCacheState
Add basic metrics to ImageCacheState and persist them, including
allocated_bytes, cached_bytes, dirty_bytes, free_bytes and hit/miss
info.

Leverage periodic_stats() timer to call update_image_cache_state.
In order to avoid outputting too much debug information, the original
statistics output log level is changed to 5.

Switch to json_spirit for encoding because encode_json encodes bool as
"true"/"false" string.

Remove rbd_persistent_cache_log_periodic_stats option because we need
to always update cache state.

[ idryomov: add cached_bytes and hits_partial; report misses and
  miss_bytes instead of respective totals; naming ]

Fixes: https://tracker.ceph.com/issues/50614
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-11 08:26:47 +02:00
Ilya Dryomov
d191b08bb6
Merge pull request #45660 from CongMinYin/fix-pwl-status
librbd/cache/pwl: correct cache state

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-31 12:30:03 +02:00
Adam King
6e4dd0e082
Merge pull request #45361 from adk3798/mds-upgrade-test
qa/suites/fs: stop looping in mds upgrade test if upgrade failed

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-03-30 18:56:45 -04:00
Adam King
7e41eb9f93
Merge pull request #45248 from adk3798/offline-thread
mgr/cephadm: offline host watcher

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-03-30 13:50:07 -04:00
Adam King
5a97bc15b7
Merge pull request #45344 from adk3798/raw-prepare-cmd
mgr/cephadm: generate one c-v raw prepare cmd per data device in raw mode

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-03-30 13:48:43 -04:00
Casey Bodley
7989a028c5
Merge pull request #45567 from cbodley/wip-55000
rgw multisite: data sync uses yield_spawn_window()

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-03-30 12:24:56 -04:00
Casey Bodley
9ce4431f26
Merge pull request #45515 from tchaikov/wip-cxx20-fixes-rgw
rgw: C++20 related fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-30 12:03:15 -04:00
Kefu Chai
fd5b4a5206
Merge pull request #45536 from tchaikov/wip-std-optional
compressor/QatAccel: s/boost::optional/std::optional/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-30 23:16:15 +08:00
Kefu Chai
4875c65998
Merge pull request #45648 from tchaikov/wip-kill-distutils
cmake: avoid using distutils

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-03-30 22:56:32 +08:00
Yuri Weinstein
daea072720
Merge pull request #45694 from amathuria/amathuri-53923-fix
osd/osd_types: Increasing decode version of scrub_duration in pg stats

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-30 07:16:44 -07:00
Kefu Chai
35f56a1850
Merge pull request #45142 from SoftIron/ceph-mgr-cephadm-python3-cherrypyTHREE
debian/control: fix python3-cherrypy*3* dependency

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-30 21:57:19 +08:00
Yin Congmin
6593e31fff librbd/cache/pwl: correct cache state
update cache state after dirty_entries or log_enties list updated.

Fixes: https://tracker.ceph.com/issues/50614

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2022-03-30 21:45:29 +08:00
Adam King
627a9df99d
Merge pull request #45217 from rkachach/fix_issue_47872
mgr/cephadm: check spec host when adding osd

Reviewed-by: Adam King <adking@redhat.com>
2022-03-30 09:01:40 -04:00
Adam King
5b49c10aaf
Merge pull request #45275 from rkachach/fix_issue_54342
mgr/cephadm: adding HostSpec validation

Reviewed-by: Adam King <adking@redhat.com>
2022-03-30 08:59:15 -04:00
Adam King
f1760bed61
Merge pull request #45534 from rkachach/fix_issue_50804
mgr/cephadm: Show warning when user provides --fsid option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-03-30 08:58:04 -04:00
Adam King
c985bd639c
Merge pull request #45692 from rkachach/fix_issue_55113
mgr/cephadm: fallback to normal sorted if cannot import natsorted

Reviewed-by: Adam King <adking@redhat,com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2022-03-30 08:48:33 -04:00
Koen Kooi
b7b381fe91 debian/control: fix python3-cherrypy*3* dependency
The trailing '3' was missed in one instance, ceph-mgr-cephadm, leading to:

	Some packages could not be installed. This may mean that you have
	requested an impossible situation or if you are using the unstable
	distribution that some required packages have not yet been created
	or been moved out of Incoming.
	The following information may help to resolve the situation:

	The following packages have unmet dependencies:
	 ceph-mgr-cephadm : Depends: python3-cherrypy but it is not installable

Which makes the installation fail.

Fixes: 78983ad0d0
Signed-off-by: Koen Kooi <koen@softiron.com>
2022-03-30 05:18:51 -07:00
Ernesto Puerta
043f7953d8
Merge pull request #45335 from rhcs-dashboard/fix-54513-master
mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-03-30 14:05:38 +02:00
Ernesto Puerta
751d3df47b
Merge pull request #45559 from rhcs-dashboard/existing-host-test-fix
mgr/dashboard: fix 01-hosts.e2e test error

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-03-30 13:00:36 +02:00
Anthony D'Atri
39e8ec9389
Merge pull request #45701 from inspur-wyq/wip-doc-typos
doc/radosgw/sync-modules.rst: fix typos
2022-03-29 23:54:42 -07:00
Anthony D'Atri
2df627751b
Merge pull request #45702 from inspur-wyq/wip-doc-my-typos
doc/cephadm/operations.rst: fix typos
2022-03-29 23:43:46 -07:00
Samuel Just
b73ed7bb39
Merge pull request #45630 from liu-chunmei/crimson-omap-cmp
crimson: add omap cmp

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-29 21:40:49 -07:00
wangyunqing
92eb799a95 doc/cephadm/operations.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2022-03-30 11:58:43 +08:00
wangyunqing
07584628d6 doc/radosgw/sync-modules.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2022-03-30 11:53:57 +08:00
Samuel Just
32ee47a864
Merge pull request #45526 from xxhdx1985126/wip-coordinate-segment-seq
crimson/os/seastore: coordinate segment seq of journal and ool segments

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-29 20:53:30 -07:00
Chunmei Liu
7244b80956 crimson: return 0 in osd_op.rval when enodata
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-29 17:44:54 -07:00
Chunmei Liu
019c4128cb crimson: add OP_OMAP_CLEAR support
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-29 17:44:40 -07:00
Chunmei Liu
3414b28278 crimson:add OMAP_CMP support
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-29 17:38:17 -07:00
Aishwarya Mathuria
3532b78901 osd/osd_types: Increasing decode version of scrub_duration in pg stats
All new fields added to pg stats after quincy RC need to have the decode field bumped up to avoid decoding errors during an upgrade from quincy RC to the quincy stable version

Fixes: https://tracker.ceph.com/issues/53923
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2022-03-29 23:40:54 +05:30
Redouane Kachach
19c07de820
mgr/cephadm: fallback to normal sorted if cannot import natsorted
Fixes: https://tracker.ceph.com/issues/55113

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-03-29 19:08:10 +02:00
Ernesto Puerta
64c6bb394b
Merge pull request #45563 from rhcs-dashboard/fix-54991-master
mgr/dashboard:  don't log 3xx as errors

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-03-29 19:03:49 +02:00
Boris Ranto
f1d17b04e4
Merge pull request #45295 from lbausch/tweak_mgr_alerts_mail
mgr/alerts: Add Message-Id and Date header to sent emails

Reviewed-by: Boris Ranto <branto@redhat.com>
2022-03-29 13:55:14 +02:00
Ernesto Puerta
406e54d8c4
mgr/dashboard: don't log 3xx as errors
Let's avoid printing these ugly/misleading/redundant messages:

```
0 [dashboard DEBUG controllers.home] frontend language from headers: ['en-us']
0 [dashboard DEBUG controllers.home] found directory for language 'en-us'
0 [dashboard DEBUG controllers.home] serving static content: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/src/pybind/mgr/dashboard/frontend/dist/en-US/styles.css
0 [dashboard ERROR exception] Internal Server Error
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/pybind/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/cherrypy/_cpdispatch.py", line 60, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/pybind/mgr/dashboard/controllers/home.py", line 134, in __call__
    return serve_file(full_path)
  File "/usr/lib/python3/dist-packages/cherrypy/lib/static.py", line 70, in serve_file
    cptools.validate_since()
  File "/usr/lib/python3/dist-packages/cherrypy/lib/cptools.py", line 117, in validate_since
    raise cherrypy.HTTPRedirect([], 304)
cherrypy._cperror.HTTPRedirect: ([], 304)
```

Fixes: https://tracker.ceph.com/issues/54991
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2022-03-29 11:52:18 +02:00
Kefu Chai
1651ce5b03
Merge pull request #45676 from rzarzynski/wip-crimson-toolset-10
ceph.spec.in: use gcc-toolset-10 for building crimson

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-29 17:03:14 +08:00
Ernesto Puerta
729e956855
Merge pull request #44014 from rhcs-dashboard/host-detail-service-column
mgr/dashboard: extend daemon actions to host details 

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-03-29 10:49:31 +02:00
Yuval Lifshitz
cd96c07c6f
Merge pull request #45537 from yuvalif/wip-yuval-multisite-test-devices
multisite/test: running the multisite test script with custom devices
2022-03-29 10:35:57 +03:00
Tim Serong
01359f7905
Merge pull request #45666 from SUSE/wip-fix-55087
ceph.spec.in: Use libthrift-devel on SUSE distros

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-29 18:01:46 +11:00
Tim Serong
c454eeae35
Merge pull request #45664 from SUSE/wip-fix-55079
ceph.spec.in: remove build directory at end of %install

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-29 17:08:09 +11:00
Radosław Zarzyński
30f7514317 ceph.spec.in: use gcc-toolset-10 for building crimson
This commit bumps up the toolset version but only to build crimson.
That is, the classical OSD stays unaffected.

The reason behind the upgrade is the following FTBFS:

```
[ 32%] Building CXX object src/seastar/CMakeFiles/seastar.dir/src/core/reactor.cc.o
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc: In constructor ‘seastar::reactor::reactor(std::shared_ptr<seastar::smp>, seastar:👽:instance&, unsigned int, seastar::reactor_backend_selector, seastar::reactor_config)’:
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc:926:90: error: use of deleted function ‘seastar::condition_variable::condition_variable()’
  926 |     , _thread_pool(std::make_unique<thread_pool>(this, seastar::format("syscall-{}", id))) {
      |                                                                                          ^
In file included from /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/include/seastar/core/reactor.hh:74,
                 from /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/src/core/reactor.cc:32:
/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11345-ga3bb1485/rpm/el8/BUILD/ceph-17.0.0-11345-ga3bb1485/src/seastar/include/seastar/core/condition-variable.hh:157:5: note: ‘seastar::condition_variable::condition_variable() noexcept’ is implicitly deleted because its exception-specification does not match the implicit exception-specification ‘’
  157 |     condition_variable() noexcept = default;
```

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-03-28 21:03:01 +02:00
Ernesto Puerta
2022a56646
Merge pull request #45647 from rhcs-dashboard/fix-55060-master
mgr/dashboard: fix api test issue with pip

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: zmc <NOT@FOUND>
2022-03-28 17:29:11 +02:00
Aashish Sharma
9719cc795e mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview
This PR intends to fix this issue

Fixes:https://tracker.ceph.com/issues/54513
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2022-03-28 18:25:25 +05:30
Xuehan Xu
34e5ebfca7 crimson/os/seastore: coordinate segment seq of journal and ool segments
the segment seq in ool segments' headers also need to be set to the
current journal segment seq, because we rely on this to judge whether a
delta needs to be replayed

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-03-28 17:48:33 +08:00
Tim Serong
19922e1875 ceph.spec.in: Use libthrift-devel on SUSE distros
Fixes: 80e82686eb
Fixes: https://tracker.ceph.com/issues/55087
Signed-off-by: Tim Serong <tserong@suse.com>
2022-03-28 20:46:26 +11:00
Ernesto Puerta
2289ad2bc3
mgr/dashboard: fix api test issue with pip
Fix
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
apache-libcloud 3.5.0 requires requests>=2.26.0, but you have requests 2.25.1 which is incompatible.
Successfully installed CherryPy-13.1.0 PyJWT-2.0.1 Routes-2.4.1 bcrypt-3.1.4 ceph-1.0.0 chardet-4.0.0 cheroot-8.6.0 idna-2.10 jaraco.functools-3.5.0 more-itertools-4.1.0 natsort-8.1.0 portend-3.1.0 pyopenssl-22.0.0 pytz-2022.1 repoze.lru-0.7 requests-2.25.1 tempora-5.0.1
```

Fixes: https://tracker.ceph.com/issues/55060
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2022-03-28 11:28:36 +02:00
Redouane Kachach
b87c966697
mgr/cephadm: check spec host when adding osd
Fixes: https://tracker.ceph.com/issues/47872

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-03-28 10:30:27 +02:00
Tim Serong
aa18cb1200 ceph.spec.in: remove build directory at end of %install
By the time we get to the end of the %install section, all the
built binaries have been installed in the build root, so we can
delete the build directory from the source tree.  This frees up
about 17GB of disk space on build hosts, which is helpful in
case other processes later in the RPM build need more disk space.

Fixes: https://tracker.ceph.com/issues/55079
Signed-off-by: Tim Serong <tserong@suse.com>
2022-03-28 19:23:18 +11:00
Kefu Chai
01b57c1f63
Merge pull request #45661 from tchaikov/doc-dpdk
doc/dpdk: reformat and reword

Reviewed-by: Chunsong Feng <fengchunsong@huawei.com>
2022-03-27 09:58:43 +08:00
Kefu Chai
352ab9b5f0 doc/dpdk: reword the root access part
the root access to system is not a must have for running a DPDK
application. so reword the "Configuring OSD DPDKStack" section.

also, manually editing /etc/passwd is not encouraged, so use
"usermod" instead. to add a directory after user's command
interpreter in /etc/passwd does not make sense. see PASSWD(5).
so drop the paragraph on editing /etc/passwd.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-27 01:00:21 +08:00
Ronen Friedman
a3bb148507
Merge pull request #45640 from ronen-fr/wip-rf-snaptrim
osd/scrub: restart snap trimming only after scrubbing is done

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-26 20:00:20 +03:00
Kefu Chai
3d2d6ffabb doc/dpdk: improve the formatting
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-27 00:46:17 +08:00