Commit Graph

137572 Commits

Author SHA1 Message Date
Adam King
adf13c82eb mgr/cephadm: asyncio based universal timeout for ssh/cephadm commands
Since we already have make use of asyncio for our ssh commands,
we can use asyncio's timeout on waiting for concurrent futures to complete
as a way to have universal timeouts on our cephadm commands.
This change also creates a contextmanager that will catch any asyncio.TimeoutError.
Using the contextmanager along with calls to the wait_async function
will catch any timeout exception raised and convert it into an appropriate
OrchetratorError including information about what and where for the timeout
if it was provided (host where run, what command). This allows us to guarantee a
background ssh command eventually returns and inform users of any
timeouts by raising a health warning or logging the error instead
of sitting idle indefinitely

Fixes: https://tracker.ceph.com/issues/54024

Signed-off-by: Adam King <adking@redhat.com>
2023-04-25 10:13:50 -04:00
Adam King
00d4203ec6 mgr/cephadm: pass --timeout to cephadm binary commands
Signed-off-by: Adam King <adking@redhat.com>
2023-04-25 10:11:39 -04:00
Adam King
819777e68f
Merge pull request #50719 from guits/update-dockerfile-keepalived
contrib: update keepalived Dockerfile

Reviewed-by: Adam King <adking@redhat.com>
2023-04-25 10:04:36 -04:00
Yingxin
23183260c5
Merge pull request #49980 from myoungwon/wip-generalize-recordsubmitter
crimson/os/seastore/journal: generalize recordsubmitter

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-04-25 09:48:35 +08:00
Guillaume Abrioux
7e93b032d7
Merge pull request #51067 from guits/fix-backward-compatible-issue-cv
ceph-volume: fix batch refactor issue
2023-04-24 10:36:42 +02:00
Guillaume Abrioux
300e3ee87e
Merge pull request #50838 from guits/osd-replacement-improvements
orchestrator: add `--no-destroy` arg to `ceph orch osd rm`
2023-04-24 10:35:51 +02:00
Guillaume Abrioux
bfa8bd15c7
Merge pull request #50745 from guits/quick-fix-cv
ceph-volume: quick fix in zap.py
2023-04-24 10:35:31 +02:00
zdover23
d9e6e1de79
Merge pull request #51178 from zdover23/wip-doc-2023-04-22-glossary-placement-groups
doc/glossary: add "Placement Groups" definition

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-04-23 17:07:28 +02:00
zdover23
f6d5a33d72
Merge pull request #51177 from zdover23/wip-doc-2023-04-22-start-documenting-ceph-first-150-lines
doc/start: edit first 150 lines of documenting-ceph

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-04-23 16:53:46 +02:00
Zac Dover
dd37f94aa4 doc/start: edit first 50 lines of documenting-ceph
Edit the first 150 lines of doc/start/documenting-ceph.rst. This is part
of an initiative to harvest the fruits of Cephalocon 2023, at which
documentation proved to be in demand to a surprising degree.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-23 16:43:39 +02:00
Zac Dover
9f37ea651f doc/glossary: add "Placement Groups" definition
Add a definition of "Placement Groups" to the Glossary.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-22 16:54:41 +02:00
Anthony D'Atri
2d2fd8d8a9
Merge pull request #51175 from zdover23/wip-doc-2023-04-21-main-page-docs-link
doc: add link to "documenting ceph" to index.rst
2023-04-22 13:24:38 +02:00
Zac Dover
155a382cb2 doc: add link to "documenting ceph" to index.rst
Add a link to the landing page of docs.ceph.com to direct documentation
contributors to documentation-related information.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-22 07:51:23 +02:00
Ilya Dryomov
92ba1c08c8
Merge pull request #50324 from pkalever/scrub_incomplete
librbd: remove previous incomplete primary snapshot after successfully creating a new one

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-04-21 18:00:55 +02:00
Samuel Just
7dd23978a6
Merge pull request #48717 from liu-chunmei/multicore-seastore
crimson/os: support multicore seastore

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-04-20 15:55:11 -07:00
chunmei-liu
ad5b0c12a7 crimson/os: make ZNS support shard metadata
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2023-04-20 20:08:00 +00:00
chunmei
c401a4af88 crimson/os: fix formatter build error
Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 20:07:55 +00:00
chunmei
91572c1799 crimson/os: fix unittest error for multicore seastore
Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 20:07:50 +00:00
chunmei
ad4f7bbf15 crimson/os: static assign segments to each shard
and make device sharded

Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 20:05:05 +00:00
chunmei
7a673d006d cirmson/os: seperate futurizedStore interface
into futurizedStore and futurizedShardStore
and modification the other parts in osd and objectstore

Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 19:58:16 +00:00
chunmei
6a121c0852 crimson/seastore: fix heap-used-after-free bug
Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 18:46:41 +00:00
Nizamudeen A
f79b94300e
Merge pull request #51153 from rhcs-dashboard/applitools-fix
mgr/dashboard: ignore Details card in applitools vrt

Reviewed-by: Pegonzal <NOT@FOUND>
2023-04-20 23:07:48 +05:30
Nizamudeen A
d36d554041
Merge pull request #51163 from rhcs-dashboard/hide-rbd-force-promote-notification
mgr/dashboard: hide notification on force promote

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-04-20 23:05:45 +05:30
Samuel Just
49bd470185
Merge pull request #50835 from cyx1231st/wip-seastar-adjust-conn-ref
crimson/net: change ConnectionRef to be a local_shared_foreign_ptr

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-04-20 09:09:39 -07:00
Pedro Gonzalez Gomez
abe1e5101c mgr/dashboard: hide notification on force promote
Fixes: https://tracker.ceph.com/issues/59500
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-04-20 17:31:52 +02:00
myoungwon oh
6b453c9b2f crimson/os/seastore/cbjournal: reset start address for replay if committed_to exceeds cbjournal_end()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 14:31:27 +00:00
myoungwon oh
7bac32cd71 test/crimson/seastore/cbjournal: call close after a unittest is done and relocate open and close related functions
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 14:31:21 +00:00
myoungwon oh
59cb8c1050 crimson/os/seastore/journal: add CircularJournalSpace
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 14:31:08 +00:00
Rishabh Dave
ac9ca5d14b
Merge pull request #51127 from rishabh-d-dave/fs-qa-caps_helper-bug
qa/cephfs/caps_helper: fix a bug in methods that generate cap string

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-04-20 19:27:19 +05:30
Rishabh Dave
9be6de9185
Merge pull request #50882 from rishabh-d-dave/fs-qa-CapTester
qa/cephfs: improvements in caps_helper

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-04-20 19:27:02 +05:30
myoungwon oh
bba25dbce6 crimson/os/seastore/journal: introduce JournalAllocator to generalize SegmentAllocator
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 13:01:27 +00:00
myoungwon oh
5abf9b5c98 crimson/os/seastore/journal: relocate RecordSubmitter related classes
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 13:01:21 +00:00
Ilya Dryomov
d13bf0da9f
Merge pull request #51159 from idryomov/wip-rbd-man-filestore
doc/man/rbd: resurrect filestore alloc_size note

Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2023-04-20 12:43:57 +02:00
Ilya Dryomov
d3558c49bf doc/man/rbd: resurrect filestore alloc_size note
Mistakenly removed in commit d79f2a8154 ("docs: warning and remove
few docs section for Filestore Update docs after filestore removal.").
The kernel client, however new, will continue to be able to talk to
FileStore OSDs for as long as they exist.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-04-20 12:08:40 +02:00
Nizamudeen A
4cd6864014
Merge pull request #50580 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-webpack-and-angular-devkit-build-angular-5.76.1
mgr/dashboard: bump webpack and @angular-devkit/build-angular in /src/pybind/mgr/dashboard/frontend 

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-04-20 15:37:28 +05:30
Nizamudeen A
f31053b5fa mgr/dashboard: ignore Details card in applitools vrt
because the details card includes the FSID and Ceph Version which can
change on different clusters.

also don't run the 800x600 resolution because for new dashboard it looks
really weird.

Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-04-20 13:10:27 +05:30
zdover23
353a81f362
Merge pull request #51140 from zdover23/wip-doc-2023-04-18-rados-operations-2-of-x
doc/rados: edit user-management (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-04-20 08:18:04 +02:00
Zac Dover
e3575bb72f doc/rados: edit user-management (2 of x)
Line-edit doc/rados/user-management.rst (2 of x). Some internal
references had to be removed, but these will be repaired when the next
part of this file is updated in a future PR.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-20 08:09:57 +02:00
zdover23
195262cf6b
Merge pull request #51143 from NitzanMordhai/wip-nitzan-filestore-removal-docs-update
docs: warning and remove few docs section for Filestore

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-04-20 07:42:58 +02:00
Nitzan Mordechai
d79f2a8154 docs: warning and remove few docs section for Filestore
Update docs after filestore removal.

Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2023-04-20 05:01:53 +00:00
Casey Bodley
1c49dc213d
Merge pull request #51068 from cbodley/wip-59269
cmake/rgw: librgw tests depend on ALLOC_LIBS

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-04-19 12:43:53 -04:00
Casey Bodley
5609383d5b
Merge pull request #51031 from cbodley/wip-59423
rgw/aio: fix recursion from tracing changes

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-04-19 11:36:15 -04:00
Yuval Lifshitz
fba5b3f898
Merge pull request #50975 from yuvalif/wip-yuval-fix-59381
rgw/lua: fix CopyFrom crash
2023-04-19 16:30:29 +03:00
Yuval Lifshitz
cfbc2129c6
Merge pull request #50965 from yuvalif/wip-yuval-fix-59383
rgw/kafka: failed to reconnect to broker after idle timeout
2023-04-19 16:29:50 +03:00
Ilya Dryomov
df2bb13d03 librbd: always refresh after creating snapshot in CreatePrimaryRequest
Up until now this was conditioned on whether the caller expressed
interest in the ID of the created snapshot and happened to work only
because CreatePrimaryRequest wasn't actually consulting any mirror
snapshot metadata.  This has just changed with unlink_peer() needing to
see an up-to-date complete flag which is set in SetImageStateRequest
following the write out of image state object(s).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-04-19 11:05:05 +02:00
Nizamudeen A
eeba8e5f43
Merge pull request #50918 from rhcs-dashboard/delete-ceph-authx
mgr/dashboard: delete-ceph-authx

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-04-19 13:10:57 +05:30
Prasanna Kumar Kalever
165c9a4e16 librbd: remove previous incomplete primary snapshot after successfully creating a new one
Problem:
-------
At a high level, creating a primary snapshot consists of three steps:

1. actually creating a snapshot in the mirror namespace
2. generating a set of image state objects with additional metadata for
   the snapshot
3. marking the snapshot as complete after the image state objects are
   written out

Depending on the circumstances, a request to create a primary snapshot
can be forwarded to rbd-mirror daemon.  If that happens and rbd-mirror
daemon gets axed for some practical reason after completing steps (1)
and/or (2) but before completing step (3), we are left with a
permanently incomplete primary snapshot because upon retrying that
primary snapshot creation request, librbd notices that such snapshot
already exists.  It does not check whether this "pre-existing" snapshot
is complete.

Solution:
--------
As part of the next mirror snapshot create (say triggered by the
scheduler) the unlink_peer() is called, it checks if there exists any
incomplete snapshot and delete them accordingly.

Fixes: https://tracker.ceph.com/issues/58887
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2023-04-19 11:47:19 +05:30
Venky Shankar
6d1eb725be Merge PR #50089 into main
* refs/pull/50089/head:
	doc: add a note for minimum compatible python version and supported distros
	tools/cephfs/top/CMakeList.txt: check the minimum compatible python version for cephfs-top

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-04-19 11:30:54 +05:30
Rishabh Dave
f0ffade052 qa/cephfs/cap_tester: simplify CapTester and its instantiation
Class CapTester contains two distinct immiscible group of methods: one
that tests MON caps and other that tests MDS caps. When using CapTester
for the former reason the instantiation neither needs mount object and
the path where files for testing will be created nor it needs to run the
method that creates files for testing rw permissions. When using
this class for latter the case is the exact opposite.

Create 2 separate classes for each of these purpose and class that
inherits both of these classes so that instantiating the class becomes
as simple as it can be.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 20:32:15 +05:30
Rishabh Dave
95c6daa45b qa/cephfs: update method caps_helper.CapTester.run_cap_tests()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 19:42:55 +05:30