Commit Graph

137771 Commits

Author SHA1 Message Date
Zac Dover
dd37f94aa4 doc/start: edit first 50 lines of documenting-ceph
Edit the first 150 lines of doc/start/documenting-ceph.rst. This is part
of an initiative to harvest the fruits of Cephalocon 2023, at which
documentation proved to be in demand to a surprising degree.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-23 16:43:39 +02:00
Zac Dover
9f37ea651f doc/glossary: add "Placement Groups" definition
Add a definition of "Placement Groups" to the Glossary.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-22 16:54:41 +02:00
Anthony D'Atri
2d2fd8d8a9
Merge pull request #51175 from zdover23/wip-doc-2023-04-21-main-page-docs-link
doc: add link to "documenting ceph" to index.rst
2023-04-22 13:24:38 +02:00
Zac Dover
155a382cb2 doc: add link to "documenting ceph" to index.rst
Add a link to the landing page of docs.ceph.com to direct documentation
contributors to documentation-related information.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-22 07:51:23 +02:00
Yaarit Hatuka
750772d71b PendingReleaseNotes: add a note about telemetry leaderboard
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2023-04-21 18:17:10 +00:00
Ilya Dryomov
92ba1c08c8
Merge pull request #50324 from pkalever/scrub_incomplete
librbd: remove previous incomplete primary snapshot after successfully creating a new one

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-04-21 18:00:55 +02:00
Samuel Just
7dd23978a6
Merge pull request #48717 from liu-chunmei/multicore-seastore
crimson/os: support multicore seastore

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-04-20 15:55:11 -07:00
chunmei-liu
ad5b0c12a7 crimson/os: make ZNS support shard metadata
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2023-04-20 20:08:00 +00:00
chunmei
c401a4af88 crimson/os: fix formatter build error
Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 20:07:55 +00:00
chunmei
91572c1799 crimson/os: fix unittest error for multicore seastore
Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 20:07:50 +00:00
chunmei
ad4f7bbf15 crimson/os: static assign segments to each shard
and make device sharded

Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 20:05:05 +00:00
chunmei
7a673d006d cirmson/os: seperate futurizedStore interface
into futurizedStore and futurizedShardStore
and modification the other parts in osd and objectstore

Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 19:58:16 +00:00
chunmei
6a121c0852 crimson/seastore: fix heap-used-after-free bug
Signed-off-by: chunmei <chunmei.liu@intel.com>
2023-04-20 18:46:41 +00:00
Nizamudeen A
f79b94300e
Merge pull request #51153 from rhcs-dashboard/applitools-fix
mgr/dashboard: ignore Details card in applitools vrt

Reviewed-by: Pegonzal <NOT@FOUND>
2023-04-20 23:07:48 +05:30
Nizamudeen A
d36d554041
Merge pull request #51163 from rhcs-dashboard/hide-rbd-force-promote-notification
mgr/dashboard: hide notification on force promote

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-04-20 23:05:45 +05:30
Samuel Just
49bd470185
Merge pull request #50835 from cyx1231st/wip-seastar-adjust-conn-ref
crimson/net: change ConnectionRef to be a local_shared_foreign_ptr

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-04-20 09:09:39 -07:00
Pedro Gonzalez Gomez
abe1e5101c mgr/dashboard: hide notification on force promote
Fixes: https://tracker.ceph.com/issues/59500
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-04-20 17:31:52 +02:00
Casey Bodley
af5f9cabb9 rgw: remove RGWUserInfo::assumed_role_arn
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-04-20 11:11:25 -04:00
Casey Bodley
9da3120505 rgw/dbstore: stop storing RGWUserInfo::assumed_role_arn
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-04-20 11:11:25 -04:00
myoungwon oh
6b453c9b2f crimson/os/seastore/cbjournal: reset start address for replay if committed_to exceeds cbjournal_end()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 14:31:27 +00:00
myoungwon oh
7bac32cd71 test/crimson/seastore/cbjournal: call close after a unittest is done and relocate open and close related functions
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 14:31:21 +00:00
myoungwon oh
59cb8c1050 crimson/os/seastore/journal: add CircularJournalSpace
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 14:31:08 +00:00
Rishabh Dave
ac9ca5d14b
Merge pull request #51127 from rishabh-d-dave/fs-qa-caps_helper-bug
qa/cephfs/caps_helper: fix a bug in methods that generate cap string

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-04-20 19:27:19 +05:30
Rishabh Dave
9be6de9185
Merge pull request #50882 from rishabh-d-dave/fs-qa-CapTester
qa/cephfs: improvements in caps_helper

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-04-20 19:27:02 +05:30
Casey Bodley
2f60200bf6 rgw/sts: AssumeRole no longer writes to user metadata
`storeARN()` was storing the role's ARN in `RGWUserInfo::assumed_role_arn`,
but that field was unused

Fixes: https://tracker.ceph.com/issues/59495

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-04-20 09:25:52 -04:00
myoungwon oh
bba25dbce6 crimson/os/seastore/journal: introduce JournalAllocator to generalize SegmentAllocator
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 13:01:27 +00:00
myoungwon oh
5abf9b5c98 crimson/os/seastore/journal: relocate RecordSubmitter related classes
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-04-20 13:01:21 +00:00
Ilya Dryomov
d13bf0da9f
Merge pull request #51159 from idryomov/wip-rbd-man-filestore
doc/man/rbd: resurrect filestore alloc_size note

Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2023-04-20 12:43:57 +02:00
Ilya Dryomov
d3558c49bf doc/man/rbd: resurrect filestore alloc_size note
Mistakenly removed in commit d79f2a8154 ("docs: warning and remove
few docs section for Filestore Update docs after filestore removal.").
The kernel client, however new, will continue to be able to talk to
FileStore OSDs for as long as they exist.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-04-20 12:08:40 +02:00
Nizamudeen A
4cd6864014
Merge pull request #50580 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-webpack-and-angular-devkit-build-angular-5.76.1
mgr/dashboard: bump webpack and @angular-devkit/build-angular in /src/pybind/mgr/dashboard/frontend 

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-04-20 15:37:28 +05:30
Zac Dover
97b751ed8f doc/rados/ops: edit user-management.rst (3 of x)
Line-edit doc/rados/user-management.rst (3 of x).

https://tracker.ceph.com/issues/58485

Follows https://github.com/ceph/ceph/pull/51140.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-20 10:25:00 +02:00
Nizamudeen A
f31053b5fa mgr/dashboard: ignore Details card in applitools vrt
because the details card includes the FSID and Ceph Version which can
change on different clusters.

also don't run the 800x600 resolution because for new dashboard it looks
really weird.

Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-04-20 13:10:27 +05:30
zdover23
353a81f362
Merge pull request #51140 from zdover23/wip-doc-2023-04-18-rados-operations-2-of-x
doc/rados: edit user-management (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-04-20 08:18:04 +02:00
Zac Dover
e3575bb72f doc/rados: edit user-management (2 of x)
Line-edit doc/rados/user-management.rst (2 of x). Some internal
references had to be removed, but these will be repaired when the next
part of this file is updated in a future PR.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-20 08:09:57 +02:00
zdover23
195262cf6b
Merge pull request #51143 from NitzanMordhai/wip-nitzan-filestore-removal-docs-update
docs: warning and remove few docs section for Filestore

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-04-20 07:42:58 +02:00
Nitzan Mordechai
d79f2a8154 docs: warning and remove few docs section for Filestore
Update docs after filestore removal.

Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2023-04-20 05:01:53 +00:00
Casey Bodley
1c49dc213d
Merge pull request #51068 from cbodley/wip-59269
cmake/rgw: librgw tests depend on ALLOC_LIBS

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-04-19 12:43:53 -04:00
Casey Bodley
5609383d5b
Merge pull request #51031 from cbodley/wip-59423
rgw/aio: fix recursion from tracing changes

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-04-19 11:36:15 -04:00
Yuval Lifshitz
fba5b3f898
Merge pull request #50975 from yuvalif/wip-yuval-fix-59381
rgw/lua: fix CopyFrom crash
2023-04-19 16:30:29 +03:00
Yuval Lifshitz
cfbc2129c6
Merge pull request #50965 from yuvalif/wip-yuval-fix-59383
rgw/kafka: failed to reconnect to broker after idle timeout
2023-04-19 16:29:50 +03:00
Ali Masarwa
1186c477fb RGW: notification | hide auto generated topics when listing topics
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-04-19 13:32:24 +03:00
Ilya Dryomov
df2bb13d03 librbd: always refresh after creating snapshot in CreatePrimaryRequest
Up until now this was conditioned on whether the caller expressed
interest in the ID of the created snapshot and happened to work only
because CreatePrimaryRequest wasn't actually consulting any mirror
snapshot metadata.  This has just changed with unlink_peer() needing to
see an up-to-date complete flag which is set in SetImageStateRequest
following the write out of image state object(s).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-04-19 11:05:05 +02:00
Nizamudeen A
eeba8e5f43
Merge pull request #50918 from rhcs-dashboard/delete-ceph-authx
mgr/dashboard: delete-ceph-authx

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-04-19 13:10:57 +05:30
Prasanna Kumar Kalever
165c9a4e16 librbd: remove previous incomplete primary snapshot after successfully creating a new one
Problem:
-------
At a high level, creating a primary snapshot consists of three steps:

1. actually creating a snapshot in the mirror namespace
2. generating a set of image state objects with additional metadata for
   the snapshot
3. marking the snapshot as complete after the image state objects are
   written out

Depending on the circumstances, a request to create a primary snapshot
can be forwarded to rbd-mirror daemon.  If that happens and rbd-mirror
daemon gets axed for some practical reason after completing steps (1)
and/or (2) but before completing step (3), we are left with a
permanently incomplete primary snapshot because upon retrying that
primary snapshot creation request, librbd notices that such snapshot
already exists.  It does not check whether this "pre-existing" snapshot
is complete.

Solution:
--------
As part of the next mirror snapshot create (say triggered by the
scheduler) the unlink_peer() is called, it checks if there exists any
incomplete snapshot and delete them accordingly.

Fixes: https://tracker.ceph.com/issues/58887
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2023-04-19 11:47:19 +05:30
Venky Shankar
6d1eb725be Merge PR #50089 into main
* refs/pull/50089/head:
	doc: add a note for minimum compatible python version and supported distros
	tools/cephfs/top/CMakeList.txt: check the minimum compatible python version for cephfs-top

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-04-19 11:30:54 +05:30
Rishabh Dave
f0ffade052 qa/cephfs/cap_tester: simplify CapTester and its instantiation
Class CapTester contains two distinct immiscible group of methods: one
that tests MON caps and other that tests MDS caps. When using CapTester
for the former reason the instantiation neither needs mount object and
the path where files for testing will be created nor it needs to run the
method that creates files for testing rw permissions. When using
this class for latter the case is the exact opposite.

Create 2 separate classes for each of these purpose and class that
inherits both of these classes so that instantiating the class becomes
as simple as it can be.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 20:32:15 +05:30
Rishabh Dave
95c6daa45b qa/cephfs: update method caps_helper.CapTester.run_cap_tests()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 19:42:55 +05:30
Rishabh Dave
ea9f13e553 qa/cephfs: move few methods such that they can be reused
Move get_mon_cap_from_keyring() and get_fsnmes_from_moncap() from class
CapTester to main namespace of caps_helper.py so that they can be
imported freely and reused by tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 19:42:55 +05:30
Rishabh Dave
ad68a55121 qa/cephfs: improve caps_helper.CapTester.run_mon_cap_tests()
This method checks if the output of the command "ceph fs ls" for client
ID it receives is same as the output printed for client.admin. Don't do
so, limit the test to only checking if "ceph fs ls --id client.x -k
keyring_file" prints fs name for which client.x has permissions.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 19:42:55 +05:30
Rishabh Dave
008dbe91e2 qa/cephfs: improve caps_helper.CapTester
Improvement #1:

CapTester.write_test_files() not only creates the test file but also
does the following for every mount object it receives in parameters -

* carefully produces the path for the test file as per parameters
  received
* generates the unique data for each test file on a CephFS mount
* creates a data structure -- list of lists -- that holds all this
  information along with mount object itself for each mount object so
  that tests can be conducted at a later point

Untangle this mess of code by splitting this method into 3 separate
methods -

1. To produce the path for test file (as per user's need).
2. To generate the data that will be written into the test file.
3. To actually create the test file on CephFS.

Improvement #2:

Remove the internal data structure used for testing -- self.test_set --
and use separate class attributes to store all the data required for
testing instead of a tuple. This serves two purpose -

One, it makes it easy to manipulate all this data from helper methods
and during debugging session, especially while using a PDB session.

And two, make it impossible to have multiple mounts/multiple "test sets"
within same CapTester instance for the sake of simplicity. Users can
instead create two instances of CapTester instances if needed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-04-18 19:42:55 +05:30