Commit Graph

129899 Commits

Author SHA1 Message Date
Kefu Chai
d020d07be9 admin/doc-requirements: pin breathe to 4.32.0
in breathe v4.33, it includes following commit

2498a43723

which specfies the app config value of "graphviz_dot". this annoys
sphinx:

WARNING: while setting up extension breathe: node class 'graphviz' is already registered, its visitors will be overridden

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/application.py", line 245, in __init__
    self.setup_extension(extension)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/application.py", line 402, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/registry.py", line 430, in load_extension
    metadata = setup(app)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/breathe/__init__.py", line 14, in setup
    renderer_setup(app)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/breathe/renderer/sphinxrenderer.py", line 2613, in setup
    app.add_config_value("graphviz_dot", "dot", "html")
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/application.py", line 535, in add_config_value
    self.config.add(name, default, rebuild, types)
  File "/home/docs/checkouts/readthedocs.org/user_builds/ceph/envs/44951/lib/python3.8/site-packages/sphinx/config.py", line 282, in add
    raise ExtensionError(__('Config value %r already present') % name)
sphinx.errors.ExtensionError: Config value 'graphviz_dot' already present

Extension error:
Config value 'graphviz_dot' already present

this issue has been reported to upstream, see
https://github.com/michaeljones/breathe/issues/803

before it is fixed upstream, let's stick with 4.32.0
which is known to work.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-02-14 18:31:39 +08:00
Volker Theile
48fff60b63 mgr/dashboard: "Please expand your cluster first" shouldn't be shown if cluster is already meaningfully running
This PR will assume that a cluster is already up and fully running. If this should not be the expected behaviour, deployment tools have to set 'INSTALLED' explicitly. Without this assumption it might happen that upgraded and fully running clusters, e.g. Octopus -> Pacific, will show the 'Expand Cluster' on first log in.

cephadm will take care that the bootstrap phase will write the necessary key to show the 'Expand cluster' page.

Fixes: https://tracker.ceph.com/issues/54215

Signed-off-by: Volker Theile <vtheile@suse.com>
2022-02-14 10:49:38 +01:00
Venky Shankar
2de993b969
Merge pull request #44971 from joscollin/wip-doc-fix-name-id
doc: mention MDS name and ID format in ceph-mds.rst

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-14 12:19:43 +05:30
Venky Shankar
43b3a7c845
Merge pull request #39315 from shenhang/fetchkey
mds: fetch single dentry instead of complete dirfrag for lookup

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-14 09:53:10 +05:30
Kefu Chai
3a1ef3fe36 include/rados/librados.h: drop duplicated function declaration
rados_mgr_command() has been declared in the very same header file
with exactly the same function signature, so there is no need to
repeat it.

in this change, the duplicated declaration of rados_mgr_command()
is dropped. the one with doxygen document is reserved.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-02-14 12:19:51 +08:00
Matan Breizman
1aaf24f503 mon/OSDMonitor: Restrict pool names beggining with a dot
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-02-13 08:45:09 +00:00
Kotresh HR
481093b7f6
Merge pull request #44308 from vshankar/wip-document-mds-locker
doc: document mds locking operation
2022-02-13 08:41:10 +05:30
Anthony D'Atri
e5b9173836
Merge pull request #44898 from taggelos/patch-1
doc/man/8: Fix spelling in user rename example
2022-02-12 02:52:16 -08:00
Yuri Weinstein
1ead2229ae
Merge pull request #44780 from sosodev/dump-blocked-ops-count
mds, osd: add dump_blocked_ops_count command

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-02-11 14:51:00 -08:00
Yuri Weinstein
66e651ce3c
Merge pull request #44752 from NitzanMordhai/wip-nitzan-osd-status
pybind/mgr: ceph osd status crash with ZeroDivisionError

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-11 14:50:31 -08:00
Yuri Weinstein
2624f51a72
Merge pull request #44588 from kamoltat/wip-ksirivad-disable-progress-by-default
pybind/mgr/progress: disable pg recovery event by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-02-11 14:49:17 -08:00
Yuri Weinstein
0549f3b07b
Merge pull request #44289 from Vicente-Cheng/fix-outgoing-bl-overflow
msg/async: fix outgoing_bl overflow and reset state_offset

Reviewed-by: Kefu Chai <kchai@redhat.com>
2022-02-11 14:48:18 -08:00
Samuel Just
ec2fb479f4
Merge pull request #44809 from cyx1231st/wip-crimson-cleanup-epm
crimson/os/seastore: allow EPM to make decisions on the general allocation path

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-11 13:36:57 -08:00
Venky Shankar
95f326d780 doc: document mds locking operation
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-02-11 18:45:33 +05:30
Milind Changire
ece89098e9 doc: describe json, pool_id and pool_name layout fields
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-02-11 16:05:54 +05:30
Milind Changire
5d9c81cbf6 qa: unit test to test getvxattr
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-02-11 16:05:54 +05:30
Milind Changire
50ff98751c mds: add json handling in setvxattr
Setting the entire layout via setfattr -n ceph.dir.layout.json
requires a json value with the name of all mandatory fields as
returned by getfattr, except 'inheritance'.
Also, the json format expects a 'pool_name' or 'pool_id'.
If both are specified, the 'pool_name' is given priority over
'pool_id' for better disambiguation.

ceph.(dir|file).layout.pool_name and ceph.(dir|file).layout.pool_id
can also be set and fetched as individual fields.

Fixes: https://tracker.ceph.com/issues/51062
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-02-11 16:05:54 +05:30
Milind Changire
518f714941 client: add new getvxattr rpc
This new op fetches ceph namespace specific extended attributes
i.e. attributes
1. ceph.dir.layout.json and ceph.file.layout.json
2. ceph.dir.layout.pool_name and ceph.file.layout.pool_name
3. ceph.dir.layout.pool_id and ceph.file.layout.pool_id
4. ceph.dir.pin, ceph.dir.pin.random and ceph.dir.pin.distributed

Fixes: https://tracker.ceph.com/issues/51062
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-02-11 16:05:23 +05:30
Sarthak0702
db3eb10fa1
Merge pull request #44849 from rhcs-dashboard/bugfix-cephfs-invalid-dir-permission
mgr/dashboard: Directories Menu Can't Use on Ceph File System Dashboard
2022-02-11 15:29:20 +05:30
caishan
4bd9a68fe8 Fix rgw data sync log info error
Signed-off-by: caishan <caishan1993@foxmail.com>
2022-02-11 17:54:18 +08:00
Milind Changire
b96c01da85 mds: add new getvxattr op
This new op fetches ceph namespace specific extended attributes
i.e. attributes  "ceph.dir.layout.json", "ceph.file.layout.json"
and "ceph.dir.pin*"

For layout attributes, value of the .json xattrs provide the
following features:
1. output in json format
2. resolves layout from the most closest ancestor inode, starting
   at self
3. adds an 'inheritance' field to show the layout inheritance
   where:
       @default - implies default system-wide layout
       @inherited - implies layout has been inherited from an ancestor
       @set - implies layout has been set for that specific inode
4. adds two fields: pool_name and pool_id to differentiate between
   the two values; especially for the situations where users have
   chosen to use purely digit sequences to name pools; since pool_id
   is system generated and its output is a decimal number as well,
   users cannot distinguish the value in the field named 'pool'

Fixes: https://tracker.ceph.com/issues/51062
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-02-11 14:00:06 +05:30
Venky Shankar
4c78eb81ad
Merge pull request #44970 from kotreshhr/fix-clone-uid-gid
mgr/volumes: Fix clone uid/gid mismatch

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-11 13:11:24 +05:30
Jos Collin
3404d4c8ed
doc: mention MDS name and ID format in ceph-mds.rst
Signed-off-by: Jos Collin <jcollin@redhat.com>
2022-02-11 10:20:47 +05:30
Yingxin Cheng
cc5619ff10 crimson/os/seastore: misc log improvements
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:51:58 +08:00
Yingxin Cheng
3b54d41c78 crimson/os/seastore: initialize segment_info_t correctly when open/empty
Previously, segment_info_t was not initialized for out-of-line segments
when open, including journal_segment_seq and the legacy out_of_line.

And the journal_segment_seq was not reset when empty.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:51:58 +08:00
Yingxin Cheng
1d6d93a6ee crimson/os/seastore: replace out_of_line by segment_type_t
Leverage segment_seq_t to identify the type of the segment.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:42:45 +08:00
Yingxin Cheng
1446c58acd crimson/os/seastore: cleanup dependency between epm and journal
OOL segments won't contain deltas and don't need to replay. There is no
need to assign segment sequence from journal.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Yingxin Cheng
d83627c066 crimson/os/seastore: cleanup dependency between epm and lba_manager
Move lba_manamger->update_mapping() from epm to transaction manager.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Yingxin Cheng
cc67f3262d crimson/os/seastore: update_mapping related cleanups
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Yingxin Cheng
32da0e0ecb crimson/os/seastore: allow EPM to make decisions on the general extent allocation path
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Yingxin Cheng
d483557c56 crimson/os/seastore: drop the unnecessary backend_type in every cached extent
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Yingxin Cheng
bcd5c73540 crimson/os/seastore/epm: cleanup unnecessary cache dependency when mark extent ool
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Yingxin Cheng
00a27c8fef crimson/os/seastore: cleanup delayed paddr
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-11 09:26:22 +08:00
Samuel Just
5f252053d8
Merge pull request #44968 from cyx1231st/wip-crimson-cleanup-cleaner-1
crimson/os/seastore/segment_cleaner: minor cleanups

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-10 14:45:59 -08:00
Neha Ojha
7809ff5cc0
Merge pull request #44964 from yaarith/pending-release-notes-perf
PendingReleaseNotes: add a note about perf channel

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-02-10 14:40:38 -08:00
Neha Ojha
fc54d1a616
Merge pull request #44963 from ljflores/wip-peering-state-docs
doc/dev: add section on using the gen_state_diagram.py script

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-02-10 14:07:31 -08:00
Casey Bodley
8cc82b6fe8 rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation
use the cluster defaults for pg_num_min

Fixes: https://tracker.ceph.com/issues/52673

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-02-10 13:40:57 -05:00
Laura Flores
b544e6986c doc/dev: add section on using the gen_state_diagram.py script
Commands sourced from Jianshen Liu's blog post on the Ceph peering state machine: https://jianshenliu.com/blog/development/ceph-peering-state-machine/

Also fixed the formatting of "PG info".

Signed-off-by: Laura Flores <lflores@redhat.com>
2022-02-10 09:12:05 -06:00
Guillaume Abrioux
2bd3dd512a
Merge pull request #44765 from guits/guits-fix-cve-cv
ceph-volume: honour osd_dmcrypt_key_size option
2022-02-10 13:38:13 +01:00
Ernesto Puerta
f1dcdf4356
Merge pull request #44808 from rhcs-dashboard/support-snmp-service-creation
mgr/dashboard: support snmp-gateway service creation from UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-02-10 12:54:00 +01:00
Kotresh HR
b3c9e6b50c mgr/volumes: Fix clone uid/gid mismatch
This is the regression caused by commit 18b85c53a.
The 'set_attrs' function sets the uid/gid of the
group to the subvolume if uid/gid is not passed.
The attrs of the clone should match the source
snapshot. Hence, don't use the 'set_attrs'
function to set only the quota attrs for the
clone.

Fixes: https://tracker.ceph.com/issues/54066
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-02-10 15:16:22 +05:30
Venky Shankar
236fcc32af
Merge pull request #44953 from aejv/spellingerror
purge_queue.py: fixed typo

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-10 12:21:47 +05:30
Yingxin Cheng
d352970d61 crimson/os/seastore/segment_cleaner: mark get_next_gc_target() private
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-10 13:22:10 +08:00
Yingxin Cheng
b81b0e4796 crimson/os/seastore/segment_cleaner: make init_mark_segment_closed() private
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-10 13:22:10 +08:00
Yingxin Cheng
f241023393 crimson/os/seastore/segment_cleaner: drop unused code
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-10 13:09:44 +08:00
Yingxin Cheng
7907253147 crimson/net/socket: add a missing header
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-10 13:09:44 +08:00
Guillaume Abrioux
5ac1ec65cb ceph-volume/activate: load the config from lv tag
When `ceph-volume lvm trigger` is called with an OSD where the tag
`ceph.cluster_name` is not 'ceph', it fails.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-02-10 02:46:59 +01:00
Adam King
46f939f057 qa/tasks/cephadm_cases: increase timeouts in test_cli.py
These seem to be failing sometimes but in my testing
sometimes these events are happening a few seconds after
we hit the timeout. Trying to see if this makes the tests
more consistent. No need to mark the test as failed
if we report something up in 34 seconds vs 25 especially
when cephadm works on a cyclic daemon refresh.

Signed-off-by: Adam King <adking@redhat.com>
2022-02-09 20:42:42 -05:00
Guillaume Abrioux
2b793952bb ceph-volume/tests: use centos/stream8 images
Since recent move from CentOS 8 to CentOS Stream 8, let's do the same here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-02-10 02:39:47 +01:00
Guillaume Abrioux
db48850745 ceph-volume/tests: add tests in util/encryption.py
this adds some unit tests in order to cover `luks_format()` and `luks_open()`
in `util/encryption.py`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-02-10 02:39:45 +01:00