Commit Graph

137542 Commits

Author SHA1 Message Date
John Mulligan
8042e3cefb mgr/dashboard: ignore type checking on mgr proxy object assignments
Add `# type: ignore` comments to two dashboard functions that attempt
to set manager properties. There appear to be two approaches to fixing
the problem:

1. The _MgrProxy object that the dashboard uses has a __getattr__ method
for pulling value from the underlying mgr object. It does not have a
__setattr__ method. This means the setting values on _MgrProxy do not
propogate down to the original mgr.
mypy detects the fact that the object doesn't have __setattr__ and
complains. One could add a __setattr__ to the proxy type and mypy
is satisfied.

2. We can just suppress the type check with the comment.

Because I have no idea why the _MgrProxy exists or why it's implemented
the way it is, I feel that 2 is simpler. It is easy enough to go back
later and clean up the comments rather than me investing a lot of time
to understand the dashboard's approach just to bump up the version of
mypy.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
e767b8643a mgr/dashboard: ignore type checking for exception handling module
Add a `# type: ignore` comment to the exception handling dashboard
module just like the instance to lines below. This module does not
already import typing, so I'm not going to add it.
This change is needed in order to run mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
bc65da0acd pybind/mgr: add a type annotation to the extra_args dict
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
e1df7d939f mgr/dashboard: add an assertion for type narrowing
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
de9e8094aa mgr/cephadm: add a type annotation to the ganesha conf template context
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
Laura Flores
8bf4784c5a
Merge pull request #50794 from ljflores/wip-crontab
qa/crontab: check older builds on teuthology/nop when necessary
2023-03-30 15:42:28 -05:00
Adam King
c36a07d425
Merge pull request #49824 from adk3798/iscsi-etc-hosts
cephadm: mount host /etc/hosts for daemon containers in podman deployments

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-03-30 12:24:32 -04:00
Adam King
0022f3f952
Merge pull request #50328 from adk3798/maintenance-mode-force
mgr/cephadm: add more aggressive force flag for host maintenance enter

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 12:20:58 -04:00
Adam King
7f2dec6312
Merge pull request #50547 from adk3798/ssh-denied-error-handling
mgr/cephadm: handle HostConnectionError when checking for valid addr

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 12:18:53 -04:00
Adam King
9f5af55ac2
Merge pull request #50607 from adk3798/keepalive-2-2-4
cephadm: bump default keepalived image to 2.2.4

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 12:16:29 -04:00
Adam King
5eee6d0b77
Merge pull request #50655 from rkachach/fix_issue_58158
mgr/cephadm: increasing container stop timeout for OSDs

Reviewed-by: Adam King <adking@redhat.com>
2023-03-30 12:14:33 -04:00
Adam King
70b53ba4b8
Merge pull request #50685 from rkachach/fix_issue_59170
mgr/cephadm: fixing ceph-exporter prometheus's job section

Reviewed-by: avanthakkar <avanjohn@gmail.com>
2023-03-30 11:54:52 -04:00
Laura Flores
7f6c3f393e qa/crontab: check older builds on teuthology/nop when necessary
Today's scheduled run failed since the newest build of main
had failed. If we add `-n 10` to the command, this will
make it so we start at the newest build and backtrack
up to 10 older builds if necessary.

A higher number than that is not necessary, as the suite
failing to run will signal to us that more than the last
10 main builds are broken in Shaman.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-03-30 10:38:14 -05:00
Redouane Kachach
ef958d47b4
mgr/cephadm: use a dedicated cephadm tmp dir to copy remote files
Fixes: https://tracker.ceph.com/issues/59189

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 17:17:53 +02:00
Casey Bodley
3c0d618311 test/rgw: ceph_test_librgw_file target also builds associated tests
this just makes it easier to build all of the tests needed by the
test_librgw_file.sh workunit

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-03-30 11:13:42 -04:00
Yaarit Hatuka
e73d94820b qa/tasks/mgr/dashboard: add leaderboard_description to api tests
leaderboard_description is a new telemetry module option.

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2023-03-30 15:05:44 +00:00
Matt Benjamin
de776bcc31
Merge pull request #50684 from yuvalif/wip-yuval-fix-59136
rgw/notifications: support bucket notification with bucket policy
2023-03-30 10:25:16 -04:00
dparmar18
310286fa18 qa: fix issue with fn unable to fetch port and ip
_get_port_ip_info() fails to fetch port and ip due to empty 'backend' key:

2023-02-24T20:49:09.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph nfs cluster info test
2023-02-24T20:49:09.471 INFO:teuthology.orchestra.run.smithi042.stdout:{
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  "test": {
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "backend": [],
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "virtual_ip": null
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  }
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:}

it then raises:

2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:    info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:IndexError: list index out of range

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
503f8679ed qa: fix helper function _check_nfs_cluster_status()
Comment in the code says to wait for two minutes as cluster
creation takes time but actually it's waiting for thirteen
minutes, it's not required to wait this long, i think a minute
here is more than enough, also switched to using safe_while().

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
3b3fa71edf qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
Fixes: https://tracker.ceph.com/issues/58758
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
7cb3e58ecb qa: fix cluster creation failure in test_nfs.py
Also adds a function _nfs_complete_cmd() that returns process obj so that stdout/stderr
can be used for evaluation(_nfs_cmd() uses raw_cluster_cmd() that returns just stdout
and it became difficult to time cluster creation errors in _test_create_cluster()).

It takes sometime to update the cluster data, therefore running the command set
(check nfs server status -> nfs cluster create test -> check cluster status) in
a loop (max six iteration with sleep of 5 secs at each iteration) fixes the issue.

Fixes: https://tracker.ceph.com/issues/58744
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
afaf251d21 qa: test export creation at filepath and symlink
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
b0bb25024d qa: added test case test_nfs_export_with_invalid_path
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
992c686a3d mgr/nfs: disallow non-existent paths when creating export
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
Dhairya Parmar
6334e10fa5 mgr/nfs/tests: mock check_cephfs_path
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
Casey Bodley
c0186fe5eb
Merge pull request #45693 from cbodley/wip-qa-rgw-scrub
qa/rgw: reenable wait-for-scrub

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2023-03-30 08:50:30 -04:00
liangmingyuan
1c0162f9e9 osd/cls_rgw: fix the duplicated record in cls_method_handle_t
Though the handle_t is not used, it looks odds when duplicated.

Signed-off-by: Mingyuan Liang <liangmingyuan@baidu.com>
2023-03-30 20:23:54 +08:00
Venky Shankar
aa9a83029a Merge PR #47649 into main
* refs/pull/47649/head:
	mds: adjust MDSRank::command_tag_path invocation of enqueue_scrub()
	doc/scrub: documented stray evaluation using recursive scrub
	qa: added testcases
	mds: make `scrub status` print flag `scrub_mdsdir`
	mds: add scrub_mdsdir to ScrubHeader
	mds: do not dump multiple JSON obj
	mds: evaluate strays while performing scrub on root path
	mds: remove inode from scrub_stack if being purged
	mds: do not scrub inode if it is purging

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-30 16:13:48 +05:30
Venky Shankar
e84f319c66 Merge PR #50053 into main
* refs/pull/50053/head:
	libcephfs: move ClearSetuid to suidsgid.cc
	libcephfs: add test cases for dropping the suid/sgid in write/truncate
	libcephfs: add test cases for dropping the suid/sgid in fallocate
	libcephfs: fix ClearSetuid incorrectly using SETATTR_MODE mask
	client: switch to clear_suid_sgid for ftruncate
	client: switch to clear_suid_sgid for _write()
	mds/client: clear the suid/sgid in fallocate path
	client: allow unprivileged users to clear suid/sgid

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2023-03-30 14:48:26 +05:30
Rishabh Dave
96f4886dc6
Merge pull request #50687 from rishabh-d-dave/fs-qa-kernel-untar-tar
qa/workunit/fs: print commands for making debugging easier

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-30 14:43:49 +05:30
Anthony D'Atri
880b0fcca1
Merge pull request #50764 from zdover23/wip-doc-2023-03-230-start-documenting-ceph-squash-formatting
doc/start: format procedure in documenting-ceph
2023-03-30 00:59:11 -04:00
Patrick Donnelly
3c434884ce
Merge PR #50692 into main
* refs/pull/50692/head:
	tools/cephfs: include lost+found in scan_links

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-29 23:01:33 -04:00
Tongliang Deng
6d9e4f7924 rgw/sse-s3: fix bucket encryption of multipart upload
Multipart upload missing encryption when we have bucket encryption
policy. Fix it by fetching bucket encryption policy and resolving
defaults at multipart init op.

Fixes: https://tracker.ceph.com/issues/59218

Signed-off-by: Tongliang Deng <dengtongliang@gmail.com>
2023-03-30 10:13:15 +08:00
Zac Dover
cb1e20f879 doc/rados/ops: edit health-checks.rst (1 of x)
Edit docs/rados/operations/health-checks.rst (1 of x).

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-03-30 11:28:33 +10:00
zdover23
8dacbe191b
Merge pull request #50742 from zdover23/wip-doc-2023-03-29-start-documenting-ceph-notify-us
doc/start: update "notify us" section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-30 11:01:29 +10:00
Zac Dover
6f3cd78f61 doc/start: format procedure in documenting-ceph
Organize three steps in a procedure into one step with three sub-steps.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-03-30 10:54:52 +10:00
Anthony D'Atri
b6bf70a1cf
Merge pull request #50744 from zdover23/wip-doc-2023-03-29-rados-ops-health-checks
doc/rados: add link to ops/health-checks.rst
2023-03-29 17:53:29 -04:00
Patrick Donnelly
0c4d835148
Merge PR #49773 into main
* refs/pull/49773/head:
	mds: add config to decide whether to mark dentry bad
	qa: add missing scan_links step for data scan recovery
	qa/tasks/cephfs: test damage to dentry's first is caught
	qa/tasks/cephfs: use rank_asok and allow specifying rank
	qa/tasks: allow specifying timeout command prefix to ceph
	mds: provide test configs for creating first corruption
	mds: catch damage to dentry's first field
	mds: add debugging for pre_cow_old_inode
	mds: cleanup code

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2023-03-29 16:15:47 -04:00
Dhairya Parmar
1356992c1c mgr/nfs/utils: add helper func to check cephfs path
this helper instantiates CephfsClient, however this was
initially planned in ExportMgr class in export.py but
due to make check failure where main python thread
experienced a dead lock which after several efforts
pointed at instantiation of CephfsClient in ExportMgr
was problematic, it was decided in order to achieve
singleton behavior, func has been added inside this
helper func that restricts instantiation using functool's
lru_cache.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 01:00:22 +05:30
zdover23
c3f1eeebbd
Merge pull request #50750 from Sainithin-bit/main
doc/rados/operations: Fix typo in erasure-code.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-03-30 05:09:15 +10:00
SAINITHIN.ARTHAM
3269fea2a2 doc/rados/operations: Fix typo in erasure-code.rst
Signed-off-by: Sainithin Artham <sai.artham.19cse@bmu.edu.in>
2023-03-30 00:05:43 +05:30
Casey Bodley
963c5a0575
Merge pull request #50633 from cbodley/wip-54102
qa/rgw: unpin centos for verify suite

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-03-29 14:16:15 -04:00
Patrick Donnelly
7ffa065a03
mds: add config to decide whether to mark dentry bad
So admin can restore access to files if necessary.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
1b4d9077af
qa: add missing scan_links step for data scan recovery
Without, the first field remains corrupt (HEAD).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
293b90fe63
qa/tasks/cephfs: test damage to dentry's first is caught
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
3ba17390e9
qa/tasks/cephfs: use rank_asok and allow specifying rank
This will use the more efficient:

    ceph tell mds.<fsname>:<rank> ...

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
dbf1a402c9
qa/tasks: allow specifying timeout command prefix to ceph
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:55 -04:00
Patrick Donnelly
fe258dc54b
mds: provide test configs for creating first corruption
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:55 -04:00
Patrick Donnelly
03407528f9
mds: catch damage to dentry's first field
When possible. Abort the MDS before it can be written to the
journal/directory.

This is part of a series to address corruption first observed in [1].
How the corruption is introduced is yet unknown.

[1] https://tracker.ceph.com/issues/38452#note-10

Fixes: http://tracker.ceph.com/issues/58482
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:55 -04:00
Patrick Donnelly
c9d36056c7
mds: add debugging for pre_cow_old_inode
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:33 -04:00