Commit Graph

137239 Commits

Author SHA1 Message Date
Ville Ojamo
d5343396aa
doc/cephfs: refer to MDS and not NFS in fs-volumes.rst
Placement should talk about MDS placement and not NFS Ganesha placement.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2023-03-31 19:01:39 +07:00
Ilya Dryomov
e1c5ea5420
Merge pull request #50793 from petrutlucian94/allperms
include: move ALLPERMS definition to compat.h

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-03-31 10:54:45 +02:00
Lucian Petrut
a2966c0817 include: move ALLPERMS definition to compat.h
The Windows CI job started to fail as some libcephfs tests that use
ALLPERMS have been moved [1] to a separate file which doesn't have
the ALLPERMS definition.

  /ceph/src/test/libcephfs/suidsgid.cc:240:36: error: ‘ALLPERMS’ was
  not declared in this scope
  240 |   ASSERT_EQ(stx.stx_mode & (mode_t)ALLPERMS, before_mode);

We'll move this definition to compat.h so that we won't have to
redefine it in each file that uses it.

Note that we're moving the Windows "fs_compat.h" include up,
ensuring that the constants used by ALLPERMS are defined.

[1] 254eeb2730

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-03-31 07:09:37 +00:00
Guillaume Abrioux
b538d6adb6
Merge pull request #49586 from mohan7427/mohan/batch_args-issue
ceph-volume: fix drive-group issue that expects the batch_args to be a string
2023-03-31 09:07:43 +02:00
zdover23
60195779a7
Merge pull request #50554 from zdover23/wip-doc-2023-03-16-rados-operations-pg-repair
doc/rados: line-edit pg-repair.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-31 16:45:06 +10:00
zdover23
f66bdb87f7
Merge pull request #45354 from dvanders/dualdoc
doc/msgr2: update dual stack status

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-03-31 16:25:31 +10:00
Nizamudeen A
7a7ab80e42
Merge pull request #50426 from rhcs-dashboard/role-create
mgr/dashboard: creation of rgw roles

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-03-31 10:42:09 +05:30
Venky Shankar
365895fb2d Merge PR #49460 into main
* refs/pull/49460/head:
	qa: fix issue with fn unable to fetch port and ip
	qa: fix helper function _check_nfs_cluster_status()
	qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
	qa: fix cluster creation failure in test_nfs.py
	qa: test export creation at filepath and symlink
	qa: added test case test_nfs_export_with_invalid_path
	mgr/nfs: disallow non-existent paths when creating export
	mgr/nfs/tests: mock check_cephfs_path
	mgr/nfs/utils: add helper func to check cephfs path

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-03-31 09:32:37 +05:30
zdover23
37280a387d
Merge pull request #50674 from zdover23/wip-doc-2023-03-26-rados-operations-health-checks-1-of-5
doc/rados/ops: edit health-checks.rst (1 of 5)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-31 09:38:03 +10:00
Laura Flores
8bf4784c5a
Merge pull request #50794 from ljflores/wip-crontab
qa/crontab: check older builds on teuthology/nop when necessary
2023-03-30 15:42:28 -05:00
Adam King
c36a07d425
Merge pull request #49824 from adk3798/iscsi-etc-hosts
cephadm: mount host /etc/hosts for daemon containers in podman deployments

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-03-30 12:24:32 -04:00
Adam King
0022f3f952
Merge pull request #50328 from adk3798/maintenance-mode-force
mgr/cephadm: add more aggressive force flag for host maintenance enter

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 12:20:58 -04:00
Adam King
7f2dec6312
Merge pull request #50547 from adk3798/ssh-denied-error-handling
mgr/cephadm: handle HostConnectionError when checking for valid addr

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 12:18:53 -04:00
Adam King
9f5af55ac2
Merge pull request #50607 from adk3798/keepalive-2-2-4
cephadm: bump default keepalived image to 2.2.4

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 12:16:29 -04:00
Adam King
5eee6d0b77
Merge pull request #50655 from rkachach/fix_issue_58158
mgr/cephadm: increasing container stop timeout for OSDs

Reviewed-by: Adam King <adking@redhat.com>
2023-03-30 12:14:33 -04:00
Adam King
70b53ba4b8
Merge pull request #50685 from rkachach/fix_issue_59170
mgr/cephadm: fixing ceph-exporter prometheus's job section

Reviewed-by: avanthakkar <avanjohn@gmail.com>
2023-03-30 11:54:52 -04:00
Laura Flores
7f6c3f393e qa/crontab: check older builds on teuthology/nop when necessary
Today's scheduled run failed since the newest build of main
had failed. If we add `-n 10` to the command, this will
make it so we start at the newest build and backtrack
up to 10 older builds if necessary.

A higher number than that is not necessary, as the suite
failing to run will signal to us that more than the last
10 main builds are broken in Shaman.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-03-30 10:38:14 -05:00
Matt Benjamin
de776bcc31
Merge pull request #50684 from yuvalif/wip-yuval-fix-59136
rgw/notifications: support bucket notification with bucket policy
2023-03-30 10:25:16 -04:00
dparmar18
310286fa18 qa: fix issue with fn unable to fetch port and ip
_get_port_ip_info() fails to fetch port and ip due to empty 'backend' key:

2023-02-24T20:49:09.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph nfs cluster info test
2023-02-24T20:49:09.471 INFO:teuthology.orchestra.run.smithi042.stdout:{
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  "test": {
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "backend": [],
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "virtual_ip": null
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  }
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:}

it then raises:

2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:    info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:IndexError: list index out of range

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
503f8679ed qa: fix helper function _check_nfs_cluster_status()
Comment in the code says to wait for two minutes as cluster
creation takes time but actually it's waiting for thirteen
minutes, it's not required to wait this long, i think a minute
here is more than enough, also switched to using safe_while().

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
3b3fa71edf qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
Fixes: https://tracker.ceph.com/issues/58758
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
7cb3e58ecb qa: fix cluster creation failure in test_nfs.py
Also adds a function _nfs_complete_cmd() that returns process obj so that stdout/stderr
can be used for evaluation(_nfs_cmd() uses raw_cluster_cmd() that returns just stdout
and it became difficult to time cluster creation errors in _test_create_cluster()).

It takes sometime to update the cluster data, therefore running the command set
(check nfs server status -> nfs cluster create test -> check cluster status) in
a loop (max six iteration with sleep of 5 secs at each iteration) fixes the issue.

Fixes: https://tracker.ceph.com/issues/58744
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
afaf251d21 qa: test export creation at filepath and symlink
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
b0bb25024d qa: added test case test_nfs_export_with_invalid_path
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
dparmar18
992c686a3d mgr/nfs: disallow non-existent paths when creating export
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
Dhairya Parmar
6334e10fa5 mgr/nfs/tests: mock check_cephfs_path
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 19:29:15 +05:30
Casey Bodley
c0186fe5eb
Merge pull request #45693 from cbodley/wip-qa-rgw-scrub
qa/rgw: reenable wait-for-scrub

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2023-03-30 08:50:30 -04:00
Venky Shankar
aa9a83029a Merge PR #47649 into main
* refs/pull/47649/head:
	mds: adjust MDSRank::command_tag_path invocation of enqueue_scrub()
	doc/scrub: documented stray evaluation using recursive scrub
	qa: added testcases
	mds: make `scrub status` print flag `scrub_mdsdir`
	mds: add scrub_mdsdir to ScrubHeader
	mds: do not dump multiple JSON obj
	mds: evaluate strays while performing scrub on root path
	mds: remove inode from scrub_stack if being purged
	mds: do not scrub inode if it is purging

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-30 16:13:48 +05:30
Venky Shankar
e84f319c66 Merge PR #50053 into main
* refs/pull/50053/head:
	libcephfs: move ClearSetuid to suidsgid.cc
	libcephfs: add test cases for dropping the suid/sgid in write/truncate
	libcephfs: add test cases for dropping the suid/sgid in fallocate
	libcephfs: fix ClearSetuid incorrectly using SETATTR_MODE mask
	client: switch to clear_suid_sgid for ftruncate
	client: switch to clear_suid_sgid for _write()
	mds/client: clear the suid/sgid in fallocate path
	client: allow unprivileged users to clear suid/sgid

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2023-03-30 14:48:26 +05:30
Rishabh Dave
96f4886dc6
Merge pull request #50687 from rishabh-d-dave/fs-qa-kernel-untar-tar
qa/workunit/fs: print commands for making debugging easier

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-30 14:43:49 +05:30
Anthony D'Atri
880b0fcca1
Merge pull request #50764 from zdover23/wip-doc-2023-03-230-start-documenting-ceph-squash-formatting
doc/start: format procedure in documenting-ceph
2023-03-30 00:59:11 -04:00
Patrick Donnelly
3c434884ce
Merge PR #50692 into main
* refs/pull/50692/head:
	tools/cephfs: include lost+found in scan_links

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-29 23:01:33 -04:00
Zac Dover
cb1e20f879 doc/rados/ops: edit health-checks.rst (1 of x)
Edit docs/rados/operations/health-checks.rst (1 of x).

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-03-30 11:28:33 +10:00
zdover23
8dacbe191b
Merge pull request #50742 from zdover23/wip-doc-2023-03-29-start-documenting-ceph-notify-us
doc/start: update "notify us" section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-30 11:01:29 +10:00
Zac Dover
6f3cd78f61 doc/start: format procedure in documenting-ceph
Organize three steps in a procedure into one step with three sub-steps.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-03-30 10:54:52 +10:00
Anthony D'Atri
b6bf70a1cf
Merge pull request #50744 from zdover23/wip-doc-2023-03-29-rados-ops-health-checks
doc/rados: add link to ops/health-checks.rst
2023-03-29 17:53:29 -04:00
Patrick Donnelly
0c4d835148
Merge PR #49773 into main
* refs/pull/49773/head:
	mds: add config to decide whether to mark dentry bad
	qa: add missing scan_links step for data scan recovery
	qa/tasks/cephfs: test damage to dentry's first is caught
	qa/tasks/cephfs: use rank_asok and allow specifying rank
	qa/tasks: allow specifying timeout command prefix to ceph
	mds: provide test configs for creating first corruption
	mds: catch damage to dentry's first field
	mds: add debugging for pre_cow_old_inode
	mds: cleanup code

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2023-03-29 16:15:47 -04:00
Dhairya Parmar
1356992c1c mgr/nfs/utils: add helper func to check cephfs path
this helper instantiates CephfsClient, however this was
initially planned in ExportMgr class in export.py but
due to make check failure where main python thread
experienced a dead lock which after several efforts
pointed at instantiation of CephfsClient in ExportMgr
was problematic, it was decided in order to achieve
singleton behavior, func has been added inside this
helper func that restricts instantiation using functool's
lru_cache.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-30 01:00:22 +05:30
zdover23
c3f1eeebbd
Merge pull request #50750 from Sainithin-bit/main
doc/rados/operations: Fix typo in erasure-code.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-03-30 05:09:15 +10:00
SAINITHIN.ARTHAM
3269fea2a2 doc/rados/operations: Fix typo in erasure-code.rst
Signed-off-by: Sainithin Artham <sai.artham.19cse@bmu.edu.in>
2023-03-30 00:05:43 +05:30
Casey Bodley
963c5a0575
Merge pull request #50633 from cbodley/wip-54102
qa/rgw: unpin centos for verify suite

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-03-29 14:16:15 -04:00
Patrick Donnelly
7ffa065a03
mds: add config to decide whether to mark dentry bad
So admin can restore access to files if necessary.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
1b4d9077af
qa: add missing scan_links step for data scan recovery
Without, the first field remains corrupt (HEAD).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
293b90fe63
qa/tasks/cephfs: test damage to dentry's first is caught
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
3ba17390e9
qa/tasks/cephfs: use rank_asok and allow specifying rank
This will use the more efficient:

    ceph tell mds.<fsname>:<rank> ...

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:56 -04:00
Patrick Donnelly
dbf1a402c9
qa/tasks: allow specifying timeout command prefix to ceph
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:55 -04:00
Patrick Donnelly
fe258dc54b
mds: provide test configs for creating first corruption
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:55 -04:00
Patrick Donnelly
03407528f9
mds: catch damage to dentry's first field
When possible. Abort the MDS before it can be written to the
journal/directory.

This is part of a series to address corruption first observed in [1].
How the corruption is introduced is yet unknown.

[1] https://tracker.ceph.com/issues/38452#note-10

Fixes: http://tracker.ceph.com/issues/58482
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:55 -04:00
Patrick Donnelly
c9d36056c7
mds: add debugging for pre_cow_old_inode
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:33 -04:00
Patrick Donnelly
17d826551c
mds: cleanup code
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-03-29 13:02:33 -04:00