Commit Graph

137425 Commits

Author SHA1 Message Date
Soumya Koduri
aeb3a74618 rgw/sync-policy: Support disabling per-bucket replication
When the zones replicate, allow disabling replication
for specific buckets using sync policy.

These are the semantics to be followed while resolving the policy
conflicts -

==================================================
zonegroup		bucket		Result
==================================================
enabled			enabled		enabled
			allowed		enabled
			forbidden	disabled
allowed			enabled		enabled
			allowed		disabled
			forbidden	disabled
forbidden		enabled		disabled
			allowed		disabled
			forbidden	disabled

In case multiple group policies are set to reflect for any sync pair
(<source-zone,source-bucket>, <dest-zone,dest-bucket>), the following
rules are applied in the order -
1) Even if one policy status is FORBIDDEN, the sync will be disabled
2) Atleast one policy should be ENABLED for the sync to be allowed.

Various cases tested are outlined here -
https://docs.google.com/document/d/19oBQA-bYxLBR4BnekA2DTwJJaTFvjAfrqAk9G3RGU0I/edit#heading=h.4qac9dpc76m

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-04-03 19:44:57 +05:30
Mark Kogan
5846a9c267 rgw : fix python script using s3cmd with error code 403 ubuntu 20.04
Fixes: https://tracker.ceph.com/issues/54104

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2023-04-03 15:18:55 +03:00
Zhang Song
0740e21e32 crimson/os/seastore: ensure transaction commit order
The previous implementation acquired the throttler first,
which could result in an unordered wakeup sequence.

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2023-04-03 19:15:21 +08:00
Ville Ojamo
e5355e3d66 doc/cephfs: fix prompts in fs-volumes.rst
Modernize prompts in example commands in file
fs-volume.rst and use a root prompt correctly instead
of using a non-privileged prompt when privileges
are needed.

Indent all JSON in example command outputs with
four spaces consistently instead of a mix of four and
two spaces.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2023-04-02 16:36:44 +07:00
Anthony D'Atri
66552a560f
Merge pull request #50818 from anthonyeleven/anthonyeleven/tweak-fs-volumes.rst
doc/cephfs: Improve fs-volumes.rst
2023-04-02 01:47:36 -04:00
Anthony D'Atri
c8e1f3ac3d doc/cephfs: Improve fs-volumes.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2023-04-01 01:05:20 -04:00
zdover23
b682861f86
Merge pull request #50670 from zdover23/wip-doc-2023-03-26-rados-operations-monitoring-1-of-3
doc/rados: edit ops/monitoring.rst (1 of 3)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-04-01 05:48:39 +10:00
Zac Dover
bc5f404a78 doc/rados: edit ops/monitoring.rst (1 of 3)
Line-edit the first third of doc/rados/operations/monitoring.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-04-01 05:23:01 +10:00
Rishabh Dave
989f7ecee5
Merge pull request #46905 from lxbsz/wip-quota
mds: align quota.max_bytes to 4MB or 4KB

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-04-01 00:12:19 +05:30
Rishabh Dave
75b12e4c00
Merge pull request #49882 from zhsgao/dump_resolve_status
mds: fix bug of dump_resolve_status

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-04-01 00:10:05 +05:30
Rishabh Dave
0e1e26e27a
Merge pull request #50024 from kotreshhr/mds-unused-function-removal
mds: Remove unused declaration of the function

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-04-01 00:06:49 +05:30
Casey Bodley
6a46f9f2e9
Merge pull request #50746 from cbodley/wip-rgw-bucket-full-sync-compat
rgw: set init_check_compat when bucket sync status doesn't exist

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2023-03-31 14:34:42 -04:00
Ali Maredia
1d1672e854
Merge pull request #50644 from cbodley/wip-59135
qa/rgw: upgrade tests restrict supported distros
2023-03-31 14:27:26 -04:00
Pedro Gonzalez Gomez
dae8168e29 mgr/dashboard: delete rgw multisite
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-03-31 20:14:42 +02:00
Casey Bodley
01096e840a
Merge pull request #50690 from cbodley/wip-59174
rgw/admin: 'data sync status' formats binary error repo entries

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-03-31 13:48:52 -04:00
Casey Bodley
df21f6d37b
Merge pull request #50462 from matutter/main
sts: Fixes get_cert_url improper url path concatenation

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
2023-03-31 13:47:48 -04:00
Casey Bodley
12f9cb5e88
Merge pull request #50623 from VVoidV/fix-rgw_admin
[RGW]fix minor bug in rgw_admin

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2023-03-31 13:47:18 -04:00
Casey Bodley
8fb26b219d
Merge pull request #50648 from cbodley/wip-58035
rgw: fix CopyObj crash after admin override

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2023-03-31 13:43:01 -04:00
Anthony D'Atri
cf40777309
Merge pull request #50817 from bluikko/patch-10
doc/cephfs: refer to placement of MDS and not NFS in fs-volumes.rst
2023-03-31 09:18:15 -04:00
Ville Ojamo
d5343396aa
doc/cephfs: refer to MDS and not NFS in fs-volumes.rst
Placement should talk about MDS placement and not NFS Ganesha placement.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2023-03-31 19:01:39 +07:00
Ilya Dryomov
e11097bc4c librbd: clear Image::list_watchers() list before populating it
The "append to the passed list" behavior is confusing and not what the
corresponding C API (rbd_watchers_list) or other similar C++ APIs (e.g.
list_lockers) do.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-03-31 12:23:45 +02:00
Ilya Dryomov
e1c5ea5420
Merge pull request #50793 from petrutlucian94/allperms
include: move ALLPERMS definition to compat.h

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-03-31 10:54:45 +02:00
Lucian Petrut
a2966c0817 include: move ALLPERMS definition to compat.h
The Windows CI job started to fail as some libcephfs tests that use
ALLPERMS have been moved [1] to a separate file which doesn't have
the ALLPERMS definition.

  /ceph/src/test/libcephfs/suidsgid.cc:240:36: error: ‘ALLPERMS’ was
  not declared in this scope
  240 |   ASSERT_EQ(stx.stx_mode & (mode_t)ALLPERMS, before_mode);

We'll move this definition to compat.h so that we won't have to
redefine it in each file that uses it.

Note that we're moving the Windows "fs_compat.h" include up,
ensuring that the constants used by ALLPERMS are defined.

[1] 254eeb2730

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-03-31 07:09:37 +00:00
Guillaume Abrioux
b538d6adb6
Merge pull request #49586 from mohan7427/mohan/batch_args-issue
ceph-volume: fix drive-group issue that expects the batch_args to be a string
2023-03-31 09:07:43 +02:00
zdover23
60195779a7
Merge pull request #50554 from zdover23/wip-doc-2023-03-16-rados-operations-pg-repair
doc/rados: line-edit pg-repair.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-31 16:45:06 +10:00
zdover23
f66bdb87f7
Merge pull request #45354 from dvanders/dualdoc
doc/msgr2: update dual stack status

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-03-31 16:25:31 +10:00
lichaochao
f9aae71af3 rgw: fix rgw cache invalidation after unregister_watch() error
When a metadata osd fails, an unregister_watch() error may occur,
resulting in an rgw cache invalidation.
By adding an unregister_done flag and when a register_watch() error ,
performing a reinit() operation again,
After the first reinit() failure, the register_watch() will be performed again

Fixes: https://tracker.ceph.com/issues/59217

Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
2023-03-31 07:54:50 +02:00
Redouane Kachach
2c46c07419
mgr/cephadm: Adding extra arguments support for RGW frontend
Fixes: https://tracker.ceph.com/issues/57931

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2023-03-31 07:27:04 +02:00
Nizamudeen A
7a7ab80e42
Merge pull request #50426 from rhcs-dashboard/role-create
mgr/dashboard: creation of rgw roles

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-03-31 10:42:09 +05:30
Venky Shankar
365895fb2d Merge PR #49460 into main
* refs/pull/49460/head:
	qa: fix issue with fn unable to fetch port and ip
	qa: fix helper function _check_nfs_cluster_status()
	qa: fix testcase 'test_cluster_set_user_config_with_non_existing_clusterid'
	qa: fix cluster creation failure in test_nfs.py
	qa: test export creation at filepath and symlink
	qa: added test case test_nfs_export_with_invalid_path
	mgr/nfs: disallow non-existent paths when creating export
	mgr/nfs/tests: mock check_cephfs_path
	mgr/nfs/utils: add helper func to check cephfs path

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-03-31 09:32:37 +05:30
zdover23
37280a387d
Merge pull request #50674 from zdover23/wip-doc-2023-03-26-rados-operations-health-checks-1-of-5
doc/rados/ops: edit health-checks.rst (1 of 5)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-31 09:38:03 +10:00
Christopher Hoffman
6a0aeadc31 librbd: Propagate EBLOCKLIST in send_acquire_lock
During send_acquire_lock, there's a case where
there's no watcher handle present and lock request is delayed.
If the client is blocklisted, the delayed request will not
continue and the call that requested lock will never complete.

The lock process will now propagate -EBLOCKLIST, to callback
instead of indefinitely delaying.

Fixes: https://tracker.ceph.com/issues/59115
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
2023-03-30 21:25:56 +00:00
John Mulligan
ff39f87701 python-common: add a dedicated tox env to run mypy
IMO it's not a good practice to overload a tox rule with multiple
different test tools. It forces the tools to share the same virtualenvs
and makes it impossible to run the tools individually. A separate mypy
env also better matches the other tox.ini files in the ceph tree.
Since the new 'mypy' env is in the default env list it will continue
to get run automatically when no specific envs are selected.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
866f14d40c mypy: update pinned mypy version to 0.981
mypy version 0.981 fixes a bug where on newer python versions mypy
doesn't properly load pyi files with keyword only arguments.
As noted in the src/mypy-constrains.txt mypy version needs to be
manually bumped periodically, and ceph is overdue for an update too.
It's never been updated since the file was added in June 2021.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
f2646dbaba python-common: fix variable name reuse to make mypy happy
The variables high and low were being used as both `str`s and regex
match objects. Rename the vars in the if block to avoid this problem.
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
3035ca6c52 src/pybind: fix type annotations for signal handler function
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
84a02d63ef cephadm: fix type annotation for list containing IP addresses
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
8042e3cefb mgr/dashboard: ignore type checking on mgr proxy object assignments
Add `# type: ignore` comments to two dashboard functions that attempt
to set manager properties. There appear to be two approaches to fixing
the problem:

1. The _MgrProxy object that the dashboard uses has a __getattr__ method
for pulling value from the underlying mgr object. It does not have a
__setattr__ method. This means the setting values on _MgrProxy do not
propogate down to the original mgr.
mypy detects the fact that the object doesn't have __setattr__ and
complains. One could add a __setattr__ to the proxy type and mypy
is satisfied.

2. We can just suppress the type check with the comment.

Because I have no idea why the _MgrProxy exists or why it's implemented
the way it is, I feel that 2 is simpler. It is easy enough to go back
later and clean up the comments rather than me investing a lot of time
to understand the dashboard's approach just to bump up the version of
mypy.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
e767b8643a mgr/dashboard: ignore type checking for exception handling module
Add a `# type: ignore` comment to the exception handling dashboard
module just like the instance to lines below. This module does not
already import typing, so I'm not going to add it.
This change is needed in order to run mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
bc65da0acd pybind/mgr: add a type annotation to the extra_args dict
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
e1df7d939f mgr/dashboard: add an assertion for type narrowing
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
John Mulligan
de9e8094aa mgr/cephadm: add a type annotation to the ganesha conf template context
This change makes this file pass mypy checking on mypy 0.981.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 16:52:53 -04:00
Laura Flores
8bf4784c5a
Merge pull request #50794 from ljflores/wip-crontab
qa/crontab: check older builds on teuthology/nop when necessary
2023-03-30 15:42:28 -05:00
Adam King
c36a07d425
Merge pull request #49824 from adk3798/iscsi-etc-hosts
cephadm: mount host /etc/hosts for daemon containers in podman deployments

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-03-30 12:24:32 -04:00
Adam King
0022f3f952
Merge pull request #50328 from adk3798/maintenance-mode-force
mgr/cephadm: add more aggressive force flag for host maintenance enter

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-03-30 12:20:58 -04:00
Adam King
7f2dec6312
Merge pull request #50547 from adk3798/ssh-denied-error-handling
mgr/cephadm: handle HostConnectionError when checking for valid addr

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 12:18:53 -04:00
Adam King
9f5af55ac2
Merge pull request #50607 from adk3798/keepalive-2-2-4
cephadm: bump default keepalived image to 2.2.4

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-03-30 12:16:29 -04:00
Adam King
5eee6d0b77
Merge pull request #50655 from rkachach/fix_issue_58158
mgr/cephadm: increasing container stop timeout for OSDs

Reviewed-by: Adam King <adking@redhat.com>
2023-03-30 12:14:33 -04:00
Adam King
70b53ba4b8
Merge pull request #50685 from rkachach/fix_issue_59170
mgr/cephadm: fixing ceph-exporter prometheus's job section

Reviewed-by: avanthakkar <avanjohn@gmail.com>
2023-03-30 11:54:52 -04:00
Laura Flores
7f6c3f393e qa/crontab: check older builds on teuthology/nop when necessary
Today's scheduled run failed since the newest build of main
had failed. If we add `-n 10` to the command, this will
make it so we start at the newest build and backtrack
up to 10 older builds if necessary.

A higher number than that is not necessary, as the suite
failing to run will signal to us that more than the last
10 main builds are broken in Shaman.

Signed-off-by: Laura Flores <lflores@redhat.com>
2023-03-30 10:38:14 -05:00