Commit Graph

135954 Commits

Author SHA1 Message Date
Kefu Chai
cf6089200d mgr/prometheus: use vendored "packaging" instead
instead of using the top-level "packaging" module, use the one
vendored by setuptools.

packaging python module provides versioning defined by PEP-440.
but python3-packaging is provided by CentOS8 powertools repo,
which is not enabled by default. and in CentOS9, this package
is provided by AppStream instead of BaseOS.

as prometheus mgr module is included by ceph-mgr-module-core,
it would be desirable if our user can install ceph-mgr-module-core
without enabling powertools or AppStream repo on a CentOS or
its derivative distros.

fortunately, setuptools vendors packaging module. and both
CentOS8 and CentOS9 provide python3-setuptools in their BaseOS
repos.

in this change, instead of using "packging" module, we use the
venderored one, which is in turn embedded in pkg_resources.
this python module is provided by python3-setuptools on CentOS
distros, and python3-pkg-resources on Debian and its derivatives.

the packaging recipes are updated accordingly to reflect the
new runtime dependency.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2023-01-11 20:31:03 +08:00
Aashish Sharma
f1727f028d mgr/dashboard: rgw server side encryption daemon name fix
The config values for the server side encryption currently are wrongly set to the mon daemon by default. This PR intends to fix this issue

Fixes: https://tracker.ceph.com/issues/58419
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2023-01-11 18:00:09 +05:30
Adam C. Emerson
04d3a4f068 rgw: SimpleRadosReadAttrsCR uses an async RADOS call
Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:46:45 -05:00
Adam C. Emerson
634673a64e rgw: Use cls_version for read/writes to global data sync status
The `radosgw-admin data sync init` command does *not* use
`cls_version` and just overwrites.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:37:41 -05:00
Adam C. Emerson
25ef265513 rgw: Reread sync status after acquiring lock in RGWDataSyncCR
If someone else got there first, we won't smash their work.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:35:58 -05:00
Adam C. Emerson
b0d401a0f4 rgw: Pull lock out of RGWInitDataSyncStatusCoroutine
RGWDataSyncCR manages the lock instead, holding it through StateInit
and StateBuildingFullSyncMaps but releasing it by StateSync.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:32:21 -05:00
Adam C. Emerson
2b3fbcf9b1 rgw: RGWContinuousLeaseCR takes arguments by value and moves them
Since we were taking them by reference and copying before, this is
strictly better. Callers that give us an RValue can skip the copy.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:26:29 -05:00
Adam C. Emerson
77deaa9b02 rgw: Get rid of some COOKIE_LEN preprocessor macros
Sticking random #defines everywhere is just atrocious style.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:26:27 -05:00
Yingxin
9ef2b8d2dd
Merge pull request #49686 from xxhdx1985126/wip-crimson-dump-metrics
crimson/admin: correct metrics dumping methods

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-01-11 13:44:23 +08:00
chunmei-liu
543b7d3334 crimson/os: make cyanstore not use proxy
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2023-01-10 21:40:00 -08:00
chunmei-liu
bc5876e3b9 crimson/os: make load_pgs() load pg on proper core
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2023-01-10 21:39:16 -08:00
chunmei-liu
7a58e875a5 crimson/os: support multicores for cyanstore
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-01-10 21:30:51 -08:00
Casey Bodley
113cda8834 rgw: add rgw_curl_tcp_keepalive option for http client requests
this can be useful to prevent long-lived connections from being dropped
due to inactivity

Fixes: https://tracker.ceph.com/issues/48402

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-01-11 00:23:40 -05:00
Shilpa Jagannath
852e60d6f3 rgw/multisite: clear objv before reading bucket sync status
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:21:27 -05:00
Soumya Koduri
e1c31b2997 rgw/bucket-sync: flush marker_tracker when the lease is lost
Also clear objv before reading the bucket sync status.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-01-11 00:17:57 -05:00
Shilpa Jagannath
d0725750b6 rgw/multisite: marker_tracker->finish() returns error
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:16:48 -05:00
Shilpa Jagannath
180d5d9783 rgw/multisite: if remote datalog is trimmed prematurely, sync status will report that it's behind the remote's max-marker even if there are no more entries to sync for each behind shard. if we get an empty listing, remove that shard from behind_shards.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:15:23 -05:00
Soumya Koduri
abc5883604 rgw/multisite: Update marker and objv after acquiring the lease
In RGWDataSyncShardCR, after acquiring the lease, reread sync status
shard object to fetch the latest marker & objv stored.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-01-11 00:13:59 -05:00
Shilpa Jagannath
9aeb2aa38d rgw/multisite: add cls versioning for tracking data sync per shard object and store it in a vector
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:13:56 -05:00
Samuel Just
68149e9c99 ceph.spec.in: remove the bcond_without cephfs_java option
This effectively disable cephfs_java for rpm builds for now.
The only user appears to be cephfs jni bindings.

Fixes: https://tracker.ceph.com/issues/58382
Signed-off-by: Samuel Just <sjust@redhat.com>
2023-01-10 20:06:06 -08:00
Xuehan Xu
702fe1ccc3 crimson/admin: correct metrics dumping methods
First level object has no key, so we have to inclose metric objects with
an outer one

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2023-01-11 02:30:06 +00:00
colemitchell
58f33d83bc
Merge pull request #49699 from zdover23/wip-doc-2023-01-11-cephadm-host-management-removing-hosts
doc/cephadm: refine "Removing Hosts"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-10 14:31:23 -05:00
colemitchell
1eb04cbc5e
Merge pull request #49703 from zdover23/wip-doc-2023-01-11-rados-operations-move-colon
doc/rados: move colon

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2023-01-10 14:24:11 -05:00
colemitchell
7fe7819c71
Merge pull request #49700 from zdover23/wip-doc-2023-01-11-css-h3-padding
doc/css: add top-bar padding for h3 html element

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2023-01-10 14:20:16 -05:00
Zac Dover
b1a53d7621 doc/rados: move colon
Move colon in add-or-rm-osds.rst so that the sentence reads properly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 05:08:35 +10:00
Zac Dover
c28cb53390 doc/css: add top-bar padding for h3 html element
Add "scroll-margin-top: 4em;" property to h3 html element.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 04:48:32 +10:00
Zac Dover
2e1fd63087 doc/cephadm: refine "Removing Hosts"
An intended edit to remove a redundant indefinite article became a
longer (but still brief) full editorial pass.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 04:29:02 +10:00
zdover23
7c9cfc9013
Merge pull request #49692 from zdover23/wip-doc-2023-01-11-header-fix
doc/css: add "span" padding to custom.css

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-11 04:10:22 +10:00
Ilya Dryomov
e55a6f618e
Merge pull request #49690 from idryomov/wip-58398
doc/man/ceph-rbdnamer: remove obsolete udev rule

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-01-10 18:44:28 +01:00
Zac Dover
dd9555b0ae doc/css: add "span" padding to custom.css
Add "scroll-top-bar: 2em;" for the "span" html element in custom.css so
that the top bar doesn't get in the way of headings bounded by the "span
element".

See also https://github.com/ceph/ceph/pull/49644.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 01:55:55 +10:00
Ilya Dryomov
8ca3bd5042 doc/man/ceph-rbdnamer: remove obsolete udev rule
Fixes: https://tracker.ceph.com/issues/58398
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-01-10 13:43:03 +01:00
Venky Shankar
9939dcc401 Merge PR #49630 into main
* refs/pull/49630/head:
	doc: document the relevance of mds_namespace mount option

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-01-10 16:41:10 +05:30
Jos Collin
86f8fa19e5
doc: document the relevance of mds_namespace mount option
Fixes: https://tracker.ceph.com/issues/57673
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-01-10 14:48:20 +05:30
Venky Shankar
6243f33d0b
Merge pull request #49664 from batrick/first-damage-repair-nosnap
tools/cephfs/first-damage: repair specific first=CEPH_NOSNAP damage

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-01-10 09:59:51 +05:30
zdover23
6c1dfe9e72
Merge pull request #49677 from zdover23/wip-doc-2023-01-10-rados-replacing-an-osd
doc/rados: link to cephadm replacing osd section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-10 10:45:39 +10:00
Adam C. Emerson
3493e044cb cls/fifo: Retry on push to nonexistent part
A racing client may delete the part we're trying to push to. Use
`assert_exists()` to check, and re-read metadata if we receive
-ENOENT.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 17:27:36 -05:00
Adam C. Emerson
bf222906b0 cls/fifo: Fix race condition on _prepare_new_head
First, make `_prepare_new_head` take the new head part number, so two
calls racing from the same push will attempt to create the same head.

Also remove the neorados FIFO since it doesn't have all the bug fixes
in the legacy version and will be rewritten in terms of `async_compose`
anyway.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 17:27:34 -05:00
Adam C. Emerson
fcaa45d32d cls/fifo: Journal is flat_set, not multimap
We don't really need the overhead and complexity of a multimap.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 16:53:51 -05:00
Adam King
a161381ba8 mgr/cephadm: fix extra container/entrypoint args with spaces
Fixes: https://tracker.ceph.com/issues/57338

Prior, doing extra container args like

- "--cpus"
- "2"

would work fine as the two args would be passed separately and
eventually placed in the final podman/docker run command
with a space between them. However, trying to do something like

- "--cpus 2"

instead would fail, as it would be translated to

--extra-container-args=--cpus 2

causing "2" to be considered its own arg, which cephadm
wouldn't know how to handle. Another way this can cause problems
is listed in the linked tracker. Either way, leaving the spaces
in the args was causing problems, and the simplest way to handle
it seems to be to just split on the original arg on the spaces
into multiple args

Signed-off-by: Adam King <adking@redhat.com>
2023-01-09 15:11:49 -05:00
Zac Dover
7b8f75ebd7 doc/rados: link to cephadm replacing osd section
Direct readers to the "Replacing an OSD" section in the cephadm
documentation, for cases in which the instructions in "Replacing an OSD"
in the RADOS documentation don't work.

https://tracker.ceph.com/issues/58401

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-10 05:11:00 +10:00
Adam C. Emerson
32b514c52b cls/fifo: Move version inc into apply_update()
Only increment the version if we make an actual change.

Return whether we have changed something or not so the OSD side can
skip writing if there's no change.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 14:00:34 -05:00
Adam C. Emerson
ade8909991 cls/fifo: Merge duplicate journal entries
Since we no longer use tags, journal entries are just an operation and
a part number. If an entry being added is already in the journal, skip
it.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 13:37:06 -05:00
Adam King
8ee799e9cc doc/cephadm: docs for extra entrypoint args
Signed-off-by: Adam King <adking@redhat.com>
2023-01-09 13:24:50 -05:00
Adam King
db8e4462c9 mgr/cephadm: support for extra entrypoint args
Args specified in the service spec to be added as
args for the entrypoint when we deploy the daemon

Fixes: https://tracker.ceph.com/issues/57944

Signed-off-by: Adam King <adking@redhat.com>
2023-01-09 13:24:50 -05:00
Adam C. Emerson
9e67a437a3 cls/fifo: Use spaceship operator to implement comparisons
Except for the version type, where only versions with the same tag are
comparable.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 13:23:44 -05:00
Adam C. Emerson
f89a6d1833 cls/fifo: Remove all use of part tags
Part tags make part creation and setting the head non-idempotent,
leading to issues where racing RGWs may get confused about the correct
tag for a part. (Or worse, potentially have the metadata header hold
different value for a part than the part's header.)

Consistently only requires that all nodes agree on the number.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 13:14:46 -05:00
Adam C. Emerson
dae8e78145 cls/fifo: Fix same_or_later() comparison
Fix logic error where disjunction was used instead of conjunction.

Fixes: https://tracker.ceph.com/issues/57562
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-09 12:59:57 -05:00
Adam King
98f6a9379e python-common: add extra_entrypoint_args to service specs
To allow users to set extra args to be passed to
the entrypoint in the podman/docker run command
built by cephadm

Signed-off-by: Adam King <adking@redhat.com>
2023-01-09 10:10:43 -05:00
Adam King
c86fc2b859
Merge pull request #48696 from adk3798/agent-testing-binary
cephadm: unit tests for CephadmAgent, MgrListener, AgentGatherer

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-01-09 10:07:59 -05:00
Casey Bodley
066e6745f5
Merge pull request #49495 from BryceCao/wip-fix-result-when-users-topics-is-null
rgw: fix result when user's topics is null

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-01-09 09:45:18 -05:00