Commit Graph

135743 Commits

Author SHA1 Message Date
Shilpa Jagannath
8ff79b1c62 rgw/multisite: don't disable versioning on archive zone upon source bucket modifications
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-13 09:12:33 -05:00
Shilpa Jagannath
bcea963d97 rgw/multisite: object should generate version id on archive zone, even when versioning
is disabled on primary

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-13 08:53:21 -05:00
Shilpa Jagannath
bfebd24591 rgw/multisite: take the bucket lease before checking if bucket sync is enabled/disabled
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-13 08:51:03 -05:00
Soumya Koduri
c55c505090 rgw/multisite: Add testcases for sync policy feature
Added test cases for the various use-cases of multisite sync
policy feature listed in
https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-01-13 08:27:47 -05:00
Yehuda Sadeh
9f774b4dec rgw: fix get sync policy handler if bucket_id is empty
Fixes: https://tracker.ceph.com/issues/48416

bucket was passed in without bucket_id, now reading entrypoint
info if needed.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2023-01-13 08:27:47 -05:00
Adam C. Emerson
a1ea877635 rgw: LatencyConcurrencyControl warns on very high latency
Limited to only warn every five minutes.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-13 08:27:39 -05:00
Adam C. Emerson
b5d5da9c4d rgw: Reduce data sync parallelism in response to RADOS lock latency
Lock latency in RGWContinuousLeaseCR gets high enough under load that
the locks end up timing out, leading to incorrect behavior.

Monitor lock latency and cut concurrent operations in half if it goes
above ten seconds.

Cut currency to one if it goes about twenty seconds.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-12 18:13:46 -05:00
Adam C. Emerson
900068ef93 rgw: SimpleRadosWriteAttrsCR uses an async RADOS call
Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-12 18:05:41 -05:00
Adam C. Emerson
aea643ada6 rgw: SimpleRadosWriteCR uses an async RADOS call
Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-12 18:03:43 -05:00
Adam C. Emerson
a366b900b5 rgw: SimpleRadosReadCR uses an async RADOS call
Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-12 17:49:55 -05:00
Adam C. Emerson
04d3a4f068 rgw: SimpleRadosReadAttrsCR uses an async RADOS call
Don't go through the 'system object' cache. This also saves us the use
of the RADOS async completion processor.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:46:45 -05:00
Adam C. Emerson
634673a64e rgw: Use cls_version for read/writes to global data sync status
The `radosgw-admin data sync init` command does *not* use
`cls_version` and just overwrites.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:37:41 -05:00
Adam C. Emerson
25ef265513 rgw: Reread sync status after acquiring lock in RGWDataSyncCR
If someone else got there first, we won't smash their work.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:35:58 -05:00
Adam C. Emerson
b0d401a0f4 rgw: Pull lock out of RGWInitDataSyncStatusCoroutine
RGWDataSyncCR manages the lock instead, holding it through StateInit
and StateBuildingFullSyncMaps but releasing it by StateSync.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:32:21 -05:00
Adam C. Emerson
2b3fbcf9b1 rgw: RGWContinuousLeaseCR takes arguments by value and moves them
Since we were taking them by reference and copying before, this is
strictly better. Callers that give us an RValue can skip the copy.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:26:29 -05:00
Adam C. Emerson
77deaa9b02 rgw: Get rid of some COOKIE_LEN preprocessor macros
Sticking random #defines everywhere is just atrocious style.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2023-01-11 01:26:27 -05:00
Casey Bodley
113cda8834 rgw: add rgw_curl_tcp_keepalive option for http client requests
this can be useful to prevent long-lived connections from being dropped
due to inactivity

Fixes: https://tracker.ceph.com/issues/48402

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-01-11 00:23:40 -05:00
Shilpa Jagannath
852e60d6f3 rgw/multisite: clear objv before reading bucket sync status
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:21:27 -05:00
Soumya Koduri
e1c31b2997 rgw/bucket-sync: flush marker_tracker when the lease is lost
Also clear objv before reading the bucket sync status.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-01-11 00:17:57 -05:00
Shilpa Jagannath
d0725750b6 rgw/multisite: marker_tracker->finish() returns error
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:16:48 -05:00
Shilpa Jagannath
180d5d9783 rgw/multisite: if remote datalog is trimmed prematurely, sync status will report that it's behind the remote's max-marker even if there are no more entries to sync for each behind shard. if we get an empty listing, remove that shard from behind_shards.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:15:23 -05:00
Soumya Koduri
abc5883604 rgw/multisite: Update marker and objv after acquiring the lease
In RGWDataSyncShardCR, after acquiring the lease, reread sync status
shard object to fetch the latest marker & objv stored.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-01-11 00:13:59 -05:00
Shilpa Jagannath
9aeb2aa38d rgw/multisite: add cls versioning for tracking data sync per shard object and store it in a vector
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2023-01-11 00:13:56 -05:00
colemitchell
58f33d83bc
Merge pull request #49699 from zdover23/wip-doc-2023-01-11-cephadm-host-management-removing-hosts
doc/cephadm: refine "Removing Hosts"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-10 14:31:23 -05:00
colemitchell
1eb04cbc5e
Merge pull request #49703 from zdover23/wip-doc-2023-01-11-rados-operations-move-colon
doc/rados: move colon

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2023-01-10 14:24:11 -05:00
colemitchell
7fe7819c71
Merge pull request #49700 from zdover23/wip-doc-2023-01-11-css-h3-padding
doc/css: add top-bar padding for h3 html element

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2023-01-10 14:20:16 -05:00
Zac Dover
b1a53d7621 doc/rados: move colon
Move colon in add-or-rm-osds.rst so that the sentence reads properly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 05:08:35 +10:00
Zac Dover
c28cb53390 doc/css: add top-bar padding for h3 html element
Add "scroll-margin-top: 4em;" property to h3 html element.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 04:48:32 +10:00
Zac Dover
2e1fd63087 doc/cephadm: refine "Removing Hosts"
An intended edit to remove a redundant indefinite article became a
longer (but still brief) full editorial pass.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 04:29:02 +10:00
zdover23
7c9cfc9013
Merge pull request #49692 from zdover23/wip-doc-2023-01-11-header-fix
doc/css: add "span" padding to custom.css

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-11 04:10:22 +10:00
Ilya Dryomov
e55a6f618e
Merge pull request #49690 from idryomov/wip-58398
doc/man/ceph-rbdnamer: remove obsolete udev rule

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-01-10 18:44:28 +01:00
Zac Dover
dd9555b0ae doc/css: add "span" padding to custom.css
Add "scroll-top-bar: 2em;" for the "span" html element in custom.css so
that the top bar doesn't get in the way of headings bounded by the "span
element".

See also https://github.com/ceph/ceph/pull/49644.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-11 01:55:55 +10:00
Ilya Dryomov
8ca3bd5042 doc/man/ceph-rbdnamer: remove obsolete udev rule
Fixes: https://tracker.ceph.com/issues/58398
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-01-10 13:43:03 +01:00
Venky Shankar
9939dcc401 Merge PR #49630 into main
* refs/pull/49630/head:
	doc: document the relevance of mds_namespace mount option

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-01-10 16:41:10 +05:30
Jos Collin
86f8fa19e5
doc: document the relevance of mds_namespace mount option
Fixes: https://tracker.ceph.com/issues/57673
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-01-10 14:48:20 +05:30
Venky Shankar
6243f33d0b
Merge pull request #49664 from batrick/first-damage-repair-nosnap
tools/cephfs/first-damage: repair specific first=CEPH_NOSNAP damage

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-01-10 09:59:51 +05:30
zdover23
6c1dfe9e72
Merge pull request #49677 from zdover23/wip-doc-2023-01-10-rados-replacing-an-osd
doc/rados: link to cephadm replacing osd section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-10 10:45:39 +10:00
Zac Dover
7b8f75ebd7 doc/rados: link to cephadm replacing osd section
Direct readers to the "Replacing an OSD" section in the cephadm
documentation, for cases in which the instructions in "Replacing an OSD"
in the RADOS documentation don't work.

https://tracker.ceph.com/issues/58401

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-10 05:11:00 +10:00
Adam King
c86fc2b859
Merge pull request #48696 from adk3798/agent-testing-binary
cephadm: unit tests for CephadmAgent, MgrListener, AgentGatherer

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-01-09 10:07:59 -05:00
Casey Bodley
066e6745f5
Merge pull request #49495 from BryceCao/wip-fix-result-when-users-topics-is-null
rgw: fix result when user's topics is null

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-01-09 09:45:18 -05:00
Guillaume Abrioux
71c3f8e31b
Merge pull request #49629 from guits/cv-fix-tox-allowlist_externals
ceph-volume/tests: add allowlist_externals to tox.ini
2023-01-09 09:59:25 +01:00
colemitchell
fe4cb1a645
Merge pull request #49665 from zdover23/wip-doc-2023-01-08-glossary-ceph-object-storage
doc/glossary: Clean up "Ceph Object Storage"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-08 14:33:46 -05:00
Zac Dover
6bede3505f doc/glossary: Clean up "Ceph Object Storage"
Remove redundant material under the "Ceph Object Storage" headword and
add a "See 'Ceph Object Store'" link. A future PR will provide a couple
of sentences that explain how object storage is what's really supporting
both CephFS and RBD.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2023-01-09 03:43:35 +10:00
Patrick Donnelly
2ade9390c6
tools/cephfs/first-damage: repair specific first=CEPH_NOSNAP damage
first==CEPH_NOSNAP is never valid. Apparently somehow the MDS may set
this in some code path yet to be discovered, even without snapshots.

Note, we probably should not necessarily automate this fix via the MDS
because we do not know if this dentry (visible in HEAD) should also be
visible before NEXT_SNAP.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-01-07 15:49:46 -05:00
Patrick Donnelly
916241fd53
tools/cephfs/first-damage.py: load CEPH_CONF
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2023-01-07 15:48:35 -05:00
Anthony D'Atri
c76e8e5538
Merge pull request #49663 from bradfitz/patch-1
doc: fix a typo
2023-01-06 23:44:12 -05:00
Brad Fitzpatrick
c670906b49
doc: fix a typo
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
2023-01-06 19:54:45 -08:00
zdover23
f87801e195
Merge pull request #49658 from zdover23/wip-doc-2023-01-15-h2-scroll-margin-top
doc/css: Add scroll-margin-top to h2 html element

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-01-07 04:02:00 +10:00
zdover23
89768e6738
Merge pull request #49653 from zdover23/wip-doc-2023-01-06-crushtool-options
doc/man: define --num-rep, --min-rep and --max-rep

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2023-01-07 03:57:32 +10:00
Daniel Gryniewicz
8df799f985
Merge pull request #49657 from dang/wip-dang-motr-build
Fix MOTRStore build

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-01-06 11:47:47 -05:00