Commit Graph

27 Commits

Author SHA1 Message Date
J. Eric Ivancich
c02906ae64 rgw: enhances rgw-restore-bucket-index script
This enhances the script to both process versioned buckets correctly
and to handle object names that begin with underscore.

If the bucket is versioned it submits each version chronologically
(based on mtime) to be reindexed in order to "replay" the modification
of objects. However mtime is not a perfect indicator. So additionally
it looks at the OLH object to determine the most recent version and
the script makes sure that it is replayed last. The order of previous
versions is likely correct, but not guaranteed to be so.

Additional logic is added to handle objects with names that begin with
underscore ('_') since that's used as a delimiter and needs to be
escaped and rados object locators are also used.

A man page for the script is added.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-08-30 14:41:15 -04:00
J. Eric Ivancich
637d461bf1 rgw: install rgw scripts with common files rather than radosgw files
Update ceph.spec.in and debian install files so
rgw-restore-bucket-index, rgw-orphan-list, rgw-gap-list,
rgw-gap-list-comparator are installed with common files.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-03-15 09:35:45 -04:00
J. Eric Ivancich
6ea11b1305 rgw: add rgw-restore-bucket-index
Adds an experimental script that allows a bucket index of a
non-versioned bucket to be restored by applying `radosgw-admin object
reindex ...` to all objects in the specified bucket. The objects in
the bucket are determined by scanning the data pool for head objects
containing the bucket's marker.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-03-09 11:00:57 -05:00
Casey Bodley
69a231888e rgw/cmake: stop building all of radosgw a shared lib
Fixes: https://tracker.ceph.com/issues/57050

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-08-08 11:10:11 -07:00
J. Eric Ivancich
3c5154f010 rgw: add rgw-gap-list-comparator tool
The rgw-gap-list tool can produce a number of false positives when the
cluster is being used during its run. One technique to minimize the
number of false positives is to run the tool twice and look for the
objects that appear in both lists. The rgw-gap-list-comparator tool is
designed to do this comparison.

Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-03 16:57:50 -05:00
J. Eric Ivancich
07b42195fb rgw: add rgw-gap-list tool
Due to a prior bug (pr: 38228) tail rados objects of some RGW objects
could have been incorrectly deleted. This tool is designed to look for
such cases. It essentially does the opposite of rgw-orphan-list,
looking for rados objects that RGW expects to be there, but which are
not to be found.

IMPORTANT: This is very experimental at this point in time, and any
"results" produced should be verified by other means.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Signed-off-by: Michael Kidd <linuxkidd@gmail.com>
2021-02-03 16:57:50 -05:00
J. Eric Ivancich
9d5e9c3031 doc/rgw: add docs for rgw-orphan-list and ceph-diff-sorted
Add man pages and documentation for both tools.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-04 13:33:56 -04:00
J. Eric Ivancich
e396064d9a rgw: add rgw-orphan-list tool & radosgw-admin bucket radoslist ...
Adds a `radosgw-admin` subcommand and walks the associated bucket
indices and manifests to generate the list of rados objects that
represent the rgw objects in the bucket(s).

Also adds a tool named `rgw-orphan-list`, which uses the radoslist
subcommand, that produces a list in a local file of what appear to be
rgw orphans.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-04 13:20:08 -04:00
Kefu Chai
1484ef2fa8 debian/radosgw.install: correct path to libradosgw.so*
it was a regression introduced by f528f173e4

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-08 11:22:01 +08:00
Kaleb S. KEITHLEY
f528f173e4 rgw: build radosgw daemon as a shared lib + small executable
Majority of radosgw is contained in libradosgw.so. (/usr)/bin/radosgw
is now a few lines that calls radosgw_Main() in libradosgw.so.

The "zipper" work to modularize storage back-ends that will require
linking to the shared library to resolve the methods that they reference.

Putting the bulk of the implementation in a shared lib also allows for
unit testing to link with the shared lib for testing.

radosgw_Main() is the C++ implementation. For C linkage, radosgw_main()
is provided.

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-12-23 12:34:44 -05:00
Kefu Chai
50707e7d62 debian: install system units using cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-28 00:23:22 +08:00
James Page
1341de99a0 debian: wrap-and-sort all files
'wrap-and-sort -a -t' debian folder to apply consistent
ordering/formatting to all files.

This makes synchronization of debian package changes
between Ceph packages and Ubuntu packages easier to
manage.

Signed-off-by: James Page <james.page@ubuntu.com>
2017-07-04 17:38:39 +01:00
Yehuda Sadeh
ea902d18e2 rgw: package radosgw-es
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2017-05-30 13:26:59 -07:00
Ali Maredia
9fd30b93f7 debian/rpm: move radosgw-admin to ceph-common
This commit moves the radosgw-admin command from
the radosgw package to ceph-common.

This allows for a better user expierence since the
radosgw-admin command can be run on any node.

Fixes: http://tracker.ceph.com/issues/19577

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-05-03 14:28:17 -04:00
Matt Benjamin
2bb6ef9d84 rgw: ldap token glue
Based on feedback from upstream RGW, introduce a new json-format
token structure representing credentials for pass-through authentication
(e.g., LDAP without Keystone's digest authentication).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-03-09 10:54:35 -05:00
Sage Weil
2d1d2efb1b debian: package radosgw-object-expirer in radosgw deb
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-03 18:41:52 -04:00
Andras Elso
8f5c665744 debian: Run wrap-and-sort from devscripts
Signed-off-by: Andras Elso <elso.andras@gmail.com>
2013-03-29 13:34:48 +01:00
Sage Weil
6103ee9e88 more radosgw_admin -> radosgw-admin
Signed-off-by: Sage Weil <sage@newdream.net>
2011-10-03 11:25:44 -07:00
Wido den Hollander
440382fb54 bash_completion: Add the bash_completion files to the Debian packages
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-07-26 14:02:02 -07:00
Laszlo Boszormenyi
4e3c201132 Tune Debian packaging for the upcoming v0.24 release.
Including switch OpenSSL dependency to Crypto++ as its being used instead of
the former; remove radosacl as its not compiled anymore and pristine clean
the source. Explicitly note this is in a 1.0 package format.
2010-12-05 22:20:48 -08:00
Sage Weil
9360126943 debian: many many cleanups
Signed-off-by: Laszlo Boszormenyi <gcs@debian.hu>
2010-11-30 09:13:54 -08:00
Sage Weil
4545f8b929 debian: fix copyright to avoid lintian check
possible-gpl-code-linked-with-openssl
2009-10-28 14:05:27 -07:00
Sage Weil
307382f302 radosgw: fix man page names, include in package 2009-10-28 12:08:54 -07:00
Sage Weil
f8fb331226 radosgw: rename binaries rgw -> radosgw 2009-10-28 11:06:36 -07:00
Sage Weil
5f7951d0e4 debian: remove rgw_aclparser from package
It's a unit test
2009-10-28 11:03:11 -07:00
Sage Weil
887a347fd9 debian: override openssl warning on radosgw 2009-10-28 10:48:39 -07:00
Sage Weil
335ea1d703 debian: create radosgw package 2009-09-10 13:48:47 -07:00