Move the JWT requirement to the test requirements file. Also remove JWT from ceph specification and debian build.
Signed-off-by: Daniel Persson <mailto.woden@gmail.com>
* refs/pull/54726/head:
PendingReleaseNotes: announce cephfs-shell avail. on rhel9
qa: test fs:shell on all distros
qa: add cephfs-shell to installed rpm packages
ceph.spec.in: enable support for cephfs-shell by default via EPEL9
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
this is due to the fact that lua-devel
cannot be installed in teuthology for rhel8
this would fail the build of some luarocks packages
Fixes: https://tracker.ceph.com/issues/63672
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Add two conditions to either disable bundling or switch to rpm based
depdencies. The default is to bundle deps from pip.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This is the MVP for a driver for RGW that operates on top of a POSIX
filesystem. It supports get, put, list, copy, multipart, external
access via the filesystem itself, and ordered bucket listings via an
LRU-based cache.
Note that this is currently a Filter, indended to run on top of dbstore.
This is because it currently doesn't have any User implementation, so it
depends on dbstore's User. Everything else is implemented in
POSIXDriver. Once there is a User implementation, this will become a
Store, instead of a Filter.
Commit messages from bucket listing cache:
rgw/posixdriver: recycle lmdb database handles as required
While LMDB workflows often do not close/return database handles,
ours continually reuses them. This requires us to close each
handle (atomically) when a cache entry is recycled.
rgw/posixdriver: don't instantiate bucket cache entries from notify events
rgw/posixdriver: incorporate lmdb-safe for now
The current inclusion is based on https://github.com/Martchus/lmdb-safe,
which is actively maintained but currently has some packaging issues the
author has agreed to accept fixes for.
For now, skip the submodule to save time and remove an external dependency.
rgw/posixdriver: fix listing of cached, empty bucket
* check lmdb enumeration result in all cases and w/better style
* add unit test for enumeration of an empty cached directory
rgw/posixdriver: nest lmdbs in a directory under the dbroot path to avoid cleanup issues
rgw/posixdriver: refactor for posix integration
* Derive BucketCache types as templates on a SAL driver and SAL
bucket pair.
* Integrate cache fills as callbacks into SAL layer (or mock, for
tests)
* Renaming and cleanups
rgw/posixdriver: add bucket cache implementation and tests
Adds free-standing cache of buckets and object names, with
bucket names (and listing attributes, upcoming) managed in
a hashed set of lmdb databases, which provides ordering and
a high-performance listing cache.
An framework for notification on new object creation (e.g.,
outside S3 workflow) is provided, and a Linux implementation
using inotify.
FindLMDB.cmake taken with attribution and license.
rgw/posixdriver: add zpp_bits serialization (FAST)
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
This enhances the script to both process versioned buckets correctly
and to handle object names that begin with underscore.
If the bucket is versioned it submits each version chronologically
(based on mtime) to be reindexed in order to "replay" the modification
of objects. However mtime is not a perfect indicator. So additionally
it looks at the OLH object to determine the most recent version and
the script makes sure that it is replayed last. The order of previous
versions is likely correct, but not guaranteed to be so.
Additional logic is added to handle objects with names that begin with
underscore ('_') since that's used as a delimiter and needs to be
escaped and rados object locators are also used.
A man page for the script is added.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
The ceph main branch daily build on openEuler has been built.
https://github.com/openeuler-mirror/ceph-daily-build
The verification includes:
- build ceph from source
- run unit test
- build RPM package
Signed-off-by: liuqinfei <lucas.liuqinfei@huawei.com>
Update ceph.spec.in and debian install files so
rgw-restore-bucket-index, rgw-orphan-list, rgw-gap-list,
rgw-gap-list-comparator are installed with common files.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Adds an experimental script that allows a bucket index of a
non-versioned bucket to be restored by applying `radosgw-admin object
reindex ...` to all objects in the specified bucket. The objects in
the bucket are determined by scanning the data pool for head objects
containing the bucket's marker.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
instead of using the top-level "packaging" module, use the one
vendored by setuptools.
packaging python module provides versioning defined by PEP-440.
but python3-packaging is provided by CentOS8 powertools repo,
which is not enabled by default. and in CentOS9, this package
is provided by AppStream instead of BaseOS.
as prometheus mgr module is included by ceph-mgr-module-core,
it would be desirable if our user can install ceph-mgr-module-core
without enabling powertools or AppStream repo on a CentOS or
its derivative distros.
fortunately, setuptools vendors packaging module. and both
CentOS8 and CentOS9 provide python3-setuptools in their BaseOS
repos.
in this change, instead of using "packging" module, we use the
venderored one, which is in turn embedded in pkg_resources.
this python module is provided by python3-setuptools on CentOS
distros, and python3-pkg-resources on Debian and its derivatives.
the packaging recipes are updated accordingly to reflect the
new runtime dependency.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
This effectively disable cephfs_java for rpm builds for now.
The only user appears to be cephfs jni bindings.
Fixes: https://tracker.ceph.com/issues/58382
Signed-off-by: Samuel Just <sjust@redhat.com>
rgw-policy-check - a program to do syntax checking on bucket policy.
This program just reads the policy into memory, so it is not
checking anything except syntax.
Signed-off-by: Marcus Watts <mwatts@redhat.com>
rgw: Fix return value of `rgw-policy-check`
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
rgw: Use ceph initialization in `rgw-policy-check`
Specifically so we can pull in the options from `ceph.conf` and similar.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
The current VDO support implementation is buried inside the common/blkdev.cc
with a simple interface used by KernelDevice. It is not easily extendable
and can not be easily used for other devices providing similar capabilities.
This patch adds a plugin system that is based in its structure on the
erasure code plugin system and moves the VDO support code into a VDO plugin.
Signed-off-by: Martin Ohmacht <mohmacht@us.ibm.com>
otherwise after enabling gcc-toolset-11, cmake fails with:
- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/g++ requires libatomic,
but it is not found
Signed-off-by: Casey Bodley <cbodley@redhat.com>
as RHEL/CentOS 9, we have the access to GCC-11, which is good enough for
compiling main HEAD even with WITH_SEASTAR=ON.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
there is chance that we are using `yum-builddep` to prepare the
build dependencies. in that case, gcc-toolset-11-build is not
installed. it's like a chicken-egg dilemma, but the point is
`yum-builddep` is able to pull in the gcc-toolset-11-build. once
gcc-toolset-11-build is installed, we will have the %enable_devtoolset11
rpm macro.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
otherwise we'd have failures like
/opt/compiler-explorer/gcc-trunk-20220808/bin/../lib/gcc/x86_64-linux-gnu/13.0.0/../../../../x86_64-linux-gnu/bin/ld:
/tmp/ccVlMbVh.o: in function `std::atomic<tagged_ptr>::store(tagged_ptr,
std::memory_order)':
/opt/compiler-explorer/gcc-trunk-20220808/include/c++/13.0.0/atomic:273:
undefined reference to `__atomic_store_16'
when generating the building system using CMake on fedora 36.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Without this, the openSUSE build fails with:
ceph-mib-17.0.0-[...].noarch.rpm: directories not owned by a package:
- /usr/share/snmp
Fixes: 183e347506
Signed-off-by: Tim Serong <tserong@suse.com>
to address following failure when generating the building system
using CMake:
```
-- Performing Test HAVE_LIBATOMIC
-- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/c++ requires libatomic,
but it is not found
```
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
before this change %enable_devtoolset11 is called only when building
with crimson on centos8.
after this change %enable_devtoolset11 is called when building on
centos8. because we've started using gcc-toolset-11 for building
rpm packages on centos8 after the C++20 migration. so, to build
with gcc-11, we need to enable it.
also, because gcc-toolset-11 is used, we have to disable
annotated_build.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
