This enhances the script to both process versioned buckets correctly
and to handle object names that begin with underscore.
If the bucket is versioned it submits each version chronologically
(based on mtime) to be reindexed in order to "replay" the modification
of objects. However mtime is not a perfect indicator. So additionally
it looks at the OLH object to determine the most recent version and
the script makes sure that it is replayed last. The order of previous
versions is likely correct, but not guaranteed to be so.
Additional logic is added to handle objects with names that begin with
underscore ('_') since that's used as a delimiter and needs to be
escaped and rados object locators are also used.
A man page for the script is added.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
The ceph main branch daily build on openEuler has been built.
https://github.com/openeuler-mirror/ceph-daily-build
The verification includes:
- build ceph from source
- run unit test
- build RPM package
Signed-off-by: liuqinfei <lucas.liuqinfei@huawei.com>
Update ceph.spec.in and debian install files so
rgw-restore-bucket-index, rgw-orphan-list, rgw-gap-list,
rgw-gap-list-comparator are installed with common files.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Adds an experimental script that allows a bucket index of a
non-versioned bucket to be restored by applying `radosgw-admin object
reindex ...` to all objects in the specified bucket. The objects in
the bucket are determined by scanning the data pool for head objects
containing the bucket's marker.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
instead of using the top-level "packaging" module, use the one
vendored by setuptools.
packaging python module provides versioning defined by PEP-440.
but python3-packaging is provided by CentOS8 powertools repo,
which is not enabled by default. and in CentOS9, this package
is provided by AppStream instead of BaseOS.
as prometheus mgr module is included by ceph-mgr-module-core,
it would be desirable if our user can install ceph-mgr-module-core
without enabling powertools or AppStream repo on a CentOS or
its derivative distros.
fortunately, setuptools vendors packaging module. and both
CentOS8 and CentOS9 provide python3-setuptools in their BaseOS
repos.
in this change, instead of using "packging" module, we use the
venderored one, which is in turn embedded in pkg_resources.
this python module is provided by python3-setuptools on CentOS
distros, and python3-pkg-resources on Debian and its derivatives.
the packaging recipes are updated accordingly to reflect the
new runtime dependency.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
This effectively disable cephfs_java for rpm builds for now.
The only user appears to be cephfs jni bindings.
Fixes: https://tracker.ceph.com/issues/58382
Signed-off-by: Samuel Just <sjust@redhat.com>
rgw-policy-check - a program to do syntax checking on bucket policy.
This program just reads the policy into memory, so it is not
checking anything except syntax.
Signed-off-by: Marcus Watts <mwatts@redhat.com>
rgw: Fix return value of `rgw-policy-check`
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
rgw: Use ceph initialization in `rgw-policy-check`
Specifically so we can pull in the options from `ceph.conf` and similar.
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
The current VDO support implementation is buried inside the common/blkdev.cc
with a simple interface used by KernelDevice. It is not easily extendable
and can not be easily used for other devices providing similar capabilities.
This patch adds a plugin system that is based in its structure on the
erasure code plugin system and moves the VDO support code into a VDO plugin.
Signed-off-by: Martin Ohmacht <mohmacht@us.ibm.com>
otherwise after enabling gcc-toolset-11, cmake fails with:
- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/g++ requires libatomic,
but it is not found
Signed-off-by: Casey Bodley <cbodley@redhat.com>
as RHEL/CentOS 9, we have the access to GCC-11, which is good enough for
compiling main HEAD even with WITH_SEASTAR=ON.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
there is chance that we are using `yum-builddep` to prepare the
build dependencies. in that case, gcc-toolset-11-build is not
installed. it's like a chicken-egg dilemma, but the point is
`yum-builddep` is able to pull in the gcc-toolset-11-build. once
gcc-toolset-11-build is installed, we will have the %enable_devtoolset11
rpm macro.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
otherwise we'd have failures like
/opt/compiler-explorer/gcc-trunk-20220808/bin/../lib/gcc/x86_64-linux-gnu/13.0.0/../../../../x86_64-linux-gnu/bin/ld:
/tmp/ccVlMbVh.o: in function `std::atomic<tagged_ptr>::store(tagged_ptr,
std::memory_order)':
/opt/compiler-explorer/gcc-trunk-20220808/include/c++/13.0.0/atomic:273:
undefined reference to `__atomic_store_16'
when generating the building system using CMake on fedora 36.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Without this, the openSUSE build fails with:
ceph-mib-17.0.0-[...].noarch.rpm: directories not owned by a package:
- /usr/share/snmp
Fixes: 183e347506
Signed-off-by: Tim Serong <tserong@suse.com>
to address following failure when generating the building system
using CMake:
```
-- Performing Test HAVE_LIBATOMIC
-- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/c++ requires libatomic,
but it is not found
```
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
before this change %enable_devtoolset11 is called only when building
with crimson on centos8.
after this change %enable_devtoolset11 is called when building on
centos8. because we've started using gcc-toolset-11 for building
rpm packages on centos8 after the C++20 migration. so, to build
with gcc-11, we need to enable it.
also, because gcc-toolset-11 is used, we have to disable
annotated_build.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
This is much simpler than explicitly requiring gcc 11, and
anyway, openSUSE Tumbleweed is up to gcc 12 now.
Signed-off-by: Tim Serong <tserong@suse.com>
gcc-toolset-11-annobin is already installed, but ceph.spec.in adds
"-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1" which needs the gcc
plugin too
resolves this failure during the cmake configure step:
-- Check for working CXX compiler: /opt/rh/gcc-toolset-11/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/gcc-toolset-11/root/usr/bin/c++ - broken
CMake Error at /usr/share/cmake/Modules/CMakeTestCXXCompiler.cmake:59 (message):
The C++ compiler
"/opt/rh/gcc-toolset-11/root/usr/bin/c++"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11497-gfae772b8/rpm/el8/BUILD/ceph-17.0.0-11497-gfae772b8/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_09421/fast && /usr/bin/gmake -f CMakeFiles/cmTC_09421.dir/build.make CMakeFiles/cmTC_09421.dir/build
gmake[1]: Entering directory '/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11497-gfae772b8/rpm/el8/BUILD/ceph-17.0.0-11497-gfae772b8/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_09421.dir/testCXXCompiler.cxx.o
/opt/rh/gcc-toolset-11/root/usr/bin/c++ -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -o CMakeFiles/cmTC_09421.dir/testCXXCompiler.cxx.o -c /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-11497-gfae772b8/rpm/el8/BUILD/ceph-17.0.0-11497-gfae772b8/x86_64-redhat-linux-gnu/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
cc1plus: fatal error: inaccessible plugin file plugin/annobin.so expanded from short plugin name annobin: No such file or directory
compilation terminated.
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Prior to this change, Fedora and RHEL 9 on s390x would incorrectly pass
the HAVE_CXX11_ATOMIC check. As a consequence, the rest of Ceph would
fail to link because of missing atomic methods "__atomic_load_16",
"__atomic_store_16", "__atomic_compare_exchange_16".
Mark this method so that it is not optimized out, even with LTO. With
this change, s390x properly fails HAVE_CXX11_ATOMIC and falls back to
HAVE_LIBATOMIC.
Move the "BuildRequires: libatomic" RPM line out from the seastar+fedora
conditional and into the main fedora+rhel conditional so that the
HAVE_LIBATOMIC check will pass.
Fixes: https://tracker.ceph.com/issues/54514
Fixes: https://tracker.ceph.com/issues/56492
Co-authored-by: Kaleb S. Keithley <kkeithle@redhat.com>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
* create the destination directory when installing the MIBs file
* no need to set the file permission bits of a directory not owned by us
* the MIBs installed by us are not config files, and should not be
marked with `%config`, they are just immutable files not supposed to
be edited by user.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
In order to support the character device of pmem usage in bluestore via
libpmem built by Ceph itself, we need to enable daxctl and ndctl
dependency. add the installation of ndctl and find it. the version of
ndctl and daxctl library requires >63. "apt-get install" meet the version
under ubuntu focal.
the installation of ndctl-devel in ceph.spec.in has not been verified.
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
