ceph.spec.in: bump gcc-toolset to 13 and use it on rhel>=8
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
both RHEL8 and RHEL9 have gcc-toolset 13, and we need to use gts-13
for building crimson, so let's enable it when building crimson,
and we need to use gts-11 when building on RHEL7. hence this change.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
since gts 13 is out, and GCC-13 brings better support of C++20 coroutines,
and because clang prefers using a newer gts when its gcc is around. so
let's bump up the gcc-toolset from 11 to 13 when building crimson. because
gcc-toolset-13 LTO triggers a linker bug resulting in a segfault in SafeTimer,
see https://tracker.ceph.com/issues/63867 , we cannot switch the classic
build to gts 13 without proving that it does not incur performance
regressions.
since annobin plugin package was renamed to
gcc-toolset-13-gcc-plugin-annobin, let's update its name accordingly.
and use -runtime subpackage instead of -build, as
macros.gcc-toolset-13-enable is now located in -runtime subpackage
since devtoolset12
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
gcc-toolset-13 lto can trigger a linker bug resulting in a segfault in
SafeTimer (and perhaps elsewhere). See
https://tracker.ceph.com/issues/63867 for details. This patch disables
lto for now now so that we can switch to gcc-toolset-13.
Fixes: https://tracker.ceph.com/issues/63867
Signed-off-by: Samuel Just <sjust@redhat.com>
because the gcc-toolset-13 always links against the `libstdc++_nonshared.a`, which misses the
simple symbols. so we have to link against libstdc++ explicitly to assure that the
executables have access to them when compiling with the gcc-toolset toolchain.
see
```
/opt/rh/gcc-toolset-13/root/usr/bin/c++ -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -rdynamic -pie CMakeFiles/ceph_scratchtool.dir/scratchtool.c.o -o ../../bin/ceph_scratchtool -Wl,-rpath,/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1977-gb0c1f7e8/rpm/el9/BUILD/ceph-19.0.0-1977-gb0c1f7e8/redhat-linux-build/lib: ../../lib/librados.so.2.0.0 ../../lib/libglobal.a ../../lib/libceph-common.so.2 ../../lib/libjson_spirit.a ../../lib/libcommon_utf8.a ../../lib/liberasure_code.a ../../lib/libextblkdev.a -lcap ../../boost/lib/libboost_thread.a ../../boost/lib/libboost_chrono.a ../../boost/lib/libboost_atomic.a ../../boost/lib/libboost_system.a ../../boost/lib/libboost_random.a ../../boost/lib/libboost_program_options.a ../../boost/lib/libboost_date_time.a ../../boost/lib/libboost_iostreams.a ../../boost/lib/libboost_regex.a /usr/lib64/libblkid.so /usr/lib64/libcrypto.so /usr/lib64/libudev.so /usr/lib64/libibverbs.so /usr/lib64/librdmacm.so /usr/lib64/libz.so ../opentelemetry-cpp/sdk/src/trace/libopentelemetry_trace.a ../opentelemetry-cpp/sdk/src/resource/libopentelemetry_resources.a ../opentelemetry-cpp/sdk/src/common/libopentelemetry_common.a ../opentelemetry-cpp/exporters/jaeger/libopentelemetry_exporter_jaeger_trace.a ../opentelemetry-cpp/ext/src/http/client/curl/libopentelemetry_http_client_curl.a /usr/lib64/libcurl.so /usr/lib64/libthrift.so -ldl /usr/lib64/librt.a -lresolv -Wl,--as-needed -latomic
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/libstdc++_nonshared.a(ios_init.o): undefined reference to symbol '_ZNSt8ios_base4InitD1Ev@@GLIBCXX_3.4'
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld:
/usr/lib64/libstdc++.so.6: error adding symbols: DSO missing from command line
```
this change prepares us for switching to gts-13.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
since we are going to use different gcc-toolset versions on rhel8 and
on rhel9, and the subpackage names of gcc-toolset changed across
gts 11 and gts 13, let's prepare for this change.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
For this, the grafana dashboards are installed directly to the
containers and later, cephadm picks it up to mount the dashboards to the
grafana container
Signed-off-by: Nizamudeen A <nia@redhat.com>
so that it can be later used by the dashboard to configure the nvmeof
through UI
and create rbd pool in UI
Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
CloudFlare engineers made some testing and realized that using
workqueues with encryption on flash devices has a bad effect.
See [1] for details.
With this patch it will make ceph-volume call crypsetup with
`--perf-no_read_workqueue` and `--perf-no_write_workqueue` options
when the device is not a rotational.
[1] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Co-Authored-by: Stefan Kooman <stefan@kooman.org>
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Move the JWT requirement to the test requirements file. Also remove JWT from ceph specification and debian build.
Signed-off-by: Daniel Persson <mailto.woden@gmail.com>
* refs/pull/54726/head:
PendingReleaseNotes: announce cephfs-shell avail. on rhel9
qa: test fs:shell on all distros
qa: add cephfs-shell to installed rpm packages
ceph.spec.in: enable support for cephfs-shell by default via EPEL9
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
this is due to the fact that lua-devel
cannot be installed in teuthology for rhel8
this would fail the build of some luarocks packages
Fixes: https://tracker.ceph.com/issues/63672
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Add two conditions to either disable bundling or switch to rpm based
depdencies. The default is to bundle deps from pip.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This is the MVP for a driver for RGW that operates on top of a POSIX
filesystem. It supports get, put, list, copy, multipart, external
access via the filesystem itself, and ordered bucket listings via an
LRU-based cache.
Note that this is currently a Filter, indended to run on top of dbstore.
This is because it currently doesn't have any User implementation, so it
depends on dbstore's User. Everything else is implemented in
POSIXDriver. Once there is a User implementation, this will become a
Store, instead of a Filter.
Commit messages from bucket listing cache:
rgw/posixdriver: recycle lmdb database handles as required
While LMDB workflows often do not close/return database handles,
ours continually reuses them. This requires us to close each
handle (atomically) when a cache entry is recycled.
rgw/posixdriver: don't instantiate bucket cache entries from notify events
rgw/posixdriver: incorporate lmdb-safe for now
The current inclusion is based on https://github.com/Martchus/lmdb-safe,
which is actively maintained but currently has some packaging issues the
author has agreed to accept fixes for.
For now, skip the submodule to save time and remove an external dependency.
rgw/posixdriver: fix listing of cached, empty bucket
* check lmdb enumeration result in all cases and w/better style
* add unit test for enumeration of an empty cached directory
rgw/posixdriver: nest lmdbs in a directory under the dbroot path to avoid cleanup issues
rgw/posixdriver: refactor for posix integration
* Derive BucketCache types as templates on a SAL driver and SAL
bucket pair.
* Integrate cache fills as callbacks into SAL layer (or mock, for
tests)
* Renaming and cleanups
rgw/posixdriver: add bucket cache implementation and tests
Adds free-standing cache of buckets and object names, with
bucket names (and listing attributes, upcoming) managed in
a hashed set of lmdb databases, which provides ordering and
a high-performance listing cache.
An framework for notification on new object creation (e.g.,
outside S3 workflow) is provided, and a Linux implementation
using inotify.
FindLMDB.cmake taken with attribution and license.
rgw/posixdriver: add zpp_bits serialization (FAST)
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
This enhances the script to both process versioned buckets correctly
and to handle object names that begin with underscore.
If the bucket is versioned it submits each version chronologically
(based on mtime) to be reindexed in order to "replay" the modification
of objects. However mtime is not a perfect indicator. So additionally
it looks at the OLH object to determine the most recent version and
the script makes sure that it is replayed last. The order of previous
versions is likely correct, but not guaranteed to be so.
Additional logic is added to handle objects with names that begin with
underscore ('_') since that's used as a delimiter and needs to be
escaped and rados object locators are also used.
A man page for the script is added.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
The ceph main branch daily build on openEuler has been built.
https://github.com/openeuler-mirror/ceph-daily-build
The verification includes:
- build ceph from source
- run unit test
- build RPM package
Signed-off-by: liuqinfei <lucas.liuqinfei@huawei.com>
Update ceph.spec.in and debian install files so
rgw-restore-bucket-index, rgw-orphan-list, rgw-gap-list,
rgw-gap-list-comparator are installed with common files.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Adds an experimental script that allows a bucket index of a
non-versioned bucket to be restored by applying `radosgw-admin object
reindex ...` to all objects in the specified bucket. The objects in
the bucket are determined by scanning the data pool for head objects
containing the bucket's marker.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
instead of using the top-level "packaging" module, use the one
vendored by setuptools.
packaging python module provides versioning defined by PEP-440.
but python3-packaging is provided by CentOS8 powertools repo,
which is not enabled by default. and in CentOS9, this package
is provided by AppStream instead of BaseOS.
as prometheus mgr module is included by ceph-mgr-module-core,
it would be desirable if our user can install ceph-mgr-module-core
without enabling powertools or AppStream repo on a CentOS or
its derivative distros.
fortunately, setuptools vendors packaging module. and both
CentOS8 and CentOS9 provide python3-setuptools in their BaseOS
repos.
in this change, instead of using "packging" module, we use the
venderored one, which is in turn embedded in pkg_resources.
this python module is provided by python3-setuptools on CentOS
distros, and python3-pkg-resources on Debian and its derivatives.
the packaging recipes are updated accordingly to reflect the
new runtime dependency.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
