CMP0097 and below are all implicitly set to new because
the minimum required CMake version is 3.16 and these
policies are older.
Signed-off-by: Christoph Grüninger <foss@grueninger.de>
At some point the debug builds for wip branches no longer had the .git
directory available so the Debug build type was unset. This meant we are
no longer doing numerous checks (like mutex ownership checks) that we
would normally be doing in the qa suite.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This is the MVP for a driver for RGW that operates on top of a POSIX
filesystem. It supports get, put, list, copy, multipart, external
access via the filesystem itself, and ordered bucket listings via an
LRU-based cache.
Note that this is currently a Filter, indended to run on top of dbstore.
This is because it currently doesn't have any User implementation, so it
depends on dbstore's User. Everything else is implemented in
POSIXDriver. Once there is a User implementation, this will become a
Store, instead of a Filter.
Commit messages from bucket listing cache:
rgw/posixdriver: recycle lmdb database handles as required
While LMDB workflows often do not close/return database handles,
ours continually reuses them. This requires us to close each
handle (atomically) when a cache entry is recycled.
rgw/posixdriver: don't instantiate bucket cache entries from notify events
rgw/posixdriver: incorporate lmdb-safe for now
The current inclusion is based on https://github.com/Martchus/lmdb-safe,
which is actively maintained but currently has some packaging issues the
author has agreed to accept fixes for.
For now, skip the submodule to save time and remove an external dependency.
rgw/posixdriver: fix listing of cached, empty bucket
* check lmdb enumeration result in all cases and w/better style
* add unit test for enumeration of an empty cached directory
rgw/posixdriver: nest lmdbs in a directory under the dbroot path to avoid cleanup issues
rgw/posixdriver: refactor for posix integration
* Derive BucketCache types as templates on a SAL driver and SAL
bucket pair.
* Integrate cache fills as callbacks into SAL layer (or mock, for
tests)
* Renaming and cleanups
rgw/posixdriver: add bucket cache implementation and tests
Adds free-standing cache of buckets and object names, with
bucket names (and listing attributes, upcoming) managed in
a hashed set of lmdb databases, which provides ordering and
a high-performance listing cache.
An framework for notification on new object creation (e.g.,
outside S3 workflow) is provided, and a Linux implementation
using inotify.
FindLMDB.cmake taken with attribution and license.
rgw/posixdriver: add zpp_bits serialization (FAST)
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Because of winpthreads issues, we had to use Boost's shared_mutex
implementation.
When using mingw-llvm, we can safely use libc++'s shared mutex
implementation.
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
* use the one shiped by the latest CMake (ab379e5054aa792df9572078dcf95bddd75f7661)
* use the new policy to use the new find strategy.
* accomodate the vanilla FindPython3 module to Ceph by:
- dropping the `cmake_policy()` calls which set the policy not supported
by 3.16.
- `include (FindPackageHandleStandardArgs)` without specifying the
relative path.
- dropping the `HANDLE_VERSION_RANGE` from `FindPackageHandleStandardArgs()` call.
this option was introduced by CMake v3.19, see
https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
but Ubuntu focal comes with CMake 3.16, which is our minimal required CMake version.
the new FindPython3 module from CMake:
* enables us to find the recent Python intepreter and development files up to
CPython 3.13.
* finds intepreter with the new `Python_FIND_STRATEGY`. the old and default
strategy always finds the most recent version with all specified name
and in all locations. so, if /usr/bin/python exists, it would accept, even
if it is a symlink to python3.9 and what we want is python3.6. while
the new policy stops at the one which satisfies the constraints.
simpler this way and less error prone.
Fixes: https://tracker.ceph.com/issues/62428
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
We're currently setting FMT_USE_TZSET=0 when building libfmt
in order to avoid the _tzset function, which is unavailable
under Mingw:
aa5769ecf1
The issue is that it still gets used by fmt/chrono.h, which is
why we'll move this definition to the top level cmake file.
Note that the Windows build is currently failing as a result of
a recent change: https://github.com/ceph/ceph/pull/52590/files
In file included from ceph/src/common/ceph_time.h:22,
from ceph/src/include/encoding.h:31,
from ceph/src/include/uuid.h:9,
from ceph/src/include/types.h:21,
from ceph/src/crush/CrushWrapper.h:14,
from ceph/src/crush/CrushCompiler.h:7,
from ceph/src/crush/CrushCompiler.cc:4:
ceph/src/fmt/include/fmt/chrono.h: In lambda function:
ceph/src/fmt/include/fmt/chrono.h:953:5: error: ‘_tzset’ was
not declared in this scope; did you mean ‘tzset’?
953 | _tzset();
| ^~~~~~
| tzset
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Unless the allocator was set on command line, we will select one based on the following order:
```
"specify memory allocator to use. currently tcmalloc, tcmalloc_minimal, \
jemalloc, and libc is supported. if not specified, will try to find tcmalloc, \
and then jemalloc. If neither of then is found. use the one in libc.")
```
with this change, cmake will explicitly message the compiler selected,
otherwise we have no option to identify the one which is being used.
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
ubbd (Userspace Backend Block Device) is a method to build block device
and handle IO by process started in userspace. That means we can provide
a generic block device to user and handle IO requests by librbd
in linux.
This way, we can allow user to use rbd image as a linux block device
supporting full image features, especial the journaling feature, which
is not supported by krbd.
For more information: https://github.com/DataTravelGuide/ubbd
Signed-off-by: Dongsheng Yang <dongsheng.yang.linux@gmail.com>
Arrow Flight integration is triggered by defining
WITH_RADOSGW_ARROW_FLIGHT=ON with the cmake invocation.
For now this assumes that grpc-plugins is installed on the system and
won't be built internally.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
The Windows client does not use libcurl for anything. Remove it to
simplify the build process.
Note, if we ever add libcurl back on Windows, we should disable unused
protocols to harden the build:
--disable-ftp --disable-ldap --disable-ldaps --disable-rtsp \
--disable-dict --disable-telnet --disable-tftp --disable-pop3 \
--disable-imap --disable-smb --disable-smtp --disable-gopher \
--disable-mqtt --disable-manual --disable-ntlm
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
To build with DAOS backend, use -DWITH_RADOSGW_DAOS=YES cmake
option. `daos-devel` rpm should be installed beforehand.
To connect to DAOS pool, add the following configuration
parameters to ceph.conf:
```
[client]
...
rgw backend store = daos
daos pool = tank
```
A pool could be created using the following command:
```
dmg pool create --size=<size> <pool_name>
```
To install `daos-devel` do:
```
sudo wget -O /etc/yum.repos.d/daos-packages.repo https://packages.daos.io/v2.0/EL8/packages/x86_64/daos_packages.repo
sudo rpm --import https://packages.daos.io/RPM-GPG-KEY
sudo yum install -y epel-release daos-server daos-client daos-devel
```
Co-authored-by: Walter Warniaha <walter.warniaha@seagate.com>
Signed-off-by: Zuhair AlSader <zuhair.alsader@seagate.com>
so the `DOWNLOAD_EXTRACT_TIMESTAMP` property of
`ExternalProject_Add()` command is set by default on CMake v3.24 and up.
it helps to set the a more accurate timestamp for the downloaded
content, hence the targets depending on the extracted content can be
rebuilt if the URL changes.
see also https://cmake.org/cmake/help/latest/policy/CMP0135.html
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
we were using a for loop for this purpose, but the for loop was unrolled
when we bumped up the required cmake version.
this change paves the road to setting "CMP0135" to "NEW". this policy
is a new one introduced by CMake v3.24.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
ccache only works for c and c++, so instead of using the universal
`RULE_LAUNCH_COMPILE` use `CMAKE_<LANG>_COMPILER_LAUNCHER` instead,
so ccache is only configured for c and c++ compilation. this is a better
solution for integrating ccache into our building system.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
When building boost, try to schedule multiple build jobs in parallel. If
provided with `-DBOOST_J=<n>`, the given number of jobst is going to be
used.
Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
in src/CMakeLists.txt, if "WITH_FUSE" is true, we always link ceph-fuse
against FUSE::FUSE no matter what FUSE_FOUND is.
to avoid the FTBFS when FUSE is not found when building ceph-fuse, we'd
better fail early by marking FUSE a must have.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
before this change, the ALLOCATOR cmake option are handled at two
difference places: one to handle ALLOCATOR option, another to add
compilation options based on ALLOCATOR. after this change:
* the ALLOCATOR option is handled in a single place
* dedup the branches handling different ALLOCATORS into a single
condition: (NOT ALLOCATOR STREQUAL "libc")
* add_compile_options() calls are consolidated into a single one
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
* CMakeLists.txt:
always pass "EXACT" to find_package(Python3).
because per cmake document, "EXACT" only takes effect when
<Package>_FIND_VERSION_COUNT is greater than 1, where <Package>
is "Python3". see also cmake/modules/FindPython/Support.cmake
* cmake/modules/AddCephTest.cmake:
drop redundant find_package(Python3) calls. since Python3 is
a mandatory requirement for building Ceph, we only need a
single call of find_package(Python3..) in the top of the source
tree. the only possible case to repeat it is to ensure that we
have the correct version of Python3 used in following CMake
script. but there is no need to repeat it if we just want to
ensure that we have a python3 interpretor in place.
* cmake/modules/Distutils.cmake:
always pass "EXACT" to find_package(Python3).
we should always pass EXACT to find_package() when finding python3,
this is a follow-up of e2babdfae8
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
The purpose of this patch is to add the initial support to
offload memory/pmem operations by sync usage through hardware path
in DML library.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
