Upstreaming / rebase of #24500
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: change io_ctx pool per storage class
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Changing free() to delete()
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: Addressing review comments
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: Fixing seg fault
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Moving CacheRequest out of librados
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: cache initialization fix
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix missing spawn.hpp compile errors
resolves compilation errors similar to:
```
[ 15%] Building CXX object src/tools/immutable_object_cache/CMakeFiles/ceph_immutable_object_cache_lib.dir/CacheController.cc.o
In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_common.h:31,
from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_rados.h:17,
from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/IoCtxImpl.h:30,
from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/RadosClient.h:35,
from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOSImpl.h:27,
from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOS.cc:37:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory
31 | #include <spawn/spawn.hpp>
| ^~~~~~~~~~~~~~~~~
compilation terminated.
src/neorados/CMakeFiles/neorados_api_obj.dir/build.make:62: recipe for target 'src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o' failed
make[3]: *** [src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o] Error 1
```
Signed-off-by: Mark Kogan <mkogan@redhat.com>
Resolving merge conflict
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Removing all logs and unnecessary comments
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Cache Read and Write working
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Initial Commit L1 Cache
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
post re-rebase merge, update fixes
Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixup! post re-rebase merge, update fixes
rgw: continuation of rgwcache branch rebase
Signed-off-by: Mark Kogan <mkogan@redhat.com>
RGW: DataCache: post merge fixes
Signed-off-by: Mark Kogan <mkogan@redhat.com>
fixes of segmentation fault caused by oid
Signed-off-by: E. Ugur Kaynar <ukaynar@bu.edu>
rgw: fixes for segmentation faults and configuration processing
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: data cache first commit
Signed-off-by: Mania Abdi <mania.abdi287@gmail.com>
rgw: cleanup addressing PR comments
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: cleanup addressing PR comments, continuation.
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: pr cleanup addressing second review round
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Addressing review comments, removing all D3N code from librados
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: for compilation err from removal of mydout() helper
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rge: addressing review comments
rgw: move d3n datacache into separate files
rgw: 1st part of datacache rebranding to d3n
fix forward declaration compile err (only with clang):
../src/rgw/rgw_cache.h:396:4: error: member access into incomplete type 'struct get_obj_data'
d->data_lock.lock();
^
../src/rgw/rgw_cache.h:365:8: note: forward declaration of 'get_obj_data'
struct get_obj_data;
^
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: addressing review comments, datacache rebranding to d3n cache
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Cleaning up unused D3N cache code
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: cont. cleaning up of rgw_obj_data()
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Removing redundant code, fix for multipart S3 objects
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix: incorrect content len on multipart get
in s3tests_boto3.functional.test_s3:test_multipart_copy_versioned
when d3n cache is disabled
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault reading from cache
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault in multisite
sync on secondary site
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix segfault in multisite teuthology tests, cont.
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: Adding drain to wait for all AIO reads to complete
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
rgw: fix for using read() by liabio or posix io per config
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: improve persistent data cache directory handling
on start create the persistent datacache directory if necessary
and add an option to evict it's content if already exists
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix possible segfault during eviction
Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing latest review comments
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: add debug logs for cache in/out flow
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: move the L2 cache functionality to separate PR
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing review comments
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address java_s3tests teuthology issues
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: do not handle compressed objects fro now
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: l2 cleanup and log fixups + post dpp
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: thread dpp thru get_obj_iterate_cb() and related
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: bypass reading versioned objects from cache
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: cleanup and fix s3tests
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: allow to enable cache only on beast
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: delete the content of the cache directory on rgw start
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: re-enable d3n cache with civetweb frontend
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: rebase post zipper 10
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address teuthoogy valgrind leaks detected
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: mitigating valgrind leaks
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: remove rgw_d3n_l1_libaio_read option
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip segfault fix
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: mitigate libaio SIGEV_THREAD cb race
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: change libaio signaling mechanism
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip cont. libaio cb thread race
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: wip libaio cb thread race
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: libaio cleanups and edge case handling fixes
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: narrow the libaio locking scope
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: add libaio req ordering mechanism
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: fix lock regression
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing reviwe comments and cleasnup
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: libaio locks cleanup
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: refactor libaio abstraction to share the ioc implementation
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: addressing latest review comments and cleanup
Signed-off-by: Mark Kogan <mkogan@redhat.com>
rgw: d3n: address review comments, cont.
Signed-off-by: Mark Kogan <mkogan@redhat.com>
Co-authored-by: Mania Abdi <mania.abdi287@gmail.com>
Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu>
Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Ali Maredia <amaredia@redhat.com>
Co-authored-by: Feng Hualong <hualong.feng@intel.com>
Add a workunit for testing the rgw object cache
by using s3cmd to write objects and then
verify the objects in the cache.
Also move the 0-install.yaml file out of tasks and
into the main dir for the rgw/verify subsuite.
Signed-off-by: Ali Maredia <amaredia@redhat.com>
qa: d3n: add debug logs
Signed-off-by: Ali Maredia <amaredia@redhat.com>
rgw: s3n: qa: fix netstat search for rgw process
Signed-off-by: Mark Kogan <mkogan@redhat.com>
