Mark Kogan 31a9255636 rgw: D3N Cache changes for Upstream ... Upstreaming / rebase of #24500 Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: change io_ctx pool per storage class Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: Changing free() to delete() Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: Addressing review comments Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: Fixing seg fault Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> Moving CacheRequest out of librados Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: cache initialization fix Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: fix missing spawn.hpp compile errors resolves compilation errors similar to: ``` [ 15%] Building CXX object src/tools/immutable_object_cache/CMakeFiles/ceph_immutable_object_cache_lib.dir/CacheController.cc.o In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_common.h:31, from /home/jenkins-build/build/workspace/ceph-pull-requests/src/rgw/rgw_rados.h:17, from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/IoCtxImpl.h:30, from /home/jenkins-build/build/workspace/ceph-pull-requests/src/librados/RadosClient.h:35, from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOSImpl.h:27, from /home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/RADOS.cc:37: /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/async/yield_context.h:31:10: fatal error: spawn/spawn.hpp: No such file or directory 31 | #include <spawn/spawn.hpp> | ^~~~~~~~~~~~~~~~~ compilation terminated. src/neorados/CMakeFiles/neorados_api_obj.dir/build.make:62: recipe for target 'src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o' failed make[3]: *** [src/neorados/CMakeFiles/neorados_api_obj.dir/RADOS.cc.o] Error 1 ``` Signed-off-by: Mark Kogan <mkogan@redhat.com> Resolving merge conflict Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> Removing all logs and unnecessary comments Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> Cache Read and Write working Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> Initial Commit L1 Cache Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> post re-rebase merge, update fixes Signed-off-by: Mark Kogan <mkogan@redhat.com> fixup! post re-rebase merge, update fixes rgw: continuation of rgwcache branch rebase Signed-off-by: Mark Kogan <mkogan@redhat.com> RGW: DataCache: post merge fixes Signed-off-by: Mark Kogan <mkogan@redhat.com> fixes of segmentation fault caused by oid Signed-off-by: E. Ugur Kaynar <ukaynar@bu.edu> rgw: fixes for segmentation faults and configuration processing Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: data cache first commit Signed-off-by: Mania Abdi <mania.abdi287@gmail.com> rgw: cleanup addressing PR comments Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: cleanup addressing PR comments, continuation. Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: pr cleanup addressing second review round Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: Addressing review comments, removing all D3N code from librados Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: for compilation err from removal of mydout() helper Signed-off-by: Mark Kogan <mkogan@redhat.com> rge: addressing review comments rgw: move d3n datacache into separate files rgw: 1st part of datacache rebranding to d3n fix forward declaration compile err (only with clang): ../src/rgw/rgw_cache.h:396:4: error: member access into incomplete type 'struct get_obj_data' d->data_lock.lock(); ^ ../src/rgw/rgw_cache.h:365:8: note: forward declaration of 'get_obj_data' struct get_obj_data; ^ Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: addressing review comments, datacache rebranding to d3n cache Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: Cleaning up unused D3N cache code Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: cont. cleaning up of rgw_obj_data() Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: Removing redundant code, fix for multipart S3 objects Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: fix: incorrect content len on multipart get in s3tests_boto3.functional.test_s3:test_multipart_copy_versioned when d3n cache is disabled Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: fix segfault reading from cache Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: fix segfault in multisite sync on secondary site Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: fix segfault in multisite teuthology tests, cont. Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: Adding drain to wait for all AIO reads to complete Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com> rgw: fix for using read() by liabio or posix io per config Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: improve persistent data cache directory handling on start create the persistent datacache directory if necessary and add an option to evict it's content if already exists Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: fix possible segfault during eviction Signed-off-by: Mark Kogan <mkogan@redhat.com> Co-authored-by: Mania Abdi <mania.abdi287@gmail.com> Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu> Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com> Co-authored-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: addressing latest review comments Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: add debug logs for cache in/out flow Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: move the L2 cache functionality to separate PR Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: addressing review comments Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: address java_s3tests teuthology issues Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: do not handle compressed objects fro now Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: l2 cleanup and log fixups + post dpp Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: thread dpp thru get_obj_iterate_cb() and related Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: bypass reading versioned objects from cache Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: cleanup and fix s3tests Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: allow to enable cache only on beast Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: delete the content of the cache directory on rgw start Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: re-enable d3n cache with civetweb frontend Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: rebase post zipper 10 Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: address teuthoogy valgrind leaks detected Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: mitigating valgrind leaks Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: remove rgw_d3n_l1_libaio_read option Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: wip segfault fix Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: mitigate libaio SIGEV_THREAD cb race Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: change libaio signaling mechanism Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: wip cont. libaio cb thread race Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: wip libaio cb thread race Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: libaio cleanups and edge case handling fixes Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: narrow the libaio locking scope Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: add libaio req ordering mechanism Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: fix lock regression Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: addressing reviwe comments and cleasnup Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: libaio locks cleanup Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: refactor libaio abstraction to share the ioc implementation Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: addressing latest review comments and cleanup Signed-off-by: Mark Kogan <mkogan@redhat.com> rgw: d3n: address review comments, cont. Signed-off-by: Mark Kogan <mkogan@redhat.com> Co-authored-by: Mania Abdi <mania.abdi287@gmail.com> Co-authored-by: E. Ugur Kaynar <ukaynar@bu.edu> Co-authored-by: Aishwarya Mathuria <amathuri@redhat.com> Co-authored-by: Ali Maredia <amaredia@redhat.com> Co-authored-by: Feng Hualong <hualong.feng@intel.com>		2021-07-06 21:36:06 +03:00
..
run-datacache.sh	qa: add workunit for testing rgw datacache	2021-07-04 14:18:06 +03:00
run-reshard.sh
run-s3tests.sh
s3_bucket_quota.pl
s3_multipart_upload.pl
s3_user_quota.pl
s3_utilities.pm
test_librgw_file.sh	test/rgw_file: override keyring if we gen keys	2021-01-25 16:46:50 -05:00
test_rgw_datacache.py	rgw: D3N Cache changes for Upstream	2021-07-06 21:36:06 +03:00
test_rgw_gc_log.sh
test_rgw_obj.sh
test_rgw_orphan_list.sh	rgw: test radosgw-admin radoslist and incomplete multiparts better	2021-04-10 02:07:53 -04:00
test_rgw_reshard.py	qa/rgw: test_rgw_reshard.py just tries common ports	2021-02-09 14:26:26 -05:00
test_rgw_throttle.sh