SampleDedupWorkerThread didn't actually use any of the facilities on
CrawlerThread and indeed left several of them uninitialized.
Also allows us to remove the CrawlerThread constructor which left
several members uninitialized.
Signed-off-by: Samuel Just <sjust@redhat.com>
To validate sample-dedup actually works, validate() runs
separated thread from sample-dedup and verifies
two following things.
1. check sample-dedup starts properly.
2. check references of all the chunk objects' in chunk tier
exists in designated base pool.
This routune repeats for max_valication_cnt times while
sample-dedup is running. If it doesn't raise any fail while the loop,
we can pretend sample-dedup works accurately.
If not, assert() will stop this test.
In case that a reference of chunk object doesn't exist in base pool,
validate() gives a second chance after repairing it (chunk-repair op)
to deal with false-positive reference inconsistency.
Signed-off-by: Sungmin Lee <sung_min.lee@samsung.com>
Daemonization is used for background execution of crawler.
Iterative execution makes crawler threads wake up at each period.
[usage]
ceph_dedup_tool --op sample-dedup --pool POOL --chunk-pool POOL \
--iterative --daemonize
Signed-off-by: JinyongHa <jy200.ha@samsung.com>
By using lower sampling ratio, runtime deduplication with lower overhead.
It tries deduplication with sampled few objects in base-pool, and
only deduplicates objects or chunks which are highly duplicated among
samples.
The option "sampling-ratio" is used for controling the ratio of the objects
to be picked.
Signed-off-by: JinyongHa <jy200.ha@samsung.com>
Create crawling threads which crawl objects in base pool and deduplicate
based on their deduplication efficiency. Crawler samples objects and finds
duplicated chunks within the samples. It regards an object which has
duplicated chunks higher than object_dedup_threshold value as an efficient
object to be deduplicated. Besides the chunk which is duplicated more than
chunk_dedup_threshold times is also deduplicated.
The commit contains basic crawling which crawls all objects in base pool
instead of sampling among the objects.
[usage]
ceph_dedup_tool --op sample-dedup --pool POOL --chunk-pool POOL \
--chunk-altorithm ALGO --fingerprint-algorithm FP \
--object-dedup-threshold <percentile> --chunk-dedup-threshold <number>
Signed-off-by: JinyongHa <jy200.ha@samsung.com>
cephadm: support quotes around public/cluster network in config passed to bootstrap
Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
mgr/dashboard: add flag to automatically deploy loki/promtail service at bootstrap
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
To construct TransactionManager after all the devices are discoverred.
Also, it makes the following cleanups possible:
* Cleanup SeaStore and TransactionManager factory methods.
* Decouple TransactionManager from SegmentManagerGroup.
* Drop the unnecessary tm_make_config_t.
* Drop the unnecessary add_device() methods.
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
otherwise after enabling gcc-toolset-11, cmake fails with:
- Performing Test HAVE_LIBATOMIC - Failed
CMake Error at cmake/modules/CheckCxxAtomic.cmake:66 (message):
Host compiler /opt/rh/gcc-toolset-11/root/usr/bin/g++ requires libatomic,
but it is not found
Signed-off-by: Casey Bodley <cbodley@redhat.com>
as RHEL/CentOS 9, we have the access to GCC-11, which is good enough for
compiling main HEAD even with WITH_SEASTAR=ON.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
mgr/cephadm: adding dynamic prometheus configuration based on http_sd_config
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
there is chance that we are using `yum-builddep` to prepare the
build dependencies. in that case, gcc-toolset-11-build is not
installed. it's like a chicken-egg dilemma, but the point is
`yum-builddep` is able to pull in the gcc-toolset-11-build. once
gcc-toolset-11-build is installed, we will have the %enable_devtoolset11
rpm macro.
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
install-deps.sh: ensure that pip ugrades to most recent version within virtualenv
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
pybind/mgr/cephadm/upgrade: allow upgrades without reducing max_mds
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
