If user enable fiemap feature in osd side, there exists three ways to make
read/write sprase:
1. normal sparse read/write op
2. clone op
3. recover op
If any op doesn't support sparse way, it will destroy the advantage of this
way and write zero hole to sparse file. Now clone op can support sparse write,
this commit implement sparse read/write for recover op.
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
With this, it no longer needs to special case the MDS,
as the MDS can just call unset_honor_osdmap_full.
Signed-off-by: John Spray <john.spray@redhat.com>
Allow users to set an artificial upper bound
on size of the memstore OSD, and report actual
usage information in fsstat.
This is a useful tool for simulating nearly-full
systems.
Signed-off-by: John Spray <john.spray@redhat.com>
The osd/osd-test-helpers.sh mon/mon-test-helpers.sh are useful to create
shell based functional tests. Since they were introduced they did not
grow significantly. As more tests are added, it becomes necessary to
have more sophisticated helpers. The ceph-helpers.sh toolbox is a merge
of the existing helpers intended to be easy to maintain and expand.
* Each function is documented
* Each function is tested
When adding to a function or looking for a new function, it is unlikely
the developer will find time to explore the entire file and understand
its structure. The documentation, body and test for each function are
grouped together to limit the scope of exploration required from the
casual developer.
The test are designed to not use internal helpers so that they can be
understood in full just by reading the test function. It means a given
test takes longer than it should (i.e. each getters tests run one mon).
In the context of testing this toolbox, simplicity is preferred over
efficiency.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
If --with-jemalloc option is not specified, then it defaults to tcmalloc.
If --with-jemalloc is specified, then --without-tcmalloc is required.
Also if a rocksdb static linking is desired --with-librocksdb-static,
do not use --with-jemalloc, as rocksdb uses tcmalloc for builds.
Support for jemalloc in package build and perf stats/heap profiler is
underway
Signed-off-by: shishir gowda <shishir.gowda@sandisk.com>
Make sure the message we got from the peer OSD is valid. Specifically,
this avoids a race like this:
- A marks down B
- B sends MOSDRepScrub
- A accepts connection from B as new
- A replies to scrub
- B crashes because msgr seq 1 < expected seq 1000+
See #8880 for the most recent fix for requests.
Fixes: #9555
Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
These do nothing but op->get_req(); pass the Message* explicitly so that
non-OpRequest callers can use them.
Signed-off-by: Sage Weil <sage@redhat.com>
Add a ceph-disk test to first setup an OSD with a separate journal
block device, then tear down the OSD (simulating a failure) and create
a new OSD which re-uses the same journal device.
Add create_dev / destroy_dev helpers that encapsulate the operations
that ensure the partition table is up to date in the kernel and the
symlinks are created as expected. In particular it makes sure the kernel
is aware that the partition table of a newly created device is
empty. If the device previously existed and the kernel was not informed
of the latest partition table updates via partprobe / partx, it may
have cached an old partition table which can create all sorts of
unexpected behaviors such as a failure to create the by-partuuid
symbolic links as described in http://tracker.ceph.com/issues/10146
Refs: #10146
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Loopback and cciss devices have an extra 'p' before the partition
number. Add exceptional handling of these devices.
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
We observe that the new /dev/disk/by-partuuid/<journal_uuid>
symlink is not always created by udev when reusing a journal
partition. Fix by not changing the uuid of a journal partition
in this case -- instead we can reuse the existing uuid (and
journal_symlink) instead. We also now assert that the symlink
exists before further preparing the OSD.
Fixes: #10146
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Tested-by: Dan van der Ster <daniel.vanderster@cern.ch>
Encapsulate the compilation steps (install dependencies, autogen.sh,
configure, make check) in the run-make-check.sh script. Update the
developer documentation to point to this script instead of multiple
steps.
It is intended as a tool to help new developer make sure their patch is
sane, it focuses on efficiency (runs make check in parallel if possible)
and coverage (enables docker based tests if possible).
http://tracker.ceph.com/issues/10265Fixes: #10265
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Instead of cloning the repository, create a work directory that has
symbolic links to the original .git, using
https://github.com/git/git/blob/master/contrib/workdir/git-new-workdir
It resolves the problem of fetching a commit that is not attached to any
ref, which is apparently not implemented in the git protocol (discussed
on irc.freenode.net#git).
http://tracker.ceph.com/issues/10264Fixes: #10264
Signed-off-by: Loic Dachary <ldachary@redhat.com>
If we ping before the watch is registered we will get ETIMEDOUT (the ping
is are a read op that will fail to see the watch registration).
Signed-off-by: Sage Weil <sage@redhat.com>
