Collapse multiple RUN statements related to systemd into a single line
to reduce the number of steps that are preserved by docker.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Add the --ref option to chose the git ref to which the working tree must
be reset instead of HEAD.
test/docker-test.sh --ref giant make
is equivalent to
test/docker-test.sh bash -c 'git reset --hard giant ; make'
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Do not force interactive mode when the bash script is specified by the
user because it should be usable to run a small script instead of
starting an interactive shell session.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
It can be removed once https://github.com/ceph/ceph/pull/3228 is merged
and xmlstarlet is found in ceph.spec.in and debian/control.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Add scripts that run make check in a container for a given operating
system version. They are a little more than oneliners and serve two
purposes:
* help new developers who do not have to figure out which options
should be used
* run make check in containers in parallel with make -j2
Signed-off-by: Loic Dachary <ldachary@redhat.com>
All run-make-check.sh arguments are transparently given to the configure
command. For instance:
run-make-check.sh --enable-root-make-check
implies
./configure --enable-root-make-check
Signed-off-by: Loic Dachary <ldachary@redhat.com>
The --enable-docker logic was that each test that needs privileges to
run uses a container to do so. The problem with this approach and
make -j8 check is that such tests compete for a single container.
The --enable-root-make-check activates tests that require privileges and
assume it is ok to use sudo to acquire privileges. The decision to run
in a container is then taken by the caller who will run something like:
docker-tests.sh ./configure --enable-root-make-check
docker-tests.sh make -j8 check
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Instead of being in the src directory. Forcing the working directory to
src is convenient to run unittests individually without the need to
change directory, but it is confusing to the user.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
If compilation is run with ./configuire --enable-root-make-check, sudo
is expected to work without human interaction.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
If any of the files in test/$os_type is newer than the creation date of
a docker image, the image is removed and rebuilt from scratch. For
instance, when a package is added to debian/control, the debian based
images are rebuilt.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
The dockerfiles must run install-deps.sh instead of duplicating the
package list that is found in ceph.spec.in and debian/control.
A directory is created for each os_type and provided as a context for
docker build to use. The former $os_type.dockerfile is moved into
$os_type/Dockerfile.in (the .in as a reminder that it will be variable
substituted).
http://tracker.ceph.com/issues/10401Fixes: #10401
Signed-off-by: Loic Dachary <ldachary@redhat.com>
jq is useful to parse json from the command line. It is however not
packaged for all supported distributions (precise has it in the backport
repository which is usually not active) and cannot be conveniently added
to debian/control because it targets all distributions and has no
conditionals.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
If the user is root, do not use sudo. The sudo package is not installed
by default on centos by default and when building from script it may be
that root is running install-deps.sh on a freshly install distribution.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Users can easily forget this. It makes shutdown potentially block, but if
they have racing callbacks they get what they ask for.
Signed-off-by: Sage Weil <sage@redhat.com>
Add a call so that callers can make sure all queued callbacks have
completed before shutting down the ioctx. This avoids a segv triggered
by the LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1
test due to the ioctx being destroyed when the in-progress callback
does a notify_ack.
Signed-off-by: Sage Weil <sage@redhat.com>
Wait for any digest updates to apply before we scrub the next chunk. This
bounds the number of repops we initiate by the size of the scrub chunk, and
it generally nicer to the cluster.
Signed-off-by: Sage Weil <sage@redhat.com>
Pass the omap data in the struct as an opaque (encoded) bufferlist. This
avoids a decode into an STL map, simplifying the crc calculation. The
win isn't huge (yet) because we still turn it into a map to pass it down
to the ObjectStore method, but that too will change in time.
Signed-off-by: Sage Weil <sage@redhat.com>
Unfortunately it's nontrivial to inject this from the usual
ceph_test_rados_api_* tests which need to run despite thrashing.
Signed-off-by: Sage Weil <sage@redhat.com>
Two things here:
1- Pass the original digest from the source across the wire, if it is
present.
2- Calculate a new digest as we receive it, and record that.
If there is a mismatch, we currently crash; need to turn this into an
EIO, most likely.
Signed-off-by: Sage Weil <sage@redhat.com>
Add a tunable to control this. Returning an EIO instead of garbled data
may not be the best approach in all environments.
Eventually we want to do something better here, like read from a replica
instead.
Signed-off-by: Sage Weil <sage@redhat.com>
If an object is being actively updated, the whole-object digest will
quickly be invalidated. On deep scrub, only record that digest if the
object is a few hours old. Otherwise, we are wasting an IO.
Signed-off-by: Sage Weil <sage@redhat.com>
If we do not have a whole-object digest, record one after a deep scrub.
Note that we make no particular attempt to avoid this on frequently
changing objects where the digest will quickly be invalidated.
Signed-off-by: Sage Weil <sage@redhat.com>
This will only trigger if we choose an auth that is known bad, but that
currently can happen, so compalin here too.
Signed-off-by: Sage Weil <sage@redhat.com>
The digest may just disagree with our best guess 'auth' shard, or it may
also disagree with the recorded oi digest. Clarify which.
Signed-off-by: Sage Weil <sage@redhat.com>
The auth may or may not know that the digest is truly bad. Note that we
can only relate scrub digests to those in oi if the scrub used a seed of
-1; older OSDs use 0.
Signed-off-by: Sage Weil <sage@redhat.com>
0 is a weak initial value for a CRC since it doesn't change with a sequence
of 0 bytes (which are relatively common). -1 is better. Use -1 when
everyone in the acting set supports it.
Signed-off-by: Sage Weil <sage@redhat.com>
