before this change, we use docker for running promtools offered by
a docker image, but this is not efficient, and quite a few developers
do not want to use docker for running "make check". this change was
introduced by #39246, the reason was that, in Ceph's CI process, we
are using Ubuntu/Bionic for running "make check" jobs, but prometheus
packaged by Bionic does not offer the "test rules" command. so, to
address problem, we are using "dnanexus/promtool:2.9.2" docker image
for verifying monitoring/prometheus/alerts/test_alerts.yml.
after this change, we use prometheus packaged by debian derivatives
instead of pulling a docker image.
* debian/control: add prometheus as a "make check" dependency
* install-deps.sh: partially revert
53a5816ded, as we don't need to
pull docker or start docker service for using promtool anymore.
* cmake: check if promtool is capable of running "test rules"
command, bail out if it is not.
see also: https://tracker.ceph.com/issues/49653
Signed-off-by: Kefu Chai <kchai@redhat.com>
we bump boost on regular basis. let's take the opportunity of moving to
focal to use boost v1.75.
v1.73 was used before this change. since both boost 1.75 and boost 1.73
install some files at the same places, we need to remove boost 1.73
before installing boost 1.75.
Signed-off-by: Kefu Chai <kchai@redhat.com>
the vanilla FindBoost.cmake pulled from cmake has couple assumptions
which do not hold in our environment. so address them case by case.
Signed-off-by: Kefu Chai <kchai@redhat.com>
WITH_ZBD is enabled for testing the build of zbd bluestore backend, and
we plan to migrate to Ubuntu/Focal for testing "make check", so need to
install libzbd when the distro version is focal.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/40200/head:
mgr/cephadm: clean up misc messages
mgr/cephadm/configcheck: do not spam info every minute
Reviewed-by: Adam King <adking@redhat.com>
in
ceph-build/ceph-perf-pull-requests/config/definitions/ceph-perf-pull-requests.yml,
we pass "vstart-base crimson-osd" as the targets argument, but the
build() function in ceph/src/script/run-make.sh fails to quote them, so
they are expanded into two argument of `test -n`. hence it breaks like
src/script/run-make.sh: line 124: test: vstart-base: binary operator expected
make will run with option(s) -j40
Unknown argument vstart-base
Unknown argument crimson-osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
to reduce the resource usage when running tests
there is an exception though, as we want to test test_config.cc with
multiple reactors.
Signed-off-by: Kefu Chai <kchai@redhat.com>
without this change the seastar based tests fail on host with 48 cores,
because the /proc/sys/fs/aio-nr used by the tests is greater than
1048576. if run-make-check.sh is used to launch the test, the default
job number is `$(nproc) / 2`, and the peak number of /proc/sys/fs/aio-nr
when running ctest was 3190848 when testing on the 48-core host.
so we need to increase fs.aio-max-nr accordingly to the available cores
on the host.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to address the failure of
ERROR: Cannot install -r requirements-lint.txt (line 2) and -r requirements-lint.txt (line 8) because these package versions have conflicting dependencies.
The conflict is caused by:
flake8 3.8.4 depends on pycodestyle<2.7.0 and >=2.6.0a1
autopep8 1.5.6 depends on pycodestyle>=2.7.0
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
Signed-off-by: Kefu Chai <kchai@redhat.com>
osd: remove a ceph_assert() from a legitimate path
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sam Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
* refs/pull/40048/head:
mgr/cephadm: stop conflicting daemon when deploying to a specific port
mgr/cephadm: make DaemonPlacement print nicer
mgr/cephadm: fix --force remove comment
mgr/cephadm/schedule: choose an IP from a subnet list
mgr/cephadm: rgw: clean up config and config-key values on removal
mgr/cephadm: rgw: drop .crt extension when storing cert in config-key
mgr/cephadm/services: allow beast/civetweb to bind to a particular IP
python-common: add 'networks' property to ServiceSpec
mgr/cephadm/schedule: match placement ip only combination with port
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
socket.getfqdn() will return the reverse lookup for 127.0.1.1, which is
the last item listed for that IP in /etc/hosts. Podman, by default, will
append the container name (ceph-$fsid-$name) to that line, which is not
a valid hostname, and not what we want the dashbaord to use for the URI
it advertises in the service map.
Pass --no-hosts to podman to disable this.
Docker does not appear to modify /etc/hosts by default--or, more
importantly, does not add the container name there.
Explicitly instruct podman (and docker) to add a
Fixes: https://tracker.ceph.com/issues/49890
Signed-off-by: Sage Weil <sage@newdream.net>
on_replica_init() might be legitimately called twice,
if the replica was waiting for updates to complete
before servicing the request.
Fixes: https://tracker.ceph.com/issues/49867
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
At the end of the lost_unfound tests add an additional wait_for_clean()
check to ensure that recoveries get enough time to complete before
proceeding and avoid failures down the line. For e.g. failure like
"Scrubbing terminated -- not all pgs were active and clean." is because
recoveries on the PGs did not get sufficient time to complete even though
they were bound to eventually complete.
Fixes: https://tracker.ceph.com/issues/49844
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
If we are deploying a daemon to bind to a specific port and there is
an existing daemon we are removing that also binds to that port, stop
it first. Unless we are both binding to different IPs.
This resolves the case where daemons bind to * and we redeploy with a
subnet to bind to. It would eventually converge before, but would
throw a bind error in the process and take longer.
Signed-off-by: Sage Weil <sage@newdream.net>
crimson/onode-staged-tree: fix tree_cursor_t::Cursor to be aware of extent duplication
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
* refs/pull/40160/head:
qa/suites/rados/cephadm/orchestrator_cli: random-distro$ -> 0-random-distro$
qa/suites/rados/cephadm/smoke-roleless: distro -> 0-distro
qa/distros/podman: install kubic once per host, in parallel
qa/suites/fs/multiclient: use clients: not all: for pexec
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
PR #40062 tweaked the behavior of lockdep to compile it out
of the code entirely for release builds. This fixes several
gtests where lockdep was force-enabled.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
