Commit Graph

124188 Commits

Author SHA1 Message Date
Kefu Chai
e7597405c3 do_cmake.sh: refactor code to set PYBUILD
so it is more consistent with rhel/centos part.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-07 17:53:09 +08:00
Nizamudeen A
7c1df692f2 mgr/dashboard: Include Network address and labels on Host Creation form
The ability to create host by specifying network address and also create
labels.

https://tracker.ceph.com/issues/50318
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-06-07 14:47:09 +05:30
Kefu Chai
6eacbdcac1 cmake: let vstart-base depend on cython_rados if NOT WIN32
to disable this custom target if NOT WIN32 without explanations is difficult
to maintain, so add a comment, and limit the scope to just
`cython_rados`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-07 16:02:20 +08:00
Kefu Chai
e1a081a2c4 cmake: do not print warning in each cmake run
cmake prints the warning of

> run-promtool-unittests is skipped due to missing promtool

everytime if the test is skipped per the test. but we only need to
get this message the first time cmake is executed. more importantly,
the value of `PROMTOOL_EXECUTABLE` is cached. so the new error message
does not provide updated information, so just stop doing so if the
cached variable is found.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-07 15:13:12 +08:00
Kefu Chai
9b1d524839 debian: mark "crimson" specific deps with "pkg.ceph.crimson"
see also https://wiki.debian.org/BuildProfileSpec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-07 12:36:19 +08:00
Kefu Chai
c0fad448e9 debian: mark "make check" deps with "pkg.ceph.check" build
instead of inventing our way for defining "make check" dependencies, use
build profile for adding "make check" specific dependencies. see
https://wiki.debian.org/BuildProfileSpec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-07 12:35:00 +08:00
Kefu Chai
cf8814d316
Merge pull request #41695 from tchaikov/wip-crimson-net-move
crimson/net: move from out_q into sent queue

Reviewed-by: Amnon Hanuhov <ahanukov@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-07 09:52:36 +08:00
Kefu Chai
76d882bd6d
Merge pull request #41708 from tchaikov/wip-seastore-open-coll
crimson/os/seastore: open_collection() returns nullptr if DNE

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-06 09:45:16 +08:00
Sage Weil
d49b3712d3 Merge PR #41665 into master
* refs/pull/41665/head:
	mgr/cephadm:fix alerts sent to wrong URL

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-06-05 16:43:36 -04:00
Sage Weil
4757a13dc6 cephadm: improve is_container_running()
The 'podman ps' command sporatically exits with 125, I suspect due
to some race/bug in podman itself.

However, our goal here is just to check if a specific container
is running.  Use inspect for that instead--it's even (a bit) faster.

Fixes: https://tracker.ceph.com/issues/51109
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-05 15:39:29 -05:00
Kefu Chai
89951a1d20
Merge pull request #40652 from ronen-fr/wip-ronenf-cscrub-class
osd/scrub: modify "classic" OSD scrub state-machine to support Crimson

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-06-06 00:06:32 +08:00
Sage Weil
6bd4df3573 common/cmdparse: fix CephBool validation for tell commands
The tell/asok validation uses a different validation path; make it work
for CephBool arguments.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-05 11:23:35 -04:00
Kefu Chai
0d38c8a668
Merge pull request #41154 from rzarzynski/wip-global-backtrace-bug-50647
global: fault handlers cope with simultaneous faults now.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-05 21:41:00 +08:00
Kefu Chai
4857c35084
Merge pull request #41604 from t-msn/wip-51030
osd/ECBackend: Fix null pointer dereference when enabling jaeger tracing

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-06-05 21:33:00 +08:00
Kefu Chai
4855e971ce
Merge pull request #41501 from aclamk/wip-bluefs-safer-flush
os/bluestore: Remove possibility of replay log and file inconsistency

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-06-05 21:26:04 +08:00
Kefu Chai
b2823a457f
Merge pull request #41506 from ceph/wip-cv-batch-fixes
ceph-volume: fix batch report and respect ceph.conf config values

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-06-05 21:23:13 +08:00
Kefu Chai
6ba7763f80
Merge pull request #41688 from tchaikov/wip-debian-rook
debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-roo…

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-05 21:17:24 +08:00
Kefu Chai
9fbd60131b crimson/os/seastore: open_collection() returns nullptr if DNE
we check for the existence of meta collection by trying to open it,
if it exists, we continue check for the superblock stored in it, if
the superblock does not exist or corrupted, we consider it as a failure.

before this change, open_collection() always return a valud Collection
even if the store does not have the collection with specified cid. this
behavior could be misleading in the use case above.

after this change, open_collection() looks up the collections stored in
root collection node for the specfied cid, and return nullptr if it does
not exist.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 18:02:45 +08:00
Kefu Chai
ca48d1cbdb crimson/os/seastore: use structured binding
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 18:02:45 +08:00
Kefu Chai
854df814d9 script/ceph-debug-docker: s/x86_64/$(arch)/
try to avoid hardwiring to a certain architecture if possible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 12:22:00 +08:00
Kefu Chai
7804a3180f pybind: add tox.ini
also add ceph_daemon to mypy.ini

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 12:10:37 +08:00
Kefu Chai
024012a943 pybind/ceph_daemon: add missing type annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 12:08:40 +08:00
Kefu Chai
9121d5f711 pybind/ceph_daemon: correct type annotation of admin_socket()
we always pass a list of string to it, but since it does not assume the
container type of string, just specify Sequence[str].

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 11:53:08 +08:00
Kefu Chai
fb50de457c pybind/ceph_daemon: fix the format string in Termsize.__repr__()
we intended to print the class name usign "%s".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 11:40:26 +08:00
Kefu Chai
2d0d87dc9b pybind/ceph_daemon: do not handle get_command_descriptions in admin_socket()
we never pass a single string to admin_socket() as its 2nd parameter.
instead, we always pass a list of string. so no need to handle it as a
special case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 11:38:18 +08:00
Kefu Chai
6103f833df common/options: extract mgr and mon options out
this change is a part of a sequence of changes like
9b5e54d996 and
dff996d9e2. because we need to audit
the consumers of an option before extracting it to the dedicated
.yaml.in file, it takes considerable time to get this done.

so let's do this piecemeal.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 10:45:26 +08:00
Patrick Donnelly
84ae38594d
qa: test nowsync option in kernel client workflows
Fixes: https://tracker.ceph.com/issues/49341
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 19:15:12 -07:00
Patrick Donnelly
88f74dbfa6
qa: deep merge top level overrides for fuse/kclient
This allows for array/dict configs like mntopts to accumulate changes
from multiple yaml fragments.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 19:15:12 -07:00
Kefu Chai
9370ac9fa8
Merge pull request #41581 from tchaikov/wip-options-mgr-mon
common/options: extract mgr and mon options out

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-06-05 10:06:07 +08:00
Kefu Chai
e8f1477da1
Merge pull request #40073 from jmolmo/delete_service_causes_osd_removal
mgr/cephadm: Warn about OSDs to remove manually when deleting an OSD service

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-06-05 08:44:42 +08:00
Sage Weil
ffce51af8e mgr/nfs: fix 'nfs export create' argument order
Put path before --readonly so that it can still be positional.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
ec293f34b1 common/cmdparse: emit proper json
Instead of '"req": "false"', emit '"req": false'.  Same for conditional.

Luckily, the clients don't really care about this change, as
ceph_argparse.py argdesc interpets the JSON like so:

            self.req = req in (True, 'True', 'true')
            self.positional = positional in (True, 'True', 'true')

Clean up command definitions to use lowercase 'false', but tolerate
both for backward compat during upgrade and to tolerate future errors.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
b6b5ecc210 mon/MonCommands: add -- seperator to example
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
8683cccd06 qa/tasks/cephfs/test_nfs: fix export create test
Everything after --readonly is non-positional.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
218f4cefea mgr: make mgr commands compat with pre-quincy mon
If the mon is not yet quincy, do not share the quincy-ism of
'positional=false'.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
295a5f2827 doc/_ext/ceph_commands: handle non-positional args in docs
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
c70a4f583c mgr: fix reweight-by-utilization cephbool flag
This was broken, since it was trying to get a CephChoices as a bool.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Sage Weil
bd242c4997 mon/MonCommands: convert some CephChoices to CephBool
These are old bool options that never got converted to CephBool.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 16:56:17 -04:00
Patrick Donnelly
8f8ba3d783
Merge PR #41697 into master
* refs/pull/41697/head:
	script: add a few more volume mounts for sepia
	script: drop ceph-fuse from docker debugging
	script: enable centos debuginfo repo for debugging
	script: update repo url for multi-arch builds
	script: fetch autobuild.asc key via HTTPS

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-04 13:07:42 -07:00
Kefu Chai
68dd82e2c2
Merge pull request #41690 from tchaikov/wip-test-alloc_aging
test/objectstore/unittest_alloc_aging: init cct

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-06-05 01:57:03 +08:00
Sage Weil
d4a6c3d009 mgr/telemetry: pass leaderboard flag even w/o ident
Allow non-identified clusters to appear in the leaderboard.
The leaderboard option still defaults to false, so the change here
is that if they opt in to leaderboard but not ident we'll see
that on the backend.

Note that a leaderboard still does not exist (yet), so this doesn't
have any immediate impact.  But if/when we do create one, it will
allow us to show big clusters (that opt in) on the leaderboard
as 'unidentified' or similar.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 12:49:40 -05:00
Daniel Pivonka
adecb2d2da mgr/cephadm: allow monitoring stack services to bind to network and port
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
2021-06-04 13:49:05 -04:00
Kefu Chai
6d0c175ae7
Merge pull request #41698 from tchaikov/wip-qa-rook
qa/suites/orch/rook/smoke: stop testing on ubuntu 18.04

Reviewed-by: Sage Weil <sage@redhat.com>
2021-06-05 01:23:35 +08:00
Kefu Chai
104f054cee qa/suites/orch/rook/smoke: stop testing on ubuntu 18.04
even rook does not really install ceph packages in the host directly, it
uses the ceph container image. but teuthology insists on checking the
existence of debian packages by querying shaman server when it sees a
teuthology facet file which includes:

os_type: ubuntu
os_version: "18.04"

but since we've stopped building ubuntu/bionic packages, teuthology
just complains when we are scheduling test suites which are composed
from facets in qa/suites/orch/rook/smoke.

in this change, the ubuntu_18.04.yaml is dropped because ubuntu/bionic
does not really increase the test coverage of ceph. it helps to test
the rook and container runtime though.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-05 01:11:14 +08:00
Patrick Donnelly
51b60e990b
script: add a few more volume mounts for sepia
We now have a few Ceph file systems with various possible mount points
depending which lab machine you're using.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 09:33:54 -07:00
Patrick Donnelly
9d71fc1bee
script: drop ceph-fuse from docker debugging
Install this on the fly as necessary...

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 09:33:30 -07:00
Patrick Donnelly
efbfd678a2
script: enable centos debuginfo repo for debugging
So we can fetch e.g. the sqlite debuginfo packages.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 09:32:52 -07:00
Patrick Donnelly
472bc091ad
script: update repo url for multi-arch builds
Brad suggested this change based on his commit [1]. Thank you!

[1] 267cce9e83

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 09:31:19 -07:00
Patrick Donnelly
6223ab14cc
script: fetch autobuild.asc key via HTTPS
Rather than relying the key being avaiable on the LRC /ceph file system.
(Someone appears to have deleted it recently.)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-04 09:30:04 -07:00
Sage Weil
c8c5071dcd qa/tasks/cephfs/test_sessionmap: reap connections immediately
We have to reap connections promptly for this test to work.

This test was broken indirectly by d51d80b323,
which moved the counter decrement to reap time instead of mark_down/stop
time.

The reaping is asynchronous, so allow for a delay in the count change.

Fixes: https://tracker.ceph.com/issues/50622
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-04 11:02:29 -04:00