Commit Graph

124015 Commits

Author SHA1 Message Date
hoamer
78bb36cb62 doc/mgr/administrator: add a more precise description for creating key
added a more precise description to handle filename when creating key for mgr

Signed-off-by: hoamer <kontakt@sebastian-neugebauer.de>
2021-06-04 16:56:56 +08:00
Kefu Chai
814fb28892 debian/control: ceph-mgr-modules-core does not Recommend ceph-mgr-rook anymore
per https://www.debian.org/doc/debian-policy/ch-relationships.html

> Recommends
>   This declares a strong, but not absolute, dependency.
>
> The Recommends field should list packages that would be found together
> with this one in all but unusual installations.

ceph-mgr-modules-core provides a set of ceph-mgr modules which are
always enabeld. but the rook module enables ceph-mgr to install and
configure a Ceph cluster using Rook. this module is very useful but
it does not have such a strong connection with ceph-mgr-modules-core.
we can always install it separately for using better intergration with
Rook.

See-also: https://tracker.ceph.com/issues/45574
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-04 16:18:23 +08:00
Aashish Sharma
2c396be7cc mgr/dashboard: introduce seperate frontend component for API docs
This PR intends to add a separate frontend component for the API docs that are currently being rendered from the python code.

Fixes: https://tracker.ceph.com/issues/50955
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-06-04 13:09:20 +05:30
Kefu Chai
8b651b3522 test/objectstore/unittest_alloc_aging: init cct
* initialize the cct use by test, otherwise g_ceph_context is
  not set at all.
* instead of using g_ceph_context, use static member variable cct.
  less dependency to the global instance.
* setup and teardown the cct for test suite, because global_init()
  initialize g_ceph_context, which cannot be set multiple times.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-04 14:07:29 +08:00
Kefu Chai
f3ec742fe2 test/objectstore: s/TearDownTestCase/TearDownTestSuite/
TearDownTestCase is deprecated by GTest. let's use the new API instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-04 14:07:23 +08:00
Kefu Chai
9ed2656d79 vstart.sh: disable restful by default
since we are going to deprecate restful in favor of dashboard's RESTful
API. see
https://lists.ceph.io/hyperkitty/list/dev@ceph.io/message/LBKLNXH7UQL7TLFU5G52Y2SYVME4RS6P/
and https://tracker.ceph.com/issues/47066, let's disable this module
by default. and change the option of --without-restful to
--with-restful.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-04 13:58:51 +08:00
Kefu Chai
dba26fc7a8
Merge pull request #41652 from tchaikov/wip-qa-asock-or
qa/tasks/admin_socket: support "foo || bar" as command

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-04 13:50:38 +08:00
Kefu Chai
c4d5121305
Merge pull request #41686 from t-msn/update-trace-doc
doc/dev: update how to use lttng/blkin trace

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-04 12:30:23 +08:00
Misono Tomohiro
53a68e27bb doc/dev: update how to use lttng/blkin trace
Update doc to reflect current status.

Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
2021-06-04 11:41:29 +09:00
Sage Weil
9b9d90bddf Merge PR #41553 into master
* refs/pull/41553/head:
	ceph-volume: replace __ with _ in device_id

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-03 22:04:55 -04:00
Sage Weil
ce033d629c Merge PR #41636 into master
* refs/pull/41636/head:
	mgr/cephadm/inventory: do not try to resolve current mgr host
	pybind/mgr/mgr_module: make get_mgr_ip() return mgr's IP from mgrmap
	mgr/restful: use get_mgr_ip() instead of hostname

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-03 22:04:32 -04:00
Kefu Chai
b5d9418c11
Merge pull request #41674 from tchaikov/wip-vstart-without-restful
vstart.sh: add an option named --without-restful

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-06-04 09:44:58 +08:00
Paul Cuzner
9d408a70c7 mgr/cephadm:fix alerts sent to wrong URL
The path_prefix in prometheus.yml was specifying an
endpoint prefix, which was invalid. This resulted in 404
errors when trying to send alerts to alertmanager and
blocked alerts being sent on to the ceph-dashboard API
receiver. This fix remves this prefix.

Fixes: https://tracker.ceph.com/issues/51073

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2021-06-04 12:52:35 +12:00
Kefu Chai
25480e2452
Merge pull request #41670 from tchaikov/wip-op-tracking-spin-off-0
crimson, common: improve const-correctness of Operation::dump()s.

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-04 07:50:44 +08:00
Kefu Chai
295afe68a2
Merge pull request #41672 from tchaikov/wip-crimson-test-handle-fut
test/crimson/seastore: always handle returned future<>

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-04 07:50:21 +08:00
Patrick Donnelly
e3c4ae389e
Merge PR #41654 into master
* refs/pull/41654/head:
	mds: do not infinitely recursively print a metric

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-06-03 13:34:54 -07:00
Patrick Donnelly
943915f8df
Merge PR #41639 into master
* refs/pull/41639/head:
	mds/scrub: write root inode backtrace at creation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-03 13:33:58 -07:00
Patrick Donnelly
a12db7941b
Merge PR #41499 into master
* refs/pull/41499/head:
	qa/tasks/mds_thrash: fix thrash iteration never skip

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-03 13:33:27 -07:00
Patrick Donnelly
a52712f955
Merge PR #41443 into master
* refs/pull/41443/head:
	test: update log-ignorelist for fs:mirror test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-03 13:23:17 -07:00
Patrick Donnelly
4e1f812461
Merge PR #39910 into master
* refs/pull/39910/head:
	test: Add test for mgr hang when osd is full
	mgr: Set client_check_pool_perm to false
	mds: Add full caps to avoid osd full check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-03 13:22:23 -07:00
Patrick Donnelly
b9bf490974
qa: update RHEL to 8.4
Fixes: https://tracker.ceph.com/issues/51082
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-03 12:44:35 -07:00
Dan Mick
2544ee6d7f
Merge pull request #41559 from dmick/wip-grafana-container
monitoring/grafana/build/Makefile: revamp for arm64 builds, pushes to docker and quay, jenkins
2021-06-03 11:32:24 -07:00
Sage Weil
45e806676b mgr/cephadm/inventory: do not try to resolve current mgr host
The CNI configuration may set up a private network for the container, which
is mapped to the hostname in /etc/hosts.  For example, my test box sets
up 10.88.0.0/24 because I was using crio + kubeadm on this host earlier
(at least I think that's why):

$ sudo podman run --rm --name test123 --entrypoint /bin/bash -it quay.ceph.io/ceph-ci/ceph:master -c "cat /etc/hosts"
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.88.0.8	f9e91bf2478f test123

In any case, we should never trust a lookup of our own hostname from inside
a container!

This isn't quite sufficient, though: if this is a single-host cluster, then
we fall back to using get_mgr_ip(). That value may be distorted by the
public_network option on the mgr, but we don't have any other good
options here, and single-node clusters are unlikely to have complex
network configs.

Refactor a bit to avoid the try/except nesting.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-03 12:11:31 -04:00
Sage Weil
72d72fc5db pybind/mgr/mgr_module: make get_mgr_ip() return mgr's IP from mgrmap
The previous approach was convoluted: we tried to do a DNS lookup on the
hostname, which would fail if /etc/hosts had an entry.  Which, with podman,
it does.  And the IP it has will vary in all sorts of weird ways.  For
example, CNI on my host means that I get a dynamic address in 10.88.0.0/24.

Avoid all of that nonsense and use the IP that is in the mgrmap.  There
may be multiple IPs (v2 + v1, or maybe even IPv4 + v6 in the future); in
that case, use the first one.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-03 12:11:31 -04:00
Sage Weil
df79946174 mgr/restful: use get_mgr_ip() instead of hostname
Now we match dashboard!

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-03 12:11:31 -04:00
Neha Ojha
11252f6117
Merge pull request #41308 from sseshasa/wip-osd-benchmark-for-mclock
osd: Run osd bench test to override default max osd capacity for mclock

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-06-03 08:39:22 -07:00
Casey Bodley
38ae161670
Merge pull request #41316 from cbodley/wip-50785
rgw: parse tenant name out of rgwx-bucket-instance

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2021-06-03 11:05:00 -04:00
Kefu Chai
a53592e484
Merge pull request #41677 from tchaikov/wip-oom
ceph.spec.in: increase the mem_per_job to 3GiB

Reviewed-by: David Galloway <dgallowa@redhat.com>
2021-06-03 22:40:26 +08:00
Casey Bodley
0f44459bc9
Merge pull request #41668 from pleiadesian/patch-bucket-chown
rgw: require bucket name in bucket chown

Reviewed-by: Or Friedmann <ofriedma@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-06-03 10:28:35 -04:00
Casey Bodley
416d60c20d
Merge pull request #41462 from yehudasa/wip-50920
rgw: auth v4 client: don't convert '+' to space

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-06-03 10:16:30 -04:00
J. Eric Ivancich
9ab7c18c40 rgw: completion of multipart upload leaves delete marker
The multipart upload meta object is deleted when the multipart upload
is completed. When the bucket is versioned, it needs to be deleted
from the bucket index rather than go through versioning delete process
that adds a delete marker to the bucket index rather than simply
removing it from the bucket index.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-06-03 09:24:26 -04:00
Kefu Chai
7577fec7fd cmake: increase the MAX_{LINK,COMPILE}_MEM
based on recent observation, quite a few C++ source file take
around more than 3.0GiB to compile. for instance,
test_mock_HttpClient.cc could take up to 6270MiB memory to compile.

so increase MAX_{LINK,COMPILE}_MEM accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 21:03:03 +08:00
Kefu Chai
38be14bc0f ceph.spec.in: increase the mem_per_job to 3GiB
to lower the number of jobs, we are experiencing build failures on
a builder with 48c96t, 193 free mem. the failures were caused by
OOM killer which kills the c++ compiler

[498376.128969] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/jenkins.service,task=cc1plus,pid=1387895,uid=1110
[498376.145288] Out of memory: Killed process 1387895 (cc1plus) total-vm:3323312kB, anon-rss:3164568kB, file-rss:0kB, shmem-rss:0kB, UID:1110
[498376.315185] oom_reaper: reaped process 1387895 (cc1plus), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[498377.882072] cc1plus invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0

before this change, we use the total memory to calculate the number
of jobs, and assume that each job takes at most 2.5GiB mem. in the
case above, the # of job is 96.

after this change, we use the free memory, and increse the mem per job
to 3.0GiB. in the case above, the # of job would be 85.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 21:02:33 +08:00
Kefu Chai
e9ac37d424
Merge pull request #41669 from tchaikov/wip-crimson-asok-dump-metrics
crimson/admin: s/perf dump_seastar/dump_metrics/

Reviewed-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-03 19:45:23 +08:00
Kefu Chai
284884218a vstart.sh: use here document to display multi-line message
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 19:43:10 +08:00
Kefu Chai
edd05d764d vstart.sh: add an option named --without-restful"
so we don't need to wait for restful module to be loaded if not working
on this mgr module.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 19:43:10 +08:00
Kefu Chai
a3dd9d51a5 vstart.sh: extract create_mgr_restful_secret() out
for better readability, and so it's easier to make this step optional if
developer is not interested in using the restful mgr module.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 19:43:10 +08:00
Alfonso Martínez
09394a1991 mgr/dashboard: simplify object locking fields in 'Bucket Creation' form
- UI: The object locking retention period must be expressed in days.
  Years value from API (as API keeps param parity with RGW API) are converted to days.
- API: add more validation and increase test coverage.

Fixes: https://tracker.ceph.com/issues/49885
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2021-06-03 12:25:45 +02:00
Sridhar Seshasayee
76420f9d59 doc: Update mclock-config-ref to reflect automated OSD benchmarking
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-06-03 14:45:21 +05:30
Jonas Pfefferle
adf8c17821 ceph-volume: data_allocate_fraction: arg_validators tests
Add tests for argument validator. Check for Nan, <=0.0 and > 1.0

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
2021-06-03 10:32:55 +02:00
Jonas Pfefferle
287ab0e049 ceph-volume: data_allocate_fraction: new validation class
Move validator for data_allocate_fraction to utils.arg_validators

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
2021-06-03 09:44:16 +02:00
Kefu Chai
94224614e0
Merge pull request #41671 from liu-chunmei/seastore-logger
crimson/seastore: cleanup ceph_subsystem_filestore to seastore

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-03 15:39:16 +08:00
Kefu Chai
ea3a00a513 test/crimson/seastore: declare return type explicitly
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 15:37:34 +08:00
Kefu Chai
b6e9822058 test/crimson/seastore: always handle returned future<>
this change also silences the [-Wunused-result] warning.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-03 15:37:34 +08:00
Jonas Pfefferle
8f249d17cc python-common: add data_allocate_fraction to DriveGroupSpec
data_allocate_fraction is used to partially allocate a drive.

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
2021-06-03 09:28:35 +02:00
Jonas Pfefferle
4d518ca44c python-common: translate to_ceph_volume: add allocation fraction
Add --data-allocate-fraction to translate from json to ceph-volume
command.

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
2021-06-03 09:28:35 +02:00
Jonas Pfefferle
9e0aaa49a8 ceph-volume: data_allocate_pct: allocate percentage
Add data-allocate-pct to ceph-volume lvm batch to allow to only
allocate a certain percentage of a drive, e.g. to reduce wear leveling
impact on SSDs.

Signed-off-by: Jonas Pfefferle <pepperjo@japf.ch>
2021-06-03 09:28:35 +02:00
Radoslaw Zarzynski
f60d0a1106 common: fix a formatting nit in OpTracker::dump_ops_in_flight().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-03 14:47:39 +08:00
Radoslaw Zarzynski
84e8a81ecd crimson: improve const-correctness of Operation::dump()s.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-03 14:47:39 +08:00
chunmei-liu
4a76d8f79d crimson/seastore: cleanup ceph_subsystem_filestore to seastore
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-06-02 23:44:35 -07:00