Commit Graph

107509 Commits

Author SHA1 Message Date
Jason Dillaman
57d373fa99 qa/workunits/rbd: override CEPH_ARGS when initializing the site name
The mirroring site name is stored in the MON config which requires
higher privledges than the standard "client.mirror" user.

Fixes: https://tracker.ceph.com/issues/44066
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-02-10 11:39:24 -05:00
Sage Weil
9169a037d9 Merge PR #33137 into master
* refs/pull/33137/head:
	mon/PGMap: fix digest summary of PGs by state
	mon/PGMap: fix compat encoding of num_pg_by_state

Reviewed-by: yanjun <yan.jun8@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-10 08:54:51 -06:00
Kefu Chai
d352315e52 Merge pull request #33035 from jdurgin/wip-target-ratio
mgr/pg_autoscaler: treat target ratios as weights

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-10 18:41:07 +08:00
Josh Durgin
fff3b704de mgr/pg_autoscaler: add unit tests for effective_target_ratio
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 13:00:00 +08:00
Josh Durgin
63f2a06568 mgr/pg_autoscaler: remove unused imports
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
37519d26bb mgr/pg_autoscaler: cleanup pool_id type
Keep it as an int so we don't have to cast back and forth.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
2050cc2a09 mgr/pg_autoscaler: correct and simplify progress tracking
Reset the progress each time we make an adjustment, and track progress
from that initial state to that new target. Previously we were also
using the wrong target: the current pg_num_target, not the new value
(pg_num_final) that we set.

Look up the pool by name, not id, in _maybe_adjust(), since that is how it is
retrieved by osdmap.get_pools_by_name().

Dedupe some logic into PgAdjustmentProgress to simplify things.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
9118065c4c PendingReleaseNotes: mention target_size_ratio change
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
0a29cf7d6c doc: update autoscaler docs for target ratio
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
9758197afa mgr/pg_autoscaler: include effective target ratio in status
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
772d7c1d3c mgr/pg_autoscaler: add warning when target bytes and ratio are both set
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
d62c121ee3 mgr/pg_autoscaler: remove target ratio warning
Since the ratios are normalized, they cannot exceed 1.0 or overcommit
combined with target_bytes.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
1ea8ce3480 qa: use ratios >1 in pg_autoscaler test
Also check for pg_num_target being set correctly, rather than pg_num,
so the test doesn't depend on merging/splitting speed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Josh Durgin
435c56a03b mgr/pg_autoscaler: treat target ratios as weights
Normalize across pools so that it's simpler to use - this way you
don't have to adjust every other pool when you add one.

Handle pools with target_bytes by taking their capacity off the top,
and dividing the rest into the pools with a target_ratio.

If both target bytes and ratio are specified, ignore bytes. This
matches the docs and makes accounting simpler.

Fixes: https://tracker.ceph.com/issues/43947

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-02-10 10:08:36 +08:00
Sage Weil
67a2a4e7aa Merge PR #33126 into master
* refs/pull/33126/head:
	doc/mgr/orchestrator_cli: update support table
	mgr/deepsea: remove
	mgr/ansible: remove

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-02-09 15:36:32 -06:00
Sage Weil
ceff81c9a5 Merge PR #33136 into master
* refs/pull/33136/head:
	cephadm: fix ceph version probe

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-02-09 15:34:59 -06:00
Sage Weil
a7d5d5cc8c cephadm: fix ceph version probe
docker returns '<no value>' if the label isn't present, in which case we
still need to run ceph -v.

Also, don't probe non-ceph (e.g., monitoring) containers.

Also, only probe each image id once.

Add a simple test.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-09 14:00:50 -06:00
Sage Weil
9dba764c02 Merge PR #33133 into master
* refs/pull/33133/head:
	qa/workunits/cephadm/test_cephadm.sh: make monitoring tests faster
	qa/workunits/cephadm/test_cephadm: 2 OSDs is enough
	cephadm: disable node-exporter cpu/memory limits for the time being

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-02-09 13:55:27 -06:00
Sage Weil
c11d031bab Merge PR #33134 into master
* refs/pull/33134/head:
	qa/workunits/cephadm/test_repos: don't try to use the refspec

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-02-09 12:48:09 -06:00
Sage Weil
b5a01b7a3b doc/mgr/orchestrator_cli: update support table
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-09 09:47:40 -06:00
Sage Weil
2e3ed251a6 mgr/deepsea: remove
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-09 09:45:36 -06:00
Sage Weil
0fff7d61b5 Merge PR #33117 into master
* refs/pull/33117/head:
	qa/suites/upgrade/nautilus-x-singleton: ensure hit sets behave across upgrade
	osd/PrimaryLogPG: use legacy timestamp rendering for hit_set objects
	include/utime: allow legacy rendering of timestamp

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-09 09:40:10 -06:00
Sage Weil
1453154c25 mon/PGMap: fix digest summary of PGs by state
This fixes the PG state summary seen in 'ceph -s' to reflect PG state bits
beyond 32 bits.

Fixes: https://tracker.ceph.com/issues/44052
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-09 09:27:53 -06:00
Sage Weil
b9359d0f54 mon/PGMap: fix compat encoding of num_pg_by_state
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-09 09:22:56 -06:00
Kefu Chai
5560680929
Merge pull request #32928 from ljishen/wip-rados-bench-latency
rados bench: fix the delayed checking of completed ops

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-02-09 18:43:38 +08:00
Kefu Chai
1867afcef0
Merge pull request #32934 from rzarzynski/wip-bl-32bytes
include, common: make ceph::bufferlist 32 bytes long on x86

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-09 18:41:03 +08:00
Kefu Chai
97182e75da
Merge pull request #33099 from howard0su/wip_ceph_fix
ceph.in: print decoded output in interactive mode

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-09 18:36:03 +08:00
Kefu Chai
f66dc746c8
Merge pull request #33101 from tchaikov/wip-thread-join
pybind/ceph_argparse: avoid int overflow

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-09 18:33:26 +08:00
Yuval Lifshitz
2277588617
Merge pull request #33068 from yuvalif/verify-pubsub-inc-sync
qa/rgw/pubsub: verify incremental sync is used in pubsub
2020-02-09 11:55:57 +02:00
Patrick Donnelly
16e58d65d4
Merge PR #32816 into master
* refs/pull/32816/head:
	mds: check inode type when deciding if filelock should be in EXCL state
	mds: don't delegate inos when handling replayed requests
	mds: process re-sent async dir operations at clientreplay stage
	mds: consider async dirops when checking directory empty
	mds: always suppress issuing caps in Locker::issue_new_caps()
	mds: try reconnect cap only when replayed request creates new inode
	mds: set cap id to 1 for newly created inode

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-08 17:51:56 -08:00
Sage Weil
36daf1cc29 qa/workunits/cephadm/test_cephadm.sh: make monitoring tests faster
The sleep 90 was way overkill.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-08 18:00:37 -06:00
Sage Weil
0594526393 qa/workunits/cephadm/test_cephadm: 2 OSDs is enough
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-08 18:00:37 -06:00
Sage Weil
99bcf59783 cephadm: disable node-exporter cpu/memory limits for the time being
Ubuntu 18.04 kernel does not support these.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-08 08:22:24 -06:00
Sage Weil
12a0faa9fa Merge PR #33092 into master
* refs/pull/33092/head:
	doc/rados/operations: adjust docs a bit
	mon/OSDMonitor: accept 'autoscale_mode' argument to 'osd pool create'

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-02-08 08:10:57 -06:00
Sage Weil
38d564cc07 Merge PR #33094 into master
* refs/pull/33094/head:
	mgr/telemetry: split entity_name only once (handle ids with dots)

Reviewed-by: Sage Weil <sage@redhat.com>
2020-02-08 08:08:11 -06:00
Sage Weil
593420d0c7 Merge PR #33037 into master
* refs/pull/33037/head:
	osd/OSD: choose more heartbeat peers from different subtrees

Reviewed-by: yanjun <yan.jun8@zte.com.cn>
2020-02-08 08:08:01 -06:00
Sage Weil
8cdbc8e8e6 qa/workunits/cephadm/test_repos: don't try to use the refspec
This is usually a sha1, and we can't reliably find packages based on
that.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-08 07:33:47 -06:00
Sage Weil
82c4bce2a1 Merge PR #33132 into master
* refs/pull/33132/head:
	qa/workunits/cephadm/test_repos: apt update
	qa/workunits/cephadm/test_repos: fix subst
	qa/suites/rados/cephadm/.../test_repos: run without ulimit and coverage tools
	qa/tasks/workunit: accept coverage_and_limits: false
	qa/suites/rados/cephadm: move ubuntu_18.04_podman to shared location
	qa/suites/rados/cephadm: fix conflicts, missing .qa link

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-02-08 07:27:27 -06:00
Sage Weil
fd17368e9c qa/workunits/cephadm/test_repos: apt update
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 22:16:29 -06:00
Sage Weil
883f82aee6 qa/workunits/cephadm/test_repos: fix subst
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 21:26:03 -06:00
Kefu Chai
957c8a0cf1
Merge pull request #33061 from tchaikov/wip-pybind-compiler-flags-patch
pybind: refactor monkey_with_compiler()

Reviewed-By: Adam Emersen <aemerson@redhat.com>
2020-02-08 10:14:10 +08:00
Sage Weil
0584db6221 Merge PR #33098 into master
* refs/pull/33098/head:
	mgr/orch,cephadm: add 'host set-addr'
	mgr/orch: include addr (and labels) in 'host ls'
	mgr/cephadm: fix 'cephadm check-host'
	mgr/cephadm: use addr to contact host
	mgr/orch: pass HostSpec to add_host
	mgr/orch: HostSpec -> HostPlacementSpec

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-02-07 19:12:43 -06:00
Sage Weil
ab71e45728 qa/suites/rados/cephadm/.../test_repos: run without ulimit and coverage tools
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 17:58:08 -06:00
Sage Weil
52cb75acf3 qa/tasks/workunit: accept coverage_and_limits: false
Allow workunits without teuthology tools (normally installed by ceph.py,
IIRC).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 17:57:45 -06:00
Sage Weil
55518208ab qa/suites/rados/cephadm: move ubuntu_18.04_podman to shared location
Also set the registries.conf file so we can pull from docker.io.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 17:49:52 -06:00
Sage Weil
cbeb24323a qa/suites/upgrade/nautilus-x-singleton: ensure hit sets behave across upgrade
Create a pool that generates hit sets before the upgrade, and ensure that
they (continue to) trim after the upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 16:44:18 -06:00
Sage Weil
8f40e2cdf7 osd/PrimaryLogPG: use legacy timestamp rendering for hit_set objects
These objects exist prior to octopus and we need to be consistent with
the naming.  Stick with the legacy form.

Fixes: https://tracker.ceph.com/issues/44024
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 16:44:18 -06:00
Sage Weil
64fdf60d64 include/utime: allow legacy rendering of timestamp
In 79d8d761cf and
ec3ddcb988
we switched to a strict ISO8660 rendering for timestamps.  In some cases,
we need to render the timestamp in the legacy form: ' ' instead of 'T',
and no time zone suffix.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 16:44:18 -06:00
Sage Weil
5dd8834c4a qa/suites/rados/cephadm: fix conflicts, missing .qa link
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-07 16:32:05 -06:00
Patrick Donnelly
53a5d1f4ec
Merge PR #33080 into master
* refs/pull/33080/head:
	qa: specify random distros in multimds

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-02-07 13:26:16 -08:00