Commit Graph

124249 Commits

Author SHA1 Message Date
Kefu Chai
75b91d49b8
Merge pull request #39624 from sebastian-philipp/mypy-812
src,qa: Upgrade to mypy 0.901

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-14 22:53:02 +08:00
Sebastian Wagner
caf793662e
Merge pull request #41632 from jmolmo/fix_osd_rm_help_msg
mgr/cephadm: fix errors in command <ceph orch osd> help and documentation

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-14 16:09:07 +02:00
Sebastian Wagner
07189c8aec
Merge pull request #41556 from adk3798/daemon-count
mgr/cephadm: fix --service-type flag in orch ls when service has id

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-14 16:08:46 +02:00
Sage Weil
9074e87611 Merge PR #41827 into master
* refs/pull/41827/head:
	qa: move dashboard e2e from cephadm -> rados suite

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-06-14 09:11:04 -04:00
Kefu Chai
5cf3f8491b
Merge pull request #41796 from rzarzynski/wip-crimson-cltreq-dbgs
crimson/osd: improve debugs in ClientRequest

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-14 21:00:04 +08:00
Radoslaw Zarzynski
c451bb8519 crimson/osd: improve debugs in OpSequencer.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-14 12:26:22 +00:00
Radoslaw Zarzynski
36b14288bd crimson/osd: should_abort_request() dumps Operation's details.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-14 12:26:22 +00:00
Radoslaw Zarzynski
6463cd9c12 crimson/osd: print prev_op_id in ClientRequest's debugs.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-14 12:26:22 +00:00
Kefu Chai
d7a70ed770
Merge pull request #41832 from inspur-wyq/patch-4
os/ObjectStore: fix typo in comments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-14 18:13:21 +08:00
Kalpesh Pandya
809571fe3d qa/tasks: Keycloak failure checking
This PR is solving an error: Directory not empty.
This was encountered during removal of the downloaded keycloak package from keycloak.py

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-06-14 13:27:18 +05:30
Venky Shankar
98a8980e86 cephfs-mirror: silence warnings when connecting via mon host
When connecting to remote cluster via mon address, the cluster
confuration file is not required. However, the mirror daemon
fills up the ceph context with the cluster name resulting in
the following warning in (systemctl) logs:

    auth: unable to find a keyring on ...

These warning messages need to be silence as they can mislead
the user.

Fixes: http://tracker.ceph.com/issues/51204
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-06-14 03:09:37 -04:00
Patrick Donnelly
b01591b790
Merge PR #41786 into master
* refs/pull/41786/head:
	mon/MonCommands: remove obsolete mds commands
	mon/MonCommands: remove obsolete mon commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:57:37 -07:00
Patrick Donnelly
31a771139d
Merge PR #41580 into master
* refs/pull/41580/head:
	cephfs: make ceph_fallocate() platform independent

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-13 19:57:05 -07:00
Patrick Donnelly
7e320919f7
Merge PR #41482 into master
* refs/pull/41482/head:
	qa: remove obsolete deactivate routines

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-06-13 19:56:34 -07:00
Patrick Donnelly
6a173d1a39
Merge PR #41481 into master
* refs/pull/41481/head:
	client: Add ceph.caps vxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-06-13 19:56:04 -07:00
Patrick Donnelly
dfce83ef12
Merge PR #41431 into master
* refs/pull/41431/head:
	mds: progress the recover queue immediately after the inode is enqueued
	mds: switch to mdr->get_client()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:54:45 -07:00
Patrick Donnelly
6a095654f4
Merge PR #41422 into master
* refs/pull/41422/head:
	qa/tasks/cephfs/test_sessionmap: reap connections immediately
	msg/async: configurable threshold for reaping dead connections

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:52:48 -07:00
Patrick Donnelly
0441b3d60f
Merge PR #41403 into master
* refs/pull/41403/head:
	mgr/volumes: Add config to insert delay at the beginning of the clone

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-13 19:51:52 -07:00
wangyunqing
7e0d79b9d1 os/ObjectStore: fix typo in comments
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2021-06-14 10:42:14 +08:00
Kefu Chai
9b6be59d6f test/pybind/test_ceph_argparse: add a test where args contains comma
to ensure the support for the new syntax of "prefix --name bon,jour"
does not break existing behavior of "prefix hello cruel,world" where value2
contains reads "hello,world", and the parsed result should be

prefix="prefix"
value=["hello", "cruel,world"]

instead of something like

prefix="prefix"
value=["hello", "cruel", "world"]

or

prefix="prefix"
value=["cruel", "world"]

the above test only applies to the test where "value" is a CephString.
if "value" is a CephChoices, the parsed argument should be

prefix="prefix"
value=["hello", "cruel", "world"]

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
49be76ce11 cmake: run nose test using "python3 -m nose"
otherwise the tests are not performed at all.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
cf4570d663 test/pybind/test_ceph_argparse: check for expected argument
instead of comparing the validated argument with empty dict, check for
its content for better coverage of testing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
3d7516f8da test/pybind/test_ceph_argparse: decorate helper method with @nottest
otherwise nose tries to run it as a test as its name contains "test", like

E.
======================================================================
ERROR: Runs validate in different arg/kargs ways.
----------------------------------------------------------------------
TypeError: arg_kwarg_test() missing 3 required positional arguments: 'prefix', 'args', and 'sig'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
77cb1fed0c test/pybind: do not test obsoleted command
"scrub" command was marked obsoleted in
e9a5ce0897, so the test_ceph_argparse
cannot retrieve its command description using "get_command_descriptions"
cli tool anymore, let's drop the related test accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:47 +08:00
Kefu Chai
b8389557ad test/pybind: drop forward and readforward from test
these two write modes were dropped in
a0a3ed324a, so update the test
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Kefu Chai
952285dcde test/pybind: do not use tab for indent
the python source file is idented using spaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Kefu Chai
0948a78ee6 pybind/ceph_argparse: validate csv if desc.N
if desc.N is not None, we should take the argument as a comma separated
values, and validate the values individually.

restructure the validate() function and its helpers to pass the
validated args if desc.N explicitly, as desc.instance.val should only
hold a single value of desc.instance type, otherwise we need to reset
it after collecting all the argument in a CSV string value is parsed.

this change would be less consistent and hackish.

and update a test to verify this behavior.

Fixes: https://tracker.ceph.com/issues/51145
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Kefu Chai
bc182888a3 test/pybind: pep8 cleanups
* do not import unused symbols
* add two empty lines before method
* aligned indents

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-14 09:50:46 +08:00
Michael Fritch
ae7271d47d
cephadm: add log warn when unable to infer mon ip addr
fixup for `--mon-ip` bootstrap test (introduced b3e2c43):
  ValueError: 'eth0' does not appear to be an IPv4 or IPv6 address

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-06-13 16:30:39 -06:00
Kefu Chai
addf4a8b59
Merge pull request #40740 from batrick/i50278
pybind/mgr: add support for common sqlite3 databases

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-13 23:30:20 +08:00
Kefu Chai
77663c7f4c
Merge pull request #41809 from linuxbox2/wip-returnvec-64
rados: increase osd_max_write_op_reply_len default to 64 bytes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-06-13 23:27:58 +08:00
Kefu Chai
3a84d3e5d6
Merge pull request #41746 from strenuous-life/wip-cephadm-osd-create
mgr/cephadm: When create osd which include db, _create_daemon will be…

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-13 23:26:46 +08:00
Kefu Chai
254109f773 pybind/ceph_argparse: do not reuse variable for holding values of different types
to appease mypy

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-13 18:11:09 +08:00
Kefu Chai
265ae0a60e pybind/ceph_argparse: define variable using type annotation
as later, it could be assigned with None or a str or an int, so make
sure its type is annotated to appease mypy.

also, do not cast it to int, as in other branches, this variable
is just a str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-13 17:27:08 +08:00
Kefu Chai
eb1e9e2410 pybind/ceph_argparse: cast to strs before joining them
we cannot concat a list of float or int by joining them with "|".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-13 17:26:12 +08:00
Kefu Chai
ce823359ce pybind/ceph_argparse: set member variables in ctor
to appease mypy, otherwise it complains that typeargs is not a member
variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-13 17:25:42 +08:00
Kefu Chai
156d25b24a pybind/ceph_argparse: remove unused method
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-13 17:25:42 +08:00
Sage Weil
3a4ae22b8f Merge PR #41781 into master
* refs/pull/41781/head:
	mgr/mgr_module:delete invalid judgment

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-12 09:02:19 -04:00
Sage Weil
f21fe13008 Merge PR #41699 into master
* refs/pull/41699/head:
	mgr/telemetry: pass leaderboard flag even w/o ident

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2021-06-12 09:01:38 -04:00
Sage Weil
ac05b3568f qa: move dashboard e2e from cephadm -> rados suite
This test fails ~20% of the time.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-12 07:52:54 -05:00
Kefu Chai
573cbb796e os/bluestore/AvlAllocator: use cbit for counting the order of alignment
no need to calculate the alignment first, cbits() would suffice. as it
counts the first set bit and the follow 0's in a number. the result
is identical to the cbit(alignment of that number).

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-12 13:16:17 +08:00
Kefu Chai
9b52ba1dd0 os/bluestore/AvlAllocator: use delegated ctor
less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-12 13:13:30 +08:00
Kefu Chai
4837166f9e os/bluestore/AvlAllocator: specialize _block_picker()
before this change AvlAllocator::_block_picker() is used by both the
best-fit mode and first-fit mode. but since we cannot achieve the
locality by searching in the area pointed by curosr in best-fit mode,
we just pass a dummy cursor to AvlAllocator::_block_picker() when
searching in the best-fit mode.

but since the range_size_tree is already sorted by the size of ranges,
if _block_picker() fails to find one by the size, we should just give
up right away, and instead try again using a smaller size.

after this change, instead of sharing AvlAllocator::_block_picker()
across both the first-fit mode and the best-fit mode, this method
is specialize to two different variants: one for first-fit, and the
simplified one for best-fit.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-12 13:13:30 +08:00
Patrick Donnelly
b24608daa2
qa: choose victim pg from rbd pool
Right now scrub_test picks any pg in ceph. Unfortunately, it picked the
.mgr pool's only pg in [1]:

	2021-05-16T11:36:35.035 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool rbd setomapval main.db-journal.0000000000000000 key val

Instead, only pick a pg in the rbd pool.

[1] /ceph/teuthology-archive/kchai-2021-05-16_11:19:39-rados-wip-kefu-testing-2021-05-16-1043-distro-basic-smithi/6117396/teuthology.log

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 20:07:22 -07:00
Patrick Donnelly
d6c66f3fa6
qa,pybind/mgr: allow disabling .mgr pool
This is mostly for testing: a lot of tests assume that there are no
existing pools. These tests relied on a config to turn off creating the
"device_health_metrics" pool which generally exists for any new Ceph
cluster. It would be better to make these tests tolerant of the new .mgr
pool but clearly there's a lot of these. So just convert the config to
make it work.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:17 -07:00
Patrick Donnelly
f5fa58cb1f
doc: document new .mgr pool
And that pool names beginning with `.` are reserved names.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:17 -07:00
Patrick Donnelly
71d2c81d41
qa: add upgrade test for devicehealth
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:17 -07:00
Patrick Donnelly
0d9032771c
qa: fix api test failures
"device_health_metrics" pool is gone -- .mgr pool is in.

I don't think the pool removal code in some test cases is necessary any
longer with recent changes to remove those warnings; so that code is
gone too.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:17 -07:00
Patrick Donnelly
da551b0b04
pybind/mgr/devicehealth: load legacy device metrics pool
Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:16 -07:00
Patrick Donnelly
abd35d4769
pybind/mgr/devicehealth: update to store metrics in sqlite
This commit just modifies the module to use the sqlite3 database for
storing health metrics. A followup commit loads the legacy pool's data.

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:16 -07:00