Commit Graph

108868 Commits

Author SHA1 Message Date
Michael Fritch
7fa4da4456
mgr/volumes: OpSmException.error -> OpSmException.errno
fixes mypy errors:

volumes/fs/async_cloner.py: note: In function "handle_clone_pending":
volumes/fs/async_cloner.py:71: error: "OpSmException" has no attribute "error"; maybe "errno"?

volumes/fs/async_cloner.py: note: In function "handle_clone_in_progress":
volumes/fs/async_cloner.py:139: error: "OpSmException" has no attribute "error"; maybe "errno"?

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:34 -07:00
Michael Fritch
a09af07cb6
mgr/volumes: remove unneeded assignment to NoneType
fixes mypy error:

volumes/fs/operations/versions/__init__.py: note: In member "get_subvolume_object" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "SubvolumeBase")

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:34 -07:00
Michael Fritch
d5c1a7cbd5
mgr/volumes: add missing OpSmException import
fixes mypy error:

volumes/fs/operations/versions/__init__.py: note: In member "upgrade_legacy_subvolume" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:56: error: Name 'OpSmException' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:34 -07:00
Michael Fritch
22b4cb9405
mgr/volumes: add missing error code
fixes mypy error:

volumes/fs/operations/versions/__init__.py: note: In member "_load_supported_versions" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:35: error: Too few arguments for "VolumeException"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
760a591839
mgr/volumes: remove stray return
fixes mypy errors:

volumes/fs/purge_queue.py: note: In function "get_trash_entry_for_volume":
volumes/fs/purge_queue.py:28: error: Name 'ret' is not defined

volumes/fs/async_cloner.py: note: In function "get_next_clone_entry":
volumes/fs/async_cloner.py:39: error: Name 'ret' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
85e57a64cb
mgr/volumes: fixup format string args
fixes mypy errors:

volumes/fs/purge_queue.py:26: error: Cannot find replacement for positional format specifier 1
volumes/fs/async_cloner.py:37: error: Cannot find replacement for positional format specifier 1

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
a3b61a3577
mgr/volumes: add missing mgr param
fixes mypy errors:

volumes/fs/operations/volume.py: note: In function "create_volume":
volumes/fs/operations/volume.py:216: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:223: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:224: error: Too few arguments for "remove_pool"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
cd97420965
mgr/volumes: assert self.fs
fixes mypy errors:

volumes/fs/operations/volume.py: note: In member "disconnect" of class "Connection":
volumes/fs/operations/volume.py:94: error: Item "None" of "Optional[Any]" has no attribute "get_addrs"
volumes/fs/operations/volume.py:95: error: Item "None" of "Optional[Any]" has no attribute "shutdown"

volumes/fs/operations/volume.py: note: In member "abort" of class "Connection":
volumes/fs/operations/volume.py:105: error: Item "None" of "Optional[Any]" has no attribute "abort_conn"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
ac02377190
mgr/volumes: skip type checking on RTimer class
Unclear why mypy does not like this:

volumes/fs/operations/volume.py: note: In member "run" of class "RTimer":
volumes/fs/operations/volume.py:118: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "interval"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "function"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "args"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "kwargs"
volumes/fs/operations/volume.py:121: error: "RTimer" has no attribute "finished"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
a611864dae
mgr/volumes: fixup threading.Timer import
Python 3.3 changed `Timer` from a factory function to a class
https://docs.python.org/3.8/library/threading.html#timer-objects

fixes mypy error:

volumes/fs/operations/volume.py:9: error: Module 'threading' has no attribute '_Timer'; maybe "Timer"?

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
f46b5059e6
mgr/volumes: fix positional str formatting
fixes mypy error:

volumes/fs/operations/group.py: note: In function "create_group":
volumes/fs/operations/group.py:135: error: Not all arguments converted during string formatting

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
6a04ecdc9c
mgr/volumes: place getters and setters next to each other
workaround for mypy issue:
https://github.com/python/mypy/issues/1465

fixes mypy errors:

volumes/fs/operations/group.py: note: In class "Group":
volumes/fs/operations/group.py:44: error: Name 'uid' already defined on line 36
volumes/fs/operations/group.py:44: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py:48: error: Name 'gid' already defined on line 40
volumes/fs/operations/group.py:48: error: "Callable[[Group], Any]" has no attribute "setter"

volumes/fs/operations/group.py: note: In function "open_group":
volumes/fs/operations/group.py:170: error: Property "uid" defined in "Group" is read-only
volumes/fs/operations/group.py:171: error: Property "gid" defined in "Group" is read-only

volumes/fs/operations/versions/subvolume_base.py: note: In class "SubvolumeBase":
volumes/fs/operations/versions/subvolume_base.py:45: error: Name 'uid' already defined on line 33
volumes/fs/operations/versions/subvolume_base.py:45: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:49: error: Name 'gid' already defined on line 37
volumes/fs/operations/versions/subvolume_base.py:49: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:53: error: Name 'mode' already defined on line 41
volumes/fs/operations/versions/subvolume_base.py:53: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
896448324f
mgr/volumes: reverse params passed to isinstace()
fixes mypy error:

volumes/fs/operations/clone_index.py: note: In member "track" of class "CloneIndex":
volumes/fs/operations/clone_index.py:38: error: Argument 2 to "isinstance" has incompatible type "Union[VolumeException, Any]"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
e2516a4806
mgr/volumes: fix arg typo '.' -> ','
fixes mypy error:

volumes/fs/fs_util.py: note: In function "get_ancestor_xattr":
volumes/fs/fs_util.py:129: error: Too few arguments for "VolumeException"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
0fda329d07
mgr/volumes: add deque type
fixes mypy error:

volumes/fs/async_job.py: note: In member "__init__" of class "AsyncJobs":
volumes/fs/async_job.py:97: error: Need type annotation for 'q'

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
aaed49a142
mgr/volumes: threading.Event.isSet no longer exists in py3
fixes mypy errors:

volumes/fs/async_job.py: note: In member "should_cancel" of class "JobThread":
volumes/fs/async_job.py:70: error: "Event" has no attribute "isSet"

volumes/fs/volume.py: note: In member "is_stopping" of class "VolumeClient":
volumes/fs/volume.py:59: error: "Event" has no attribute "isSet"

volumes/fs/volume.py: note: In member "list_fs_volumes" of class "VolumeClient":
volumes/fs/volume.py:106: error: "Event" has no attribute "isSet"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
6907c440e1
mgr/volumes: assert JobThread class
fixes mypy error:

volumes/fs/async_job.py: note: In member "run" of class "JobThread":
volumes/fs/async_job.py:43: error: "Thread" has no attribute "should_cancel"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
5b9d5e2e43
mgr/volumes: import VolumeException
fixes mypy errors:

volumes/fs/operations/index.py: note: In member "track" of class "Index":
volumes/fs/operations/index.py:19: error: Name 'VolumeException' is not defined

volumes/fs/operations/index.py: note: In member "untrack" of class "Index":
volumes/fs/operations/index.py:22: error: Name 'VolumeException' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
23efb9240e
mgr/volumes: use SafeConfigParser with py2
Python3.2 renamed `SafeConfigParser` -> `ConfigParser`

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
f213b646db
mgr/volumes: use python version during configparser import
fixes mypy error:

volumes/fs/operations/versions/metadata_manager.py:8: error: Name 'configparser' already defined (by an import)

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
322019beef
mgr/volumes: add Dict type
fixes mypy errors:

volumes/fs/operations/op_sm.py:39: error: "object" has no attribute "get"
volumes/fs/operations/op_sm.py:49: error: "object" has no attribute "get"

volumes/fs/operations/lock.py: note: In member "__init__" of class "GlobalLock":
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__exit__"

volumes/fs/operations/lock.py: note: In member "lock_op" of class "GlobalLock":
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__exit__"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:31 -07:00
Michael Fritch
8478b72dc3
mgr/volumes: import errno
fixes mypy errors:

volumes/fs/operations/op_sm.py:36: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:39: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:46: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:49: error: Name 'errno' is not defined
volumes/fs/operations/template.py:5: error: Name 'errno' is not defined
volumes/fs/operations/template.py:14: error: Name 'errno' is not defined
volumes/fs/operations/template.py:23: error: Name 'errno' is not defined
volumes/fs/operations/template.py:32: error: Name 'errno' is not defined
volumes/fs/operations/template.py:42: error: Name 'errno' is not defined
volumes/fs/operations/template.py:45: error: Name 'errno' is not defined
volumes/fs/operations/template.py:62: error: Name 'errno' is not defined
volumes/fs/operations/template.py:74: error: Name 'errno' is not defined
volumes/fs/operations/template.py:85: error: Name 'errno' is not defined
volumes/fs/operations/template.py:94: error: Name 'errno' is not defined
volumes/fs/operations/template.py:103: error: Name 'errno' is not defined
volumes/fs/operations/template.py:112: error: Name 'errno' is not defined
volumes/fs/operations/template.py:121: error: Name 'errno' is not defined
volumes/fs/operations/template.py:130: error: Name 'errno' is not defined
volumes/fs/operations/template.py:139: error: Name 'errno' is not defined
volumes/fs/operations/template.py:148: error: Name 'errno' is not defined
volumes/fs/operations/template.py:158: error: Name 'errno' is not defined
volumes/fs/operations/template.py:169: error: Name 'errno' is not defined
volumes/fs/operations/template.py:180: error: Name 'errno' is not defined
volumes/fs/operations/index.py:18: error: Name 'errno' is not defined
volumes/fs/operations/index.py:21: error: Name 'errno' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:31 -07:00
Michael Fritch
fef6450834
mgr/volumes: add mgr/volumes to the mypy checks
Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:26 -07:00
Sage Weil
8a84ddeaa8 Merge PR #33531 into master
* refs/pull/33531/head:
	qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests
	mgr/test_orchestrator: refactor listing services/daemons
	mgr/dashboard: list services and daemons

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-03-05 13:20:21 -06:00
Sage Weil
2dc112e0aa Merge PR #33718 into master
* refs/pull/33718/head:
	mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2020-03-05 13:20:10 -06:00
Sage Weil
8eb58e5bba Merge PR #33745 into master
* refs/pull/33745/head:
	mgr/cephadm: upgrade: fix daemons missing image_id

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:18:34 -06:00
J. Eric Ivancich
03a3355db5
Merge pull request #33702 from ivancich/wip-bucket-listing-clear-results
rgw: cls_bucket_list_(un)ordered should clear results collection

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-03-05 12:37:51 -05:00
J. Eric Ivancich
c52039a9e8 rgw: cls_bucket_list_(un)ordered should clear results collection
Each call to cls_bucket_list_(un)ordered should have an empty
collection to populate with results. Rather than rely on the caller to
insure this, it's more reliable to have these functions do the clear.

Additionally in some cases, a reserve call was added to the collection
to pre-allocate the space needed for the expected number of
results. This will potentially result in fewer re-allocations plus
copies.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-03-05 11:39:36 -05:00
J. Eric Ivancich
44a61a048a
Merge pull request #33693 from yaozongyou/fix-rgw-check-bucket-empty-bug
rgw: clear ent_list for each loop of bucket list

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-03-05 11:32:21 -05:00
J. Eric Ivancich
2551bce86d
Merge pull request #33398 from clwluvw/subuser-policy
rgw: Fix bug on subuser policy identity checker

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2020-03-05 10:55:24 -05:00
Alfonso Martínez
eaa0d84a13 mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite
Fixes: https://tracker.ceph.com/issues/44405
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2020-03-05 16:22:55 +01:00
Patrick Donnelly
bacfa9f5a7
Merge PR #33687 into master
* refs/pull/33687/head:
	mgr/volume: adapt arg passing to ServiceSpec

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-03-05 06:27:51 -08:00
Sage Weil
cdc778d351 Merge PR #33728 into master
* refs/pull/33728/head:
	mgr/orch: factor out nice_delta
	mgr/orch: show spec age in 'orch ls'
	mgr/cephadm: store timestamp with specs
	mgr/orch: add created timestamp to ServiceDescription
	mgr/orch: include uptime in 'orch ps'
	mgr/orch: include AGE column in 'orch ps'
	mgr/cephadm: populate new DaemonDescription timestamps
	mgr/orch: add new timestamps in DaemonDescription
	cephadm: include timestamps for configured, created
	cephadm: include timestamps for started, deploy

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-05 08:05:49 -06:00
Sage Weil
e8c268d1d4 Merge PR #33708 into master
* refs/pull/33708/head:
	ceph_test_cls_hello: set RETURNVEC on the expected EINVAL request

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-03-05 08:02:21 -06:00
Sage Weil
bb126cf9ab qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests
The ps output names daemons like 'type.foo', e.g., 'mgr.x'.  Now that
the test_orchestrator impl is less bonkers this needs to be adjusted to
match reality.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 08:00:42 -06:00
Sage Weil
e8078acf3c mgr/cephadm: upgrade: fix daemons missing image_id
If a daemon isn't running, we don't know the image_id (hash), so we skip.
But it's also possible to have a running daemon that doesn't report an
image_id... like right after we deploy it when the container hasn't
started up yet.  Skip those too.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 07:52:41 -06:00
Sage Weil
2b5008ce23 Merge PR #33729 into master
* refs/pull/33729/head:
	doc/releases: Octopus is not stable yet

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-05 07:31:23 -06:00
Sage Weil
94e77cc9c4 mgr/orch: factor out nice_delta
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:27 -06:00
Sage Weil
59127f69a1 mgr/orch: show spec age in 'orch ls'
Also rearrange the columns a bit.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:27 -06:00
Sage Weil
0e260d299f mgr/cephadm: store timestamp with specs
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:27 -06:00
Sage Weil
d417730232 mgr/orch: add created timestamp to ServiceDescription
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:27 -06:00
Sage Weil
f9c65b49ab mgr/orch: include uptime in 'orch ps'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:26 -06:00
Sage Weil
20c4a9783d mgr/orch: include AGE column in 'orch ps'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:26 -06:00
Sage Weil
c6cabac263 mgr/cephadm: populate new DaemonDescription timestamps
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:26 -06:00
Sage Weil
05dbfe1c64 mgr/orch: add new timestamps in DaemonDescription
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:26 -06:00
Sage Weil
a683065cd6 cephadm: include timestamps for configured, created
created = when daemon was first created
configured = when the daemonw as last (re)configured

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:26 -06:00
Sage Weil
99aadb094a cephadm: include timestamps for started, deploy
started == when the container started
deployed == when the container was (most recently) (re)deployed

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 06:42:26 -06:00
Sage Weil
2a1d9e97ff Merge PR #33733 into master
* refs/pull/33733/head:
	mgr/cephadm: if we had no record of deps, and deps are [], do not reconfig

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-05 06:34:48 -06:00
Sage Weil
683fc0f80c Merge PR #33734 into master
* refs/pull/33734/head:
	mgr/cephadm: do not refresh daemon and device inventory as often

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-05 06:34:38 -06:00
Sage Weil
b7fd2e8d1e Merge PR #33736 into master
* refs/pull/33736/head:
	mgr/cephadm: upgrade crash collectors too
	mgr/cephadm: fix service rm when there are no daemons

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-05 06:34:18 -06:00