Commit Graph

108894 Commits

Author SHA1 Message Date
Kyr Shatskyy
e7473b631d qa/tasks/mgr: fix imports for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
dcf85e4328 qa/tasks/keystone: get rid of cStringIO for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
06f22a3097 qa/tasks/kclient: fix import for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
09ae86aa9e qa/tasks/devstack: use six.StringIO for py3 compat
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
b23ae40a8b qa/tasks/devstack: get rid of cStringIO
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
b0bafd1416 qa/tasks/s3tests_java: get rid of cStringIO
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
478083e731 qa/tasks/s3tests: py3 compat
- use string.ascii_uppercase instead string.uppercase
- use six.ensure_str for bytes when required
- use six.ensure_binary if needed
- get rid of dict.itervalues in favor of dict.values
- get rid of cStringIO.StringIO in favor io.BytesIO

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
683421ea24 qa/tasks/util/rgw: get rid of cString for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
4cf205f285 qa/tasks/rgw_logsocket: get rid of cStringIO for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
f0c5f56ef8 qa/tasks/ragweed: get rid of cString for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
b3e42cf30c qa/tasks/ragweed: get rid of itervalues for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
0a224991cf qa/tasks/radosgw_admin: get rid of cStringIO for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
08b83007dc qa/tasks/radosgw_admin: fix tab inconsistancy
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
eec9479f6d qa/tasks/rgw: fix imports for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
db7ae8eff6 qa/tasks/rbd_fio: get rid of StringIO for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
7b72097cd1 qa/tasks/rbd: get rid of cStringIO for py3
Use io.BytesIO and six.ensure_str for py3 compatibility
instead of cStringIO.StringIO

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
ac2fa95b21 qa/tasks/ceph_fuse: fix import for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
875b45f3c9 qa/tasks/cephfs/kernel_mount: get rid of StringIO.StringIO for py3
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Kyr Shatskyy
9f6c764f10 qa/tasks/cephfs: get rid of StringIO for py3
Use io.BytesIO and six.ensure_str for py3 compatibility

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2020-03-06 10:19:55 +01:00
Ramana Raja
6b45b3b6c9
Merge pull request #33775 from tchaikov/wip-44454
qa/tasks/ceph.py: quote "<kind>" in command line

Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-03-06 12:31:12 +05:30
Patrick Donnelly
e20ad96dbf
Merge PR #33674 into master
* refs/pull/33674/head:
	mgr/volumes: OpSmException.error -> OpSmException.errno
	mgr/volumes: remove unneeded assignment to `NoneType`
	mgr/volumes: add missing OpSmException import
	mgr/volumes: add missing error code
	mgr/volumes: remove stray return
	mgr/volumes: fixup format string args
	mgr/volumes: add missing `mgr` param
	mgr/volumes: assert self.fs
	mgr/volumes: skip type checking on RTimer class
	mgr/volumes: fixup threading.Timer import
	mgr/volumes: fix positional str formatting
	mgr/volumes: place getters and setters next to each other
	mgr/volumes: reverse params passed to `isinstace()`
	mgr/volumes: fix arg typo '.' -> ','
	mgr/volumes: add `deque` type
	mgr/volumes: `threading.Event.isSet` no longer exists in py3
	mgr/volumes: assert JobThread class
	mgr/volumes: import VolumeException
	mgr/volumes: use `SafeConfigParser` with py2
	mgr/volumes: use python version during configparser import
	mgr/volumes: add `Dict` type
	mgr/volumes: import errno
	mgr/volumes: add `mgr/volumes` to the mypy checks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-03-05 20:52:46 -08:00
Kefu Chai
da736c22c5 qa/tasks/ceph.py: quote "<kind>" in command line
otherwise bash will intepret "kind" as a file when handling command like
```
sudo zgrep <kind> /var/log/ceph/valgrind/* /dev/null | sort | uniq
```
and try to feed its content to zgrep, and write the output of zgrep
to /var/log/ceph/valgrind/*. this is not the intended behavior. what we
what to do is to pass "<kind>" as an argument to zgrep, along with
the globbed files names which matches "/var/log/ceph/valgrind/*".

in this change, "<kind>" is quoted as in the command line. it's also
what `pipes.quote()` does before the change of
35cf5131e7.

this addresses the regression introduced by
35cf5131e7.

Fixes: https://tracker.ceph.com/issues/44454
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-06 12:17:42 +08:00
Sage Weil
77425b3c4a Merge PR #33766 into master
* refs/pull/33766/head:
	mgr/cephadm: fix placement when existing + specified don't overlap

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-05 18:58:00 -06:00
Sage Weil
d316156b1e Merge PR #33744 into master
* refs/pull/33744/head:
	qa/tasks/radosbench: fix usage of -O

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-05 16:41:48 -06:00
Sage Weil
718471b7e4 mgr/cephadm: fix placement when existing + specified don't overlap
If we have a daemon on A, and our spec is (count=2, hosts=[B]), we should
always return [A,B], but we sometimes were returning [B,B].

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-05 14:51:39 -06:00
Michael Fritch
7fa4da4456
mgr/volumes: OpSmException.error -> OpSmException.errno
fixes mypy errors:

volumes/fs/async_cloner.py: note: In function "handle_clone_pending":
volumes/fs/async_cloner.py:71: error: "OpSmException" has no attribute "error"; maybe "errno"?

volumes/fs/async_cloner.py: note: In function "handle_clone_in_progress":
volumes/fs/async_cloner.py:139: error: "OpSmException" has no attribute "error"; maybe "errno"?

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:34 -07:00
Michael Fritch
a09af07cb6
mgr/volumes: remove unneeded assignment to NoneType
fixes mypy error:

volumes/fs/operations/versions/__init__.py: note: In member "get_subvolume_object" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:70: error: Incompatible types in assignment (expression has type "None", variable has type "SubvolumeBase")

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:34 -07:00
Michael Fritch
d5c1a7cbd5
mgr/volumes: add missing OpSmException import
fixes mypy error:

volumes/fs/operations/versions/__init__.py: note: In member "upgrade_legacy_subvolume" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:56: error: Name 'OpSmException' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:34 -07:00
Michael Fritch
22b4cb9405
mgr/volumes: add missing error code
fixes mypy error:

volumes/fs/operations/versions/__init__.py: note: In member "_load_supported_versions" of class "SubvolumeLoader":
volumes/fs/operations/versions/__init__.py:35: error: Too few arguments for "VolumeException"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
760a591839
mgr/volumes: remove stray return
fixes mypy errors:

volumes/fs/purge_queue.py: note: In function "get_trash_entry_for_volume":
volumes/fs/purge_queue.py:28: error: Name 'ret' is not defined

volumes/fs/async_cloner.py: note: In function "get_next_clone_entry":
volumes/fs/async_cloner.py:39: error: Name 'ret' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
85e57a64cb
mgr/volumes: fixup format string args
fixes mypy errors:

volumes/fs/purge_queue.py:26: error: Cannot find replacement for positional format specifier 1
volumes/fs/async_cloner.py:37: error: Cannot find replacement for positional format specifier 1

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
a3b61a3577
mgr/volumes: add missing mgr param
fixes mypy errors:

volumes/fs/operations/volume.py: note: In function "create_volume":
volumes/fs/operations/volume.py:216: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:223: error: Too few arguments for "remove_pool"
volumes/fs/operations/volume.py:224: error: Too few arguments for "remove_pool"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
cd97420965
mgr/volumes: assert self.fs
fixes mypy errors:

volumes/fs/operations/volume.py: note: In member "disconnect" of class "Connection":
volumes/fs/operations/volume.py:94: error: Item "None" of "Optional[Any]" has no attribute "get_addrs"
volumes/fs/operations/volume.py:95: error: Item "None" of "Optional[Any]" has no attribute "shutdown"

volumes/fs/operations/volume.py: note: In member "abort" of class "Connection":
volumes/fs/operations/volume.py:105: error: Item "None" of "Optional[Any]" has no attribute "abort_conn"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
ac02377190
mgr/volumes: skip type checking on RTimer class
Unclear why mypy does not like this:

volumes/fs/operations/volume.py: note: In member "run" of class "RTimer":
volumes/fs/operations/volume.py:118: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "finished"
volumes/fs/operations/volume.py:119: error: "RTimer" has no attribute "interval"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "function"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "args"
volumes/fs/operations/volume.py:120: error: "RTimer" has no attribute "kwargs"
volumes/fs/operations/volume.py:121: error: "RTimer" has no attribute "finished"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
a611864dae
mgr/volumes: fixup threading.Timer import
Python 3.3 changed `Timer` from a factory function to a class
https://docs.python.org/3.8/library/threading.html#timer-objects

fixes mypy error:

volumes/fs/operations/volume.py:9: error: Module 'threading' has no attribute '_Timer'; maybe "Timer"?

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
f46b5059e6
mgr/volumes: fix positional str formatting
fixes mypy error:

volumes/fs/operations/group.py: note: In function "create_group":
volumes/fs/operations/group.py:135: error: Not all arguments converted during string formatting

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:33 -07:00
Michael Fritch
6a04ecdc9c
mgr/volumes: place getters and setters next to each other
workaround for mypy issue:
https://github.com/python/mypy/issues/1465

fixes mypy errors:

volumes/fs/operations/group.py: note: In class "Group":
volumes/fs/operations/group.py:44: error: Name 'uid' already defined on line 36
volumes/fs/operations/group.py:44: error: "Callable[[Group], Any]" has no attribute "setter"
volumes/fs/operations/group.py:48: error: Name 'gid' already defined on line 40
volumes/fs/operations/group.py:48: error: "Callable[[Group], Any]" has no attribute "setter"

volumes/fs/operations/group.py: note: In function "open_group":
volumes/fs/operations/group.py:170: error: Property "uid" defined in "Group" is read-only
volumes/fs/operations/group.py:171: error: Property "gid" defined in "Group" is read-only

volumes/fs/operations/versions/subvolume_base.py: note: In class "SubvolumeBase":
volumes/fs/operations/versions/subvolume_base.py:45: error: Name 'uid' already defined on line 33
volumes/fs/operations/versions/subvolume_base.py:45: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:49: error: Name 'gid' already defined on line 37
volumes/fs/operations/versions/subvolume_base.py:49: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"
volumes/fs/operations/versions/subvolume_base.py:53: error: Name 'mode' already defined on line 41
volumes/fs/operations/versions/subvolume_base.py:53: error: "Callable[[SubvolumeBase], Any]" has no attribute "setter"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
896448324f
mgr/volumes: reverse params passed to isinstace()
fixes mypy error:

volumes/fs/operations/clone_index.py: note: In member "track" of class "CloneIndex":
volumes/fs/operations/clone_index.py:38: error: Argument 2 to "isinstance" has incompatible type "Union[VolumeException, Any]"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
e2516a4806
mgr/volumes: fix arg typo '.' -> ','
fixes mypy error:

volumes/fs/fs_util.py: note: In function "get_ancestor_xattr":
volumes/fs/fs_util.py:129: error: Too few arguments for "VolumeException"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
0fda329d07
mgr/volumes: add deque type
fixes mypy error:

volumes/fs/async_job.py: note: In member "__init__" of class "AsyncJobs":
volumes/fs/async_job.py:97: error: Need type annotation for 'q'

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
aaed49a142
mgr/volumes: threading.Event.isSet no longer exists in py3
fixes mypy errors:

volumes/fs/async_job.py: note: In member "should_cancel" of class "JobThread":
volumes/fs/async_job.py:70: error: "Event" has no attribute "isSet"

volumes/fs/volume.py: note: In member "is_stopping" of class "VolumeClient":
volumes/fs/volume.py:59: error: "Event" has no attribute "isSet"

volumes/fs/volume.py: note: In member "list_fs_volumes" of class "VolumeClient":
volumes/fs/volume.py:106: error: "Event" has no attribute "isSet"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
6907c440e1
mgr/volumes: assert JobThread class
fixes mypy error:

volumes/fs/async_job.py: note: In member "run" of class "JobThread":
volumes/fs/async_job.py:43: error: "Thread" has no attribute "should_cancel"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
5b9d5e2e43
mgr/volumes: import VolumeException
fixes mypy errors:

volumes/fs/operations/index.py: note: In member "track" of class "Index":
volumes/fs/operations/index.py:19: error: Name 'VolumeException' is not defined

volumes/fs/operations/index.py: note: In member "untrack" of class "Index":
volumes/fs/operations/index.py:22: error: Name 'VolumeException' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
23efb9240e
mgr/volumes: use SafeConfigParser with py2
Python3.2 renamed `SafeConfigParser` -> `ConfigParser`

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
f213b646db
mgr/volumes: use python version during configparser import
fixes mypy error:

volumes/fs/operations/versions/metadata_manager.py:8: error: Name 'configparser' already defined (by an import)

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:32 -07:00
Michael Fritch
322019beef
mgr/volumes: add Dict type
fixes mypy errors:

volumes/fs/operations/op_sm.py:39: error: "object" has no attribute "get"
volumes/fs/operations/op_sm.py:49: error: "object" has no attribute "get"

volumes/fs/operations/lock.py: note: In member "__init__" of class "GlobalLock":
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:27: error: "object" has no attribute "__exit__"

volumes/fs/operations/lock.py: note: In member "lock_op" of class "GlobalLock":
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__enter__"
volumes/fs/operations/lock.py:35: error: "object" has no attribute "__exit__"

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:31 -07:00
Michael Fritch
8478b72dc3
mgr/volumes: import errno
fixes mypy errors:

volumes/fs/operations/op_sm.py:36: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:39: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:46: error: Name 'errno' is not defined
volumes/fs/operations/op_sm.py:49: error: Name 'errno' is not defined
volumes/fs/operations/template.py:5: error: Name 'errno' is not defined
volumes/fs/operations/template.py:14: error: Name 'errno' is not defined
volumes/fs/operations/template.py:23: error: Name 'errno' is not defined
volumes/fs/operations/template.py:32: error: Name 'errno' is not defined
volumes/fs/operations/template.py:42: error: Name 'errno' is not defined
volumes/fs/operations/template.py:45: error: Name 'errno' is not defined
volumes/fs/operations/template.py:62: error: Name 'errno' is not defined
volumes/fs/operations/template.py:74: error: Name 'errno' is not defined
volumes/fs/operations/template.py:85: error: Name 'errno' is not defined
volumes/fs/operations/template.py:94: error: Name 'errno' is not defined
volumes/fs/operations/template.py:103: error: Name 'errno' is not defined
volumes/fs/operations/template.py:112: error: Name 'errno' is not defined
volumes/fs/operations/template.py:121: error: Name 'errno' is not defined
volumes/fs/operations/template.py:130: error: Name 'errno' is not defined
volumes/fs/operations/template.py:139: error: Name 'errno' is not defined
volumes/fs/operations/template.py:148: error: Name 'errno' is not defined
volumes/fs/operations/template.py:158: error: Name 'errno' is not defined
volumes/fs/operations/template.py:169: error: Name 'errno' is not defined
volumes/fs/operations/template.py:180: error: Name 'errno' is not defined
volumes/fs/operations/index.py:18: error: Name 'errno' is not defined
volumes/fs/operations/index.py:21: error: Name 'errno' is not defined

Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:31 -07:00
Michael Fritch
fef6450834
mgr/volumes: add mgr/volumes to the mypy checks
Fixes: https://tracker.ceph.com/issues/44393
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-05 13:19:26 -07:00
Sage Weil
8a84ddeaa8 Merge PR #33531 into master
* refs/pull/33531/head:
	qa/tasks/mgr/test_orchestrator_cli: fix test_ps* tests
	mgr/test_orchestrator: refactor listing services/daemons
	mgr/dashboard: list services and daemons

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-03-05 13:20:21 -06:00
Sage Weil
2dc112e0aa Merge PR #33718 into master
* refs/pull/33718/head:
	mgr/dashboard: fix tasks.mgr.dashboard.test_rgw suite

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2020-03-05 13:20:10 -06:00