-t is an alias of "--teuthology-branch" which defaults to master. we
have a "py2" branch in teuthology. that branch is compatible with
python2, and the teuthology-worker is executed in a Python2 environment,
so all qa suites tested using "py2" branch of teuthology should work
fine as long as they are Python2 compatible. currently, all LTS branches
are compatible with Python2, and only master is also compatible with
Python3. but we plan to migrate master and octopus' qa suites to
python3 to ease the pain of maintainance in long term.
to achieve this goal, in this change, all non-master branches are now
tested using the "py2" teuthology branch.
once the python3 compatible changes are backported to octopus, "-t 2"
options will be removed from the command lines testing octopus.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Also rename 3-compat_client/mimic -> pacific to match the min_client
setting (the naming here confuses me...).
Signed-off-by: Sage Weil <sage@redhat.com>
This happens because mgrs may start before mons, or before mons have
published a new mgrmap that disables orchestrator_cli.
Signed-off-by: Sage Weil <sage@redhat.com>
v2 was introduced in nautilus, and we don't support mimic -> pacific
upgrades (only mimic -> octopus). This test can be removed!
Signed-off-by: Sage Weil <sage@redhat.com>
in tasks/module_selftest.yaml, `TestModuleSelftest.test_telegraf()` is
called. but we fail to prepare a unix domain socket to which the telegraf
module can send stats. and telegraf module does not catch
FileNotFoundError exception, so the exception is populated to ceph-mgr
and is found by the test, hence the test is marked a failure whenever
telegraf is tested.
in this change,
* catch this exception, so it won't be caught by ceph-mgr
* whitelist the error message, so the test can pass
Signed-off-by: Kefu Chai <kchai@redhat.com>
we cannot rely on the order in which items are arranged in a dict, the
order varies from version to another. in Python2, it happens to work,
and we can always have the self-signed cert added first. but in Python3,
it does not. and an exception is thrown
```
teuthology.exceptions.ConfigError: ssl: ca root not found for
certificate rgw.client.0
```
in this change, before creating certs, the settings are reordered so
that the self-signed ones are created first.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to address the test failures like
```
2020-04-07T15:44:58.693 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: ceph pg dump
pgs
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: pgid
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh: line 498: pgid: command not found
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
as the caller might want to `len(manager.get_osd_status()['raw'])`, and
`len()` does not accept a `filter` object.
also, the filtered osd statuses are printed out using `self.log()`, so
we should materialize the `filter` object before sending it to logging
facility. otherwise we will have something like:
```
2020-04-08T02:58:37.001 INFO:tasks.ceph.ceph_manager.ceph:<filter object at 0x7f5a080e1518>
```
in the logging message.
Signed-off-by: Kefu Chai <kchai@redhat.com>
to address the test failures like
```
2020-04-07T15:44:58.693 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: ceph pg dump
pgs
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_failed: pgid
2020-04-07T15:44:58.694 INFO:tasks.workunit.client.0.smithi049.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh: line 498: pgid: command not found
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
if `key.name` is not set, boto fills it with md5, in that case, it comes
from `base64.b64encode()`. so we need to make sure it's str before
passing it to shell.
the same applies to `key.get_xml_acl()`, as its return value comes
directly from something like
```
response = self.connection.make_request('GET', self.name, key_name,
query_args=query_args,
headers=headers)
body = response.read()
# ...
return body
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
we assume that boto.Key.get_content_as_string() returns str instead of
bytes, and compare the return value with a string, so, to ensure that
lhs and rhs are both strings, we need to decode the returned content.
since we always store strings composed with ASCII, it's safe to use
'ascii' to decode them.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in Python3, dict.items() returns a view instead of an instance of list,
so we have to materialize the view for changing the dict being iterated.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this file is a template for `boto.cfg` which only contains text, so we
can just open it in text mode, and the file-like object will be passed
to run() as stdin, which is able to consume strings.
Signed-off-by: Kefu Chai <kchai@redhat.com>
base64.b64encode() returns bytes in Python3, and we will pass it as a
commandline parameter later on, which will be quoted. so the command
line can be consumed by shell. so before sending it to shell, we should
convert it to string.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in Python3, json.loads() expects a string, while
HTTPConnection.getresponse() returns a byte-like object, so we need to
coerce it to str first.
Signed-off-by: Kefu Chai <kchai@redhat.com>
in Python2, ConfigParser is almost the same as RawConfigParser, which
allows set non-string values, but in Python3, ConfigParser.set() only
accepts strings as value of option.
since we do not use "cpar" as an internal storage for options, it does
not matter what type of options we set using ConfigParser as long as it
can be consumed by tempest. boolean settings are translated to "true" or
"false". see also
https://docs.openstack.org/tempest/latest/sampleconf.html
Signed-off-by: Kefu Chai <kchai@redhat.com>
in Python3, some methods offered by `string` module are now member
method of `str` class, and `string.uppercase` is renamed to
`string.ascii_uppercase` in Python2 and Python3. so let's update
accordingly.
Signed-off-by: Kefu Chai <kchai@redhat.com>
this change should address the failure of
```
2020-04-06T03:07:59.152 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_wip-py3/teuthology/contextutil.py", line 32, in nested
vars.append(enter())
File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__
return next(self.gen)
File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/rgw.py", line 266, in configure_compression
rgwadmin(ctx, client, cmd=['user', 'list'], check_status=True)
File "/home/teuthworker/src/github.com_tchaikov_ceph_wip-qa-py3/qa/tasks/util/rgw.py", line 43, in rgwadmin
j = json.loads(out)
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
