Commit Graph

2697 Commits

Author SHA1 Message Date
Kefu Chai
ea9c14c322
Merge pull request #38020 from mdw-at-linuxbox/teuth-fix-x509-san
qa/tasks/openssl_keys.py: add subjectAltName to certificates

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-12 12:18:16 +08:00
Patrick Donnelly
cd62a8494b
Merge PR #38322 into master
* refs/pull/38322/head:
	test_cephfs_shell: add tests for ls command

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-12-11 18:59:42 -08:00
Patrick Donnelly
0c663e7d4e
Merge PR #38511 into master
* refs/pull/38511/head:
	qa/tasks/cephfs/nfs: Check if host ip is in cluster info output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-11 09:47:35 -08:00
Sebastian Wagner
56db4fbed8
Merge pull request #38009 from ideepika/wip-48058-cephadm
qa/tasks/cephadm: update_info_yam() only if ctx.archive is valid

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-12-10 15:13:42 +01:00
Jason Dillaman
6f4e3b0dd1
Merge pull request #36551 from CongMinYin/immutable_object_cache_throttle
tools: add throttle mechanism to immutable object cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-12-10 09:07:16 -05:00
Ernesto Puerta
f977d1acf2
Merge pull request #38435 from votdev/issue_48449_test_standby
mgr/dashboard: test_standby* (tasks.mgr.test_dashboard.TestDashboard) failed locally

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-12-10 13:25:03 +01:00
Patrick Donnelly
69e073aed3
Merge PR #38407 into master
* refs/pull/38407/head:
	qa: allow arbitrary mount options on kclient mounts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-12-09 11:29:10 -08:00
Varsha Rao
db50dd29c3 qa/tasks/cephfs/nfs: Check if host ip is in cluster info output
Along with host IP, sometimes Docker container IP's shows up in 'hostname -I'
output. Since this output is variable. Just check if host IP is present in the
cluster info output.

Fixes: https://tracker.ceph.com/issues/48491
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-12-09 16:52:50 +00:00
Patrick Donnelly
7ba3669e88
Merge PR #38419 into master
* refs/pull/38419/head:
	vstart_runner: print result line unless opt_rotate_log is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-08 07:35:23 -08:00
Jeff Layton
4181742a3b qa: allow arbitrary mount options on kclient mounts
With this, we can pass a list of mount options to the kclient that will
be added onto the default ones. This is necessary to support testing
with fscache enabled (and other features activated by mount options).

Fixes: https://tracker.ceph.com/issues/6373
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2020-12-04 18:22:28 -05:00
Ernesto Puerta
4ae467e979
Merge pull request #38298 from bk201/wip-43165
mgr/dashboard: refactor /api/orchestrator/* endpoints

Reviewed-by: Courtney Caldwell <ccaldwel@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-12-04 18:24:46 +01:00
Volker Theile
9d4282e53a mgr/dashboard: test_standby* (tasks.mgr.test_dashboard.TestDashboard) failed locally
Fixes: https://tracker.ceph.com/issues/48449

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-12-04 14:39:37 +01:00
Tarang Sharma
8e1afc35f8 test_cephfs_shell: add tests for ls command
Signed-off-by: Tarang Sharma <tarangnsharma@gmail.com>
2020-12-04 00:03:19 +05:30
Deepika Upadhyay
4da3b23b89 qa/tasks/ceph: rename s/update_archive_setting/update_info_yaml
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-03 22:08:52 +05:30
Deepika Upadhyay
6d7ba1180e qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility
in extention for: 9536625558

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-03 21:23:12 +05:30
Deepika Upadhyay
22b832e94e qa/tasks/cephadm: refactor
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-03 21:23:12 +05:30
Deepika Upadhyay
2f0209ae43 qa/tasks/cephadm: update_info_yaml() only if ctx.archive is valid
related to: 6cd8e4b55883b2d282f101f11521a2759b3db564
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-12-03 21:23:11 +05:30
Yin Congmin
29690a338b tools: add throttle mechanism to immutable object cache
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2020-12-03 19:14:44 +08:00
Rishabh Dave
b0cc7ac1a8 vstart_runner: print result line unless opt_rotate_log is set
Unless the option to rotate logs is set, don't delete the final result
line printed by unittest.

Fixes: https://tracker.ceph.com/issues/48447
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-12-03 16:20:30 +05:30
Marcus Watts
1c18808b07 qa/tasks/openssl_keys.py: add subjectAltName to certificates
Get rid of this annoying teuthology log message which appears
many many times:

.../urllib3/connection.py:395: SubjectAltNameWarning: Certificate
for <some_host> has no `subjectAltName`, falling back to check for a
`commonName` for now. This feature is being removed by major browsers and
deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497
for details.)

I'm also adding the ip address, which also allows https://IPaddress/
This is part of the standard and works with most clients, but python
ignores this.  C'est la vie.

Fixes: https://tracker.ceph.com/issues/48177
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2020-12-02 15:44:35 -05:00
Patrick Donnelly
9b8fc10031
Merge PR #37708 into master
* refs/pull/37708/head:
	qa/suites/fs: enable thrashing in multifs environment
	qa/workunits/fs/snaps: allow tests to be run
	qa/tasks/{kclient,ceph_fuse}: allow mounting
	qa/tasks: allow per file system config setting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-02 12:07:17 -08:00
Patrick Donnelly
5c9fbc7948
Merge PR #38093 into master
* refs/pull/38093/head:
	qa: add debug information for client address for kclient
	qa/kernel_mount.py: rename _read_debug_file to read_debug_file

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-30 20:04:18 -08:00
Kefu Chai
e59c088e3b
Merge pull request #38281 from tchaikov/wip-mgr-no-kv-options
mgr: do not migrate conf from config-key store to new-style conf

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-01 07:12:43 +08:00
Ali Maredia
1bfdd2c1c2
Merge pull request #37503 from TRYTOBE8TME/wip-rgw-sts-web-identity
Wip rgw sts web identity
2020-11-29 11:24:05 -05:00
Kalpesh Pandya
64db3a9aca qa: STS Web Identity task files
This commit comprises of 3 things:
1) Keycloak task (to bring up keycloak in teuthology inorder to execute Assume_Role_With_Web_Identity tests).
2) The required changes for this in s3tests.py which consists of writing some parameters to configuration file.
3) The yaml files which executes these tasks.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2020-11-29 04:52:19 +05:30
Kefu Chai
8cdc8c73bd
Merge pull request #38286 from votdev/issue_48360_disable_tlsv1
mgr/dashboard: Disable TLS 1.0 and 1.1

Reviewed-by: Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2020-11-28 16:14:03 +08:00
Ramana Raja
a38b836589 qa/suites/fs: enable thrashing in multifs environment
Fixes: https://tracker.ceph.com/issues/15134
Co-authored-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-11-27 15:55:01 +05:30
Kefu Chai
09f1e7ffff
Merge pull request #38241 from Rethan/update_url_from_master_to_lastest
doc/dev: use http://docs.ceph.com/en/latest/ instead of /docs/master/ for docs

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2020-11-27 14:00:12 +08:00
Volker Theile
5cf222b6d2 mgr/dashboard: Disable TLS 1.0 and 1.1
Disable these TLS versions because of security issues.

Fixes: https://tracker.ceph.com/issues/48360

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-11-26 15:40:15 +01:00
Kiefer Chang
812c105a43
mgr/dashboard: refactor /api/orchestrator/* endpoints
- API changes:
  - Move `/api/orchestrator/identify_device` to
    `/api/host/<hostname>/identify_device`.
  - Move `/api/orchestartor/inventory` to `/ui-api/host/inventory`. This
    UI API provides a shortcut to get all inventories.
  - Add `/api/host/<hostname>/inventory` for getting a host's inventory.
  - Add inventory schema to improve OpenAPI doc.
- Backend unittests:
  - Refactor: Remove duplicated orchestrator patch calls.
  - Add unittest for identify device.

Fixes: https://tracker.ceph.com/issues/43165
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-11-26 15:44:11 +08:00
Kefu Chai
70e99e76b5 mgr: do not migrate conf from config-key store to new-style conf
since all module options are using the new-style config framework.
the migration is offered for the use case of upgrade from luminous to mimic,
since pacific can only be upgraded from octopus. the mimic monitors are alreay
able to populate the configurations to mgr, not to mention the octopus
monitors, so there is no need to migrate the options stored in config-key
store anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-25 23:30:15 +08:00
haoyixing
0e7e036aa7 doc/dev: use http://docs.ceph.com/en/latest/ instead of /docs/master/ for docs
Several links under http://docs.ceph.com/docs/master/ were unable to access.
Change them to http://docs.ceph.com/en/lastest so we can access them directly.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
2020-11-24 12:49:47 +08:00
Ilya Dryomov
0e4bc27722 qa/tasks/ceph: resurrect log compression
Commit 9536625558 ("qa/tasks/ceph: use Cluster.sh() and Remote.sh()
when appropriate") dropped run.wait(), which waits for all given
processes to exit.  This resulted in errors like

  INFO:teuthology.orchestra.run.smithi107.stderr:tar: ./objectstore_tool..log: file changed as we read it
  INFO:teuthology.orchestra.run.smithi107.stderr:tar: ./ceph-client.admin.175125.log: File removed before we read it

as the task moved on to archiving semi-corrupted and uncompressed logs,
filling up the lab cluster.

Revert that hunk, as Cluster.sh() is useless here -- we don't need
stdout or stderr, but very much need parallel execution and wait for
the compression to finish.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-11-24 00:51:54 +01:00
Ernesto Puerta
e238c6f6e7
Merge pull request #37740 from s0nea/wip-dashboard-ceph-input-validation
mgr/dashboard: validate username while creation

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-20 17:45:44 +01:00
Ramana Raja
a232d273c1 qa/workunits/fs/snaps: allow tests to be run
... filesystems other than 'cephfs'.

It is not required to set  'allow_new_snaps' to True to allow snapshot
to be created on a filesystem. Remove that setting.

Remove 'fs/snaps/snaptest-0.sh' that is racy when running in parallel on
an another client that mounted the same file system. Include a similar
test in qa/tasks/cephfs/test_snapshots.py

Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-11-20 13:23:21 +05:30
Ramana Raja
8c11cdf5dc qa/tasks/{kclient,ceph_fuse}: allow mounting
... a specific filesystem.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-11-20 13:23:21 +05:30
Ramana Raja
7016a2001d qa/tasks: allow per file system config setting
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-11-20 13:23:21 +05:30
Patrick Donnelly
cfabba6668
Merge PR #34552 into master
* refs/pull/34552/head:
	tests: add snap schedule tests
	snap-schedule: reject invalid retention specs
	mgr/snap_schedule: log snapshot name on creation
	pybind / snap_schedule: sanely dump schedules and retention specs
	pybind/snap_schedule: config option to allow minute granularity snaps
	pybind / snap_schedule: remove 'SNAP_SCHED_TESTING' check

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-18 07:54:15 -08:00
Neha Ojha
1523bf9bdb
Merge pull request #38107 from ceph/wip-mgr-progress-fix-48217
qa/mgr/test_progress: add _get_osd_in_out_events to account for osd marked in/out events

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-18 07:15:22 -08:00
Kefu Chai
739550f365
Merge pull request #38004 from tchaikov/wip-48058
qa/tasks/ceph: update_archive_setting() only if ctx.archive is valid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-18 09:58:47 +08:00
Josh Durgin
c2b9152eac
Merge pull request #37488 from kamoltat/wip-mgr-progress-turn-off-option
mgr/progress: progress module on/off option

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-11-17 13:37:30 -08:00
Patrick Donnelly
f6639c80ed
Merge PR #35749 into master
* refs/pull/35749/head:
	Update Release notes for multimds scrub
	qa/cephfs: log-ignorelist scrub errors
	qa/cephfs: Add more tests for multimds scrub
	qa/cephfs: add tests for multimds scrub
	qa/cephfs: update existing scrub test cases
	mds: don't skip validating disk state of symlink
	mds: abort/pause/resume scrubs in multiple mds
	mds: track scrub status in multiple mds
	mds: remove on_finish from {CInode,CDir}::scrub_info_t
	Continuation: don't delete self while there are in-processing stages
	mds: auth pin CInode when validating its disk state
	mds: rdlock file/nest lock when accumulating stats of subtree dirfrags
	mds: multiple mds scrub support
	include/frag: add encode/decode functions for fragset_t
	mds: remove object can't be scrubbed immediately from scrub stack
	mds: prevent dirfrag scrub/fragment from running at the same time
	mds: change scrub traverse from post-order to breadth-first search
	mds: make both CInode and CDir as entities of scrub
	mds: remove ScrubStack::scrubstack

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-17 07:33:20 -08:00
Venky Shankar
a8c8b3ade2 tests: add snap schedule tests
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-11-17 08:39:09 -05:00
Kamoltat
52fe9dbdae qa/mgr/test_progress: fix bug 48217
Fixes a failing test case regarding osd coming back
after being marked out. The old test case wasn't accounting
for a specific event, therefore this resulted in the failure.
The fix basically accounts for a specific event of osd being
marked in/out.

Fixes: https://tracker.ceph.com/issues/48217

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2020-11-17 07:54:35 +00:00
Xiubo Li
cf6797828a qa: add debug information for client address for kclient
Fixes: https://tracker.ceph.com/issues/48242
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-11-17 09:44:57 +08:00
Casey Bodley
c10a7240b6
Merge pull request #38105 from ofriedma/wip-barbican-qa-issue
qa/rgw: fix "cannot create secret" on barbican test

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-11-16 09:53:08 -05:00
Tatjana Dehler
fbf1c37220 mgr/dashboard: validate username while creation
When creating a user the username is not checked if
it's valid from the Ceph perspective (`CephString`).
The commit adds a decorator to check if the input
values in the API are valid from the Ceph perspective
by calling the `valid()` method of the Ceph-defined
datatypes (`ceph_argparse.py`).

Fixes: https://tracker.ceph.com/issues/46548
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-11-16 12:12:25 +01:00
Or Friedmann
7951391ce2 qa/rgw: fix "cannot create secret" on barbican test
fix "cannot create secret" on barbican test

Signed-off-by: Or Friedmann <ofriedma@redhat.com>

Fixes: https://tracker.ceph.com/issues/47799
2020-11-16 11:53:54 +02:00
Xiubo Li
66be9cd291 qa/kernel_mount.py: rename _read_debug_file to read_debug_file
Fixes: https://tracker.ceph.com/issues/48053
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-11-16 13:19:51 +08:00
Kamoltat
993bb02b30 mgr/progress: introduce turn off/on feature
progress module can be turned off/on by using
the commands: 'progress off' and 'progress on'

As well as refractoring teuthology test suite
to prevent future bugs that can possibly occur

fixes: https://tracker.ceph.com/issues/47238

Signed-off-by: kamoltat <ksirivad@redhat.com>
2020-11-16 03:46:42 +00:00