Commit Graph

897 Commits

Author SHA1 Message Date
Stephan Müller
c98c8055c2 mgr/dashboard: Unset compression arguments for pools
The dashboard backend can now unset all set compression arguments if the
compression mode is switched to 'unset'. In the case of 'unset' Ceph
itself will only delete the 'compression_mode' argument, not all other
set arguments. The other arguments that should be removed, too, are
added to the update arguments in order to delete all set arguments.

Fixes: https://tracker.ceph.com/issues/36355
Signed-off-by: Stephan Müller <smueller@suse.com>
2018-10-09 15:56:47 +02:00
Patrick Nawracay
a4956b40b2 mgr/dashboard: Fix errors when clicking on new OSD
Fixes: http://tracker.ceph.com/issues/36245

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-10-08 10:45:58 +02:00
Patrick Nawracay
0f0de0ce2a mgr/dashboard: Add type hints to QA helper
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-10-08 10:42:14 +02:00
Ricardo Dias
43a72f0e49
Merge pull request #24350 from ricardoasmarques/wip-json-decode
mgr/dashboard: Check content-type before decode json response

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-10-04 10:08:37 +01:00
Patrick Donnelly
bae625e666
Merge PR #21566 into master
* refs/pull/21566/head:
	test: add test for mds drop cache command
	mds: command to trim mds cache and client caps
	mds: implement journal flush as asynchronous context execution
	mds: cleanup some asok commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-01 12:48:16 -07:00
Patrick Donnelly
b07a184859
Merge PR #24044 into master
* refs/pull/24044/head:
	qa: tolerate failed rank while waiting for state

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2018-10-01 10:43:35 -07:00
Ricardo Marques
731dcefbeb mgr/dashboard: Check content-type before decode json response
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-10-01 11:47:38 +01:00
Patrick Donnelly
3a10d74f3a
qa: add timeout to cleaning up workunit sandbox
If there is a bug preventing rm from completing, the workunit will get stuck.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-30 18:12:19 -07:00
Patrick Donnelly
7a64eb9dfb
qa: add timeout to kclient umount
Otherwise QA sits forever waiting for the kclient to umount when there is a
problem.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-30 18:12:15 -07:00
Patrick Donnelly
d4b8f94cf8
qa: do not cleanup sandbox on error
Otherwise the command will hang if the mount is broken.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-30 18:12:11 -07:00
Venky Shankar
43d1b8e174 test: add test for mds drop cache command
Fixes: http://tracker.ceph.com/issues/23362
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2018-09-28 00:07:59 -04:00
Lenz Grimmer
75c1d8eacd
Merge pull request #23351 from tspmelo/wip-rbd-trash
mgr/dashboard: Add support for RBD Trash

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-09-27 11:04:46 +02:00
Mykola Golub
78881934c0
Merge pull request #22043 from dillaman/wip-24139
rbd-mirror: bootstrap needs to handle local image id collision

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-26 22:49:02 +03:00
Jason Dillaman
5c317aef31 qa/tasks/rbd_mirror_thrash: let daemon gracefully shut down if possible
Otherwise, try to capture a core dump to discover what was blocking the
shutdown process.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-25 16:08:59 -04:00
Patrick Donnelly
7bcab914bb
Merge PR #24252 into master
* refs/pull/24252/head:
	qa: cleanup workunit dir for each unit
	qa: use sudo to cleanup workspace

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-09-25 07:36:16 -07:00
Tiago Melo
6bb951b1bf mgr/dashboard: Add RBD Trash endpoints
Fixes: http://tracker.ceph.com/issues/24272

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-09-25 14:02:58 +01:00
Patrick Donnelly
0dd623c8aa
Merge PR #23187 into master
* refs/pull/23187/head:
	test: make rank argument mandatory when running journal_tool
	cephfs-journal-tool: make "--rank" argument mandatory
	cephfs-journal-tool: pass local arg vector for Journal actions
	cephfs-journal-tool: dump to per rank output file wherever necessary

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-24 14:46:14 -07:00
Patrick Donnelly
51b27b7e3d
Merge PR #23530 into master
* refs/pull/23530/head:
	qa/vstart_runner: fix daemons list
	PendingReleaseNotes: note multifs support in libcephfs
	test/cephfs: add pybind test for mount_root
	pybind/cephfs: enable passing filesystem name to mount
	libcephfs: add ceph_select_filesystem
	common: add doc strings to client_mds_namespace
	client: allow passing fs name to mount()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Conflicts:
    PendingReleaseNotes
2018-09-24 14:42:13 -07:00
Patrick Donnelly
70844f3f55
qa: cleanup workunit dir for each unit
This was wrongly dropped and moved to the finalizer.

Introduced-by: de824f74dd
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-24 11:29:10 -07:00
Patrick Donnelly
1eaf78a754
qa: use sudo to cleanup workspace
Files in scratch_tmp may not be owned by ubuntu.

Fixes: http://tracker.ceph.com/issues/36165
Introduced-by: de824f74dd
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-24 11:02:49 -07:00
Lenz Grimmer
4d3f896c89
Merge pull request #23568 from rhcs-dashboard/wip-24573-landing-page
mgr/dashboard: New Landing Page

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-09-24 15:32:48 +02:00
Patrick Donnelly
9e9f3ead36
Merge PR #24157 into master
* refs/pull/24157/head:
	qa: cleanup parallel execution of fsstress

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-09-21 13:01:14 -07:00
Patrick Donnelly
de824f74dd
qa: cleanup parallel execution of fsstress
Two instances of fsstress clobber each other. Just build it in the local sandbox.

Fixes: http://tracker.ceph.com/issues/24177

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-21 11:34:20 -07:00
Laura Paduano
3db50d5aca
Merge pull request #24219 from p-na/fix-test-scrub
mgr/dashboard: Possible fix for some dashboard timing issues

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-09-21 17:24:32 +02:00
Patrick Nawracay
8daffe86a6 mgr/dashboard: Fix for some dashboard timing issues
Specifically fixes the recurringly occurring `test_osd.py` error on the
`test_scrub` method. But this change should also prevent other issues of
the same kind. Issues of "same kind" are issues which occurr due to
tests which do not immediately result in a clean cluster status and
aren't manually programmed to wait for it.

Fixes: http://tracker.ceph.com/issues/36107

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-09-21 16:01:24 +02:00
Venky Shankar
f65193d955 test: make rank argument mandatory when running journal_tool
Also, fix a bunch of quirky journal_tool invocations that pass
"--rank" argument as the command argument rather than passing it
as function argument.

Fixes: https://tracker.ceph.com/issues/24780
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2018-09-21 06:09:39 -04:00
Kefu Chai
4b0e2c8ed4 qa: fix typos
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-21 12:41:42 +08:00
Lenz Grimmer
01c2ea52c1
Merge pull request #24128 from ricardoasmarques/wip-optional-user-password
mgr/dashboard: User password should be optional

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-09-19 16:44:51 +02:00
Ricardo Marques
49ba4a7b84 mgr/dashboard: User password should be optional
Fixes: https://tracker.ceph.com/issues/36031

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-09-19 13:49:22 +01:00
alfonsomthd
f9fbc25f60 mgr/dashboard: New Landing Page
Updated integration tests to check data from new python code

Fixes: https://tracker.ceph.com/issues/24573

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-09-19 08:54:41 +02:00
John Spray
da5b25db64 qa/vstart_runner: fix daemons list
This was missing a cluster name prefix that
was added at some point, and consequently
calls to iter_daemons_of_role were returning
no daemons.

This was causing e.g. TestVolumeClient.test_data_isolated
to fail when run in vstart_runner.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-17 14:00:36 -04:00
Rick Chen
4abb79f159 mgr/diskprediction: add prototype diskprediction module
This module is written by Rick Chen <rick.chen@prophetstor.com> and
provides both a built-in local predictor and a cloud mode that queries
a cloud service (provided by ProphetStor) to predict device failures.

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-17 08:20:57 -05:00
Patrick Donnelly
ea43ce7aae
Merge PR #21992 into master
* refs/pull/21992/head:
	qa: extend timeout for SessionMap flush

Reviewed-by: John Spray <john.spray@redhat.com>
2018-09-16 14:30:28 -07:00
Kefu Chai
59e6fbbe70
Merge pull request #24024 from p-na/fix-flaky-qa-tests
mgr/dashboard: Fix flaky QA tests

Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-09-13 21:52:35 +08:00
Patrick Donnelly
644a535ee6
qa: extend timeout for SessionMap flush
Fixes: https://tracker.ceph.com/issues/24129

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-12 12:52:00 -07:00
Patrick Donnelly
e52976e64d
qa: tolerate failed rank while waiting for state
Fixes: http://tracker.ceph.com/issues/35828

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-12 06:37:13 -07:00
Patrick Nawracay
079cfeae4d mgr/dashboard: Cleanup
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-09-12 10:16:54 +02:00
Patrick Nawracay
49ab1315a3 mgr/dashboard: Fix flaky QA tests
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-09-12 10:16:54 +02:00
John Spray
0551d0e14b
Merge pull request #23497 from noahdesu/insights
mgr/insights: insights reporting module

Reviewed-by: John Spray <john.spray@redhat.com>
2018-09-11 15:56:10 +01:00
John Spray
b18bee1b19
Merge pull request #23146 from jcsp/wip-progress
mgr/progress: improve+test OSD out handling

Reviewed-by: Noah Watkins <nwatkins@redhat.com>
2018-09-11 15:37:51 +01:00
Kefu Chai
510d9e1345
Merge pull request #23723 from xiexingguo/wip-list-missing
osd/PrimaryLogPG: rename list_missing -> list_unfound command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-09-11 20:25:21 +08:00
John Spray
5ecd69099d qa: add tests for progress module
Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-11 11:21:35 +01:00
vasukulkarni
10f1c4c9de
Merge pull request #23602 from smanjara/wip-test-netem
qa: Task to emulate network delay and packet drop between two given h…
2018-09-10 09:57:10 -07:00
Sage Weil
09ee3f3538 Merge PR #20469 into master
* refs/pull/20469/head:
	osd/PG: remove warn on delete+merge race
	osd: base project_pg_history on is_new_interval
	osd: make project_pg_history handle concurrent osdmap publish
	osd: handle pg delete vs merge race
	osd/PG: do not purge strays in premerge state
	doc/rados/operations/placement-groups: a few minor corrections
	doc/man/8/ceph: drop enumeration of pg states
	doc/dev/placement-groups: drop old 'splitting' reference
	osd: wait for laggy pgs without osd_lock in handle_osd_map
	osd: drain peering wq in start_boot, not _committed_maps
	osd: kick split children
	osd: no osd_lock for finish_splits
	osd/osd_types: remove is_split assert
	ceph-objectstore-tool: prevent import of pg that has since merged
	qa/suites: test pg merging
	qa/tasks/thrashosds: support merging pgs too
	mon/OSDMonitor: mon_inject_pg_merge_bounce_probability
	doc/rados/operations/placement-groups: update to describe pg_num reductions too
	doc/rados/operations: remove reference to lpgs
	osd: implement pg merge
	osd/PG: implement merge_from
	osdc/Objecter: resend ops on pg merge
	osd: collect and record pg_num changes by pool
	osd: make load_pgs remove message more accurate
	osd/osd_types: pg_t: add is_merge_target()
	osd/osd_types: pg_t::is_merge -> is_merge_source
	osd/osd_types: adding or substracting invalid stats -> invalid stats
	osd/PG: clear_ready_to_merge on_shutdown (or final merge source prep)
	osd: debug pending_creates_from_osd cleanup, don't use cbegin
	ceph-objectstore-tool: debug intervals update
	mgr/ClusterState: discard pg updates for pgs >= pg_num
	mon/OSDMonitor: fix long line
	mon/OSDMonitor: move pool created check into caller
	mon/OSDMonitor: adjust pgp_num_target down along with pg_num_target as needed
	mon/OSDMonitor: add mon_osd_max_initial_pgs to cap initial pool pgs
	osd/OSDMap: set pg[p]_num_target in build_simple*() methods
	mon/PGMap: adjust SMALLER_PGP_NUM warning to use *_target values
	mon/OSDMonitor: set CREATING flag for force-create-pg
	mon/OSDMonitor: start sending new-style pg_create2 messages
	mon/OSDMonitor: set last_force_resend_prenautilus for pg_num_pending changes
	osd: ignore pg creates when pool FLAG_CREATING is not set
	mgr: do not adjust pg_num until FLAG_CREATING removed from pool
	mon/OSDMonitor: add FLAG_CREATING on upgrade if pools still creating
	mon/OSDMonitor: prevent FLAG_CREATING from getting set pre-nautilus
	mon/OSDMonitor: disallow pg_num changes while CREATING flag is set
	mon/OSDMonitor: set POOL_CREATING flag until initial pool pgs are created
	osd/osd_types: add pg_pool_t FLAG_POOL_CREATING
	osd/osd_types: introduce last_force_resend_prenautilus
	osd/PGLog: merge_from helper
	osd: no cache agent or snap trimming during premerge
	osd: notify mon when pending PGs are ready to merge
	mgr: add simple controller to adjust pg[p]_num_actual
	mon/OSDMonitor: MOSDPGReadyToMerge to complete a pg_num change
	mon/OSDMonitor: allow pg_num to adjusted up or down via pg[p]_num_target
	osd/osd_types: make pg merge an interval boundary
	osd/osd_types: add pg_t::is_merge() method
	osd/osd_types: add pg_num_pending to pg_pool_t
	osd: allow multiple threads to block on wait_min_pg_epoch
	osd: restructure advance_pg() call mechanism
	mon/PGMap: prune merged pgs
	mon/PGMap: track pgs by state for each pool
	osd/SnapMapper: allow split_bits to decrease (merge)
	os/bluestore: fix osr_drain before merge
	os/bluestore: allow reuse of osr from existing collection
	os/filestore: (re)implement merge
	os/filestore: add _merge_collections post-check
	os: implement merge_collection
	os/ObjectStore: add merge_collection operation to Transaction
2018-09-07 15:55:21 -05:00
Ilya Dryomov
478aca82eb
Merge pull request #23976 from idryomov/wip-cram-git-clone
qa/tasks/cram: tasks now must live in the repository

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-09-07 19:57:42 +02:00
Sage Weil
6bd682f53d ceph-objectstore-tool: prevent import of pg that has since merged
We currently import a portion of the PG if it has split.  Merge is more
complicated, though, mainly because COT is operating in a mode where it
fast-forwards the PG to the latest OSDMap epoch, which means it has to
implement any transformations to the PG (split/merge) independently.
Avoid doing this for merge.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-07 12:09:05 -05:00
Sage Weil
0b59b7a688 qa/tasks/thrashosds: support merging pgs too
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-07 12:09:05 -05:00
Ilya Dryomov
592f566b4e qa/tasks/cram: tasks now must live in the repository
Commit 0d8887652d ("qa/tasks/cram: use suite_repo repository for all
cram jobs") removed hardcoded git.ceph.com links, but as it turned out
it is still used for nightlies.  There is no good way to accommodate
the different URL schemes, so let's get rid of URLs altogether.

Fixes: https://tracker.ceph.com/issues/27211
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-09-06 22:32:39 +02:00
Ilya Dryomov
e1c89b51c8 qa/tasks/workunit: factor out overrides and refspec logic
Allow for reuse in the cram task.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-09-06 22:31:55 +02:00
xie xingguo
85ba2f0a82 osd/PrimaryLogPG: s/list_missing/list_unfound/
Also:
- Do not print **offset** until specified
- Count missing objects correctly (used to be primary's local missing)

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-09-06 09:52:20 +08:00