Commit Graph

3799 Commits

Author SHA1 Message Date
Sage Weil
247586a0be Merge PR #22596 into master
* refs/pull/22596/head:
	os/bluestore: use vector instead of set for zombies
	os/bluestore: reuse zombie OpSequencers by collection id
	qa/suites/rados/objecstore/backends/objectstore: capture coredumps
	os/bluestore: more debug output
	os/bluestore: print cnode from _open_collections
	os/bluestore: print cnode on fsck
	qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-06-19 13:51:09 -05:00
Sage Weil
55f390d0eb qa/suites/rados/objecstore/backends/objectstore: capture coredumps
Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-19 13:49:57 -05:00
Sage Weil
353c906ca7
Merge pull request #22606 from yuriw/wip-yuriw-crontab
qa/tests - added powercycle suite to run on weekly basis on master and mimic
2018-06-19 12:59:22 -05:00
Patrick Donnelly
111c515ab0
Merge PR #22569 into master
* refs/pull/22569/head:
	qa/cephfs: add test for "dump inode"
	mds: add dump inode command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-19 07:13:30 -07:00
Sage Weil
0f38951ea3 Merge PR #22582 into master
* refs/pull/22582/head:
	mon: destroy-new -> purge-new
	mon/OSDMonitor: fix NEW flag adjustment in destroy[-new]

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-06-19 07:37:39 -05:00
Sage Weil
3cd7d5eb22 Merge PR #22343 into master
* refs/pull/22343/head:
	qa/standalone remove ceph-disk from activate_osd helper
	cmake: remove subman.sh tests
	test remove ceph-disk directory
	debian: remove ceph_detect_init python files from base
	qa/standalone remove virtualenv paths for ceph-disk and ceph-detect-init
	debian: remove ceph-disk ceph-detect-init python files
	rpm: remove ceph-disk ceph-detect-init python files
	alpine: remove ceph-disk ceph-detect-init python files
	alpine: remove ceph-osd and parttypeuuid udev rules
	debian: remove ceph-osd and parttypeuuid udev rules
	rpm: remove ceph-osd and parttypeuuid udev rules
	ceph-helpers.sh: remove ceph-disk, set up osds directly
	CMakeLists.txt: add back CEPH_BUILD_VIRTUALENV
	alpine: remove ceph-disk, add ceph-volume in APKBUILD.in
	upstart: remove ceph-disk activation call
	doc/install add anchor for manual osd deployment in freebsd guide
	doc/dev remove ceph-disk from freebsd guide, link to manual reference
	doc/dev/config-key remove ceph-disk references
	doc/dev remove ceph-disk.rst
	doc/dev: change ceph-disk suite examples for ceph-deploy
	doc/man_index: remove ceph-disk, ceph-detect-init refs
	doc/install: remove ceph-disk from freebsd examples
	doc/rados remove ceph-disk from man references
	doc/man remove ceph-disk ref from ceph-volume-systemd
	doc/man: update reference from ceph-disk to ceph-volume
	doc/man: remove ceph-disk, ceph-detect-init from cmake
	doc/man/ceph-volume remove doc reference to ceph-disk
	doc/man: remove ceph-disk, ceph-detect-init
	qa/suites: remove ceph-disk
	qa/run-standalone.sh: remove requirement for ceph-detect-init virtualenv
	qa/workunits: remove ceph-detect-init from rbdmapfile test
	qa/workunits: remove ceph-detect-init from ceph-helpers-root.sh
	qa/workunits: remove ceph-disk
	build: remove ceph-disk from freebsd script
	cmake: remove ceph-disk, ceph-detect-init tox tests
	init-ceph: remove ceph-disk
	cmake: remove top-level entries for ceph-disk, ceph-detect-init
	debian: remove ceph-detect-init references
	debian: remove ceph-disk references
	src: remove ceph-detect-init tool
	rpm: remove ceph-disk, ceph-detect-init from spec file
	test: remove subman script
	script: remove subman script
	udev: remove parttypeuuid rules for ceph-disk
	tool remove ceph-disk from ps-ceph.pl
	upstart: remove ceph-disk conf file
	systemd: remove ceph-disk from CMakeLists
	systemd: remove ceph-disk service
	udev: remove ceph-disk rules
	src: remove ceph-disk tool
2018-06-19 07:07:55 -05:00
David Zafman
ea75e03d22 test: run-standalone move aside cores found before testing starts
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-18 14:10:10 -07:00
David Zafman
fe09fc5e9d test: Fail immediately if some operations fail
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-18 14:09:14 -07:00
David Zafman
33538aca35 test: Fix standalone main usage
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-18 14:09:14 -07:00
David Zafman
f886ebba08 test: Fix some function desciptions
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-18 14:09:14 -07:00
David Zafman
39fc43556f test: Put files in private test directory
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-18 14:08:23 -07:00
Yuri Weinstein
ae1ad9c0b4 qa/tests - added powercycle suite to run on weekly basis on master and mimic
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-06-18 12:56:18 -07:00
Lenz Grimmer
ae4f4d315a
Merge pull request #22416 from votdev/get_rgw_api_user_via_rest
mgr/dashboard: Get user ID via RGW Admin Ops API

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-06-18 17:17:33 +02:00
taodd
0bd385f30b qa/cephfs: add test for "dump inode"
Signed-off-by: dongdong tao <tdd21151186@gmail.com>
2018-06-16 17:46:04 +08:00
Sage Weil
761f68dc0d mon: destroy-new -> purge-new
What we actually want is a purge, not a destroy.  Destroy leaves the OSD
ID in used and allows it to be recreated.  What ceph-volume wants is to
purge all trace of the failed OSD setup.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-15 14:11:01 -05:00
Volker Theile
ed0750699e mgr/dashboard: Get user ID via RGW Admin Ops API.
The RGW API user id (set via 'ceph dashboard set-rgw-api-user-id <xxx>') is optional but the user ID is required internally for some situations. Because of that the user ID is requested via a RGW Admin Ops API call if it is not configured via CLI.

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-06-15 16:24:33 +02:00
Patrick Nawracay
d7de1fa067 mgr/dashboard/backend: Enable get/set of cluster-wide OSD settings
Add ability to list, set and unset cluster-wide OSD flags.

Flags can be listed and changed through the `/api/osd/flags` API
resource. By using a GET request, the list is retrieved. By using a PUT
request, the flags are updated (all at once). Flags not contained in the
data of the PUT are removed, additional once are added. Note that the
PUT requests require a JSON body with the data contained as value of the
'flags' key like so:

    {"flags": ["flag1", "flag2", ...]}

Fixes: http://tracker.ceph.com/issues/24056

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-06-15 10:26:29 +02:00
Sage Weil
70ba58e9e8 Merge PR #22554 into master
* refs/pull/22554/head:
	qa/standalone/ceph-helpers.sh: Fixing comment for wait_for_health()
	tests: Protecting rados bench against endless loop
	qa/standalone/ceph-helpers.sh: Defining custom timeout for wait_for_clean()

Reviewed-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-06-14 13:04:56 -05:00
Lenz Grimmer
d785010a35
Merge pull request #22303 from ricardoasmarques/wip-help-menu
mgr/dashboard: Add help menu entry

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-06-14 15:56:39 +02:00
Lenz Grimmer
27f6012ff4
Merge pull request #21460 from s0nea/wip-dashboard-config-options
mgr/dashboard: Config options integration (read-only) depends on #22422

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-06-14 12:59:47 +02:00
Erwan Velu
2ce480b8fd qa/standalone/ceph-helpers.sh: Fixing comment for wait_for_health()
wait_for_health doesn't check if the cluster is making progress. So
let's adjust the comment accordingly.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-06-14 11:06:52 +02:00
Erwan Velu
e6e10246c6 tests: Protecting rados bench against endless loop
If the cluster dies during the rados bench, the maximum running time is
no more considered and all emitted aios are pending.

rados bench never quits and the global testing timeout (3600 sec : 1
hour) have to be reach to get a failure.

This situation is dramatic for a background test or a CI run as it locks
the whole job for too long for an event that will never occurs.

This ideal solution would be having 'rados bench' considering a failure
once the timeout is reached when aios are pending.

A possible workaround here is to put use the system command 'timeout'
before calling rados bench and fail if rados didn't completed on time.

To avoid side effects, this patch is doubling rados timeout. If rados
didn't completed after twice the expected time, it have to fail to avoid
locking the whole testing job.

Please find below the way it worked on a real test case.
We can see no IO after t>2 but despite timeout=4 the bench continue.
Thanks to this patch, the bench is stopped at t=8 and return 1.

5: /home/erwan/ceph/src/test/smoke.sh:55: TEST_multimon:  timeout 8 rados -p foo bench 4 write -b 4096 --no-cleanup
5: hints = 1
5: Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 4 seconds or 0 objects
5: Object prefix: benchmark_data_mr-meeseeks_184960
5:   sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
5:     0       0         0         0         0         0           -           0
5:     1      16      1144      1128   4.40538   4.40625  0.00412965   0.0141116
5:     2      16      2147      2131   4.16134   3.91797  0.00985654   0.0109079
5:     3      16      2147      2131   2.77424         0           -   0.0109079
5:     4      16      2147      2131    2.0807         0           -   0.0109079
5:     5      16      2147      2131   1.66456         0           -   0.0109079
5:     6      16      2147      2131   1.38714         0           -   0.0109079
5:     7      16      2147      2131   1.18897         0           -   0.0109079
5: /home/erwan/ceph/src/test/smoke.sh:55: TEST_multimon:  return 1
5: /home/erwan/ceph/src/test/smoke.sh:18: run:  return 1

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-06-14 11:06:52 +02:00
Erwan Velu
62d2646c30 qa/standalone/ceph-helpers.sh: Defining custom timeout for wait_for_clean()
The wait_for_clean() is using the default timeout aka 300sec = 5mn.

wait_for_clean() is trying to find a clean status within that timeout
_or_ reset its counter if any progress got made in between loops.

In a case where the cluster is sane, the recovery should be made in
shorter than 5mn but it the cluster died, waiting for 5mn for nothing is
unefficient.

This patch is about defining a custom timeout for a wait_for_clean() not
to wait much more that 1m30 (90sec). If no progress is made in that
period, there is very few chance this will read the a valid state
anyhow.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-06-14 11:06:52 +02:00
Jason Dillaman
ad89b795f0
Merge pull request #21969 from zhongyimao/interlock_fast-diff/object-map
rbd: interlock object-map/fast-diff features together

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-06-13 16:22:04 -04:00
Alfredo Deza
5b3a540045 qa/standalone remove ceph-disk from activate_osd helper
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:27 -04:00
Alfredo Deza
aa4f5569c3 qa/standalone remove virtualenv paths for ceph-disk and ceph-detect-init
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:27 -04:00
Dan Mick
50f2b72f2f ceph-helpers.sh: remove ceph-disk, set up osds directly
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2018-06-13 15:16:26 -04:00
Alfredo Deza
5f0d542cbd qa/suites: remove ceph-disk
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:24 -04:00
Alfredo Deza
a8095ee7e0 qa/run-standalone.sh: remove requirement for ceph-detect-init virtualenv
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:24 -04:00
Alfredo Deza
c954a71660 qa/workunits: remove ceph-detect-init from rbdmapfile test
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:24 -04:00
Alfredo Deza
c0a3ddc6c3 qa/workunits: remove ceph-detect-init from ceph-helpers-root.sh
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:23 -04:00
Alfredo Deza
ce4f679d66 qa/workunits: remove ceph-disk
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-06-13 15:16:23 -04:00
Tatjana Dehler
9ebf5eb650 mgr/dashboard: add config options to documentation page
This commit adds the config options stored by the MON database to the
configuration documentation page.
One can filter for these config options by setting the 'Source' filter
to 'mon' on the configuration documentation page.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-06-13 15:39:24 +02:00
Sage Weil
23dd6279bb qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore
If we fail, preserve the data directory.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-13 06:55:18 -05:00
Yuri Weinstein
bb5a296fe8
Merge pull request #22531 from yuriw/wip-yuriw-crontab
qa/tests: removed all jewel runs and reduced runs on ovh
2018-06-12 12:58:43 -07:00
Yuri Weinstein
f8b4a1d99b qa/tests: removed all jewel runs and reduced runs on ovh
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-06-12 09:49:09 -07:00
Ricardo Marques
a97991d480 mgr/dashboard: Summary endpoint should return ceph version
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-06-12 11:46:45 +01:00
Mao Zhongyi
8d5bee2d46 rbd: interlock object-map/fast-diff features together
Since we only support Jewel and later releases, which both support
object-map and fast-diff, enabling/disabling object-map should always
enable/disable fast-diff.

Signed-off-by: Mao Zhongyi <maozy.fnst@cn.fujitsu.com>
2018-06-12 09:25:55 +08:00
Sage Weil
1e64c86d1a
Merge pull request #22496 from liewegas/wip-localpool-config
qa/workunits/mgr/test_localpool.sh: use new config syntax
2018-06-11 16:08:18 -05:00
Sage Weil
9154a392b0
Merge pull request #22460 from liewegas/wip-upgrade-distro
qa/suites/rados/upgrade: remove stray link
2018-06-10 13:42:23 -05:00
Sage Weil
dadf4c769b qa/workunits/mgr/test_localpool.sh: use new config syntax
Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-10 13:08:52 -05:00
Sage Weil
826d36dea8 Merge PR #22485 into master
* refs/pull/22485/head:
	qa: wait longer for osd to flush pg stats

Reviewed-by: Sage Weil <sage@redhat.com>
2018-06-09 14:21:37 -05:00
Kefu Chai
af4793f606
Merge pull request #22429 from liewegas/wip-osd-destroy-new
mon: add 'osd destroy-new' command that only destroys NEW osd slots

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>
2018-06-09 22:00:47 +08:00
Kefu Chai
16b84f7332 qa: wait longer for osd to flush pg stats
it's the test_create_from_mon()'s counterpart of 38074726.

Fixes: http://tracker.ceph.com/issues/24321
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-09 13:51:20 +08:00
David Zafman
4ffa5a46d2
Merge pull request #22428 from dzafman/wip-24396-fix
test: Use a file that should be on all OSes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-06-08 20:28:44 -07:00
Sage Weil
697d98dafc qa/suites/rados/upgrade: remove stray link
Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-07 11:25:55 -05:00
Ricardo Marques
091e01ca71 mgr/dashboard: Login failure should return HTTP 400
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-06-06 11:45:09 +01:00
Sage Weil
547d96dc01 mon: add 'osd destroy-new' command that only destroys NEW osd slots
ceph-volume may run into a problem and want to clean up, but we do not
want to give it blanket access to the 'osd destroy' command.  Instead,
make an 'osd destroy-new' that can only create new OSDs (ones that are
in the process of being created but have never booted yet).

Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-05 17:36:06 -05:00
David Zafman
c1e96ae7cb test: Use a file that should be on all OSes
Also, create temporary files in test specific dir and remove

Caused by: 154330fd68

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-05 11:27:12 -07:00
Sage Weil
43fdb22d7a
Merge pull request #22396 from liewegas/wip-24396
osd/PrimaryLogPG: fix on_local_recover crash on stray clone

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-06-05 11:09:44 -05:00