Commit Graph

3848 Commits

Author SHA1 Message Date
Josh Durgin
9106dc56c2
Merge pull request #22761 from fullerdj/wip-djf-24686
osd/filestore: Change default filestore_merge_threshold to -10

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-02 17:36:00 -07:00
Patrick Donnelly
0e50b1217d
Merge PR #22718 into master
* refs/pull/22718/head:
	qa/workunits/suites/ffsb.sh: fix strncpy bug in ffsb

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-29 10:23:38 -07:00
Douglas Fuller
75f55f2dfc osd/filestore: Change default filestore_merge_threshold to -1
Performance evaluations of medium to large size Ceph clusters have
demonstrated negligible performance impact from unnecessarily deep
directory hierarchies but significant performance impact from filestore
split and merge activity. Disable merges by default.

Fixes: http://tracker.ceph.com/issues/24686
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-06-29 11:45:12 -04:00
David Zafman
663d96e934
Merge pull request #22727 from dzafman/wip-21664
qa/standalone/scrub: When possible show side-by-side diff in addition to regular diff

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-06-28 19:59:21 -04:00
David Zafman
3ff56a82a4
Merge pull request #22763 from dzafman/wip-remove-sudo
qa: Don't use sudo when moving logs

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-06-28 18:37:24 -04:00
David Zafman
23ed63e15f
Merge pull request #22441 from ErwanAliasr1/evelu-makecheck
Improving make check reliability

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
2018-06-28 14:55:12 -04:00
David Zafman
808c628304 qa: Don't use sudo when moving logs
Caused by: f0964beac5b8b54bc44a11bfb50bdea1f2ae8825

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-28 09:17:06 -07:00
David Zafman
dee4574a66
Merge pull request #22702 from dzafman/wip-standalone-log
qa: For teuthology copy logs to teuthology expected location

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-06-28 01:57:25 -04:00
Josh Durgin
b8536e44d5
Merge pull request #22726 from yuriw/wip-yuriw-crontab
qa/tests - added mimic-p2p suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-06-27 16:13:51 -07:00
Patrick Donnelly
4309d7ce1f
Merge PR #22560 into master
* refs/pull/22560/head:
	qa/fs/upgrade/snaps: whitelist inconsistent rstat
	mds: track snapshot count in rstat
	mds: add MAY_SNAPSHOT in MDSAuthCaps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-27 14:19:03 -07:00
Patrick Donnelly
5494e7229a
Merge PR #22719 into master
* refs/pull/22719/head:
	qa: fix symlinks indirectly pointing at qa to .qa
	qa: add .qa helper link

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-06-27 12:50:33 -07:00
Patrick Donnelly
f28cc3a082
Merge PR #22725 into master
* refs/pull/22725/head:
	qa/workunits/suites/blogbench.sh: use correct dir name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-06-27 11:13:37 -07:00
Mykola Golub
40617ad18b
Merge pull request #22673 from dillaman/wip-24558
rbd: basic support for images within namespaces

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-06-27 11:40:23 +03:00
Yan, Zheng
162b1ed5f9 qa/fs/upgrade/snaps: whitelist inconsistent rstat
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-06-27 11:14:34 +08:00
David Zafman
ebb05b2542 test: When possible show side-by-side diff in addition to regular diff
Fixes: https://tracker.ceph.com/issues/21664

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-26 18:23:07 -07:00
Yuri Weinstein
298065df29 qa/tests - added mimic-p2p suite
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-06-26 16:05:44 -07:00
Neha Ojha
24513c39f1 qa/workunits/suites/blogbench.sh: use correct dir name
Signed-off-by: Neha Ojha <nojha@redhat.com>
2018-06-26 18:30:38 -04:00
Patrick Donnelly
b39f9d06dc
qa: fix symlinks indirectly pointing at qa to .qa
Building on the previous commit.

Command used:

$ find suites/ -type l -and -not -name .qa -execdir ~/fix.sh {} \;

fix.sh:
    #!/bin/bash

    link="$(readlink "$1")"

    echo $link
    dirlink="$(dirname "$link")"
    baselink="$(basename "$link")"

    while true; do
        echo $dirlink
        if [ "$dirlink" -ef ~/ceph/qa ]; then
            ln -nsf ".qa/$baselink" "$1"
            exit
        else
            baselink="$(basename "$dirlink")/$baselink"
            dirlink="$(dirname "$dirlink")"
            if [ "$dirlink" -ef . ]; then
                break
            fi
        fi
    done

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-26 11:48:38 -07:00
Patrick Donnelly
716db6e2fd
qa: add .qa helper link
This utilizes the recent feature in teuthology [1] to skip hidden files in
suites when building the job matrix.

Idea of this change is to enable referring to the top-level qa directory in a
position-independent way such that copies of a suite to another location do not
break any symlinks.

[1] https://github.com/ceph/teuthology/pull/1185

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-26 11:33:48 -07:00
Lenz Grimmer
cfcfa10a5b
Merge pull request #22672 from tspmelo/wip-encode-uri
mgr/dashboard: Add support for URI encode

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-06-26 17:03:29 +02:00
Casey Bodley
be5a6d7f34
Merge pull request #22312 from ZVampirEM77/wip-default-zonegroup-apiname
rgw: fix CreateBucket with BucketLocation parameter failed under default zonegroup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-06-26 09:58:16 -04:00
Tiago Melo
f21d0da5a3 mgr/dashboard: Add support for URI encode
Created a decorator and pipe to help encode special URI components in the
frontend.

Modified the backend request handler to decode all the string args.

fixes: http://tracker.ceph.com/issues/24621

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-06-26 14:35:15 +01:00
Yan, Zheng
843cddaaf8 qa/workunits/suites/ffsb.sh: fix strncpy bug in ffsb
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/24579
2018-06-26 20:29:21 +08:00
Ricardo Dias
cc21d270c2
qa/tasks/mgr/dashboard: test_cluster_conf: restore original value of mon_allow_pool_delete
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-06-26 12:28:54 +01:00
Ricardo Dias
76f89edee0
mgr/dashboard: auth: return user permissions on login
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-06-26 12:28:54 +01:00
Ricardo Dias
01a1a1ff84
qa/tasks/mgr/dashboard: Adapted tests to work with new authentication system
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-06-26 12:28:54 +01:00
David Zafman
f0964beac5 qa: For teuthology copy logs to teuthology expected location
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-06-25 18:06:01 -07:00
Patrick Donnelly
9829f56c07
Merge PR #22532 into master
* refs/pull/22532/head:
	qa/suites: Add supported-random-distro$ links.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-25 15:48:34 -07:00
Erwan Velu
57df91380b qa/standalone/ceph-helpers.sh: Setup ulimit in setup()
If ulimit is set to a 1024 value, ceph-osd will segfault with the
following error :
    filestore(td/smoke/0)  error (24) Too many open files not handled on operation 0x55565d1fd004 (2182.1.0, or op 0, counting from 0)

This patch is about to insure that before setting up ceph daemons in tests, a valid ulimit value is setup.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-06-25 22:09:14 +02:00
Erwan Velu
7b0d1c8b8a qa/standalone/ceph-helpers.sh: Thinner resolution in get_timeout_delays()
get_timeout_delays() is a generic function to compute delays for a long
period of time without saturating the CPU is busy loops.

It works pretty fine when the delay is short like having the following
series when requesting a 20seconds timeout : "0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ".
Here the maximum between two loops is 7.3 which is perfectly fine.

When the timeout reaches 300sec, the same code produces the following
series : "0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 "
In such example there is delays which are nearly 2 minutes !

That is not efficient as the expected event, between two loops, could
arrive just after this long sleep occurs making a minute+ sleep for
nothing. On a local system that could be ok while on a CI, if all jobs
run like CI the overall is pretty unefficient by generating useless CPU
waits.

This patch is about adding a maximum acceptable delay time between two
loops while keeping the same rampup behavior.

On the same 300 seconds delay example, with MAX_TIMEOUT set to 10, we
now have the following series: "0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3"
We can see that the long 12/25/51/102/95 values vanished and being
replaced by a series of 10 seconds. It's up to every test defining the
probability of having a soonish event to complete.

The MAX_TIMEOUT is set to 15seconds.
Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-06-25 22:09:14 +02:00
Sage Weil
dd6ad72b90 mgr/devicehealth: add self-test
Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-23 17:01:55 -05:00
Enming Zhang
ad6edd4dd2 qa: fix for s3tests.functional.test_s3.test_bucket_get_location failed
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
2018-06-23 17:09:54 +08:00
Yuri Weinstein
ef91aef78e qa/tests - added all supported distro to the mix, so we can track stats across diff OSes
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-06-22 09:13:11 -07:00
Jason Dillaman
3e8624f157 rbd: add support for namespaces
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-06-22 10:43:51 -04:00
Ilya Dryomov
8b3f3c8d38
Merge pull request #22665 from idryomov/wip-krbd-lvcreate-args
qa: krbd_msgr_segments.t: filter lvcreate output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-06-22 15:51:20 +02:00
Casey Bodley
4181b03d8c
Merge pull request #22249 from cbodley/wip-qa-rgw-verify-tasks
qa/rgw: reorganize verify tasks

Reviewed-by: Sage Weil <sage@redhat.com>
2018-06-22 09:43:25 -04:00
Josh Durgin
a83a8b3aa9
Merge pull request #22666 from yuriw/wip-yuriw-crontab
qa/tests - added luminous-p2p suite to the schedule

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-06-21 17:02:36 -07:00
Yuri Weinstein
9dae20e08d qa/tests - added luminous-p2p suite to the schedule
(Merge after https://github.com/ceph/ceph/pull/22474 is merged)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-06-21 15:21:26 -07:00
Sage Weil
cf7beba662
Merge pull request #22536 from yuriw/wip-yuriw-distros-master
qa/tests: changed disto symlink to point to new way using supported OS'es
2018-06-21 17:08:12 -05:00
Ilya Dryomov
645252d732 qa: krbd_msgr_segments.t: filter lvcreate output
Some versions of lvm emit a log message

  Using default stripesize 64.00 KiB.

which fails the test.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-06-21 20:51:23 +02:00
Volker Theile
a98bca6a2f mgr/dashboard: Replace RGW proxy controller
Fixes: http://tracker.ceph.com/issues/24436

To fully support the role based authentication/authorization system it is necessary to replace the RGW proxy controller by separate controllers for RGW user and bucket.

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-06-21 14:07:51 +02:00
Mykola Golub
80a51c9714
Merge pull request #22608 from dillaman/wip-24410
librbd: namespace create/remove/list support

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-06-21 11:02:25 +03:00
Zack Cerza
6487184ebb
Merge pull request #22647 from yuriw/wip-yuriw-teuthology-master
qa/tests - added all supported distro
2018-06-20 13:52:34 -06:00
Ilya Dryomov
69619b6966 qa: krbd_exclusive_option.sh: bump lock_timeout to 60 seconds
Avoid sporadic failures in combination with msgr-failures/many.yaml,
where assert_locked() might take over 10 seconds.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-06-20 18:22:36 +02:00
Yuri Weinstein
bd8882c22f qa/tests - added all supported distro
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-06-20 09:17:39 -07:00
Sage Weil
247586a0be Merge PR #22596 into master
* refs/pull/22596/head:
	os/bluestore: use vector instead of set for zombies
	os/bluestore: reuse zombie OpSequencers by collection id
	qa/suites/rados/objecstore/backends/objectstore: capture coredumps
	os/bluestore: more debug output
	os/bluestore: print cnode from _open_collections
	os/bluestore: print cnode on fsck
	qa/suites/rados/objecstore: preserve data dir for ceph_test_objecstore

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-06-19 13:51:09 -05:00
Sage Weil
55f390d0eb qa/suites/rados/objecstore/backends/objectstore: capture coredumps
Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-19 13:49:57 -05:00
Sage Weil
353c906ca7
Merge pull request #22606 from yuriw/wip-yuriw-crontab
qa/tests - added powercycle suite to run on weekly basis on master and mimic
2018-06-19 12:59:22 -05:00
Patrick Donnelly
111c515ab0
Merge PR #22569 into master
* refs/pull/22569/head:
	qa/cephfs: add test for "dump inode"
	mds: add dump inode command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-06-19 07:13:30 -07:00
Sage Weil
0f38951ea3 Merge PR #22582 into master
* refs/pull/22582/head:
	mon: destroy-new -> purge-new
	mon/OSDMonitor: fix NEW flag adjustment in destroy[-new]

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-06-19 07:37:39 -05:00