Commit Graph

83408 Commits

Author SHA1 Message Date
Yehuda Sadeh
930f3b85e2 qa/tasks/ragweed: don't reference fastcgi
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
2d23084092 qa/rgw: ragweed: change base branch for upgrade testing
Change directory structure to reflect base version of upgrade test.
Also set base version to kraken.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
0882d306f6 qa/tasks/rgw/upgrade: don't restart the entire cluster at once
and also whitelist 'wrongly marked me down'

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
52d8f0d82f qa/tasks: add ragweed tasks
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
3bf80dd171 ceph/qa/suites: add rgw/upgrade/ragweed
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
c2e2d8a380 qa/tasks/ragweed: only init config when doing prepare
and only remove users when doing check.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
831645c3cf qa/tasks/ragweed: configurable stages
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
cb1660d566 qa/tasks/ragweed: point at ceph.conf in config
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
7bf2296bdd qa/tasks/ragweed: set system flag on system user
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Yehuda Sadeh
82c1295196 qa/tasks: add ragweed task
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-19 14:45:51 -08:00
Kefu Chai
d051bf0441 cmake: disable DOWNLOAD_NO_PROGRESS if cmake ver is lower than 3.1
see https://cmake.org/cmake/help/v3.1/module/ExternalProject.html and
https://cmake.org/cmake/help/v3.0/module/ExternalProject.html. the
former has DOWNLOAD_NO_PROGRESS, while the latter does not.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-20 06:40:15 +08:00
Jason Dillaman
bc80c8c7f9 qa/workunits/rbd: remove sanity check in journal.sh test
When the OSDs are being concurrently thrashed, this can result in
sporadic failures due to the admin socket disappearing.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-19 15:21:46 -05:00
Jason Dillaman
1374acdf1a qa/workunits/rbd: correct permission test handling for clone v2 images
Fixes: http://tracker.ceph.com/issues/23043
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-19 15:16:40 -05:00
Jason Dillaman
7b9161eb05 qa/workunits/rbd: utilize RBD profiles in permissions.sh test case
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-19 14:53:27 -05:00
Igor Fedotov
f798b13a04 os/bluestore: do not account DB volume space in total one reported by statfs method
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-02-19 13:11:44 -06:00
Sage Weil
8709cbc2ec os/bluestore: prevent statfs available from going negative
Fixes: https://tracker.ceph.com/issues/23040
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-19 13:10:36 -06:00
Ilya Dryomov
8fa9dec271 qa: krbd_fallocate.sh: add -ENOENT handling test case
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-02-19 20:02:57 +01:00
Greg Farnum
57ecc1ef1c mon: mark manager beacons as no_reply
We never reply to manager beacons, and we have to mark them
that way or else forwarded messages pile up pending replies
and things eventually block.

Fixes: http://tracker.ceph.com/issues/22114
Reported-by: Hongpeng Lu <ludehp@163.com>

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2018-02-19 09:32:13 -08:00
Boris Ranto
aae7a21138 mgr/prometheus: Expose OSD Flags
Signed-off-by: Boris Ranto <branto@redhat.com>
2018-02-19 17:30:46 +01:00
Boris Ranto
6cefd4832f mgr/prometheus: Fix pg_* counts
Currently, the pg_* counts are not computed properly. We split the
current state by '+' sign but do not add the pg count to the already
found pg count. Instead, we overwrite any existing pg count with the new
count. This patch fixes it by adding all the pg counts together for all
the states.

It also introduces a new pg_total metric for pg_total that shows the
total count of PGs.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-02-19 17:30:46 +01:00
Boris Ranto
965aaad527 mgr/prometheus: Skip bogus entries
The osd data can contain bogus '-' entries, skip these when populating
osd metadata and disk occupation.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-02-19 17:30:46 +01:00
Jason Dillaman
c42c54d0ec test/librbd: utilize unique pool for cache tier testing
Fixes: http://tracker.ceph.com/issues/11502
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-19 10:25:15 -05:00
Sage Weil
93fd56ed03 osd/PrimaryLogPG: cancel all objecter ops atomically
We want to avoid a situation like this:

- start proxy op A (epoch E)
- start proxy op B (epoch E)
- start proxy op C (epoch E)
- objecter sends none of these because target is down in epoch E
- osdmap update to E+1
- pg cancels requeues A, B
- objecter updates to E+1
- objecter sends C
- pg cancels/requeues C

Note that the key thing is that operations on each object are canceled
atomically.  On the interval change we do it all at once.  In the other
cases, we cancel everything on the given object together.

Fixes: http://tracker.ceph.com/issues/22123
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-19 08:54:58 -06:00
Sage Weil
7459d4a479 osdc/Objecter: allow multiple ops to be canceled atomically
This allows multiple cancellations without worrying about a race with an
(objecter) osdmap update.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-19 08:52:51 -06:00
Jason Dillaman
11c1a0784c rbd: don't overwrite the error code from the remove action
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-19 09:35:31 -05:00
Mykola Golub
01a2719549 librbd: fix structure size check in rbd_mirror_image_get_info/status
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-02-19 12:31:10 +02:00
Igor Fedotov
d87cb07c8e os/bluestore: avoid frequent allocator dump on bluefs rebalance failure
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-02-19 13:27:07 +03:00
Kefu Chai
1b3d19a46b
Merge pull request #20031 from tchaikov/wip-doc-macos
doc/dev/macos: add doc for building on MacOS
2018-02-19 15:21:50 +08:00
Yan, Zheng
9d271696b7 mds: fix check of underwater dentries
Underwater dentry is dentry that is dirty in our cache from journal
replay, but had already been flushed to disk before the mds failed.
To decide if an dentry is underwater, original code compares dirty
dentry's version to on-disk dirfrag's version. This method is racy
because CDir::log_mark_dirty() can increase dirfrag's version without
adding log event. After mds failover, version of dirfrag from journal
replay can be less than on-disk dirfrag's version. So newly dirtied
dentry can be equal to or less than the on-disk dirfrag's version.

The race can cause incorrect fragstat/rstat

Fixes: http://tracker.ceph.com/issues/23032
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2018-02-19 11:28:24 +08:00
Kefu Chai
939db758e4 common/str_list: s/boost::string_view/std::string_view
since string_view is include by standard library, we can now switch from
boost::string_view to std::string_view.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-19 10:55:08 +08:00
Kefu Chai
3b10d16235
Merge pull request #20474 from wjwithagen/wjw-wip-pthread_set_name_2
include: compat.h, fix the return result of pthread_set_name()

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-02-19 10:45:31 +08:00
Patrick Donnelly
039165276e
Merge PR #20168 into master
* refs/pull/20168/head:
	Handle dir->dentries.empty()
	client: lookup_ino when i1 becomes NULL

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-02-18 10:36:34 -08:00
Patrick Donnelly
14da9d77b1
Merge PR #20424 into master
* refs/pull/20424/head:
	qa: ignore io pause warnings in mds-full test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-02-18 10:36:30 -08:00
Patrick Donnelly
81d0a52a2a
Merge PR #20428 into master
* refs/pull/20428/head:
	qa: ignore MON_DOWN while thrashing mons

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-02-18 10:36:25 -08:00
Patrick Donnelly
852b8491e8
Merge PR #20470 into master
* refs/pull/20470/head:
	qa: ignore full cluster warnings for mds-full

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-02-18 10:36:21 -08:00
Willem Jan Withagen
23074ce4a7 include: compat.h, fix the return result of pthread_set_name()
On FreeBSD the function type is void(). Make it compatible with return
value on other platforms.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-02-18 14:45:56 +01:00
Marcus Watts
1fc69243bd Allow swift acls to be deleted.
The openstack "swift" command deletes acls by submitting an
acl with an empty string.  The existing logic uses c++ strings,
which can't distinguish between an empty string and a non-existant
string.  Additional, the strings are coming from RGWEnv which
supplies C strings not c++.  Using C strings instead makes it
trivial to pass "non-existance" (as a null string).  It also avoids
some type conversion with string copying.

Fixes: http://tracker.ceph.com/issues/22897

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2018-02-18 01:10:23 -05:00
Patrick Donnelly
27b6bfbd81
qa: ignore full cluster warnings for mds-full
Fixes: http://tracker.ceph.com/issues/23033

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-02-17 16:09:57 -08:00
Sage Weil
448b6967db qa/suites/rados/objectstore: increase open file limit
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-17 10:17:57 -06:00
Sage Weil
d034945ba2 os: do not inject read EIO on meta pool objects
Avoid EIO on, say, osdmaps until we fix
http://tracker.ceph.com/issues/23029

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-17 10:17:39 -06:00
huanwen ren
3230864fb1 mds/CDir: cleanup in the lookup_exact_snap()
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2018-02-17 21:22:49 +08:00
dongdong tao
63efbd9280 cephfs-journal-tool: support purge queue
Add option "--journal" to identify the journal type.
this patch can fully support mode "journal" and "header" for purge queue.
As mode "event" is only specific for "mdlog", "journal" and "header" mode only contain
pure "Journaler" operation.

Signed-off-by: dongdong tao <tdd21151186@gmail.com>
2018-02-17 15:00:57 +08:00
Kefu Chai
483f14cd80
Merge pull request #20274 from adamemerson/wip-socket-to-me!
common/admin_socket: various cleanups

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-02-17 10:05:08 +08:00
Sage Weil
08324f7118 test_filestore_idempotent_sequence: put txn object in each collection
The transactions are idependent in each collection/sequencer, so we
can't record to a single txn object with racing transactions.  Fix
it by doing one in each collection, and when reading the latest op,
use the highest txn value we see.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-16 16:50:04 -06:00
Adam C. Emerson
4914607e7a common/admin_socket: Use unique_ptr instead of new/delete
Mostly for the default hooks, plus one Formatter.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-16 14:31:35 -05:00
Adam C. Emerson
5f7a0f5f74 common/admin_socket: Cleanup path cleanup
Don't use strdup/free explicitly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-16 14:31:35 -05:00
Adam C. Emerson
1328166ddd config/admin_socket: Remove PFL_* stuff since nobody's using it
If we want to, we have other ways of getting data out of a thread.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-16 14:31:35 -05:00
Adam C. Emerson
d26916b476 common/admin_socket: Use one map instead of three
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-16 14:31:31 -05:00
Sage Weil
03e1fae32c ceph_test_objectstore: fix split ordering
We need to flush between split.  This requirement unfortunately
doesn't quite go away with the FileStore tracking.

Also, flush for each batch.  This is just because the test environment
may have a low open file ulimit.  (The old code did apply_transaction,
so it's functionally equivalent to this.)

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-16 12:37:30 -06:00
Adam C. Emerson
c47d3dea85 common/admin_socket: Use std::mutex/condition_variable/thread
And fix a bug where we exit in one code-path without releasing the
lock.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-02-16 13:37:12 -05:00