Commit Graph

90856 Commits

Author SHA1 Message Date
Kefu Chai
83fbd116e2 auth: switch from Mutex to ceph::mutex
to be aligned with our direction to use std::mutex or its wrapper. see
https://trello.com/c/aUSezBRH/365-cephmutex-etc-for-release-builds

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-27 21:34:05 +08:00
Kefu Chai
272afb9bbe auth,common: switch to ceph::mutex, etc
in this change, along with LockPolicy, src/common/lock_* are completely
removed.

instead of using LockPolicy based template specialization of LockMutex,
etc, it would be simpler if we can just rely on WITH_SEASTAR
preprocessor macro to tell if we are compiling code for crimson or not.
but please bear in mind, we cannot link against the plain libceph-common
in crimson anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-27 21:34:05 +08:00
Kefu Chai
c45113ba3c common/ceph_mutex: add shared_mutex to ceph namespace
for the sake of completeness, also pave the road to ditch
SharedMutex<LockPolicy>. please note, in code used by crimson, we are
not using shared_mutex or SharedMutex yet. so we are not adding
the dummy shared_mutex for WITH_SEASTAR.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-27 21:34:05 +08:00
Kefu Chai
c316f82180 msg/async: keep connection alive only actually sending
When connection stuck into odd state, we need to let connection timeout.
If send_message could update last_active, it won't play the role as
expected.
So we move last_active updated to the place actually should.

Signed-off-by: Haomai Wang <haomai@xsky.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-27 20:40:30 +08:00
Mykola Golub
14f07dbd39 test/rbd_mirror: race in WaitingOnLeaderReleaseLeader
Fixes: http://tracker.ceph.com/issues/36236
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-09-27 14:32:30 +03:00
Sage Weil
bc86b06a67 qa/suites/rados/thrash-erasure-code: add clay 4+2
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-27 18:56:21 +08:00
Myna V
2a3e56b22c erasure-code: add clay codes
Introducing Clay codes to ceph. These codes are optimal
in terms of network, disk traffic utilized during repair of a lost chunk.
This feature also provides gradual increase in network, disk traffic with
respect to the number of lost chunks.

Authors: Myna, Elita.
Fixes: http://tracker.ceph.com/issues/19278
Signed-off-by: Myna V <mynaramana@gmail.com>
2018-09-27 18:56:21 +08:00
Lenz Grimmer
15165f4de2
Merge pull request #24193 from rhcs-dashboard/coverage-disabled-in-api-tests
mgr/dashboard: disable backend tests coverage

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-09-27 12:24:01 +02:00
Kefu Chai
09cc126704
Merge pull request #24250 from jecluis/wip-mon-increase-ck-default
mon/config-key: increase max key entry size

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-09-27 18:14:17 +08:00
Lenz Grimmer
59772fe91a
Merge pull request #24200 from jecluis/wip-dashboard-fix-ac
mgr/dashboard: no side-effects on failed user creation

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-09-27 11:19:37 +02:00
Lenz Grimmer
75c1d8eacd
Merge pull request #23351 from tspmelo/wip-rbd-trash
mgr/dashboard: Add support for RBD Trash

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-09-27 11:04:46 +02:00
Mykola Golub
925273d270
Merge pull request #24293 from dillaman/wip-36220
librbd: object copy state machine might dereference a deleted object

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-27 09:02:00 +03:00
Kefu Chai
fb1a75c6c1
Merge pull request #22659 from theanalyst/cmake-pybind-ssl
build: mgr: check for python's ssl version linkage

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-09-27 10:55:16 +08:00
Nathan Cutler
0e4c765a00
Merge pull request #24243 from smithfarm/wip-bci-rate-limit
tools: backport-create-issue: rate-limit to avoid seeming like a spammer

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-09-27 00:18:49 +02:00
Nathan Cutler
a38af571c2
Merge pull request #23131 from smithfarm/wip-15100
qa: add test that builds example librados programs

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
2018-09-27 00:17:48 +02:00
Ali Maredia
a296b56bcc rgw: no DoutPrefixProvider as arg for RGWRemoteDataLog::init
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2018-09-26 16:36:56 -04:00
Ali Maredia
9619e05288 rgw: pass DoutPrefixProvider into the entrypoints for trim
Also add a DoutPrefixProvider to all of the
cascading classes that need one as a result

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2018-09-26 16:36:56 -04:00
Casey Bodley
0e4ba992b7 rgw: add global prefix provider for radosgw-admin
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9a5dbc81b774d0be4b51f39e37bcfa15f3b36db8)
2018-09-26 16:36:55 -04:00
Ali Maredia
fc178b2b5d rgw: remove unused variable RGWRados *store
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2018-09-26 16:36:55 -04:00
Ali Maredia
92eb106a75 rgw: replace ldout with ldpp_dout in coroutine classes
The only classes in rgw_sync.cc that remain
with ldouts are:
- RGWBackoffControlCR
- RGWShardCollectCR
- RGWAsyncMetaStoreEntry
- RGWAsyncMetaRemoveEntry
- PurgePeriodLogsCR
- take_min_status
- MetaMasterTrimShardCollectCR
- MetaMasterStatusCollectCR
- MetaMasterTrimCR
- MetaPeerTrimCR
- MetaTrimPollCR

These classes do not have RGWMetaSyncEnv members.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2018-09-26 16:36:55 -04:00
Ali Maredia
6eb18c2a6f rgw: replace ldouts with ldpp_dout in RGWRemoteMetaLog
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2018-09-26 16:36:55 -04:00
Ali Maredia
ba6b01cd1b rgw: MetaSyncStatusManager implements DoutPrefixProvider
There are no member functions of RGWMetaSyncStatusManager
with ldouts to replace with ldpp_dout.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2018-09-26 16:36:55 -04:00
Jason Dillaman
1a9c3be2ce librbd: object copy state machine might dereference a deleted object
Fixes: http://tracker.ceph.com/issues/36220
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-26 15:58:23 -04:00
Mykola Golub
78881934c0
Merge pull request #22043 from dillaman/wip-24139
rbd-mirror: bootstrap needs to handle local image id collision

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-26 22:49:02 +03:00
Mykola Golub
9ff70326da
Merge pull request #24253 from dillaman/wip-24516
librbd: potential object map race with copyup state machine

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-26 22:48:29 +03:00
Yuri Weinstein
3c89ae73c8
Merge pull request #23625 from XiangxiangMeng/fix-rgw-bucket-policy-not
rgw: policy: fix NotAction, NotPricipal, NotResource does not take effect

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2018-09-26 09:02:36 -07:00
Yuri Weinstein
0fd7413b79
Merge pull request #24062 from nrdmn/fix-quota
rgw: fix max-size in radosgw-admin and REST Admin API

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-09-26 09:00:26 -07:00
Yuri Weinstein
a303596bb9
Merge pull request #24177 from cbodley/wip-24595
rgw: apply quota config to users created via external auth

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2018-09-26 08:59:51 -07:00
Kefu Chai
aa56412be5
Merge pull request #24199 from aclamk/fio-rados-manual
test/fio: Added tips for compilation of fio with 'rados' engine.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-09-26 21:57:18 +08:00
Lenz Grimmer
2be394ab6f
Merge pull request #24273 from tspmelo/wip-html-linter
mgr/dashboard: Add html-linter

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-09-26 15:45:52 +02:00
Casey Bodley
8ff2ecaa83
Merge pull request #23534 from cbodley/wip-26897
rgw multisite: enforce spawn_window for data full sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-26 09:28:30 -04:00
Jason Dillaman
1e874403bf librbd: keep IO blocked until after snapshot object map created
The IO was being unblocked before object map was created, allowing
a potential copyup request to fail to update a still-to-be-created
object map.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-26 08:34:33 -04:00
Jason Dillaman
5a1cb46987 librbd: do not invalidate object map if update races with copyup
The copyup state machine needs to iterate over all object maps to update
the existence for the object. If an snapshot is being removed concurrently,
it's possible to invalidate the object map for the image.

Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-26 08:34:32 -04:00
Tiago Melo
00ab84abaf mgr/dashboard: Update frontend linting scripts and documentation
Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-09-26 12:15:43 +01:00
Tiago Melo
8256e82ece mgr/dashboard: Add html linter and fix lint errors
Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-09-26 12:01:24 +01:00
Yan, Zheng
f39e9aa573 msg: clear message middle when clearing encoded message buffer
Otherwise it may trigger ceph_assert(middle.length() == 0) in
Message::encode()

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-09-26 18:58:16 +08:00
Kefu Chai
78116ce147 bluestore/NVMe: use PCIe selector as the path name
as the latest SPDK deprecated and removed spdk_pci_get_device(),
we cannot use the PCI device's serial number as its identifier and the
filename of the file representing the device. in this change, we
are using the PCI device's selector instead, it is also used by SPDK as
part of the transport id.

Tested-by: Tone Zhang <tone.zhang@arm.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-26 17:47:38 +08:00
Willem Jan Withagen
0654099d4b common: fix missing include boost/noncopyable.hpp
Clang complains:
```
In file included from /home/jenkins/workspace/ceph-master/src/osd/ReplicatedBackend.cc:15:
In file included from /home/jenkins/workspace/ceph-master/src/osd/ReplicatedBackend.h:18:
In file included from /home/jenkins/workspace/ceph-master/src/osd/PGBackend.h:27:
In file included from /home/jenkins/workspace/ceph-master/src/osd/PGTransaction.h:25:
/home/jenkins/workspace/ceph-master/src/common/inline_variant.h:88:64: error: expected class name
struct generic_visitor : boost::static_visitor<Result>, boost::noncopyable
                                                               ^
1 error generated.
gmake[2]: *** [src/osd/CMakeFiles/osd.dir/build.make:141: src/osd/CMakeFiles/osd.dir/ReplicatedBackend.cc.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-09-26 10:30:57 +02:00
Patrick Donnelly
9a01c3ae10
Merge PR #24203 into master
* refs/pull/24203/head:
	ceph-fuse: add to selinux profile

Reviewed-by: Boris Ranto <branto@redhat.com>
2018-09-25 16:01:36 -07:00
Yehuda Sadeh
84fa9d4ae0
Merge pull request #24221 from cbodley/wip-36034
rgw multisite: uses local DataChangesLog to track active buckets for trim

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-25 15:02:01 -07:00
Yehuda Sadeh
008316a127
Merge pull request #23988 from cbodley/wip-35851
rgw: dont access rgw_http_req_data::client of canceled request

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-25 13:43:17 -07:00
Casey Bodley
99279f7246
Merge pull request #23986 from cbodley/wip-35715
rgw: fix leak of curl handle on shutdown

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-25 16:40:52 -04:00
Casey Bodley
8622d8a241
Merge pull request #24011 from cbodley/wip-qa-rgw-multisite-speedup
qa/rgw: reduce number of multisite log shards

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-25 16:39:51 -04:00
Yehuda Sadeh
d32d0d7f45
Merge pull request #23987 from cbodley/wip-35812
rgw: RGWAsyncGetBucketInstanceInfo does not access coroutine memory

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-09-25 13:20:59 -07:00
Jason Dillaman
5c317aef31 qa/tasks/rbd_mirror_thrash: let daemon gracefully shut down if possible
Otherwise, try to capture a core dump to discover what was blocking the
shutdown process.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-25 16:08:59 -04:00
Jason Dillaman
dca9e3e252 qa/workunits/rbd: wait max 2 hrs for all stress images to sync
Sporadically the rbd-mirror fsx stress test would fail due to very
slow sync times due to overloaded clusters. Attempt to wait for all
images to be replicated before proceeding with the comparison.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-25 16:08:59 -04:00
Jason Dillaman
d04a7679c0 qa/workunits/rbd: exclude rbd-mirror sync-point snaps from comparison
This is a temporary workaround to tracker ticket issue #36185

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-25 16:08:25 -04:00
Alfredo Deza
ead3c7e7d1 ceph-volume tests.util verify new blkid API calls
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-09-25 15:27:47 -04:00
Alfredo Deza
3a000e4c2b
Merge pull request #24274 from ceph/wip-batch-report-fix
ceph-volume: fix journal and filestore data size in `lvm batch --report`
2018-09-25 15:26:04 -04:00
Alfredo Deza
dbde6a5b61 ceph-volume util.disk expand blkid's API calls
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-09-25 15:25:13 -04:00