Commit Graph

91619 Commits

Author SHA1 Message Date
Patrick Donnelly
60c63f71f6
Merge PR #24533 into master
* refs/pull/24533/head:
	qa: add timeouts for remote ops for client mounts

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-10-23 14:46:34 -07:00
Lenz Grimmer
f24b46ee4d
Merge pull request #24679 from tspmelo/wip-36528
mgr/dashboard: Fix HttpClient Module imports in unit tests

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 23:23:18 +02:00
Lenz Grimmer
549e00286e
Merge pull request #24632 from Devp00l/wip-issue-36468
mgr/dashboard: Handle class objects as regular objects in KV-table

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 23:22:05 +02:00
Josh Durgin
b148a231b4 test/librados/aio: remove unused callbacks
That callbacks are called is covered by many other tests.
These particular m_safe and m_complete fields were unused.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2018-10-23 16:19:51 -04:00
Josh Durgin
75a776c0eb test/librados/aio: wait for all completions properly
Using global semaphores was error-prone and unnecessary. Unbalanced
usage resulted in a race like:

aio_write object foo -> dropped on full OSD
aio_read object foo -> -ENOENT

The intent of the test was to wait for the write to complete, so
do that explicitly for each aio test and remove the semaphores.

Fixes: http://tracker.ceph.com/issues/24587
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2018-10-23 16:08:54 -04:00
Alfredo Deza
7ba945534f
Merge pull request #24674 from sileht/master
ceph-volume: extract flake8 config

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-10-23 15:04:54 -05:00
Zack Cerza
57040391f5 mgr/dashboard: Redirect /block to /block/rbd
Fixes: https://tracker.ceph.com/issues/36401

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-10-23 13:27:40 -06:00
Matt Benjamin
16dcd2ee6f
Merge pull request #23242 from linuxbox2/wip-rgw-inflight-ops
rgw: set default objecter_inflight_ops = 24576
2018-10-23 11:21:10 -04:00
Willem Jan Withagen
d7faa0de0b common: Fix function const, when not all serial() are const.
And then Clang complains:
```
/home/jenkins/workspace/ceph-master/src/common/blkdev.cc:700:13: error: out-of-line definition of 'serial' does not match any declaration in 'Bl
kDev'
int BlkDev::serial(char *serial, size_t max)
            ^~~~~~
/home/jenkins/workspace/ceph-master/src/common/blkdev.h:50:7: note: member declaration does not match because it is const qualified
  int serial(char *serial, size_t max) const;
      ^                                ~~~~~
1 error generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-10-23 15:32:56 +02:00
Sage Weil
39304caf95 common: remove unused ioprio thread/wq infrastructure
This is all now obsolete, with no remaining users.

Move ceph_gettid into Thread.{cc,h}

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-23 08:29:02 -05:00
xie xingguo
5bcac35213 mon/OSDMonitor: do not remove device class still referenced by ec-profiles
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 21:17:56 +08:00
xie xingguo
4bc54587a1 mon/OSDMonitor: make "ceph osd crush class rm" idempotent
Removing a non-existent device class should be generally okay.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 21:17:56 +08:00
Kefu Chai
86af98f42d
Merge pull request #24692 from tchaikov/wip-cmake-with-asan
cmake: fixes to enable WITH_ASAN with clang and GCC

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-23 19:26:59 +08:00
Lenz Grimmer
966a1e9629
Merge pull request #24664 from tspmelo/wip-36513
mgr/dashboard: Fix missing failed restore notification

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 12:59:06 +02:00
Lenz Grimmer
80dd99baa3
Merge pull request #24637 from ricardoasmarques/fix-rbd-actions-disable
mgr/dashboard: Fix RBD actions disable

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 12:35:53 +02:00
Lenz Grimmer
709e65e395
Merge pull request #24612 from Tinatinak/fix_bug24548
mgr/dashboard: Fixed documentation link on RGW page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 12:28:57 +02:00
Tiago Melo
fba0097a8c mgr/dashboard: Fix HttpClient Module imports in unit tests
Some of the tests were importing the wrong HttpClient Module and that was
causing random errors.

Fixes: https://tracker.ceph.com/issues/36528

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-10-23 10:41:51 +01:00
Tiago Melo
2f4ce95725 mgr/dashboard: Fix missing failed restore notification
Fixes: http://tracker.ceph.com/issues/36513

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-10-23 10:34:03 +01:00
Yan Jun
1e98c72dfc mon: drop repeated 'goodchars' and add osd crush ls testcase
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2018-10-23 16:32:45 +08:00
xie xingguo
6289e72116 mgr/DaemonServer: "osd safe-to-destroy" - more verbose output
For multiple inputs, we want to avoid a situation like:

```
$ ceph osd safe-to-destroy 0 1 2
Error EBUSY: OSD(s) 1,2 have 96 pgs currently mapped to them
$ ceph osd safe-to-destroy 0
Error EBUSY: OSD(s) 0 last reported they still store some PG data, and not all PGs are active+clean; we cannot be sure they aren't still needed.
```

The first command seems to be implying osd.0 is safe-to-destroy whereas it isn't.
This patch instead gives a complete report of each osd if multiple osds are specified.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 15:22:44 +08:00
Lenz Grimmer
5727f9b455
Merge pull request #24544 from votdev/bug_24729
mgr/dashboard: Confirmation modal doesn't close

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 08:50:14 +02:00
Lenz Grimmer
087958f329
Merge pull request #24699 from rhcs-dashboard/add-tslint-rule
mgr/dashboard: Add TSLint rule "no-unused-variable"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 08:48:54 +02:00
Lenz Grimmer
3105e967f5
Merge pull request #24694 from tspmelo/wip-36544
mgr/dashboard: Fix problem with ErasureCodeProfileService

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 08:47:55 +02:00
Mykola Golub
5dd0599bdf
Merge pull request #24696 from dillaman/wip-36542
qa/tasks/qemu: use unique clone directory to avoid race with workunit

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-10-23 09:41:43 +03:00
Nathan Cutler
7349218acb build/ops: drop sgdisk build/runtime dependencies
sgdisk was a dependency of ceph-disk, which has been ripped out.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-10-23 07:56:13 +02:00
Sage Weil
a350131d16 Merge PR #24698 into master
* refs/pull/24698/head:
	Merge PR #24697 into nautilus
	ceph_test_msgr: fix authorizer behavior
	Merge pull request #24667 from liewegas/wip-ec-thrash-full
	Merge PR #24689 into nautilus
	qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 22:46:23 -05:00
Kefu Chai
b0c8102cfb
Merge pull request #24691 from iridescent-rsy/fix_typo
common: fix typos in BackoffThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-23 10:56:13 +08:00
Sage Weil
a0b5445bc8 Merge PR #24697 into nautilus
* refs/pull/24697/head:
	ceph_test_msgr: fix authorizer behavior

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2018-10-22 20:45:40 -05:00
Brad Hubbard
a20ba3dcf7
Merge pull request #24652 from badone/wip-undefined-gperftool-symbols-2
rpm: Use updated gperftools-libs at runtime

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2018-10-23 09:37:35 +10:00
Jason Dillaman
b765a86345 include/types: fixed compile warning for signed/unsigned comparison
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
d0a7423f1a osd/PrimaryLogPG: uncommitted dup ops should respond with logged return code
Fixes: http://tracker.ceph.com/issues/36408
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
bd6b20cd26 osd/PrimaryLogPG: propagate error return codes on object copy_get ops
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
3b66716a4b osd/PGLog: optionally record error return codes for extra_reqids
When a cache tier promotes an object with one or more error PG log
entries, these errors need to be propagated and recorded for dup
op detection.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
a4e40331d5 osd/osd_types: include PG log return codes in object copy data
If the base tier records an error against an operation, the cache
tier currently might incorrectly respond with a success return code.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Sage Weil
020bd7b5f3 os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put
A                             B
SharedBlobSet::lookup()
  takes lock
  nref is not 0
                              SharedBlob::put()
                                --nref
returns SharedBlobRef,
  ++nref
                                takes cache lock
                                SharedBlobSet::remove
                                  takes lock
                                  removes
                                deletes SharedBlob

-> A ends up with a ref to deleted SharedBlob

Fix by verifying that nref is still zero in SharedBlobSet::remove(),
while we are holding the SharedBlobSet::lock.  The lock ensures that we
have increased the ref for the lookup before entering remove, so we can
verify that nref is still zero before removing it.  If not, we have
raced, and put() bails out and does nothing.

Fixes: http://tracker.ceph.com/issues/36526
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-22 14:45:37 -05:00
Casey Bodley
e515d92067 rgw: aws:SecureTransport uses rgw_transport_is_secure()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Casey Bodley
8f9464310b rgw: SSE operations check rgw_transport_is_secure()
Fixes: http://tracker.ceph.com/issues/27221

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Casey Bodley
73d355fc51 rgw: add helper function rgw_transport_is_secure()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Casey Bodley
d3dd5a996d common: add config option rgw_trust_forwarded_https
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Ilya Dryomov
cefbc82e36 test/log: drop redundant test case
After the recent logging rework, ManyGatherLog and
ManyGatherLogStringAssign are identical barring the string.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-10-22 20:09:24 +02:00
Ilya Dryomov
5f25c0ab30 common/StackStringStream: don't reserve before every insert
Unlike ConcreteEntry, MutableEntry can be appended to.  Reserving the
exact number of elements before every append is harmful: vector will
likely reallocate each time and grow linearly instead of geometrically.
This results in quadratic behaviour when we spill past the preallocated
capacity and doesn't benefit the fast path in any way.

The new test case takes half a second with this patch and many hours
spinning in memmove without this patch.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-10-22 20:09:10 +02:00
alfonsomthd
85570639cb mgr/dashboard: Add TSLint rule
- TSlint no-unused-variable rule added.
- Cleanup: unused imports and variables.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-10-22 19:38:38 +02:00
Lenz Grimmer
9d79acdaac
Merge pull request #24591 from tspmelo/wip-testing-module
mgr/dashboard: Unit Tests cleanup

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-22 19:33:53 +02:00
Sage Weil
1a01cf4872 ceph_test_msgr: fix authorizer behavior
Fixes breakage from this PR 2152d8ffb7.

Fixes: http://tracker.ceph.com/issues/36495
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-22 10:06:06 -05:00
Kefu Chai
3d0d24a228 include/ceph_assert.h: do not pack assert params if WITH_ASAN
we pack the asset() params for smaller code size, but this creates a
inlined `assert_data_ctx` instance for every compilation unit which
call ceph_assert() defined in .h .

__PRETTY_FUNCTION__ is likely to be referenced by `assert_data_ctx`
sections which are included by different compiled object files. if the
ceph_assert() call is used by header file, then there will be multiple
`assert_data_ctx` sections sharing the same identifier. these sections are
defined as "COMDAT" group sections, i.e. common data sections. when linker
see multiple COMDAT sections with the same identifer, it will simply discard
the duplicated ones, and only keep a single copy of them. without enabling
ASan, GCC can always handle this problem just fine. but the dedup feature
does not work well with ASan. if ASan is enabled, and we link the objects
with the wrong order, some references will be pointing to the discarded
sections.

to address this issue, we could audit the link command line and inspect
all .o files to make sure they are properly ordered. but this is
non-trivial. as a workaround, in this change, the assert params are not
packed, and sent to the  __ceph_assert_fail() overrides which accepts
unpacked params directly, so the COMDAT section is not created.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 23:01:36 +08:00
Jason Dillaman
5d56014c61 qa/tasks/qemu: use unique clone directory to avoid race with workunit
If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 10:44:40 -04:00
Josh Durgin
36ca230776
Merge pull request #24667 from liewegas/wip-ec-thrash-full
qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 07:39:26 -07:00
Kefu Chai
4af71e7c00
Merge pull request #23103 from ifed01/wip-ifed-bluefs-migrate
os/bluestore: allow ceph-bluestore-tool to coalesce, add and migrate BlueFS backing volumes

Reviewed-by:  Sage Weil <sage@redhat.com>
2018-10-22 22:33:08 +08:00
Kefu Chai
09c31bbea3
Merge pull request #23090 from mingshuaiwang/master
OSD: ceph-osd parent process need to restart log service after fork

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-10-22 22:30:41 +08:00
Sage Weil
883fc4d122 Merge PR #24689 into nautilus
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:20:50 -05:00