Commit Graph

91670 Commits

Author SHA1 Message Date
Timofey Titovets
e298e756a0 os/bluestore/KernelDevice: Honor discard_granularity
Kernel ABI:
A discard_granularity of 0 means
that the device does not support discard functionality.

Also discard_granularity is RO attribute, while rotational are RW.
That means what rotational can be overridden in user space if needed.

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2018-10-23 17:35:24 +03:00
Willem Jan Withagen
d7faa0de0b common: Fix function const, when not all serial() are const.
And then Clang complains:
```
/home/jenkins/workspace/ceph-master/src/common/blkdev.cc:700:13: error: out-of-line definition of 'serial' does not match any declaration in 'Bl
kDev'
int BlkDev::serial(char *serial, size_t max)
            ^~~~~~
/home/jenkins/workspace/ceph-master/src/common/blkdev.h:50:7: note: member declaration does not match because it is const qualified
  int serial(char *serial, size_t max) const;
      ^                                ~~~~~
1 error generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-10-23 15:32:56 +02:00
Sage Weil
39304caf95 common: remove unused ioprio thread/wq infrastructure
This is all now obsolete, with no remaining users.

Move ceph_gettid into Thread.{cc,h}

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-23 08:29:02 -05:00
xie xingguo
5bcac35213 mon/OSDMonitor: do not remove device class still referenced by ec-profiles
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 21:17:56 +08:00
xie xingguo
4bc54587a1 mon/OSDMonitor: make "ceph osd crush class rm" idempotent
Removing a non-existent device class should be generally okay.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 21:17:56 +08:00
Kefu Chai
86af98f42d
Merge pull request #24692 from tchaikov/wip-cmake-with-asan
cmake: fixes to enable WITH_ASAN with clang and GCC

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-23 19:26:59 +08:00
Lenz Grimmer
966a1e9629
Merge pull request #24664 from tspmelo/wip-36513
mgr/dashboard: Fix missing failed restore notification

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 12:59:06 +02:00
Lenz Grimmer
80dd99baa3
Merge pull request #24637 from ricardoasmarques/fix-rbd-actions-disable
mgr/dashboard: Fix RBD actions disable

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 12:35:53 +02:00
Lenz Grimmer
709e65e395
Merge pull request #24612 from Tinatinak/fix_bug24548
mgr/dashboard: Fixed documentation link on RGW page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 12:28:57 +02:00
Tiago Melo
fba0097a8c mgr/dashboard: Fix HttpClient Module imports in unit tests
Some of the tests were importing the wrong HttpClient Module and that was
causing random errors.

Fixes: https://tracker.ceph.com/issues/36528

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-10-23 10:41:51 +01:00
Tiago Melo
2f4ce95725 mgr/dashboard: Fix missing failed restore notification
Fixes: http://tracker.ceph.com/issues/36513

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-10-23 10:34:03 +01:00
Yan Jun
1e98c72dfc mon: drop repeated 'goodchars' and add osd crush ls testcase
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2018-10-23 16:32:45 +08:00
xie xingguo
6289e72116 mgr/DaemonServer: "osd safe-to-destroy" - more verbose output
For multiple inputs, we want to avoid a situation like:

```
$ ceph osd safe-to-destroy 0 1 2
Error EBUSY: OSD(s) 1,2 have 96 pgs currently mapped to them
$ ceph osd safe-to-destroy 0
Error EBUSY: OSD(s) 0 last reported they still store some PG data, and not all PGs are active+clean; we cannot be sure they aren't still needed.
```

The first command seems to be implying osd.0 is safe-to-destroy whereas it isn't.
This patch instead gives a complete report of each osd if multiple osds are specified.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 15:22:44 +08:00
Lenz Grimmer
5727f9b455
Merge pull request #24544 from votdev/bug_24729
mgr/dashboard: Confirmation modal doesn't close

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 08:50:14 +02:00
Lenz Grimmer
087958f329
Merge pull request #24699 from rhcs-dashboard/add-tslint-rule
mgr/dashboard: Add TSLint rule "no-unused-variable"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-23 08:48:54 +02:00
Lenz Grimmer
3105e967f5
Merge pull request #24694 from tspmelo/wip-36544
mgr/dashboard: Fix problem with ErasureCodeProfileService

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-23 08:47:55 +02:00
Mykola Golub
5dd0599bdf
Merge pull request #24696 from dillaman/wip-36542
qa/tasks/qemu: use unique clone directory to avoid race with workunit

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-10-23 09:41:43 +03:00
Nathan Cutler
7349218acb build/ops: drop sgdisk build/runtime dependencies
sgdisk was a dependency of ceph-disk, which has been ripped out.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-10-23 07:56:13 +02:00
Sage Weil
a350131d16 Merge PR #24698 into master
* refs/pull/24698/head:
	Merge PR #24697 into nautilus
	ceph_test_msgr: fix authorizer behavior
	Merge pull request #24667 from liewegas/wip-ec-thrash-full
	Merge PR #24689 into nautilus
	qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 22:46:23 -05:00
Kefu Chai
b0c8102cfb
Merge pull request #24691 from iridescent-rsy/fix_typo
common: fix typos in BackoffThrottle

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-23 10:56:13 +08:00
Sage Weil
a0b5445bc8 Merge PR #24697 into nautilus
* refs/pull/24697/head:
	ceph_test_msgr: fix authorizer behavior

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2018-10-22 20:45:40 -05:00
Brad Hubbard
a20ba3dcf7
Merge pull request #24652 from badone/wip-undefined-gperftool-symbols-2
rpm: Use updated gperftools-libs at runtime

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2018-10-23 09:37:35 +10:00
Gregory Farnum
3e22376353
Merge branch 'master' into bug_fix_1 2018-10-22 16:11:48 -07:00
Jason Dillaman
b765a86345 include/types: fixed compile warning for signed/unsigned comparison
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
d0a7423f1a osd/PrimaryLogPG: uncommitted dup ops should respond with logged return code
Fixes: http://tracker.ceph.com/issues/36408
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
bd6b20cd26 osd/PrimaryLogPG: propagate error return codes on object copy_get ops
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
3b66716a4b osd/PGLog: optionally record error return codes for extra_reqids
When a cache tier promotes an object with one or more error PG log
entries, these errors need to be propagated and recorded for dup
op detection.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Jason Dillaman
a4e40331d5 osd/osd_types: include PG log return codes in object copy data
If the base tier records an error against an operation, the cache
tier currently might incorrectly respond with a success return code.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 15:50:45 -04:00
Sage Weil
020bd7b5f3 os/bluestore: fix race between SharedBlobSet::lookup and SharedBlob::put
A                             B
SharedBlobSet::lookup()
  takes lock
  nref is not 0
                              SharedBlob::put()
                                --nref
returns SharedBlobRef,
  ++nref
                                takes cache lock
                                SharedBlobSet::remove
                                  takes lock
                                  removes
                                deletes SharedBlob

-> A ends up with a ref to deleted SharedBlob

Fix by verifying that nref is still zero in SharedBlobSet::remove(),
while we are holding the SharedBlobSet::lock.  The lock ensures that we
have increased the ref for the lookup before entering remove, so we can
verify that nref is still zero before removing it.  If not, we have
raced, and put() bails out and does nothing.

Fixes: http://tracker.ceph.com/issues/36526
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-22 14:45:37 -05:00
Casey Bodley
e515d92067 rgw: aws:SecureTransport uses rgw_transport_is_secure()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Casey Bodley
8f9464310b rgw: SSE operations check rgw_transport_is_secure()
Fixes: http://tracker.ceph.com/issues/27221

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Casey Bodley
73d355fc51 rgw: add helper function rgw_transport_is_secure()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Casey Bodley
d3dd5a996d common: add config option rgw_trust_forwarded_https
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-10-22 14:56:22 -04:00
Ilya Dryomov
cefbc82e36 test/log: drop redundant test case
After the recent logging rework, ManyGatherLog and
ManyGatherLogStringAssign are identical barring the string.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-10-22 20:09:24 +02:00
Ilya Dryomov
5f25c0ab30 common/StackStringStream: don't reserve before every insert
Unlike ConcreteEntry, MutableEntry can be appended to.  Reserving the
exact number of elements before every append is harmful: vector will
likely reallocate each time and grow linearly instead of geometrically.
This results in quadratic behaviour when we spill past the preallocated
capacity and doesn't benefit the fast path in any way.

The new test case takes half a second with this patch and many hours
spinning in memmove without this patch.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-10-22 20:09:10 +02:00
alfonsomthd
85570639cb mgr/dashboard: Add TSLint rule
- TSlint no-unused-variable rule added.
- Cleanup: unused imports and variables.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-10-22 19:38:38 +02:00
Lenz Grimmer
9d79acdaac
Merge pull request #24591 from tspmelo/wip-testing-module
mgr/dashboard: Unit Tests cleanup

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-10-22 19:33:53 +02:00
Sage Weil
1a01cf4872 ceph_test_msgr: fix authorizer behavior
Fixes breakage from this PR 2152d8ffb7.

Fixes: http://tracker.ceph.com/issues/36495
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-22 10:06:06 -05:00
Kefu Chai
3d0d24a228 include/ceph_assert.h: do not pack assert params if WITH_ASAN
we pack the asset() params for smaller code size, but this creates a
inlined `assert_data_ctx` instance for every compilation unit which
call ceph_assert() defined in .h .

__PRETTY_FUNCTION__ is likely to be referenced by `assert_data_ctx`
sections which are included by different compiled object files. if the
ceph_assert() call is used by header file, then there will be multiple
`assert_data_ctx` sections sharing the same identifier. these sections are
defined as "COMDAT" group sections, i.e. common data sections. when linker
see multiple COMDAT sections with the same identifer, it will simply discard
the duplicated ones, and only keep a single copy of them. without enabling
ASan, GCC can always handle this problem just fine. but the dedup feature
does not work well with ASan. if ASan is enabled, and we link the objects
with the wrong order, some references will be pointing to the discarded
sections.

to address this issue, we could audit the link command line and inspect
all .o files to make sure they are properly ordered. but this is
non-trivial. as a workaround, in this change, the assert params are not
packed, and sent to the  __ceph_assert_fail() overrides which accepts
unpacked params directly, so the COMDAT section is not created.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 23:01:36 +08:00
Jason Dillaman
5d56014c61 qa/tasks/qemu: use unique clone directory to avoid race with workunit
If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 10:44:40 -04:00
Josh Durgin
36ca230776
Merge pull request #24667 from liewegas/wip-ec-thrash-full
qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 07:39:26 -07:00
Kefu Chai
4af71e7c00
Merge pull request #23103 from ifed01/wip-ifed-bluefs-migrate
os/bluestore: allow ceph-bluestore-tool to coalesce, add and migrate BlueFS backing volumes

Reviewed-by:  Sage Weil <sage@redhat.com>
2018-10-22 22:33:08 +08:00
Kefu Chai
09c31bbea3
Merge pull request #23090 from mingshuaiwang/master
OSD: ceph-osd parent process need to restart log service after fork

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-10-22 22:30:41 +08:00
Sage Weil
883fc4d122 Merge PR #24689 into nautilus
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:20:50 -05:00
Sage Weil
ae583f5dde Merge PR #24689 into master
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
	Merge PR #24625 into nautilus
	qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:19:46 -05:00
Sage Weil
90c3c2032c os/bluestore: show compress and buffered from WriteContext
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-22 08:29:27 -05:00
Sage Weil
6e14a50e20 os/bluestore: fix rename race with trim on replacement onode at old name
- rename from foo to bar
 - foo onode is moved to bar in onode_map
 - keys removed at position foo as part of txc
 - new onode for foo is installed at foo in map
...
- cache trims foo
...
- new txn B does get_onode on foo, reads old foo (now bar) onode into foo ***
- txn A commits
-> onode cache has foo with stale bar content

Fix by holding a ref to the replacement foo onode so that get_onode cannot
read stale metadata out of kvdb before txn A commits.

Fixes: http://tracker.ceph.com/issues/36541
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-22 08:29:26 -05:00
Ricardo Marques
48eb0a336b mgr/dashboard: Fix RBD actions disable
Fixes: https://tracker.ceph.com/issues/36403

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-10-22 14:02:37 +01:00
Volker Theile
7a44726645 mgr/dashboard: Confirmation modal doesn't close
Fixes: https://tracker.ceph.com/issues/24729

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-10-22 13:29:38 +02:00
Tiago Melo
f775e9844c mgr/dashboard: Fix problem with ErasureCodeProfileService
ErasureCodeProfileService was being provided twice and that was causing
problems in production mode.

Fixes: https://tracker.ceph.com/issues/36544

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-10-22 11:57:01 +01:00