Commit Graph

91486 Commits

Author SHA1 Message Date
Sage Weil
883fc4d122 Merge PR #24689 into nautilus
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:20:50 -05:00
Sage Weil
ae583f5dde Merge PR #24689 into master
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
	Merge PR #24625 into nautilus
	qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:19:46 -05:00
Ricardo Marques
48eb0a336b mgr/dashboard: Fix RBD actions disable
Fixes: https://tracker.ceph.com/issues/36403

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-10-22 14:02:37 +01:00
Volker Theile
7a44726645 mgr/dashboard: Confirmation modal doesn't close
Fixes: https://tracker.ceph.com/issues/24729

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-10-22 13:29:38 +02:00
Tiago Melo
f775e9844c mgr/dashboard: Fix problem with ErasureCodeProfileService
ErasureCodeProfileService was being provided twice and that was causing
problems in production mode.

Fixes: https://tracker.ceph.com/issues/36544

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-10-22 11:57:01 +01:00
Kefu Chai
399923c71a
Merge pull request #20004 from mogeb/steady-clock-tools-rados
librados: use steady clock for rados_mon_op_timeout

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 17:46:44 +08:00
Kefu Chai
298da11351
Merge pull request #24658 from tchaikov/wip-18202-rebased
blkdev: Rework API and add FreeBSD support

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-10-22 17:44:07 +08:00
Kefu Chai
5e12cef930 include/ceph_assert: always use __PRETTY_FUNCTION__ for C++
we've moved to GCC-7, no need to check for ancient compiler versions

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 16:53:05 +08:00
Kefu Chai
fd58e5d4ad cmake,ceph.in: preload libasan if WITH_ASAN
we need to preload libasan.so as the python exectuable is not likely to
be compiled with ASan enabled.
see:
https://github.com/google/sanitizers/wiki/AddressSanitizerAsDso#asan-and-ld_preload

just to ease the use of ASan, for fine-tuned behaviour, use
`ASAN_OPTIONS`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
e851462977 ceph.in: extract get_cmake_variables()
so it can be reused

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
669853e018 cmake: should compile libzstd with -fPIC
otherwise we will have

/usr/bin/ld: libzstd/lib/libzstd.a(error_private.c.o): relocation
R_X86_64_32S against `.rodata' can not be used when making a shared
object; recompile with -fPIC

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
b605210b97 cmake: pass Sanitizers flags to linker for linking .so
see
https://github.com/google/sanitizers/wiki/AddressSanitizer#using-addresssanitizer

to be specific,

> In order to use AddressSanitizer you will need to compile and link your
> program using `clang` with the `-fsanitize=address` switch.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
38e7686a37 cmake: pass cflags to disutils using CC instead of CFLAGS
in python's distutils.ccompiler, linker_exe is composed using CC instead
of LDFLAGS. the latter only effects how it builds (shared) library.

and put CMAKE_C_FLAGS into the cflags for the compiler for building
python C extensions, it's more consistent this way. more importantly,
if we build with ASan enabled, the canary program, a.k.a. rados_dummy.c,
won't link without proper CFLAGS.

without this change, rados.so fails to build with errors like:

/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_stack_free_10'
/usr/bin/ld: /var/ssd/ceph/build/lib/librados.so: undefined reference to
`__asan_report_exp_store8'
...
...

clang: error: linker command failed with exit code 1 (use -v to see
invocation)

Link Error: RADOS library not found
make[3]: ***
[src/pybind/rados/CMakeFiles/cython_rados.dir/build.make:57:
src/pybind/rados/CMakeFiles/cython_rados] Error 1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Kefu Chai
3ac8c8dca7 common/TextTable: define endrow
otherwise "cmake -DWITH_ASAN=ON -DCMAKE_BUILD_TYPE=Debug" will fail to
build with

/usr/bin/ld: //var/ssd/ceph/build/lib/libceph-common.so.0: undefined
reference to `TextTable::endrow'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-22 14:40:03 +08:00
Shiyang Ruan
99ce7cf48b common: fix typos in BackoffThrottle
Signed-off-by: Shiyang Ruan <ruansy.fnst@cn.fujitsu.com>
2018-10-22 13:09:20 +08:00
Yan, Zheng
19d2cecd97 PendingReleaseNotes: note about cephfs client state reclaim
Fixes: http://tracker.ceph.com/issues/36394
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-10-22 11:57:08 +08:00
Sage Weil
b678356594 qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
Fixes 95b7d2340c

Fixes: http://tracker.ceph.com/issues/36485
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-21 10:52:38 -05:00
myoungwon oh
10993596d9 src/test: fix unordered manifest-unset op
- manifest unset op to foo-chunk object
 - remove manifest flag
 - commit
 - send an ack to a client
 - send decrement mesages ("chunk_put") to old chunks (bar-chunk)

Current unit test(ManifestUnset) send "chunk_read" command (to bar-chunk)
in order to see whether chunk's reference count is decreased.
But, as described above, "chunk_read" event can be triggered after a client
(test application) receives an ack. Therefore, there is a corner case
such as bar-chunk (in chunk pool) receives "chunk_read" first instead of "chunk_put"

Reference count model of dedup/tiering is based on false-positive (#24230).
So decreasing reference count is not guaranteed. If reference mismatch occur,
chunk-scrub (this is WIP) will fix it.
One guaranteed thing is that existing manifest flag is removed.

So, the solution of this commit is just re-send unset op, and then
chenk that return value is -EOPNOTSUPP (this means manifest flags is removed).

Fixes: http://tracker.ceph.com/issues/24485
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2018-10-21 18:05:40 +09:00
Sage Weil
98fc7ebc99 Merge PR #24184 into master
* refs/pull/24184/head:
	mgr/DaemonServer: remove any upmaps on merging PGs
	mgr/DaemonServer: prevent merge if either pg is remapped|upmap
	mgr/DaemonServer: move pending merge check for more consistent code
	qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
	mgr/DaemonServer: add option to bypass careful throttling for thrasher
	PendingReleaseNotes: note about mgr/balancer/max_misplaced change
	mgr/DaemonServer: remove stale/misleading check
	mgr/DaemonServer: throttle pgp_num changes based on misplaced %
	mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
	mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
	mon/PGMap: add get_recovery_stats()
	mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
	pybind/mgr/mgr_module: add get_option()
	mgr/DaemonServer: allow pg_num increases that abort pending merges
	mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
	mon/OSDMonitor: make pgp_num track pg_num more consistently

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-20 16:40:22 -05:00
Sage Weil
0db59b6c09 Merge PR #24654 into master
* refs/pull/24654/head:
	osd: remove unused parameter 'dev' in OSD::mkfs function

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-20 15:39:56 -05:00
Sage Weil
ba3679c60b mgr/DaemonServer: remove any upmaps on merging PGs
Remove any pg_upmap[_items] on pgs that are merging to ensure that they
land on the same OSDs.

This is a bit sloppy: we *could* set the source upmap to match the target
upmap (vs potentially moving both PGs to a third location, and/or then
having the balancer move the resulting PG somewhere else again), but for
now assume upmaps are not a common case and Keep It Simple.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
550dcd53bb mgr/DaemonServer: prevent merge if either pg is remapped|upmap
Remapping means they could be on different OSDs.

Fixes: http://tracker.ceph.com/issues/36166
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
5bb9820b2d mgr/DaemonServer: move pending merge check for more consistent code
No functional change, but this makes the code simpler to read.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
86ae8fb6b8 qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
Thrash such that we still exercise the careful throttling in the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
70ec5bda23 mgr/DaemonServer: add option to bypass careful throttling for thrasher
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Josh Durgin
ba3252544c
Merge pull request #20581 from chrone81/patch-1
doc: Fix EC k=3 m=2 profile overhead calculation example.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-19 15:23:55 -07:00
Gregory Farnum
c61acc5b31
Merge pull request #22349 from gregsfortytwo/wip-24368-osd-restarts
systemd: only restart 3 times in 30 minutes, as fast as possible

Reviewed-by:  Sage Weil <sage@redhat.com>
2018-10-19 13:00:07 -07:00
Ilya Dryomov
b7a62742fc msg/async: fix is_queued() semantics
Before AsyncConnection was split into two classes as part of the
multi-protocol refactor, we only had AsyncConnection::is_queued().
It checked both out_q and outcoming_bl because out_q was part of
AsyncConnection.

out_q is now part of ProtocolV1.  AsyncConnection should no longer be
concerned with out_q, only with outcoming_bl.  Checking whether out_q
is empty in _try_send() is particuarly wrong because if the write is
fininished (i.e. outcoming_bl is empty) but out_q has something in it,
the write callback isn't invoked.

Although probably not strictly necessary, this commit preserves the
semantics of connection->is_queued() in Protocol.cc.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-10-19 20:25:12 +02:00
Mykola Golub
cb48963385
Merge pull request #24646 from dillaman/wip-36500
rbd-mirror: always attempt to restart canceled status update task

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-10-19 18:19:56 +03:00
Casey Bodley
61f7c3f7d7
Merge pull request #24670 from falcon78921/wip-doc-multisite-link
doc: updated wording from federated to multisite

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-19 11:06:35 -04:00
Casey Bodley
9312c59174
Merge pull request #24595 from cbodley/wip-24505
rgw: user stats account for resharded buckets

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-10-19 10:00:42 -04:00
Jason Dillaman
d0c6c6be78
Merge pull request #24673 from tchaikov/wip-librbd-test-os-cleanup
librbd,test: remove unused context_cb() function, silence GCC warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-10-19 09:47:54 -04:00
Matt Benjamin
598185a95d
Merge pull request #24645 from cbodley/wip-36496
cls/user: cls_user_remove_bucket writes modified header
2018-10-19 09:39:03 -04:00
Sage Weil
0708e73c26 Merge PR #24611 into master
* refs/pull/24611/head:
	mgr/diskprediction: Solve predictor model file not backward compatibility

Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-19 08:31:46 -05:00
Sage Weil
a9b4ed6c01 Merge PR #24623 into master
* refs/pull/24623/head:
	mgr: Mutex::Locker -> std::lock_guard
	kv: Mutex::Locker -> std::lock_guard
	librados: Mutex::Locker -> std::lock_guard
	mon: Mutex::Locker -> std::lock_guard
	osd: Mutex::Locker -> std::lock_guard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2018-10-19 08:31:29 -05:00
Kefu Chai
ccb0489895
Merge pull request #24605 from tchaikov/wip-use-ccache-the-old-way
cmake: fix compilation with distcc and other compiler wrappers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-19 20:57:04 +08:00
Alan Somers
e31acf6567 blkdev: FreeBSD support
Signed-off-by: Alan Somers <asomers@gmail.com>
2018-10-19 19:37:42 +08:00
Alan Somers
bde6c0de1c blkdev: overhaul API for better portability
* Turn the API into a class so it will work with GoogleMock
* Take file descriptors for all methods, instead of a mix of file
  descriptors, path names, and canonical device names.  It's more
  consistent and it will work better with FreeBSD.
* Split get_device_by_fd into block_device_partition and
  block_device_wholedisk
* Remove dead code

Signed-off-by: Alan Somers <asomers@gmail.com>
2018-10-19 19:35:23 +08:00
Alan Somers
2fa3845d8f blkdev: move test_get_blkdev_size to test_get_blkdev_props
Also, enhance blkdev test program to show more properties than just size

Signed-off-by: Alan Somers <asomers@gmail.com>
2018-10-19 19:35:23 +08:00
Alan Somers
025f6c90d8 blkdev: Add dedicated accessors for all property functions
Hide get_block_device_string_property and get_block_device_int_property
to lower risk of misuse.

Signed-off-by: Alan Somers <asomers@gmail.com>
2018-10-19 19:35:23 +08:00
Alan Somers
559461b284 blkdev: add an enum type for block device properties
Signed-off-by: Alan Somers <asomers@gmail.com>
2018-10-19 19:35:23 +08:00
Kefu Chai
80376d32be
Merge pull request #21155 from runsisi/wip-auth-cleanup
auth/cephx: minor code cleanup

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-19 19:21:17 +08:00
Tim Serong
d08f290bcb
Merge pull request #24672 from SUSE/wip-orchestrator-result-property
mgr/orchestrator: use result property in Completion classes
2018-10-19 21:19:10 +10:00
Mehdi Abaakouk
f4a985b092 ceph-volume: extract flake8 config
It's preferable to use [flake8] section to configure flake8.

So external tools/editor can read the configuration.

Signed-off-by: Mehdi Abaakouk <sileht@sileht.net>
2018-10-19 13:18:47 +02:00
Lenz Grimmer
94aefee3b0
Merge pull request #24314 from rhcs-dashboard/dashboards
mgr/dashboard: Grafana dashboard updates and additions

Reviewed-by: Boris Ranto <branto@redhat.com>
2018-10-19 12:42:23 +02:00
Lenz Grimmer
203e3d2e60
Merge pull request #24628 from Devp00l/wip-issue-36445
mgr/dashboard: Add missing test requirement "werkzeug"

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2018-10-19 11:57:12 +02:00
Kefu Chai
3797d1f654
Merge pull request #24669 from falcon78921/wip-docs-36527
doc: add spacing to subcommand references

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-19 17:32:59 +08:00
Kefu Chai
39fcba396b
Merge pull request #19386 from ganeshmaharaj/qat-ceph-crypto
crypto: QAT based Encryption for RGW

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-19 17:02:56 +08:00
Kefu Chai
7200b16544 librbd: remove unused context_cb() function
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-19 16:33:42 +08:00
Kefu Chai
533064aa58 test/objectstore: silence "-Wsign-compare" warnings
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-10-19 16:33:42 +08:00