Commit Graph

103548 Commits

Author SHA1 Message Date
Yuri Weinstein
7b4f9a083f
Merge pull request #31132 from smithfarm/wip-42474
build-integration-branch: take PRs in chronological order
2019-10-24 10:43:47 -07:00
Junyoung, Sung
678c14259f do_cmake: Disable shell command printing feature before cmake execution
Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
2019-10-25 02:17:59 +09:00
Nathan Cutler
ceb3027930 build-integration-branch: change shebang to python3
The word "python" in the shebang means Python 2, but Python 2 is deprecated and
should no longer be used.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-10-24 18:54:55 +02:00
Nathan Cutler
bf564b5f1d build-integration-branch: take PRs in chronological order
Fixes: https://tracker.ceph.com/issues/42474
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-10-24 18:34:47 +02:00
huangjun
712a39e5c9 crush: remove invalid upmap items
We can not cancel in verify_upmap if remap an osd to different root bucket,

cluster topology:
osd.0 ~ osd.29 belongs to datacenter 1
osd30 ~ osd.59 belongs to datacenter 2
crush rule:
  take datacenter 1
  chooseleaf 2 host
  emit
  take datacenter 2
  chooseleaf 2 host
  emit
The pg's primary osd in datacenter 1.

We should cancel the pgid from upmap_items like below
1) from [26,12,54,46] to [30,12,54,46]
pg_upmap_items [26,30]

2) from [16,25,53,31] to [40,25,53,0]
pg_upmap_items [16,20,31,0]

Signed-off-by: huangjun <huangjun@xsky.com>
2019-10-25 00:29:21 +08:00
Junyoung, Sung
87ada5fcc8 do_cmake: Warn user about slow debug performance only for not set
Signed-off-by: Junyoung, Sung <junyoung.sung@navercorp.com>
2019-10-25 00:48:59 +09:00
Willem Jan Withagen
2853a48385 rgw: Select the std::bitset to resolv ambiguity
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-10-24 17:09:56 +02:00
Sage Weil
bb84373070 mon/MonClient: fix mon tell to older mons
For older mons, we need to call into send_command so that we trigger the
compat code that futzes with our mon sessions.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-24 08:00:32 -05:00
Sage Weil
8b56c456b5 Merge PR #30848 into master
* refs/pull/30848/head:
	mon/PGMap: drop indentation on 'df' human output

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-24 06:20:29 -05:00
Kefu Chai
56070205e6 cmake: support Seastar_DPDK=ON option
seastar does not build any 3rd party libraries. but we do. currently, we
build DPDK/SPDK when they are enabled for the messenger backend and
bluestore backend.

in this change, the same function is reused for preparing the DPDK
libraries if `Seastar_DPDK` is enabled. probably we should outsource
this work to install-deps.sh in future, but before that, let's keep
this as a builtin "feature".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 17:54:05 +08:00
Kefu Chai
a123c85256 cmake: enable Finddpdk to find DPDK to be built
find_package(dpdk...) is used by seastar to find DPDK if Seastar_DPDK is
enabled. but `build_dpdk()` only exposes `dpdk::dpdk`, and it does not
set the variables like `dpdk_INCLUDE_DIRS` and `dpdk_LIBRARIES` when it
gets called. so we need to adapt Finddpdk to query them from `dpdk::dpdk`
target.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 17:54:05 +08:00
Nathan Cutler
54020b75bc
Merge pull request #31112 from smithfarm/wip-distros-sle
qa/distros: add SLE-12-SP3 and SLE-15-SP1

Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
2019-10-24 11:28:41 +02:00
Nathan Cutler
33a9e4a671 qa/distros: add SLE-12-SP3 and SLE-15-SP1
Ceph luminous is known to run on SLE-12-SP3 and nautilus on SLE-15-SP1, so add
these two to qa/distros/all.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-10-24 09:47:22 +02:00
Kefu Chai
aa25f6202f cmake: define dpdk_LIBRARIES properly
dpdk_LIBRARIES should not be a list of dpdk library target, it should be
a list of paths to them. this also align with the definition of
`Finddpdk.cmake` in seastar project. so, if `Seastar_DPDK` is defined,
we should be able to offer the expected `dpdk_LIBRARIES` for seastar, as
our `Finddpdk.cmake` has higher priority than the one in seastar, and is
used when `find_package(dpdk...)` is called.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Kefu Chai
d82ea5d738 cmake: refactor BuildDPDK
* extract `do_export_dpdk()` into a separated function
* no need to check for `dpdk-ext` target before calling `build_dpdk()`,
  as the name of this target should be hidden from the caller of this
  function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 15:15:51 +08:00
Kefu Chai
097de62bf9
Merge pull request #31069 from tchaikov/wip/crimson/replace-ceph-with-crimson
crimson: s/ceph/crimson/ in namespace names

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-10-24 13:57:11 +08:00
Kefu Chai
bd023211d9 test/crimson: silence deprecated warnings
* do not discard returned future<>
* do not use variadic future<>

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 12:37:30 +08:00
Kefu Chai
4589fff6bf crimson: s/ceph/crimson/ in namespace names
to help differentiate the symbols shared by classic and crimson osd,
after this change, all crimson code will live in "crimson::" namespaces.
and in a follow-up change, all classic code used by crimson will live in
"ceph::" namespaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 12:37:30 +08:00
Kefu Chai
acff723c32 crimson/osd/osd_operation.h: s/ceph_assert/assert/
so we don't need to include `include/ceph_assert.h` for it. and
presumably, `ceph_assert()` is for fail the application early when it is
not able to recover from a failure, while `assert()` is for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-10-24 12:23:17 +08:00
Neha Ojha
d79325b07f
Merge pull request #31066 from ideepika/wip-doc-adds-cmake-options
doc: Adds cmake build options for optionally skipping few components

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-24 09:29:36 +05:30
Sage Weil
29c97547a9 Merge PR #30859 into master
* refs/pull/30859/head:
	auth: EACCES, not EPERM
	mon: shunt old tell commands from cli interface to asok
	mon: allow mgr to tell mon.foo smart
	mon: include quorum features in quorum_status
	qa/workunits/mon/caps.sh: fix test
	ceph_test_rados_api_cmd: fix MonDescribe test
	Merge branch 'vstart-fs-auth' of git://github.com/batrick/ceph into wip-cleanup-mon-asok
	test/pybind/test_ceph_argparse: fix tests
	vstart: add volume client keys to keyring
	vstart: use fs authorize to create master client key
	vstart: redirect some output to stderr
	vstart: output command strings to stderr
	qa/workunits/cephtool/test.sh: fix 'quorum enter' caller
	qa: change mon_status calls to quorum_status or tell commands
	mon: fix 'heap ...' command
	mon: consolidate 'sync force' commands
	mon: allow asok commands to return an error code
	mon: move 'quorum enter|exit' and 'mon_status' to asok
	mon: fix 'smart' asok command
	mon: remove old 'config set' and 'injectargs'

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-10-23 21:05:42 -05:00
Sage Weil
bf09a04d22 Merge PR #31094 into master
* refs/pull/31094/head:
	ceph-daemon: remove redundant --privileged
	test_ceph_daemon: test unit, enter, shell
	ceph-daemon: drop exec
	ceph-daemon: fix exit code for run, shell, enter, exec
	ceph-daemon: allow optional command for 'enter'
	ceph-daemon: fix LANG for 'enter' command
	ceph-daemon: allow shell to take optional command
	qa/suites/rados/singleton-nomsgr/ceph-daemon: run test_ceph_daemon.sh
	qa/standalone/test_ceph_daemon.sh: add new functional tests
	test_ceph_daemon.sh: use newer image
	ceph-daemon: unconditionally enable and start crash unit
	ceph-daemon: fix crash unit cleanup
	ceph-daemon: include 'crash' unit/item in 'ls' output
	ceph-daemon: fix 'ls'
	mgr/orchestrator: s/sdd/ssd/
	mgr/ssh: remove stdout/stderr kludges
	ceph-daemon: fix ceph-volume command to write stdout to stdout

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-10-23 19:46:06 -05:00
Gregory Farnum
e190825f26
Merge pull request #30650 from athanatos/sjust/wip-dmclock-server-only
dmclock server side refactor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-10-23 13:35:13 -07:00
Samuel Just
6eaf324083 PendingReleaseNotes: note mClockScheduler config changes
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-10-23 13:33:59 -07:00
Samuel Just
2157ac7d14 osd/: remove legacy schedulers
The consensus seems to be that PrioritizedQueue is strictly worse than
WeightedPriorityQueue.

mClockClientQueue and mClockClassQueue are superceded by
mClockScheduler.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-10-23 13:33:59 -07:00
Samuel Just
9ec24d3bfe osd/scheduler: introduce mClockScheduler
mClockScheduler schedules items based on op_scheduler_class
with configured mclock parameters.  Items which should be
scheduled immediately (op_scheduler_class::immediate) are placed
into a single queue as it's not clear that there's a reason to
differentiate among them.

A subsequent patch will add support for client provided mclock
params and dmclock request state.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-10-23 13:33:55 -07:00
Sage Weil
c6773f4c4c Merge PR #31057 into master
* refs/pull/31057/head:
	common/thread: Fix race condition in make_named_thread

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2019-10-23 15:20:56 -05:00
Sage Weil
adf22b9e59 Merge PR #31054 into master
* refs/pull/31054/head:
	qa/suites/upgrade/*-x-singleton: suppress TOO_FEW_PGS warning

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-10-23 15:20:43 -05:00
Sage Weil
62452dc191 Merge PR #30882 into master
* refs/pull/30882/head:
	ceph-daemon: deploy a logrotate.d config file for each cluster

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-10-23 15:18:26 -05:00
Sage Weil
7d2bd68461 ceph-daemon: remove redundant --privileged
The only caller, command_shell, passes this.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
d7bd029b51 test_ceph_daemon: test unit, enter, shell
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
86b2c8dd60 ceph-daemon: drop exec
It's not identical to enter.  enter seems more intuitive to me, but that
may be because I'm not a longtime docker user.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
9c3e270eec ceph-daemon: fix exit code for run, shell, enter, exec
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
c475186856 ceph-daemon: allow optional command for 'enter'
Default to interactive shell; otherwise run a specific command, just like
the 'shell' command.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
e427e72d2d ceph-daemon: fix LANG for 'enter' command
We did the same thing for 'shell' in 02d4f8c097

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
75c1ebe606 ceph-daemon: allow shell to take optional command
This is different from 'exec' in that it creates a new container instead
of entering an existing one.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
47777b9c0d qa/suites/rados/singleton-nomsgr/ceph-daemon: run test_ceph_daemon.sh
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
202d615d38 qa/standalone/test_ceph_daemon.sh: add new functional tests
- sudo as needed
- clean up afterward

There is still a bit of missing coverage, but this captures most of it.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 15:08:55 -05:00
Sage Weil
a779cb8fd0 test_ceph_daemon.sh: use newer image
Not sure what the difference between latest-master and latest-master-devel
is, but the -devel image is way out of date.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
31c974e5de ceph-daemon: unconditionally enable and start crash unit
Even if it exists, (re)try to enable and start it.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
8a19fbfbde ceph-daemon: fix crash unit cleanup
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
ebf1a7b19f ceph-daemon: include 'crash' unit/item in 'ls' output
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
0c318ed2b2 ceph-daemon: fix 'ls'
This broke when we added the crash service.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
22d83b0b0d mgr/orchestrator: s/sdd/ssd/
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
a4b7201d6e mgr/ssh: remove stdout/stderr kludges
ceph-daemon is now well-behaved with respect to stdout vs stderr.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
fb0899b8dc ceph-daemon: fix ceph-volume command to write stdout to stdout
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 14:45:34 -05:00
Sage Weil
ef3c42cd64 auth: EACCES, not EPERM
EPERM means an operation requires superuser; EACCES more generally means
permission denied.  We use EACCES elsewhere for ceph auth denials; use it
here too for consistency.  This fixes qa/workunits/mon/caps.sh.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 13:39:46 -05:00
Patrick Donnelly
be1fa70b84
qa: do not check pg count for new data_isolated volume
We don't need to specify the number of PGs for a new data pool anymore
since b1b821f608 and other related
changes. The related health warnings are also deprecated/gone. So this
no longer needs to be done.

Fixes: b1b821f608
Fixes: https://tracker.ceph.com/issues/42436
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-10-23 11:14:28 -07:00
Casey Bodley
250a65e045
Merge pull request #30997 from cbodley/wip-qa-rgw-objectstores
qa/rgw: drop some objectstore types

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-10-23 11:37:32 -04:00
Sage Weil
3010d3a531 Revert "test: librados startup/shutdown racer test"
This reverts commit 9b6ed03e60.

Reverting this until we have a fix for https://tracker.ceph.com/issues/42026

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-23 10:00:43 -05:00