Commit Graph

106051 Commits

Author SHA1 Message Date
Kefu Chai
bad8fdf762 make-dist: package boost v1.72 instead of v1.67
to be consistent with the version used by "make check"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Kefu Chai
e461bda6c9 cmake: build boost v1.72 instead of v1.67
so we can drop two patches which are already included by v1.72

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Kefu Chai
eae47868b8 crimson: do not use small_vector
if `std::is_nothrow_move_constructible<small_vector<peer_shard_t, 2>>`
also depends on if the allocator throws when moving from a small_vector
to a new one, as we need to allocate memory for the destination
small_vector.

so, in this change, the optimization for small_vector is dropped,
because the default allocator used by small_vector throws.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Kefu Chai
f91adafa4b install-deps.sh: use "apt" for checking install status
as "dpkg -s <package-name>" always returns 0 even if the <package> is
removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Kefu Chai
ea64c4909f common,rgw: workaround for boost 1.72
see also https://github.com/boostorg/coroutine/issues/46

will need to remove this change once we require boost >= 1.73

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-24 01:22:07 +08:00
Jason Dillaman
b61f83b6f0 librbd: diff iterate with fast-diff now correctly includes parent
When whole-object and include-parent options are enabled, the
diff will now include the parent image diffs. Previously, the
parent image diffs were not included when fast-diff was enabled
but was included when fast-diff was disabled.

Fixes: https://tracker.ceph.com/issues/42248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-23 11:34:07 -05:00
Sage Weil
50d247f5a2 mgr/cephadm: fix =[] default arg to _create_daemon
This is a no-no (pass by reference, not value)!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 10:29:52 -06:00
Sage Weil
2ae3f66263 qa/suites/rados/cephadm: enable mgr debugging
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-23 10:01:35 -06:00
Jason Dillaman
dc885574d9 librbd: bump minor version to match octopus
This will assist layered projects like QEMU and libvirt
determine which features are supported.

Fixed: https://tracker.ceph.com/issues/43401
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-23 10:51:33 -05:00
Jason Dillaman
9e29914a6a librbd: remove pool objects when removing a namespace
Avoid leaving RBD pool objects within a pool namespace that is
being deleted.

Fixes: https://tracker.ceph.com/issues/43378
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-23 10:46:09 -05:00
liushi
ec3a696a5d mgr: close restful socket after exec
Signed-off-by: liushi <liu.shi@navercorp.com>
2019-12-23 17:09:29 +08:00
Xie Xingguo
37f771cffb
Merge pull request #32368 from xiexingguo/wip-balancer-efficiency-4
osd/OSDMap: consider overfull osds only when trying to do upmap

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-12-23 16:08:23 +08:00
Xie Xingguo
ce888bbe16
Merge pull request #32289 from xiexingguo/wip-balancer-efficiency-2
mgr/balancer: eliminate usage of MS infrastructure for upmap mode

Reviewed-by: Yan Jun <yan.jun8@zte.com.cn>
2019-12-23 16:07:21 +08:00
Kefu Chai
d560af7d5c install-deps.sh: install boost-1.72 for bionic
and uninstall boost-1.67 before installing boost-1.72

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-23 15:21:47 +08:00
Chunmei Liu
6e65ebadce vstart.sh: move extra_seastar_agrs up in vstart.sh
add extra_seastar_args in --mkfs

Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2019-12-22 22:52:17 -08:00
Jason Dillaman
8542da0991 rbd-mirror: clear init/shutdown context before completion
This fixes a potential issue where the init fails and shut down
is called with the original init context still set.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-22 17:16:17 -05:00
Sage Weil
5d133d545b cephadm: create /var/run/ceph/$fsid as needed
The get_container_mounts method only passed this through if it already
exists so that shell etc commands will work even when this dir isn't
present yet.  Normally the systemd unit creates it.  This may not be the
case on initial bootstrap, of even later if a deploy happens but a unit
hasn't started yet.

Precreate it during bootstrap and other times before we do a deplot so
that the unit.run command has this passed through.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 14:20:40 -06:00
Sage Weil
5131b1b880 cephadm: fix v1/v2 ip/addrv handling; explicitly check bind to ip:port
- Handle the various v1 and v2 cases
- Handle --mon-ip with a port
- Explicitly test bind to the IP + port combination

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 12:06:42 -06:00
Sage Weil
858430fbbb Merge PR #32389 into master
* refs/pull/32389/head:
	test_cephadm.sh: pass --fsid to shell command

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-12-22 12:00:05 -06:00
Jason Dillaman
2bca9ee96c rbd-mirror: consolidate prepare local/remote image steps to bootstrap
Now that the bootstrap state machine is simplier, it makes it cleaner to
move the prepare steps into the bootstrap state machine. Once snapshot
mirror support is added, these prepare state can be swapped out depending
on the mirroring type.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-22 12:51:12 -05:00
Sage Weil
1701498e23 cephadm: restart non-ceph daemons on reconfig
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 11:00:33 -06:00
Sage Weil
e97b54c0ec mgr/cephadm/test_cephadm: fix expected output
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 11:00:33 -06:00
Sage Weil
b12aa3a838 mgr/cephadm: fix remapping of hosts and services
This was nondeterministically remapping services to different hosts
depending on which hosts had stale caches because the 'hosts' list order
didn't match 'results' + 'in_cache'.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 11:00:33 -06:00
Sage Weil
5087305b82 Merge PR #32286 into master
* refs/pull/32286/head:
	cephadm: correct ipv6 support in port open detection

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-12-22 10:56:57 -06:00
Sage Weil
e07330c084 mgr/cephadm: add 'reconfig' service action
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 10:50:23 -06:00
Sage Weil
40b70c632d cephadm: move mon creation config bits into mgr/cephadm
There's no need for mgr/cephadm to pass --mon-* arguments to
/usr/bin/cephadm just so it can add bits to the config file when the
config file is also being passed in from mgr/cephadm.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 10:50:23 -06:00
Sage Weil
8dbb4be2c7 cephadm: fix bootstrap message 'ssh' -> 'cephadm'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 10:50:23 -06:00
Sage Weil
a0fc82af23 cephadm: add --reconfig option for 'deploy'
We could make this a totally separate command from 'deploy', but so much
of the code path is shared that this is a lot simpler.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 10:50:23 -06:00
Sage Weil
b77f0c74a5 mgr/orchestrator: drop 'reload' service[-instance] action
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 10:50:23 -06:00
Kefu Chai
70c29729f2
Merge pull request #32388 from wjwithagen/wjw-fix-boost_spirit-isblank-assert
test: Do not test unicode if boost::spirit >= 1.72

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-23 00:40:31 +08:00
Kefu Chai
a6f4234b04 install-deps.sh: drop trusty and xenial support
since we only build on bionic on ubuntu distros, there is no need
to build on trusty and xenial anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-23 00:02:17 +08:00
Kefu Chai
be192fba4a cmake: update FindBoost.cmake for 1.72
adapted from
https://github.com/Kitware/CMake/blob/master/Modules/FindBoost.cmake
commit e51232b739b3eeae642be8a7e89079f7c2e8c1b2

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-22 23:56:52 +08:00
Willem Jan Withagen
05fe3fc9c8 test: Do not test unicode if boost::spirit >= 1.72
Testing with Boost 1.72 unittest_confutil crashes with:
```
[ RUN      ] ConfUtils.ReadFiles2
Assertion failed: (strict_ischar(ch)), function isblank, file /usr/local/include/boost/spirit/home/support/char_encoding/standard.hpp, line 129.
Abort
```

Which is due to the fact that in Boost 1.72 most of the character
testing has been garded with an assert to make sure char holds an
ascii char.
Like:
```
        static bool
        isblank BOOST_PREVENT_MACRO_SUBSTITUTION (int ch)
        {
            BOOST_ASSERT(strict_ischar(ch));
            return (ch == ' ' || ch == '\t');
        }
```
And ReadFile2 explicitly tests for strings with char value > 0x7f.
So that is certainly going to crash.

Now I can imagine that we would like to be able to have users use
different encodings for things like logfiles.
But for now that is not going to work with the boost:spirit parser

Fixes: https://tracker.ceph.com/issues/43406
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-12-22 16:08:10 +01:00
Sage Weil
d61066804a test_cephadm.sh: pass --fsid to shell command
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-22 08:48:50 -06:00
Sage Weil
effe9c9a3f Merge PR #32261 into master
* refs/pull/32261/head:
	mgr/orchestrator: use full device path for blinking lights (if available)
	mgr: report device by-path paths too
	common/blkdev: include by-path path for each device in metadata
	common/blkdev: factor get_device_metadata out of mon, osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-12-21 19:59:37 -06:00
Sage Weil
73a56e36c3 Merge PR #32333 into master
* refs/pull/32333/head:
	qa/quites/rados/singleton-flat/valgrind-leaks: specify centos8
	qa/suites/rados: test cephadm on centos and ubuntu both
	qa/workunits/cephadm/test_cephadm.sh: detect python3 and python2

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-12-21 19:59:18 -06:00
Sage Weil
a3a1e3e8ac qa/suites/rados/thrash-old-clients: stick to el7
Old distros don't have packages for bionic.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Sage Weil
65b4025635 qa/tasks/cephadm: check cluster log; support log-whitelist
Mostly just lifted from ceph.py

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Sage Weil
50294ecc65 qa/suites/rados/thrash-old-clienets: python-foo to python3-foo
Jewel and Hammer don't have python-rgw tho.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Sage Weil
efc3df1160 qa/suites/rados/thrash-old-clients: add new exclude_packages
Due to 6f5fb95408

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Sage Weil
1c7bd1d1f9 qa/suites/rados/thrash-old-clients: use cephadm
- deploy cluster with cephadm so we can run a octopus+ cluster and also
  install client packages that are ancient.
- move client.2 back onto the third node, since packages no longer
  conflict.
- test on centos 7.x (i picked 6), since the old releases all built on
  that release.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Sage Weil
4ad4118a1d mon/ConfigMonitor: make legacy mon addr/port parseable by legacy code
If it's a v1 addr on the old default 6789 port, print it as a legacy-style
addr (no v1: prefix) so that legacy code can parse it.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 19:53:38 -06:00
Jason Dillaman
da62228467 rbd-mirror: bootstrap now uses the prepare replay state machine
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-21 11:34:54 -05:00
Jason Dillaman
d711e6b769 rbd-mirror: extract pre-journal replay handling to new state machine
The new state machine will check for resync/sync states and split-brain
conditions.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-21 11:29:18 -05:00
Jason Dillaman
2d6f91d8a7 rbd-mirror: bootstrap now uses the create local image state machine
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-21 11:29:18 -05:00
Jason Dillaman
638be259f2 rbd-mirror: extract journal local image creation to new state machine
This state machine simplifies the existing bootstrap state machine steps
in that it will always unregister as a first step (might not be registered).
It also take advantage of the previous fix for REPLAYING vs SYNCING state
so that it always uses an initial SYNCING state.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-21 11:29:18 -05:00
Jason Dillaman
8c33391a62 rbd-mirror: reset bootstrap state to replaying if local image promoted
The previous logic had a race in that it would never run if the local
image was force-promoted or if the remote image was demoted and the
local image was promoted before the rbd-mirror daemon could catch the
change of state.

This change resets the mirror peer state from syncing to replaying if
the local image is no longer connected to the remote image. This
prevents an unexpected (and corrupted) re-sync from the remote.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-21 11:29:18 -05:00
Jason Dillaman
53455eebdf rbd-mirror: delay retrieval of remote tag class on bootstrap
This will help move the journal-related bootstrap logic together
as a preliminary step for handling snapshot mirroring.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-12-21 11:29:18 -05:00
Sage Weil
d96b6fd1c5 qa/quites/rados/singleton-flat/valgrind-leaks: specify centos8
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 09:03:55 -06:00
Sage Weil
47350be466 qa/suites/rados: test cephadm on centos and ubuntu both
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-21 09:03:55 -06:00