Commit Graph

108777 Commits

Author SHA1 Message Date
Kefu Chai
3ed5f0aea4 cmake: build used source
to address the link failures when compiling using clang

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-03 11:47:11 +08:00
Kefu Chai
6468deebad crimson: do not capture unused variable
this silences the warning of:
```
../src/crimson/osd/osdmap_gate.cc:48:38: warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
  std::for_each(first, last, [epoch, this](auto& blocked_requests) {
                                   ~~^~~~
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-03 11:47:11 +08:00
Kiefer Chang
f9d285f632
cephadm: fix failure when getting keyring for deploying daemons
Fixes: https://tracker.ceph.com/issues/44390
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-03-03 11:39:20 +08:00
bk203
52deebaa5c doc: update Zabbix template reference
The old link references a 2017 version of the template, I experienced problems using this version of the template with the latest version of Ceph. Ceph would report "Failed to send data to Zabbix", by importing the newer 2019 version of the template within Zabbix Ceph could again send data (due to changed Zabbix Trapper item keys). Propose to replace the link for a link referencing the master branch of the template so the newest version is always referenced in the docs.

Signed-off-by: Mathijs Smit <m.smit@goldenvalue.nl>
2020-03-03 10:32:57 +08:00
Kefu Chai
a91c3112a9
Merge pull request #33621 from aclamk/fix_mempool_btree
test/test_mempool: test accounting for btree_map

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-03 10:09:49 +08:00
Patrick Donnelly
b58c14a2d8
Merge PR #33363 into master
* refs/pull/33363/head:
	mds: add perf counters for openfiletable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-03-02 17:01:48 -08:00
Patrick Donnelly
4e90fbbcdf
Merge PR #33491 into master
* refs/pull/33491/head:
	mount.ceph: add "fs=<fs_name>" mount options support

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-03-02 16:59:51 -08:00
Patrick Donnelly
61681cffd2
Merge PR #33506 into master
* refs/pull/33506/head:
	client: add client_fs mount option support

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-03-02 16:58:02 -08:00
Patrick Donnelly
a001aa1212
Merge PR #33538 into master
* refs/pull/33538/head:
	mds: just delete MDSIOContextBase during shutdown
	mds: add comment explaining unusual locking for MDSIOContextBase

Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-03-02 16:55:09 -08:00
Sage Weil
2e39d77632 mgr/orch: fix SPEC alignment in 'orch ls'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 18:16:34 -06:00
Sage Weil
8ee0991e5b mgr/orch: include spec ref in ServiceDescription
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 18:16:34 -06:00
Casey Bodley
7aaa02dc32 denc: add denc_traits for enum class
enum traits cast to/from underlying type, and use the underlying type's
traits for encode/decode

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-02 16:09:23 -05:00
Casey Bodley
bd1b21ed7b denc: remove conditional underlying_type_t from ceph_le* traits
an enum's underlying type must be integral, so it's not possible for
ceph_le<T> to match for enums. for example, this:

  enum class enum_class_16 : ceph_le16 {};

fails to compile with:
underlying type ‘ceph_le16’ {aka ‘ceph_le<short unsigned int>’} of
‘enum_class_16’ must be an integral type

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-02 16:09:23 -05:00
Eric Jackson
a71ad57101 Refactor, prepare for other adoptions
Move the prometheus adopt out of command_adopt. refactor skip pull,
legacy check and fsid lock.

Signed-off-by: Eric Jackson <swiftgist@gmail.com>
2020-03-02 15:49:29 -05:00
Casey Bodley
e1d9ba3cf9 test: add failing denc tests for enums
all but the 8-bit enums fail to compile because they don't match the
traits specialization for underlying_type

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-03-02 15:41:09 -05:00
Jason Dillaman
725d963207 rbd-mirror: move resetting of snapshot replayer rescan variable
The `m_image_updated` boolean should be reset at the start of the
state checking loop now that we scan the local image meta and check
for forced-promotion of the local image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 15:34:22 -05:00
Sage Weil
367c402dcc Merge PR #33601 into master
* refs/pull/33601/head:
	qa/tasks/thrashosds-health: disable osd_max_markdown behavior
	osd: derr on do_shutdown cases

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-02 14:27:58 -06:00
Sage Weil
12927f492e mgr/cephadm: reduce log level for NodeAssignment
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 14:23:45 -06:00
Sage Weil
b60b51d582 mgr/cephadm: make NodeAssignment return a simple host list
We shouldn't modify the PlacementSpec in place (or at all).  Instead,
simply return a list of the hosts where we want to locate daemons.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 14:23:45 -06:00
Sage Weil
b4f040fb48 mgr/cephadm: track last config time and dependencies
Every time we deploy or reconfig a service, record the dependencies
and service dependency list.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 14:22:21 -06:00
Sage Weil
505f9a782d Merge PR #33647 into master
* refs/pull/33647/head:
	vstart.sh: fix install of cephadm ssh keys from ~/.ssh

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-02 14:14:52 -06:00
Sage Weil
dc4ff2b4e2 Merge PR #33648 into master
* refs/pull/33648/head:
	qa/suites/cephadm: move orchestrator_cli test into rados/cephadm

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-02 14:14:40 -06:00
Sage Weil
e766c57a4e Merge PR #33658 into master
* refs/pull/33658/head:
	cephadm: remove remaining crash_keyring cruft
	vstart.sh: deploy crash containers
	cephadm: bootstrap: deploy crash on every host
	mgr/cephadm: deploy crash daemons
	mgr/orch: add crash hooks
	cephadm: do not magically deploy 'crash' container

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-02 14:14:28 -06:00
Sage Weil
0752a15d52 cephadm: remove remaining crash_keyring cruft
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 13:23:56 -06:00
Sage Weil
8e80f5f678 compressor/lz4: rebuild if buffer is not contiguous
In older versions of lz4 (specifically < 1.8.2) bit errors
can be introduced when compressing from fragmented memory.  The lz4
bug was fixed by this lz4 commit:

commit af127334670a5e7b710bbd6adb71aa7c3ef0cd72
Author: Yann Collet <cyan@fb.com>
Date:   Sat May 5 18:24:11 2018 -0700

    fixed frametest error

    The error can be reproduced using following command :
    ./frametest -v -i100000000 -s1659 -t31096808

    It's actually a bug in the stream LZ4 API,
    when starting a new stream
    and providing a first chunk to complete with size < MINMATCH.
    In which case, the chunk becomes a dictionary.
    No hash was generated and stored,
    but the chunk is accessible as default position 0 points to dictStart,
    and position 0 is still within MAX_DISTANCE.
    Then, next attempt to read 32-bits from position 0 fails.

    The issue would have been mitigated by starting from index 64 KB,
    effectively eliminating position 0 as too far away.

    The proper fix is to eliminate such "dictionary" as too small.
    Which is what this patch does.

This is a workaround to rebuild our input buffer into a continguos buffer
if it is not already contiguous.

Fixes: https://tracker.ceph.com/issues/39525
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 13:01:22 -06:00
Sage Weil
8c8fb59dcf qa/objectstore: test all bluestore compression algorithms
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 13:01:22 -06:00
Dan van der Ster
1b1c71a2c2 test/compressor: test round trip of an osdmap
Check if the compressors can compress/decompress a bufferlist which is not word
aligned, such as a freshly-encoded osdmap.

Related-to: https://tracker.ceph.com/issues/39525
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2020-03-02 13:01:18 -06:00
Sage Weil
a7ea259f24 Merge PR #33633 into master
* refs/pull/33633/head:
	msg: add get_{pid,random}_nonce() helpers
	msg/Policy: make stateless_server default to anon (again)
	osd: drop broken 'poison pill'

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-02 12:04:29 -06:00
Jason Dillaman
918a48d1a5
Merge pull request #33642 from trociny/wip-run-rbd-tests-pool-init
test/run-rbd-tests: properly initialize newly created rbd pool

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 12:17:09 -05:00
Sage Weil
ddb8ab3dd3 qa/suites/cephadm: move orchestrator_cli test into rados/cephadm
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 10:52:24 -06:00
Sage Weil
b179dcef40 vstart.sh: deploy crash containers
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 10:32:46 -06:00
Sage Weil
ac554ce62e cephadm: bootstrap: deploy crash on every host
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 10:30:51 -06:00
Sage Weil
e2c43297d1 mgr/cephadm: deploy crash daemons
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 10:30:51 -06:00
Sage Weil
bec4138cee mgr/orch: add crash hooks
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 10:30:51 -06:00
Sage Weil
1ad9291337 cephadm: do not magically deploy 'crash' container
We'll soon do this explicitly with a persistent spec instead.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-02 10:30:51 -06:00
Sage Weil
0dd11a5aa5 Merge PR #33523 into master
* refs/pull/33523/head:
	mgr/orch: ServiceSpec: drop 'count'
	mgr/rook: use spec.placement.count (instead of spec.count)
	mgr/cephadm: make HostAssignment make sense
	mgr/orch: PlacementSpec: do not combine all_hosts with anything else
	mgr/orch: use PlacementSpec.from_strings() for all CLI commands

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-03-02 10:30:03 -06:00
Jason Dillaman
b4a830a23a librbd: clean-up special casing for enabling mirroring on creating image
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Jason Dillaman
d6f4ce2a38 qa: test cases for snapshot-based mirroring
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Jason Dillaman
238e284a61 rbd-mirror: simple snapshot split-brain detection
If the local image has a primary demotion and the remote does not
have a matching non-primary demotion snapshot, we should throw a
split-brain error.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Jason Dillaman
bcb7d6698b librbd: refresh image after creating primary mirror snapshot
If RPC was used to create the snapshot, the local image context will
not yet have the snapshot id and will therefore return CEPH_NOSNAP.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Jason Dillaman
25c2ffe145 librbd: acquire exclusive lock from peer when removing
This solves an issue with snapshot-based mirroring when the
rbd-mirror daemon is the exclusive lock owner. For other cases,
it still checks for watchers before proceeding.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Jason Dillaman
5cd074be9c librbd: fixed issue with deep-copy incorrectly setting parent spec
When not syncing the full image, the destination image would receive
the incorrect source parent spec instead of the expected destination
parent spec.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:44 -05:00
Jason Dillaman
df00b07385 rbd: do not show mirroring mode if not enabled
The 'info' action was incorrectly showing the mirroring mode
when mirroring was not enabled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:43 -05:00
Jason Dillaman
5af1578d10 librbd: allow force promotions to create necessary orphan snapshot
The rbd-mirror daemon needs to accept the RPC request to create an
orphan snapshot so that it knows to shut-down the replayer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:43 -05:00
Jason Dillaman
3e7642617a librbd: forced promotion of snapshot mirrored image needs to create orphan
Tweak the state machine to create an orphan record if the current
state is not demoted.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:43 -05:00
Jason Dillaman
106c103829 rbd-mirror: properly handle promoted/demoted snapshot-based images
We should return -EREMOTEIO if we don't have any primary images to sync to
since we want display a warning. Additionally, don't attempt to sync a
remote snapshot against a primary (demoted) local snapshot since it would
have an invalid primary snapshot id.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:43 -05:00
Jason Dillaman
d649218750 librbd: improve debug logging on primary mirror snapshot creation
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:43 -05:00
Jason Dillaman
7c34b0c702 cls/rbd: fixed typo in mirror image state ostream operator
The non-primary demoted state was incorrectly outputting demoted
twice.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:43 -05:00
Jason Dillaman
bfc064e4cb rbd-mirror: do not re-register the ASOK if the image replayer isn't running
Avoid the possibility of a race when shutting down and post-unregister a status
update causes the ASOK to become re-registered.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-03-02 10:53:42 -05:00
Casey Bodley
affc79bdcf
Merge pull request #32660 from cbodley/wip-rgw-admin-zone-shards
rgw: Increase the default number of RGW bucket shards

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-03-02 09:38:04 -05:00