Commit Graph

121707 Commits

Author SHA1 Message Date
Kefu Chai
80c4a1cd37 common: disable journald logging backend if struct msghdr is not found
* cmake/modules/CephChecks.cmake: detect the existence of struct msghdr,
  define HAVE_MSGHDR if it is found
* src/common/CMakeLists.txt: do not compile journald.cc if HAVE_MSGHDR
  is FALSE. as in that case, we cannot use sendmsg() to write to
  journald unix domain socket
* src/test/CMakeLists.txt, src/test/common/CMakeLists.txt: disable test
  exercising journald logging backend if HAVE_MSGHDR is not defined
* src/common/Journald.h: define a dummy JournaldLogger and a dummy
  JournaldClusterLogger when HAVE_MSGHDR is not defined, in order to
  minimize the change in Log.h and Log.cc, otherwise the source code of
  Log.h and Log.cc would be segmented into smaller chunks by
  `ifdef HAVE_MSGHDR` macros.
* src/include/config-h.in.cmake: define a new macro named
  HAVE_MSGHDR.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-06 14:46:33 +08:00
Kefu Chai
9f94d27752
Merge pull request #40582 from a16bitsysop/32bit
src/common/buffer.cc: change cast to static_cast

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-06 14:35:18 +08:00
Kefu Chai
8c37039c95
Merge pull request #40578 from tchaikov/wip-cmake-pmem
cmake: require libpmem 1.7 and cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-04-06 14:31:51 +08:00
Kefu Chai
4ae1ab90fe
Merge pull request #40572 from athanatos/sjust/wip-48613
osd/PeeringState: fix acting_set_writeable min_size check

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-06 14:30:20 +08:00
Kefu Chai
57ab82be38
Merge pull request #40475 from tchaikov/wip-qa-focal
qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2021-04-06 14:28:42 +08:00
Kefu Chai
38946dab0d
Merge pull request #40591 from tchaikov/wip-dencoder
tools/ceph-dencoder: link against libtcmalloc

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-06 10:31:07 +08:00
Samuel Just
642a1c1654 osd/PeeringState: fix acting_set_writeable min_size check
acting.size() >= pool.info.min_size is meant to check min_size against
acting set participants, but acting is a vector with placeholders.
actingset is the representation with placeholders removed.

The upshot of this bug is that the activation process will basically
ignore min_size for an ec pool allowing writes in cases where it
shouldn't.  PastIntervals::check_new_interval, however, performs
the check correctly, and will therefore discount intervals in which
we really did serve writes as not writeable.  This can trigger many
different problem conditions including but not limited to:
  - Unfound objects due to accepting a last_update with insufficient
    osds
  - Lost writes
  - Crashes due to peering rules being violated

This bug was originally introduced with recovery below min_size in
e5a96fd, and then preserved through refactors in 749a13d and 95bec9.

7cb818a exposed it with with expansion of recovery below min_size
to include ec pools (acting.size() is sufficient for replicated
pools).

Fixes: https://tracker.ceph.com/issues/48613
Fixes: https://tracker.ceph.com/issues/48417
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-04-05 17:43:18 -07:00
Sage Weil
0f1aa79a4a Merge PR #40599 into master
* refs/pull/40599/head:
	rpm: add missing % in %dir directive

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-05 16:50:23 -04:00
Yuval Lifshitz
bede749630
Merge pull request #40598 from TRYTOBE8TME/wip-rgw-kafka-tests-fix
src/rgw: Issue #50138 fix
2021-04-05 21:30:40 +03:00
Sage Weil
ee2e6e715e Merge PR #40025 into master
* refs/pull/40025/head:
	cephadm: use split cgroup strategy for podman
	cephadm: use class to represent container engine

Reviewed-by: Adam King <adking@redhat.com>
2021-04-05 14:14:31 -04:00
Sage Weil
6200ba4139 Merge PR #40459 into master
* refs/pull/40459/head:
	mgr/cephadm: don't cleanup the daemon keyring on failed redeploy

Reviewed-by: Adam King <adking@redhat.com>
2021-04-05 14:14:22 -04:00
Sage Weil
91eb165167 Merge PR #40508 into master
* refs/pull/40508/head:
	mgr/cephadm: fix orch host add with multiple labels and no addr

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-05 14:14:12 -04:00
root
903b941742 rgw/test: stop kafka consumer if tests fails
This PR fixes https://tracker.ceph.com/issues/50138 by stopping the kafka consumer if the test fails.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-04-05 21:14:17 +05:30
Nathan Cutler
404f8b2ce2 rpm: add missing % in %dir directive
de6c8250a6 added an explicit %dir directive for
a new directory added to the ceph-common package, but -- due to a typo --
neglected to include the "%". As a result, RPM builds started to fail with:

Processing files: ceph-common-17.0.0-2787.gde6c8250.el8.x86_64
error: File must begin with "/": {_libdir}/ceph/denc/

RPM build errors:
    File must begin with "/": {_libdir}/ceph/denc/

Fixes: de6c8250a6
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-04-05 17:30:04 +02:00
Kefu Chai
b9e0d4b8b3
Merge pull request #40584 from smithfarm/wip-denc-fix
rpm: make ceph-common own the new denc directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-05 23:09:14 +08:00
Yuval Lifshitz
d9b3fd3b01
Merge pull request #40585 from yuvalif/wip-yuval-fix-50135
rgw/multisite: handle case when empty marker is provided
2021-04-05 18:04:00 +03:00
Kefu Chai
9ef8055175
Merge pull request #39738 from SMIL-Infra/journald
common: enable log to journald

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-05 23:01:27 +08:00
Kefu Chai
d2c0f56b92
Merge pull request #40579 from tchaikov/wip-mgr-typing
pybind/mgr/mgr_util: fix typing annotation

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-05 21:29:13 +08:00
Sage Weil
d49bc22a10 Merge PR #40500 into master
* refs/pull/40500/head:
	doc/cephadm: remove keepalived_user from haproxy docs

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-05 09:16:34 -04:00
Kefu Chai
37a015a9ed cmake: dedup components when finding pmem
otherwise we always find libpmem twice if libpmem is explicitly
specified when find_package()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 17:33:10 +08:00
Kefu Chai
6978584e99 cmake: find pmem once instead twice
less noise in cmake output

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 17:33:09 +08:00
Kefu Chai
195d1e8c8e tools/ceph-dencoder: print human-readable error message
if $CEPH_LIB is not specified.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 16:30:05 +08:00
Kefu Chai
2365f265b1 tools/ceph-dencoder: link against libtcmalloc
to address the failure like

src/tcmalloc.cc:332] Attempt to free invalid pointer 0x5627da496130
Aborted

we need to link and load libtcmalloc before it is loaded by a shared
library, otherwise the memory chunk allocated by libc will be freed
by tcmalloc after it is loaded.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-05 16:30:04 +08:00
Yuval Lifshitz
fccf75eee3 rgw/multisite: handle case when empty marker is provided
marker is potional, however, it may also be provided empty

Fixes: https://tracker.ceph.com/issues/50135

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-04-04 17:19:03 +03:00
Kefu Chai
55564dbd01 qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04
resurrect 835f2f5f51, which was reverted by
1f885b6c2f

so all tests for `pacific` will be tested on the latest ubuntu version

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-04 21:15:05 +08:00
Nathan Cutler
de6c8250a6 rpm: make ceph-common own the new denc directory
2d3c6561b4 introduced a new library directory
"%{_libdir}/ceph/denc/" in ceph-common but did not explicitly state that it
should be owned by the package. This caused OBS builds to fail as follows:

[ 5515s] ceph-common-17.0.0-2786.1.x86_64.rpm: directories not owned by a package:
[ 5515s]  - /usr/lib64/ceph/denc

Fixes: 2d3c6561b4
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-04-04 11:02:11 +02:00
Duncan Bellamy
b45a6e957f src/common/buffer.cc: change cast to static_cast in ceph::buffer::list::page_aligned_appender::_refill
part of fixing compilation on 32bit platforms

Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>
2021-04-04 08:28:19 +01:00
Kefu Chai
5e1c42082e pybind/mgr/mgr_util: fix typing annotation
and refactor lock_timeout_log() a little bit to drop `locked`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-04 10:02:58 +08:00
Kefu Chai
7e1ef0c0c9 cmake: require libpmem 1.7
detect the libpmem libraries' version using pkg-config

please note, the version is not identical to the ones defined in
libpmem.h or libpmemobj/base.h

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 23:59:32 +08:00
Kefu Chai
6dfc80737d cmake: consolidate the find logic in Findpmem.cmake
less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 23:58:53 +08:00
Kefu Chai
7fb0569ec3
Merge pull request #40124 from adk3798/format-help
pybind/ceph_argparse: print --format flag name in help descs

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-04-03 22:27:23 +08:00
Kefu Chai
9541a97605
Merge pull request #40316 from batrick/i49605
pybind/mgr/volumes: avoid deadlock in ceph-mgr Finisher thread

Reviewed-by: Kotresh HR <khiremat@redhat.com>
2021-04-03 22:24:22 +08:00
Kefu Chai
9d00ec5bcc
Merge pull request #40555 from jecluis/wip-cephadm-fix-apparmor.master
cephadm: ignore apparmor if profiles file is empty

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-04-03 22:19:40 +08:00
Kefu Chai
889d850ba6
Merge pull request #38500 from pdvian/wip-print-weight
crush/CrushCompiler: print weight with uniform precision

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:19:07 +08:00
Kefu Chai
f5c859ea06
Merge pull request #38258 from badone/wip-tracker-48336-monmaptool-enforce-ip-better
monmaptool: Don't call set_port on an invalid address

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:18:06 +08:00
Kefu Chai
cf868b7a1b
Merge pull request #40518 from Rethan/fix-client-message-throttle-max
osd: allow osd_client_message_size throttle set to zero

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:16:49 +08:00
Kefu Chai
a661556363
Merge pull request #40444 from TheJJ/ensure-fsck-transactions
os/bluestore: ensure fsck repair transactions succeed

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-04-03 22:14:19 +08:00
Kefu Chai
e8fb63cf41
Merge pull request #40457 from lihuibng/crush-bug
crush/CrushLocation: do not print logging message in constructor

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:13:28 +08:00
Kefu Chai
5468c31d29
Merge pull request #40488 from lxbsz/typo
mgr: fix the desc typo for mds PyCFunction methods

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:12:59 +08:00
Kefu Chai
f26515fef7
Merge pull request #40168 from amathuria/wip-amathuria-fix-mon-trimming
mon: Modifying trim logic to change paxos_service_trim_max dynamically

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:12:26 +08:00
Kefu Chai
c644ae1446
Merge pull request #40498 from guits/guits-quick_fix
cephadm: fix a typo

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-03 22:11:50 +08:00
Kefu Chai
fdb92c3b5b
Merge pull request #40538 from tchaikov/wip-msg-type-cleanup
msg/msg_types: let ceph_entity_name::parse() accept string_view

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2021-04-03 22:10:12 +08:00
Kefu Chai
185b0fd1d2
Merge pull request #40556 from liewegas/mon-no-config-indent
mon/ConfigMonitor: do not indent 'config dump' output

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:08:58 +08:00
Kefu Chai
9d4505eb5c
Merge pull request #40574 from badone/wip-submanager-contatiner-failure-tracker-50118
install-deps: Use dnf config-manager

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:02:47 +08:00
Kefu Chai
bf56cddce5 cmake: use .pc for hints for finding pmem libraries
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:00:17 +08:00
Brad Hubbard
2e1ee019d2 install-deps: Use dnf config-manager
Using subscription-manager will fail in a container so use dnf
config-manager which should work on bare metal as well as in a
container.

Fixes: https://tracker.ceph.com/issues/50118

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-04-03 16:57:50 +10:00
Kefu Chai
8c6b9e45c9 msg/msg_types: remove entity_name_t::parse(const char*...)
it can be replaced with entity_name_t::parse(string_view)

also refactor entity_name_t::parse(string_view) a little bit, to
embed the logic of `entity_name_t::parse(const char*...)` in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 13:03:43 +08:00
Kefu Chai
af167c074c
Merge pull request #40435 from tchaikov/wip-48893
common/pick_address: Allow binding on loopback iface

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-03 11:13:45 +08:00
Kefu Chai
7f01d36a2c common/pick_address: Allow binding on loopback iface
in 6147c09171, "lo" is also skipped when
daemon is trying to find an address to bind. but that change reverts the
fix of 201b592043, to address the problem.

an option named "ms_bind_exclude_lo_iface" is added, it defaults to
"true". but it can be changed to false to allow daemon to bind on "lo".

Fixes: https://tracker.ceph.com/issues/50012
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 10:06:14 +08:00
Kefu Chai
6d0ed81f79 common/pick_address: pass string by reference
to silence warnings from clang-tidy.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-03 10:06:13 +08:00