Commit Graph

122095 Commits

Author SHA1 Message Date
Kefu Chai
117838c91d
Merge pull request #40866 from hualongfeng/pmdk_in_dist
make-dist: add pmdk to dist tarball

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-15 16:43:28 +08:00
Feng Hualong
14c2a2e59f cmake: use src/pmdk for building pmdk if it exists
so we can build with pmdk enabled if the dist tarball
contains pmdk

Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-04-15 10:40:06 +08:00
Feng Hualong
9d958d0b9d make-dist: add pmdk to dist tarball
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-04-15 10:40:06 +08:00
J. Eric Ivancich
6710f28746
Merge pull request #40862 from ivancich/wip-rgw-reshard-contention-logging
rgw: during reshard lock contention, adjust logging

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-04-14 16:13:17 -04:00
Casey Bodley
dbede38c7e
Merge pull request #40821 from cbodley/wip-50218
qa/rgw: use symlinks to specify distro

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-04-14 14:45:00 -04:00
J. Eric Ivancich
6d3dee3779 rgw: during reshard lock contention, adjust logging
When RGW fails to get a lock on a reshard log, we log it in such a way
that it looks like an error. Instead we'll make sure that the log
message is informational.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-04-14 14:18:30 -04:00
Ilya Dryomov
f3a4166379 Merge branch 'CVE-2021-20288' into master
* CVE-2021-20288:
  qa/standalone: default to disable insecure global id reclaim
  qa/suites/upgrade/octopus-x: disable insecure global_id reclaim health warnings
  qa/tasks/ceph[adm].conf[.template]: disable insecure global_id reclaim health alerts
  cephadm: set auth_allow_insecure_global_id_reclaim for mon on bootstrap
  mon/HealthMonitor: raise AUTH_INSECURE_GLOBAL_ID_RENEWAL[_ALLOWED]
  auth/cephx: ignore CEPH_ENTITY_TYPE_AUTH in requested keys
  auth/cephx: rotate auth tickets less often
  mon: fail fast when unauthorized global_id (re)use is disallowed
  auth/cephx: option to disallow unauthorized global_id (re)use
  auth/cephx: make cephx_decode_ticket() take a const ticket_blob
  auth/AuthServiceHandler: keep track of global_id and whether it is new
  auth/AuthServiceHandler: build_cephx_response_header() is cephx-specific
  auth/AuthServiceHandler: drop unused start_session() args
  mon/MonClient: drop global_id arg from _add_conn() and _add_conns()
  mon/MonClient: reset auth state in shutdown()
  mon/MonClient: preserve auth state on reconnects
  mon/MonClient: claim active_con's auth explicitly
  mon/MonClient: resurrect "waiting for monmap|config" timeouts

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-14 19:58:49 +02:00
Kefu Chai
cbc7f72784
Merge pull request #40614 from smithfarm/wip-39556
make-dist: refuse to run if script path contains a colon

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 22:41:35 +08:00
Gregory Farnum
ef6f36075e
Merge pull request #40839 from gregsfortytwo/wip-mon-quorum-leader
mon: MMonProbe: direct MMonJoin messages to the leader, instead of the first mon

Reviewed-by: Sam Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-14 07:41:05 -07:00
Kefu Chai
ba68020cea
Merge pull request #40807 from tchaikov/wip-common-include
common/Formatter: include used header

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-14 22:39:45 +08:00
Kefu Chai
c7cca2b86e
Merge pull request #40830 from gregsfortytwo/wip-sqlite3-build
mgr: do not require sqlite3.h if WITH_LIBCEPHSQLITE=OFF

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 22:38:55 +08:00
Kefu Chai
9b1c3e24f6
Merge pull request #40840 from tchaikov/wip-silence-gcc-warnings
tools: silence gcc warnings

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 22:35:09 +08:00
Sage Weil
ff97629375 Merge PR #40735 into master
* refs/pull/40735/head:
	mgr/orchestractor: rgw realm and zone flags must both be provided

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-14 09:53:31 -04:00
Sage Weil
8cc5dc75d7 Merge PR #40734 into master
* refs/pull/40734/head:
	mgr/cephadm: make prometheus scrape ingress haproxy
	doc/cephadm: remove big warning about stability
	doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
	mgr/cephadm: ingress: add optional virtual_interface_networks
	doc/cephadm/rgw: clean up example spec
	mgr/cephadm/services/ingress: less verbose about prepare_create
	doc/cephadm/rgw: add note about which ethernet interface is used
	cephadm: make keepalived unit fiddle sysctl settings
	mgr/orchestrator: report external endpoints from 'orch ls'
	mgr/orchestrator: drop - when no ports
	doc/cephadm/rgw: update docs for ingress service
	mgr/cephadm: use per_host_daemon feature in scheduler
	mgr/cephadm/schedule: add per_host_daemon_type support
	mgr/cephadm: HA_RGW -> Ingress
	mgr/cephadm: include daemon_type in DaemonPlacement
	mgr/cephadm: update list-networks to report interface names too
	mgr/orchestrator: streamline 'orch ps' PORTS formatting
	mgr/cephadm/schedule: handle multiple ports per daemon
	mgr/cephadm/utils: resolve_ip(): prefer IPv4

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-14 09:39:38 -04:00
Kefu Chai
e09f8d7dd0
Merge pull request #40149 from tchaikov/wip-cmake-job-pool
cmake: use ninja job pool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 15:06:50 +08:00
Kefu Chai
061affa38e
Merge pull request #40785 from tchaikov/wip-doc-list-spacing
doc/_themes: remove spacing after `ul li p`

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-14 12:42:31 +08:00
Kefu Chai
e2875647b8 tools/osdmaptool: mark unused variable [[maybe_unused]]
to silence warning from GCC when performing release build, like:

../src/tools/osdmaptool.cc: In function ‘int main(int, const char**)’:
../src/tools/osdmaptool.cc:472:9: warning: variable ‘r’ set but not used [-Wunused-but-set-variable]
  472 |     int r = clock_gettime(CLOCK_MONOTONIC, &round_start);
      |         ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 11:11:08 +08:00
Kefu Chai
25ce28a470 doc/radosgw: fix formatting of command line block
should add an empty line after the directive.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 09:33:37 +08:00
Kefu Chai
6dea67ac2d admin: require sphinx>=3.2.1
loosen the required sphinx version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 09:03:03 +08:00
Kefu Chai
aa66a4950b doc/_themes: remove spacing after ul li p
in the latest document generated from RtD, the spacing after `ul li p`
elements is set to 24px as the plain `p` elements. but this the lists
more sparse and difficult to read.

in this change, the spacing is restored to 0 as it was in old theme.css
in sphinx_rtd_theme.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 09:03:03 +08:00
Greg Farnum
ffa7ff35b4 mon: MMonProbe: direct MMonJoin messages to the leader, instead of the first mon
When monitors are joining a cluster, they may send an MMonJoin message to place
themselves correctly in the map in either handle_probe_reply() or
finish_election(). These messages must be sent to the leader -- monitors do not
forward each other's messages.

Unfortunately, this scenario was missed when converting the monitors to support
connectivity-based elections, and they're sending these messages to
quorum.begin(). Fix this by including an explicit leader in MMonProbe (that the
new monitor may reference in handle_probe_reply) and using the leader
value in both locations.

Fixes: https://tracker.ceph.com/issues/50345

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2021-04-13 23:29:39 +00:00
Gregory Farnum
05861cabb2
Merge pull request #40835 from gregsfortytwo/wip-stretch-mon-state
Fix issues with in-memory monitor stretch state

Reviewed-by: Sam Just <sjust@redhat.com>
2021-04-13 15:38:01 -07:00
Sage Weil
7a30e656b9 mgr/cephadm: make prometheus scrape ingress haproxy
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 18:22:31 -04:00
Sage Weil
d72c61b850 doc/cephadm: remove big warning about stability
It's the first item on the toctree that follows.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
5347726119 doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
bbf6a12752 mgr/cephadm: ingress: add optional virtual_interface_networks
It may be that the virtual IP we want to use is not in the same network
as any existing IPs on the host.  In that case, allow the spec to specify
a list of networks to match against existing IPs so that a match will
identify an ethernet interface to use.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
99b9f032de doc/cephadm/rgw: clean up example spec
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
f7977c551d mgr/cephadm/services/ingress: less verbose about prepare_create
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
702829f7df doc/cephadm/rgw: add note about which ethernet interface is used
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
2f33c6ebbc cephadm: make keepalived unit fiddle sysctl settings
No need to make the user adjust these manually.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
3f55c708b3 mgr/orchestrator: report external endpoints from 'orch ls'
Add a PORTS column and report the external/virtual IP (and port(s)) from
'orch ls' output.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
60562414e1 mgr/orchestrator: drop - when no ports
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
ef2d92aab2 doc/cephadm/rgw: update docs for ingress service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
6dcd0597bf mgr/cephadm: use per_host_daemon feature in scheduler
This only affects ingress, at least for now.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
db9f1930fe mgr/cephadm/schedule: add per_host_daemon_type support
This will be used to schedule a per-host keepalived alongside other
services.

Implement this as a final stage for place() that puts one per host and
also takes existing/stray daemons into consideration.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
0894773e95 mgr/cephadm: HA_RGW -> Ingress
This is mostly a rename, with some simplification and cleanup.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
d7b4a51a52 mgr/cephadm: include daemon_type in DaemonPlacement
Initially, this will always match the service_type.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
1897d1cd15 mgr/cephadm: update list-networks to report interface names too
Also, minor fix in the ipv6 addr reporting: ignore networks that aren't in CIDR
form (no /).

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
f93c555c24 mgr/orchestrator: streamline 'orch ps' PORTS formatting
"*:8000 *:8100" -> "*:8000,8100"

FWIW this matches the internal rendering used by DaemonPlacement

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
9256f1c374 mgr/cephadm/schedule: handle multiple ports per daemon
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
2e7808ccf8 mgr/cephadm/utils: resolve_ip(): prefer IPv4
On my system the first item in hte list is
'fe80::408d:35e7:510:e9fe%eno1np0'.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Patrick Donnelly
44f157742e
Merge PR #40833 into master
* refs/pull/40833/head:
	test/librados/tier_cxx: use non-deprecated wait_for_complete

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-13 13:26:43 -07:00
Samuel Just
50edcbcd81 test/librados/tier_cxx: use non-deprecated wait_for_complete
Fixes: https://tracker.ceph.com/issues/50342
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-04-13 18:18:17 +00:00
Yuval Lifshitz
8f57c508df
Merge pull request #40798 from yuvalif/wip-yuval-fix-49800
rgw/test: use 'localhost' for amqp ssl test
2021-04-13 19:00:49 +03:00
Casey Bodley
8cb94edcb1 qa/rgw: rgw/crypt uses supported-random-distro$
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-13 10:44:05 -04:00
Ilya Dryomov
7eb9c5ddb2 Merge branch 'master' into wip-unauthorized-gids
Sync up with master up to commit 3d8e73b266 ("Merge pull request
#40731 from tchaikov/wip-yamlize-options").  Specifically, bring in
src/common/options.cc yamlization and move new auth-related options
into src/common/options/global.yaml.in.

Conflicts:
	src/common/options.cc
	src/common/options/global.yaml.in

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-13 15:42:06 +02:00
Kefu Chai
ebed7e83c2
Merge pull request #40832 from wjwithagen/wjw-fix-y2c.py
common: make y2c.py work on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-13 21:14:44 +08:00
Willem Jan Withagen
1851621793 common: make y2c.py work on FreeBSD
1) make reference to python3 indepedant of explicit path
2) add required py-yaml module to install list

fixes: #40731
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-13 10:17:01 +02:00
Greg Farnum
adbaeeec95 mon: set_healthy_stretch_mode in update_from_paxos, not random leader calls!
Add header comment describing how this works now.

Fixes: https://tracker.ceph.com/issues/50308

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2021-04-13 06:39:10 +00:00
Greg Farnum
73b461a4f3 mon: maintain stretch_recovery_triggered in new OSDMon::set_*_stretch_mode
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2021-04-13 06:39:10 +00:00