Commit Graph

122117 Commits

Author SHA1 Message Date
Michael Fritch
e1023aa65e
doc/rgw: update to python3 print syntax
python3 requires parens for a `print` statement

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-04-15 15:38:16 -06:00
Mykola Golub
dc94eeddc1
Merge pull request #40718 from CongMinYin/fix-dead-lock-in-pwl
librbd/cache/pwl: Fix dead lock issue when pwl initialization failed

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-04-15 21:05:00 +03:00
Kefu Chai
f5bc82e076
Merge pull request #40875 from batrick/i50378
test: bump DecayCounter.steady acceptable error

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-16 00:25:57 +08:00
Kefu Chai
4394f75fbd
Merge pull request #40847 from tchaikov/wip-doc-build
admin/build-doc: use htmldir builder for livehtml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-15 23:06:15 +08:00
Patrick Donnelly
2293099bdf
Merge PR #40845 into master
* refs/pull/40845/head:
	systemd: remove `ProtectClock=true` for `ceph-osd@.service`

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-15 07:30:11 -07:00
Kefu Chai
e3459154f6
Merge pull request #40843 from tchaikov/wip-cmake-rgw-civetweb.h
cmake: update civetweb.h on demand

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-04-15 22:08:43 +08:00
Patrick Donnelly
9a369ef119
test: bump DecayCounter.steady acceptable error
The test is often run on heavily loaded machines which will effect the
precision of the test.

Fixes: https://tracker.ceph.com/issues/50378
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-15 06:58:44 -07:00
Kefu Chai
5df2530fc6
Merge pull request #40873 from tchaikov/wip-50374
vstart.sh: disable "auth_allow_insecure_global_id_reclaim"

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-15 21:33:12 +08:00
Kefu Chai
c945f59d78
Merge pull request #40867 from tchaikov/wip-50339
test/cls_cas: allow multi hobjects tracked by cls_cas

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Sam Just <sjust@redhat.com>
2021-04-15 21:24:38 +08:00
Kefu Chai
77a8376d07 vstart.sh: disable "auth_allow_insecure_global_id_reclaim"
to silence the health warning of "mons are allowing insecure global_id
reclaim", which prevents the cluster from being active+clean. couple
tests are expecting a warning free cluster before they starts.

as this option is enabled by default for appeasing the old clients, but when it
comes to most of upstream testing, we can just disable it.

Fixes: https://tracker.ceph.com/issues/50374
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-15 21:10:36 +08:00
Ilya Dryomov
11b72cce08
Merge pull request #40196 from ceph/wip-15210-notes
doc/releases/octopus.rst: 15.2.10 Release Notes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-15 13:36:22 +02:00
Ilya Dryomov
2e445fc888 doc/releases/releases: add 15.2.10 release
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-15 13:22:22 +02:00
David Galloway
aa4a53a3c1 doc/releases/octopus.rst: 15.2.10 Release Notes
Signed-off-by: David Galloway <dgallowa@redhat.com>
2021-04-15 12:57:44 +02:00
Kefu Chai
117838c91d
Merge pull request #40866 from hualongfeng/pmdk_in_dist
make-dist: add pmdk to dist tarball

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-15 16:43:28 +08:00
Kefu Chai
972bb2e1e6 test/cls_cas: allow multi hobjects tracked by cls_cas
in d2737fd41a, we started to use multiset
for tracking the references of hobject for snapshot support. as the same
hobject maps to multiple snapshots. and we don't want to consider
different snapshots as the same entry tracked by cls_cas.

but cls_cas.dup_get() still tries to verify that the `get` operation
is able to dedup the same referenced "source". but this does not apply
to "by_object" trunk ref type anymore.

since we cannot check/choose the chunk ref type used by OSD from the
client of the cls_cas, in this change, cls_cas.dup_get() is updated
to adapt the change solely for "by_object". otherwise we could skip
this test for "by_object" type and/or define another test for other
chunk ref types.

Fixes: https://tracker.ceph.com/issues/50339
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-15 11:36:43 +08:00
Feng Hualong
14c2a2e59f cmake: use src/pmdk for building pmdk if it exists
so we can build with pmdk enabled if the dist tarball
contains pmdk

Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-04-15 10:40:06 +08:00
Feng Hualong
9d958d0b9d make-dist: add pmdk to dist tarball
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-04-15 10:40:06 +08:00
J. Eric Ivancich
6710f28746
Merge pull request #40862 from ivancich/wip-rgw-reshard-contention-logging
rgw: during reshard lock contention, adjust logging

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-04-14 16:13:17 -04:00
Casey Bodley
dbede38c7e
Merge pull request #40821 from cbodley/wip-50218
qa/rgw: use symlinks to specify distro

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-04-14 14:45:00 -04:00
J. Eric Ivancich
6d3dee3779 rgw: during reshard lock contention, adjust logging
When RGW fails to get a lock on a reshard log, we log it in such a way
that it looks like an error. Instead we'll make sure that the log
message is informational.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-04-14 14:18:30 -04:00
Ilya Dryomov
f3a4166379 Merge branch 'CVE-2021-20288' into master
* CVE-2021-20288:
  qa/standalone: default to disable insecure global id reclaim
  qa/suites/upgrade/octopus-x: disable insecure global_id reclaim health warnings
  qa/tasks/ceph[adm].conf[.template]: disable insecure global_id reclaim health alerts
  cephadm: set auth_allow_insecure_global_id_reclaim for mon on bootstrap
  mon/HealthMonitor: raise AUTH_INSECURE_GLOBAL_ID_RENEWAL[_ALLOWED]
  auth/cephx: ignore CEPH_ENTITY_TYPE_AUTH in requested keys
  auth/cephx: rotate auth tickets less often
  mon: fail fast when unauthorized global_id (re)use is disallowed
  auth/cephx: option to disallow unauthorized global_id (re)use
  auth/cephx: make cephx_decode_ticket() take a const ticket_blob
  auth/AuthServiceHandler: keep track of global_id and whether it is new
  auth/AuthServiceHandler: build_cephx_response_header() is cephx-specific
  auth/AuthServiceHandler: drop unused start_session() args
  mon/MonClient: drop global_id arg from _add_conn() and _add_conns()
  mon/MonClient: reset auth state in shutdown()
  mon/MonClient: preserve auth state on reconnects
  mon/MonClient: claim active_con's auth explicitly
  mon/MonClient: resurrect "waiting for monmap|config" timeouts

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-14 19:58:49 +02:00
Kefu Chai
4b6a3f8161 doc/_ext: extract confval into an extension
better readability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-15 00:37:45 +08:00
Kefu Chai
4968baa252 admin/doc-requirements.txt: pin sphinx and sphinx_rtd_theme to a certain version
to ensure that we can have reproducible builds

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 23:24:38 +08:00
Kefu Chai
cbc7f72784
Merge pull request #40614 from smithfarm/wip-39556
make-dist: refuse to run if script path contains a colon

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 22:41:35 +08:00
Gregory Farnum
ef6f36075e
Merge pull request #40839 from gregsfortytwo/wip-mon-quorum-leader
mon: MMonProbe: direct MMonJoin messages to the leader, instead of the first mon

Reviewed-by: Sam Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-14 07:41:05 -07:00
Kefu Chai
ba68020cea
Merge pull request #40807 from tchaikov/wip-common-include
common/Formatter: include used header

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-14 22:39:45 +08:00
Kefu Chai
c7cca2b86e
Merge pull request #40830 from gregsfortytwo/wip-sqlite3-build
mgr: do not require sqlite3.h if WITH_LIBCEPHSQLITE=OFF

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 22:38:55 +08:00
Kefu Chai
9b1c3e24f6
Merge pull request #40840 from tchaikov/wip-silence-gcc-warnings
tools: silence gcc warnings

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 22:35:09 +08:00
Wong Hoi Sing Edison
85bc551b17
systemd: remove ProtectClock=true for ceph-osd@.service
Ceph 16.2.0 Pacific by https://github.com/ceph/ceph/commit/9a84d5a introduce following new systemd restriction:

    ProtectClock=true
    ProtectHostname=true
    ProtectKernelLogs=true
    RestrictSUIDSGID=true

BTW, `ceph-osd@.service` failed with `ProtectClock=true` unexpectly, also see:

  - <https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/TNBGGNN6STGDKARAQTQCIPTU4KLIVJQV/>
  - <https://serverfault.com/questions/1059317/bluestore-var-lib-ceph-osd-ceph-2-block-read-bdev-label-failed-to-open-var-l>

This PR intruduce:

  - Remove `ProtectClock=true` for our systemd service templates

Fixes: https://tracker.ceph.com/issues/50347
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
2021-04-14 22:19:49 +08:00
Sage Weil
ff97629375 Merge PR #40735 into master
* refs/pull/40735/head:
	mgr/orchestractor: rgw realm and zone flags must both be provided

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-14 09:53:31 -04:00
Sage Weil
8cc5dc75d7 Merge PR #40734 into master
* refs/pull/40734/head:
	mgr/cephadm: make prometheus scrape ingress haproxy
	doc/cephadm: remove big warning about stability
	doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
	mgr/cephadm: ingress: add optional virtual_interface_networks
	doc/cephadm/rgw: clean up example spec
	mgr/cephadm/services/ingress: less verbose about prepare_create
	doc/cephadm/rgw: add note about which ethernet interface is used
	cephadm: make keepalived unit fiddle sysctl settings
	mgr/orchestrator: report external endpoints from 'orch ls'
	mgr/orchestrator: drop - when no ports
	doc/cephadm/rgw: update docs for ingress service
	mgr/cephadm: use per_host_daemon feature in scheduler
	mgr/cephadm/schedule: add per_host_daemon_type support
	mgr/cephadm: HA_RGW -> Ingress
	mgr/cephadm: include daemon_type in DaemonPlacement
	mgr/cephadm: update list-networks to report interface names too
	mgr/orchestrator: streamline 'orch ps' PORTS formatting
	mgr/cephadm/schedule: handle multiple ports per daemon
	mgr/cephadm/utils: resolve_ip(): prefer IPv4

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-14 09:39:38 -04:00
Kefu Chai
d867d07890 doc/_ext: claim ceph_releses ext to be parallel read safe
so we can pass `-j auto` to sphinx-build to speed up the build

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 18:41:38 +08:00
Kefu Chai
c145444b91 include/rados: document parameters
less warnings from sphinx

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 18:41:38 +08:00
Kefu Chai
f4b0742a35 admin/build-doc: use htmldir builder for livehtml
otherwise livehtml uses the default html builder. so we can copy and
paste the partial URL to cross check the rendering result.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 18:41:38 +08:00
Kefu Chai
e09f8d7dd0
Merge pull request #40149 from tchaikov/wip-cmake-job-pool
cmake: use ninja job pool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-14 15:06:50 +08:00
Kefu Chai
061affa38e
Merge pull request #40785 from tchaikov/wip-doc-list-spacing
doc/_themes: remove spacing after `ul li p`

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-14 12:42:31 +08:00
Kefu Chai
5fa591a765 cmake: update civetweb.h on demand
instead of using a dedicated target for updating civetweb.h, use
add_custom_command() with OUTPUT option.

so the generated rule does not rerun the command specified by the
civetweb_h target every time we build the targets depending on it.

use the header file as the dependency helps to improve the readability
as there is one less link in the dependency chain, and by specifying the
OUTPUT, cmake is able to figure out the dependency on the header file so
it does not try to regenerate it every time.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 12:17:50 +08:00
Kefu Chai
e2875647b8 tools/osdmaptool: mark unused variable [[maybe_unused]]
to silence warning from GCC when performing release build, like:

../src/tools/osdmaptool.cc: In function ‘int main(int, const char**)’:
../src/tools/osdmaptool.cc:472:9: warning: variable ‘r’ set but not used [-Wunused-but-set-variable]
  472 |     int r = clock_gettime(CLOCK_MONOTONIC, &round_start);
      |         ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 11:11:08 +08:00
Kefu Chai
25ce28a470 doc/radosgw: fix formatting of command line block
should add an empty line after the directive.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 09:33:37 +08:00
Kefu Chai
6dea67ac2d admin: require sphinx>=3.2.1
loosen the required sphinx version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 09:03:03 +08:00
Kefu Chai
aa66a4950b doc/_themes: remove spacing after ul li p
in the latest document generated from RtD, the spacing after `ul li p`
elements is set to 24px as the plain `p` elements. but this the lists
more sparse and difficult to read.

in this change, the spacing is restored to 0 as it was in old theme.css
in sphinx_rtd_theme.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-14 09:03:03 +08:00
Greg Farnum
ffa7ff35b4 mon: MMonProbe: direct MMonJoin messages to the leader, instead of the first mon
When monitors are joining a cluster, they may send an MMonJoin message to place
themselves correctly in the map in either handle_probe_reply() or
finish_election(). These messages must be sent to the leader -- monitors do not
forward each other's messages.

Unfortunately, this scenario was missed when converting the monitors to support
connectivity-based elections, and they're sending these messages to
quorum.begin(). Fix this by including an explicit leader in MMonProbe (that the
new monitor may reference in handle_probe_reply) and using the leader
value in both locations.

Fixes: https://tracker.ceph.com/issues/50345

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2021-04-13 23:29:39 +00:00
Gregory Farnum
05861cabb2
Merge pull request #40835 from gregsfortytwo/wip-stretch-mon-state
Fix issues with in-memory monitor stretch state

Reviewed-by: Sam Just <sjust@redhat.com>
2021-04-13 15:38:01 -07:00
Sage Weil
7a30e656b9 mgr/cephadm: make prometheus scrape ingress haproxy
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 18:22:31 -04:00
Sage Weil
d72c61b850 doc/cephadm: remove big warning about stability
It's the first item on the toctree that follows.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
5347726119 doc/cepham/compatibility: rgw-ha -> ingress; note possibility of breaking changes
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
bbf6a12752 mgr/cephadm: ingress: add optional virtual_interface_networks
It may be that the virtual IP we want to use is not in the same network
as any existing IPs on the host.  In that case, allow the spec to specify
a list of networks to match against existing IPs so that a match will
identify an ethernet interface to use.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
99b9f032de doc/cephadm/rgw: clean up example spec
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
f7977c551d mgr/cephadm/services/ingress: less verbose about prepare_create
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
702829f7df doc/cephadm/rgw: add note about which ethernet interface is used
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00