Commit Graph

122022 Commits

Author SHA1 Message Date
Sage Weil
2f33c6ebbc cephadm: make keepalived unit fiddle sysctl settings
No need to make the user adjust these manually.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:53:08 -04:00
Sage Weil
3f55c708b3 mgr/orchestrator: report external endpoints from 'orch ls'
Add a PORTS column and report the external/virtual IP (and port(s)) from
'orch ls' output.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
60562414e1 mgr/orchestrator: drop - when no ports
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
ef2d92aab2 doc/cephadm/rgw: update docs for ingress service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
6dcd0597bf mgr/cephadm: use per_host_daemon feature in scheduler
This only affects ingress, at least for now.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
db9f1930fe mgr/cephadm/schedule: add per_host_daemon_type support
This will be used to schedule a per-host keepalived alongside other
services.

Implement this as a final stage for place() that puts one per host and
also takes existing/stray daemons into consideration.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
0894773e95 mgr/cephadm: HA_RGW -> Ingress
This is mostly a rename, with some simplification and cleanup.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
d7b4a51a52 mgr/cephadm: include daemon_type in DaemonPlacement
Initially, this will always match the service_type.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
1897d1cd15 mgr/cephadm: update list-networks to report interface names too
Also, minor fix in the ipv6 addr reporting: ignore networks that aren't in CIDR
form (no /).

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
f93c555c24 mgr/orchestrator: streamline 'orch ps' PORTS formatting
"*:8000 *:8100" -> "*:8000,8100"

FWIW this matches the internal rendering used by DaemonPlacement

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
9256f1c374 mgr/cephadm/schedule: handle multiple ports per daemon
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Sage Weil
2e7808ccf8 mgr/cephadm/utils: resolve_ip(): prefer IPv4
On my system the first item in hte list is
'fe80::408d:35e7:510:e9fe%eno1np0'.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 16:52:42 -04:00
Kefu Chai
ebed7e83c2
Merge pull request #40832 from wjwithagen/wjw-fix-y2c.py
common: make y2c.py work on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-13 21:14:44 +08:00
Willem Jan Withagen
1851621793 common: make y2c.py work on FreeBSD
1) make reference to python3 indepedant of explicit path
2) add required py-yaml module to install list

fixes: #40731
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-13 10:17:01 +02:00
Kefu Chai
3d8e73b266
Merge pull request #40731 from tchaikov/wip-yamlize-options
common: extract options into yaml

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-13 11:38:49 +08:00
Kefu Chai
f9e1c6bc23
Merge pull request #40739 from jdurgin/wip-alienstore-debug
crimson/os/alienstore: use bluestore debug prefix

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-13 11:06:53 +08:00
Sage Weil
dcd90a1c8d Merge PR #40626 into master
* refs/pull/40626/head:
	qa/suites/rados/objectstore: separate store_test tests
	qa/standalone: split osd/ into 2 directories

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-12 22:38:49 -04:00
Kefu Chai
bfdca7fa01 common/options: s/immutable-objet-cache/immutable-object-cache/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-13 09:15:19 +08:00
Kefu Chai
5ddda38da4 common: extract options into yaml
extract the options in common/options.cc into separate .yaml.in
files, and preprocess them using CMake before translating them into .cc
files using a python script.

this change paves the road to render the options using sphinx, and
will allow us to further annotate the options to include more metadata.

also, a this YAML file can be consumed by applications like dashboard
and Sphinx to consume these metadata in a simpler way.

* use @variable-name@ for substituting the variables in .yaml.in file
* use cmake variable of `mgr_disabled_modules` instead of C macro
  to define `mgr_disabled_modules` in global.yaml.in
* debian/control, ceph.spec.in, win32_deps_build.sh: add python3-yaml
  as build dep
* add y2c.py (short for YAML to C++) to translate .yaml to .cc file
* common/options/*.yaml.in: extract and split options into .yaml.in
  files, the subvars in it is then replaced with CMake variables,
  and copied to the corresponding .yaml files
* include/config-h.in.cmake: remove MGR_DISABLED_MODULES, as it
  is not a CMake variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-13 09:13:47 +08:00
Kefu Chai
3bee4b0261 cmake: use the same name for macros and cmake variables
for two reasons,

* consolidate the namings
* pave the road to yamlize options where we will use cmake variables
  to substitude the @<variable-name>@ in .in files instead of relying
  on C/C++ macros

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-13 09:13:46 +08:00
Kefu Chai
bed1b32974 cmake: introduce WITH_EC_ISA_PLUGIN
instead of checking "HAVE_NASM_X64_AVX2 OR HAVE_ARMV8_SIMD" everywhere,
use a single cached variable of WITH_EC_ISA_PLUGIN. so it's more
consistent when checking the availability of ec_isa plugin.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-13 09:13:46 +08:00
Sage Weil
d4ac4f3b78 qa/suites/rados/objectstore: separate store_test tests
This takes 5 hours currently.

- Separate out filestore and memstore into separate task (~1 hr)
- Split bluestore into -a and -b (a tests exclude SynethicMatrixC,
  b tests include it)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-12 16:27:29 -05:00
Ilya Dryomov
ca822b9a8d
Merge pull request #40576 from idryomov/wip-no-cephxv2-for-unmap
qa/suites/krbd: don't require CEPHX_V2 for unmap subsuite

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2021-04-12 21:23:06 +02:00
Yuval Lifshitz
a4a2b90032
Merge pull request #40800 from yuvalif/wip-yuval-fix-50291
rgw/amqp/test: fix mock prototype for librabbitmq-0.11.0
2021-04-12 22:19:10 +03:00
Patrick Donnelly
ca43c819c6
Merge PR #40651 into master
* refs/pull/40651/head:
	doc/cephadm: fix a typo

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-12 12:11:05 -07:00
Samuel Just
3f6f46d844
Merge pull request #40067 from myoungwon/wip-49726
osd: avoid for the two copy to cancel each other

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-12 11:50:06 -07:00
Sage Weil
e98923de9b Merge PR #40546 into master
* refs/pull/40546/head:
	SECURITY.md: Create SECURITY.md

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-12 12:36:48 -04:00
Sage Weil
3798cc6592 Merge PR #40726 into master
* refs/pull/40726/head:
	doc: Add GPG Keys

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-12 11:53:02 -04:00
Sage Weil
463bb3d2ef Merge PR #40736 into master
* refs/pull/40736/head:
	mgr/cephadm: rewrite/simplify describe_service
	mgr/orchestrator: report osds as osd.unmanaged as appropriate
	mgr/orchestrator: remove IMAGE ID from 'orch ls'

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-04-12 11:45:50 -04:00
Ernesto Puerta
648f9e11a6
Merge pull request #40721 from rhcs-dashboard/49925-fix-nfs-ganesha
mgr/dashboard: fix errors when creating NFS export.

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-04-12 17:33:52 +02:00
Sage Weil
0f65e5cffa qa/standalone: split osd/ into 2 directories
The whole osd/ directory takes 3 hours to run.  Of that, about half is
osd-backfill*:

2021-04-05T20:38:55.932 INFO:tasks.workunit:Running workunit osd/osd-backfill-prio.sh...
2021-04-05T20:47:27.184 INFO:tasks.workunit:Running workunit osd/osd-backfill-recovery-log.sh...
2021-04-05T20:55:59.497 INFO:tasks.workunit:Running workunit osd/osd-backfill-space.sh...
2021-04-05T21:48:47.549 INFO:tasks.workunit:Running workunit osd/osd-backfill-stats.sh...
2021-04-05T22:17:09.197 INFO:tasks.workunit:Running workunit osd/osd-bench.sh...

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-12 09:59:17 -05:00
Kefu Chai
2a7e54d5c4
Merge pull request #40814 from tchaikov/wip/ceph-object-corpus/pacific
ceph-object-corpus: pick up 16.2.0-90-g50f1821b4c

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-12 22:40:13 +08:00
J. Eric Ivancich
505b3f2108
Merge pull request #40801 from ivancich/wip-radoslist-incomplete-multipart-parts-marker
rgw: radoslist incomplete multipart parts marker

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-04-12 10:34:50 -04:00
Kefu Chai
13f3129f54 ceph-object-corpus: pick up 16.2.0-90-g50f1821b4c
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 21:49:28 +08:00
Kefu Chai
7b385894ba
Merge pull request #40806 from rhcs-dashboard/fix-mailmap-master
mailmap: sort alphabetically & add Pere and Waad

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 21:47:04 +08:00
Kefu Chai
6676cf92e1
Merge pull request #40656 from tchaikov/wip-qa-upgrade-focal
qa/suites: test upgrade/octopus-x on focal instead bionic

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-12 21:42:43 +08:00
Kefu Chai
1ee844f579
Merge pull request #40811 from tchaikov/wip-gen-corpus
script/gen-corpus.sh: set CEPH_CONF

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-12 21:40:29 +08:00
Hardik Vyas
b1d084d196
doc: Add GPG Keys
Replaced my GPG key with ceph.com and David's GPG keys

Signed-off-by: Hardik Vyas <hvyas@redhat.com>
2021-04-12 18:47:03 +05:30
Kefu Chai
00379cc418
Merge pull request #40795 from wjwithagen/wjw-fix-ceph-dencoder
tools: do not unload plugins during destruction.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 19:47:02 +08:00
Kefu Chai
5242487486 qa/workunits/cls: add executable bit to script
all the scripts except for test_cls_cas.sh under qa/workunits/cls
are executable. to be more consistent, add the executable bit to
test_cls_cas.sh as well.

also, these scripts are launched by src/script/gen-corpus.sh directly,
so it's convenient just call them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 19:28:47 +08:00
Kefu Chai
fdb4f83448 script/gen-corpus.sh: set CEPH_CONF
if we happen to run this script on a host where /etc/ceph/ceph.conf is
available, ceph CLI would use it instead. so, point it to $PWD/ceph.conf
instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 19:28:47 +08:00
Kefu Chai
6a1c173773
Merge pull request #40797 from wjwithagen/wjw-fix-monmap-retval
test: Undo the FreeBSD specific retval test

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 18:35:47 +08:00
Ernesto Puerta
73b8373899
mailmap: add Dashboard members: Waad and Pere
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-12 11:22:36 +02:00
Ernesto Puerta
ab458dc29c
mailmap: sort alphabetically
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-12 11:19:25 +02:00
Kefu Chai
8fd3607a0a
Merge pull request #40794 from wjwithagen/wjw-fix-promtool
test: Run Dockers only on Linux platforms

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 17:06:47 +08:00
Kefu Chai
caf364db5e
Merge pull request #38120 from kiizawa/wip-cls-remote-read
osd: allow remote read by calling cls method from within cls context

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-12 16:42:52 +08:00
Willem Jan Withagen
130dbc7725 test: Undo the FreeBSD specific retval test
Changes to the socket code now result in returning EINVAL
In the past ENOENT was returned which is the FreeBSD error code
if DNS lookup does not work.

And that change is probably because somewhere in the code that
errorcode is not passed verbatim from the systemcall, but is
rewritten in extra evaluation.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-12 10:36:28 +02:00
Willem Jan Withagen
78d301098f tools: do not unload plugins during destruction.
FreeBSD ceph-dencoder crashes in the exit() calls, due to
invalid pointer references during the release process of
the loaded libraries.

Often this is signaled by libc reporting:
  __cxa_thread_call_dtors: dtr 0x47efc0 from unloaded dso, skipping

The cause for this is different behaviour between FreeBSD and Linux:
https://groups.google.com/g/bsdmailinglist/c/22ncTZAbDp4/m/Dii_pII5AwAJ
_The FreeBSD implementation here looks racy. If one thread dlcloses an
object while another thread is exiting, we can end up calling a
function at an invalid memory address. It also looks as if it may
be possible to unload one library, load another at the same address,
and end up executing entirely the wrong code, which would have some
serious security implications.

The GNU/Linux equivalent of this function locks the DSO in memory
until all references to it have gone away. A call to dlclose() on
GNU/Linux will not actually unload the library until all threads
with destructors in that library have been unloaded. I believe
that this reuses the same reference counting mechanism that
allows the same library to be dlopened and dlclosed multiple times.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-12 10:27:02 +02:00
Kefu Chai
9d9eb283dd
Merge pull request #37016 from zhangdaolong/subcommon-bulefs-import
os/bluestore:Add subcommand bluefs-import in ceph-bluestore-tool.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 16:26:34 +08:00
Kefu Chai
dcdd790dd6
Merge pull request #40644 from SMIL-Infra/cleanup-slash
cephadm: cleanup extra slash in runtime dir

Reviewed-by: Adam King <adking@redhat.com>
2021-04-12 16:23:48 +08:00