Commit Graph

122171 Commits

Author SHA1 Message Date
Daniel Pivonka
c0803f8f27 mgr/orchestractor: rgw realm and zone flags must both be provided
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
2021-04-12 10:19:53 -04:00
Kefu Chai
13f3129f54 ceph-object-corpus: pick up 16.2.0-90-g50f1821b4c
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 21:49:28 +08:00
Kefu Chai
7b385894ba
Merge pull request #40806 from rhcs-dashboard/fix-mailmap-master
mailmap: sort alphabetically & add Pere and Waad

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 21:47:04 +08:00
Kefu Chai
6676cf92e1
Merge pull request #40656 from tchaikov/wip-qa-upgrade-focal
qa/suites: test upgrade/octopus-x on focal instead bionic

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-12 21:42:43 +08:00
Kefu Chai
1ee844f579
Merge pull request #40811 from tchaikov/wip-gen-corpus
script/gen-corpus.sh: set CEPH_CONF

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-12 21:40:29 +08:00
Hardik Vyas
b1d084d196
doc: Add GPG Keys
Replaced my GPG key with ceph.com and David's GPG keys

Signed-off-by: Hardik Vyas <hvyas@redhat.com>
2021-04-12 18:47:03 +05:30
Kefu Chai
00379cc418
Merge pull request #40795 from wjwithagen/wjw-fix-ceph-dencoder
tools: do not unload plugins during destruction.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 19:47:02 +08:00
Kefu Chai
5242487486 qa/workunits/cls: add executable bit to script
all the scripts except for test_cls_cas.sh under qa/workunits/cls
are executable. to be more consistent, add the executable bit to
test_cls_cas.sh as well.

also, these scripts are launched by src/script/gen-corpus.sh directly,
so it's convenient just call them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 19:28:47 +08:00
Kefu Chai
fdb4f83448 script/gen-corpus.sh: set CEPH_CONF
if we happen to run this script on a host where /etc/ceph/ceph.conf is
available, ceph CLI would use it instead. so, point it to $PWD/ceph.conf
instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 19:28:47 +08:00
Kefu Chai
6a1c173773
Merge pull request #40797 from wjwithagen/wjw-fix-monmap-retval
test: Undo the FreeBSD specific retval test

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 18:35:47 +08:00
Kefu Chai
03770f7556 include/denc: include used header
to address the FTBFS like:

/home/kchai/ceph/src/include/denc.h: In member function ‘void DencDumper<T>::dump() const’:
/home/kchai/ceph/src/include/denc.h:121:60: error: ‘O_BINARY’ was not declared in this scope
     int fd = ::open(fn, O_WRONLY|O_TRUNC|O_CREAT|O_CLOEXEC|O_BINARY, 0644);
                                                            ^~~~~~~~
/home/kchai/ceph/src/include/denc.h:121:60: note: the macro ‘O_BINARY’ had not yet been defined
In file included from /home/kchai/ceph/src/include/statlite.h:14,
                 from /home/kchai/ceph/src/include/types.h:41,
                 from /home/kchai/ceph/src/auth/Crypto.h:19,
                 from /home/kchai/ceph/src/auth/Crypto.cc:21:

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 17:43:14 +08:00
Kefu Chai
7c2f58ea5e mon,osd: always init local variable
this change silences the warning like:

../src/mon/Monitor.cc: In member function ‘void Monitor::handle_command(MonOpRequestRef)’:
../src/mon/Monitor.cc:3703:55: warning: ‘osd’ may be used uninitialized in this function [-Wmaybe-uninitialized]
 3703 |     uint64_t seq = mgrstatmon()->get_last_osd_stat_seq(osd);
      |                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 17:35:34 +08:00
Kefu Chai
ff79cae5a1 common/Formatter: include used header
otherwise it fails to build with gcc-toolset-10, like:

../src/common/Formatter.cc: In member function ‘virtual void ceph::XMLFormatter::close_section()’:
../src/common/Formatter.cc:449:8: error: ‘transform’ is not a member of ‘std’
  449 |   std::transform(section.begin(), section.end(), section.begin(),
      |        ^~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 17:29:49 +08:00
Ernesto Puerta
73b8373899
mailmap: add Dashboard members: Waad and Pere
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-12 11:22:36 +02:00
Ernesto Puerta
ab458dc29c
mailmap: sort alphabetically
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-12 11:19:25 +02:00
Kefu Chai
8fd3607a0a
Merge pull request #40794 from wjwithagen/wjw-fix-promtool
test: Run Dockers only on Linux platforms

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 17:06:47 +08:00
Kefu Chai
caf364db5e
Merge pull request #38120 from kiizawa/wip-cls-remote-read
osd: allow remote read by calling cls method from within cls context

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-12 16:42:52 +08:00
Willem Jan Withagen
130dbc7725 test: Undo the FreeBSD specific retval test
Changes to the socket code now result in returning EINVAL
In the past ENOENT was returned which is the FreeBSD error code
if DNS lookup does not work.

And that change is probably because somewhere in the code that
errorcode is not passed verbatim from the systemcall, but is
rewritten in extra evaluation.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-12 10:36:28 +02:00
Willem Jan Withagen
78d301098f tools: do not unload plugins during destruction.
FreeBSD ceph-dencoder crashes in the exit() calls, due to
invalid pointer references during the release process of
the loaded libraries.

Often this is signaled by libc reporting:
  __cxa_thread_call_dtors: dtr 0x47efc0 from unloaded dso, skipping

The cause for this is different behaviour between FreeBSD and Linux:
https://groups.google.com/g/bsdmailinglist/c/22ncTZAbDp4/m/Dii_pII5AwAJ
_The FreeBSD implementation here looks racy. If one thread dlcloses an
object while another thread is exiting, we can end up calling a
function at an invalid memory address. It also looks as if it may
be possible to unload one library, load another at the same address,
and end up executing entirely the wrong code, which would have some
serious security implications.

The GNU/Linux equivalent of this function locks the DSO in memory
until all references to it have gone away. A call to dlclose() on
GNU/Linux will not actually unload the library until all threads
with destructors in that library have been unloaded. I believe
that this reuses the same reference counting mechanism that
allows the same library to be dlopened and dlclosed multiple times.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-12 10:27:02 +02:00
Kefu Chai
9d9eb283dd
Merge pull request #37016 from zhangdaolong/subcommon-bulefs-import
os/bluestore:Add subcommand bluefs-import in ceph-bluestore-tool.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-12 16:26:34 +08:00
Kefu Chai
dcdd790dd6
Merge pull request #40644 from SMIL-Infra/cleanup-slash
cephadm: cleanup extra slash in runtime dir

Reviewed-by: Adam King <adking@redhat.com>
2021-04-12 16:23:48 +08:00
Kefu Chai
7a158fcb2b
Merge pull request #40658 from tchaikov/wip-systemd
cmake: s/HAVE_MSGHDR/WITH_SYSTEMD/

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-12 16:09:33 +08:00
Nathan Cutler
cb520aa914 make-dist: refuse to run if script path contains a colon
Colons are not permissible in POSIX paths and their presence is known to break
the script.

Fixes: https://tracker.ceph.com/issues/39556
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-04-12 09:50:51 +02:00
Yuval Lifshitz
2ba598ec4c rgw/amqp/test: fix mock prototype for librabbitmq-0.11.0
also use extern C for to get compilation errors when
function prototype change

Fixes: https://tracker.ceph.com/issues/50291

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-04-12 09:11:05 +03:00
pcuzner
fd63c45a97
Merge pull request #40635 from pcuzner/prometheus_add_pool_metadata
mgr/prometheus:Improve the pool metadata
2021-04-12 12:02:03 +12:00
Yuval Lifshitz
91d20a48b9
Merge pull request #39944 from yuvalif/wip-yuval-fix-49650
rgw/notifications: delete bucket notification object when empty
2021-04-11 21:52:06 +03:00
Willem Jan Withagen
2386023ca8 test: Run Dockers only on Linux platforms
Running a docker alternative only works if the platform
   is Linux

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-04-11 20:44:24 +02:00
Kefu Chai
c4e8185b52
Merge pull request #40786 from tchaikov/wip-script-bit
build-integration-branch: retry when running into network failures

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-12 00:43:46 +08:00
Yuval Lifshitz
0fcb3372b4 rgw/test: use 'localhost' for amqp ssl test
also move the amqp ssl tests to 'test_bn.py'
this fix combines commit: 1418bcc1dc
with commit: 979335f60a

Fixes: https://tracker.ceph.com/issues/49800

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-04-11 18:16:10 +03:00
Kefu Chai
34e4e4870e build-integration-branch: retry when running into network failures
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-11 22:28:05 +08:00
zhangdaolong
39118421dc os/bluestore/bluestore_tool: Add subcommand blufs-import
Examples
 ceph-bluestore-tool bluefs-import --path /var/lib/ceph/osd/ceph-1 --input-file ./db/CURRENT --dest-file db/CURRENT

Signed-off-by: zhang daolong <zhangdaolong@fiberhome.com>
2021-04-11 17:10:48 +08:00
Kefu Chai
83eb7a3dff build-integration-branch: define variables for pr_number and friends
so they can be reused later.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-11 12:00:07 +08:00
Sage Weil
58d9e90425 mgr/cephadm: rewrite/simplify describe_service
The prior implementation first tried to fabricate services based on the
running daemons, and then filled in defined services on top.  This led
to duplication and a range of small errors.

Instead, flip this around: start with the services that are defined,
and only fill in 'unmanaged' services where we need to.

Drop the osd kludges and instead rely on DaemonDescription.service_id to
return the right thing.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-10 09:12:23 -04:00
Sage Weil
d9497684f7 Merge PR #40577 into master
* refs/pull/40577/head:
	cephadm: normalize unqualified repo digests to docker.io
	mgr/cephadm/upgrade: normalize unqualified target image

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-04-10 09:01:58 -04:00
Sage Weil
542b81b8f1 Merge PR #40537 into master
* refs/pull/40537/head:
	cephadm:persist the grafana.db file

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-10 09:01:27 -04:00
J. Eric Ivancich
d76e65bcd3 rgw: test radosgw-admin radoslist and incomplete multiparts better
Make sure there are more than 1000 incomplete multiparts and also make
sure one of the incomplete multiparts has at least 1000 parts. This
test is done indirectly through rgw-orphan-list, which invokes
`radosgw-admin radoslist`.

Also, clean up shell flags, so script output is less verbose.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-04-10 02:07:53 -04:00
Patrick Donnelly
4803e9f5d5
Merge PR #40653 into master
* refs/pull/40653/head:
	mon: check mdsmap is resizeable before promoting standby-replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-09 20:08:51 -07:00
Patrick Donnelly
3e849e7349
Merge PR #40642 into master
* refs/pull/40642/head:
	client: don't allow access to MDS-private inodes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-04-09 20:06:21 -07:00
Patrick Donnelly
4d29e591dd
Merge PR #40481 into master
* refs/pull/40481/head:
	qa: test standby-replay with fs:workloads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-09 20:04:32 -07:00
Patrick Donnelly
76ac006875
Merge PR #40431 into master
* refs/pull/40431/head:
	qa/cephfs: remove create_keyring_file from cephfs_test_case.py
	qa/cephfs: don't use sudo to write files in /tmp

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-04-09 20:02:27 -07:00
Patrick Donnelly
d302b72691
Merge PR #40389 into master
* refs/pull/40389/head:
	mds: reject lookup ino requests for mds dirs
	test: add test for invalid lookup of mdsdir

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-04-09 19:58:40 -07:00
靳登科
b3ed88ef61 mgr/volumes: remove unused semicolon
Signed-off-by: jindengke <jindengke@inspur.com>
2021-04-10 09:04:47 +08:00
Neha Ojha
b4f291a6b4
Merge pull request #40723 from zdover23/wip-doc-second-attempt-mclock-rewrite-second-half-2021-Apr-09
doc/rados: rewrite mclock docs (2 of 2)

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-09 17:24:41 -07:00
Josh Durgin
91104375cd
Merge pull request #40738 from jdurgin/wip-librados-docs
include/librados: fix doxygen syntax for docs build

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-09 16:48:13 -07:00
Josh Durgin
a381c7dc56 crimson/os/alienstore: use bluestore debug prefix
Filestore is never accurate. Since we only intend to use bluestore
with alien mode, it's not worth introducing a separate debug
subsystem.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-09 18:15:43 -04:00
Josh Durgin
70b8f16a2c include/librados: fix doxygen syntax for docs build
The docs build is now warning about these like:

WARNING: Unparseable C cross-reference: '[in]'
Invalid C declaration: Expected identifier in nested name.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-09 18:01:35 -04:00
Sage Weil
e07a738304 cephadm: normalize unqualified repo digests to docker.io
A RepoDigests returned by docker|podman image inspect can either include
the docker.io/ prefix or not.  For reasons that aren't entirely clear,
this may vary between hosts in a cluster.  However, ceph/ceph@sha256:abc...
is the same thing as docker.io/ceph/ceph@sha256:abc..., and should be
treated as such.  Otherwise, upgrade can get into a loop where it pulls
the image on a new host, finds the other variant of the repodigests,
sees no overlap, updates target_digests, and restarts.  (It will then
find the first variant again on the first host and loop.)

Avoid this by normalizing any docker.io digests by always including the
docker.io/ prefix.

Note that it is technically possible that this assumption is wrong: it
may be that the image that already exists on the local host is from a
different registry in registries.conf's unqualified-search-registries.
However, we don't know which, since this is a search list.  In practice,
it should be exceeding rare that an image that *we* are installing using
a fully-qualified image name will end up having an unqualified repodigest
in the local registry.  Hopefully!

Fixes: https://tracker.ceph.com/issues/50114
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-09 16:53:37 -04:00
Sage Weil
38f84520ee mgr/cephadm/upgrade: normalize unqualified target image
If we get an unqualified target image, assume it's docker.io.  This
ensures that we're passing a fully-qualified target to docker|podman on
the various hosts and don't end up with something different based on the
per-host search path for unqualified image names.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-09 16:53:37 -04:00
Zac Dover
f3180ad722 doc/rados: rewrite mclock docs (2 of 2)
This is my second attempt to rewrite the
second half of the mclock docs. The first attempt
is enshrined in https://github.com/ceph/ceph/pull/40571,
in which I got cute with git and got burned.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-04-10 06:49:43 +10:00
Sage Weil
5adef5f766 mgr/orchestrator: report osds as osd.unmanaged as appropriate
If there is no osdspec_affinity or service_name (from unit.meta), then
report as 'osd.unmanaged'.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-09 16:23:20 -04:00