Commit Graph

119016 Commits

Author SHA1 Message Date
Lucian Petrut
5c008725db cmake,win32*.sh: Windows DLL support
At the moment, the Windows binaries can only be linked statically.
This is less than ideal since:

* the resulting binaries can be quite large, especially when
including debug symbols
* librados and librbd cannot be used directly by 3rd party apps
* the build duration is increased

In order to do a dynamic build, we'll:

* add an option to win32_build.sh
* fix link order
* dynamically link boost
* disable the "-pie" flag when using Mingw, which generates incorrect
  executable entry points
* by default, cmake generates import libs for executables. The issue
  is that for rados.exe, it generates librados.dll.a, thus overriding
  the librados.dll import library, which breaks the build process.
  We'll configure cmake to skip generating import libs for executables.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-01-26 08:41:51 +00:00
Kefu Chai
f16ac13c13 rbd/bench: include used headers
should help with fixing following failure:

src/tools/rbd/action/Bench.cc:20:26: error: aggregate 'std::atomic<bool>
terminating' has incomplete type and cannot be defined

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-26 16:09:28 +08:00
Matt Benjamin
d3a78fa196
Merge pull request #37250 from cbodley/wip-qa-librgw
qa/rgw: add librgw_file unit tests to rgw/verify task
2021-01-25 22:10:37 -05:00
Harish Munjulur
46dee40a29
Merge pull request #38852 from adamemerson/wip-thou-shalt-not-unwind
Thou Shalt Not Unwind
2021-01-25 15:22:38 -08:00
Harish Munjulur
d358f62397
Merge pull request #38905 from cbodley/wip-48874
rgw: fix trailing null in object names of multipart reuploads
2021-01-25 15:17:05 -08:00
Neha Ojha
6533027912
Merge pull request #39063 from yuriw/wip-yuriw-crontab-master
qa/tests: added pacific branch to the mix

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-25 14:11:53 -08:00
Casey Bodley
61de548c42
Merge pull request #39009 from mdw-at-linuxbox/wip-master-keystonebindep
qa/tasks/keystone.py: centos-8 bindep fix

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-01-25 17:04:36 -05:00
Neha Ojha
962127575e
Merge pull request #38962 from aclamk/wip-aclamk-bluestore-tool-reshard-fix
Fix ceph-bluestore-tool reshard command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-25 14:03:38 -08:00
Neha Ojha
bbd504eb85
Merge pull request #38965 from aclamk/wip-aclamk-bluestore-tool-sharding-show
Add command 'show-sharding' to ceph-bluestore-tool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-01-25 14:02:32 -08:00
Matt Benjamin
bf8f32600e test/rgw_file: bypass one bad-deletes test due to intermittent failure
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-01-25 16:46:58 -05:00
Matt Benjamin
9bac99361f test/rgw_file: override keyring if we gen keys
Run with ceph.keyring not that of client.${CEPH_ID}.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-01-25 16:46:50 -05:00
Matt Benjamin
247d00817d test/rgw_file: elaborate test cycle
Ensure that all delete phases are run so that script can be
re-run when desired.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
 Please enter the commit message for your changes. Lines starting
2021-01-25 16:46:41 -05:00
Matt Benjamin
a50ff83640 rgw_file: delete test bucket when do_delete
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-01-25 16:46:33 -05:00
Matt Benjamin
e6d54e7d79 rgw_file: add librgw unit test arguments
Also adjust some unit test files to avoid trivial failures.
Fixes missing commit after rgw_write(), as well as conditional
cleanup.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-01-25 16:46:25 -05:00
Casey Bodley
12b5b9506c qa/rgw: librgw workunit creates a test user
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-01-25 16:46:18 -05:00
Casey Bodley
cefd31b6f6 cmake: install the ceph_test_librgw_file_* targets
these need to be installed in order to be included in packages for
testing in teuthology

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-01-25 16:46:11 -05:00
Casey Bodley
85641ac3e6 qa/rgw: add librgw_file unit tests to rgw/verify task
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-01-25 16:46:04 -05:00
Ilya Dryomov
65948736a4 qa/suites/krbd: add msgr2 modes to most subsuites
basic, rbd and rbd-nomount subsuites are expanded to run with each
of ms_mode=legacy, ms_mode=crc and ms_mode=secure.  This increases
the total number of jobs in the suite from 100 to 220.

fsx, singleton and thrash subsuites choose ms_mode at random (from
the above plus ms_mode=prefer-crc).

unmap and wac subsuites remain msgr1-only.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-25 21:17:44 +01:00
Ilya Dryomov
fd5f5722a2 doc: deprecate [no]cephx_require_signatures map options
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-25 21:17:44 +01:00
Ilya Dryomov
08f714964b krbd: add support for msgr2
Recognize ms_mode map option and filter initial monitor addresses
accordingly: if ms_mode is not given or ms_mode=legacy, discard v2
addresses, otherwise discard v1 addresses.

Note that nothing was discarded (i.e. v2 addresses were passed to
the kernel) previously.  The intent was to preserve that behaviour
in case ms_mode is not given, allowing to change the kernel default
in the future.  However, it turns out that mount.ceph helper has
been misguidedly discarding v2 addresses since commit eae0127513
("mount.ceph: fork a child to get info from local configuration"),
so that ship has sailed.

Fixes: https://tracker.ceph.com/issues/48976
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-25 21:17:44 +01:00
Harish Munjulur
80bf477285
Merge pull request #37694 from BryceCao/wip-add-MD5-in-forward_request
rgw : add MD5 in forward_request
2021-01-25 11:50:45 -08:00
Harish Munjulur
29c9eb32e9
Merge pull request #38977 from adamemerson/wip-48929
rgw: Fix spurious error on empty datalog shard
2021-01-25 11:48:56 -08:00
Neha Ojha
71a650858a
Merge pull request #38958 from badone/wip-osd-df-tree-accounting-error
An empty bucket or OSD is not an error

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-25 10:17:19 -08:00
Neha Ojha
478ef57393
Merge pull request #38992 from ifed01/wip-ifed-fix-48776-2
os/bluestore: be more protective in _deferred_queue

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-25 10:10:15 -08:00
Ilya Dryomov
ed3913f02c doc: don't mention osdtimeout option in mount.ceph man page
It was deprecated over 8 years ago and will be entirely removed in
kernel 5.12.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-25 18:44:56 +01:00
Kefu Chai
7a5d0ecb1d
Merge pull request #39051 from tchaikov/wip-install-deps-libzbd
install-deps.sh: install libzbd-dev when WITH_ZBD is set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-26 01:00:52 +08:00
Yuri Weinstein
dc07a51d4b qa/tests: added pacific branch to the mix
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-25 08:40:31 -08:00
Sage Weil
963f54aebf cephadm: retry for 30s instead of 10s
No real reason not to wait longer, and I'm worried about things not coming
up quickly enough on slower or loady machines.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-25 08:48:44 -06:00
Sage Weil
c1cee0a952 cephadm: try 'mgr stat' before 'mgr dump' to get the epoch
This is less noise in the debug log.  (Also, there seems to be a problem
when the mgr stat output gets too big.)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-25 08:48:44 -06:00
Sage Weil
bb702eda19 mon: add 'mgr stat' command
This is analogous to 'osd stat' and 'mds stat'.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-25 08:48:44 -06:00
Kefu Chai
e7e3b86762 install-deps.sh: install libzbd-dev when WITH_ZBD is set
to avoid zbd backend bitrot, and for build test it in our "make check"
builds.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-25 14:13:15 +08:00
Kefu Chai
141969d7c4 pybind/mgr: define CLUSTER_LOG_PRIO* using enum.IntEnum
so the clog type / prio can be typed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 22:16:36 +08:00
Kefu Chai
7a2e460d34 pybind/mgr/mgr_module: add poll param to CLICommand
and add it to the output of dump_cmd() call, if "poll" is missing
in the output of dump_cmd() call, ceph CLI consider the it as False.

but it'd be better to be explicit. so this field is always set.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 22:16:36 +08:00
Kefu Chai
1453f0ff1f mgr,pybind/mgr: use bool for poll in COMMANDS
use correctly typed variable helps with readability and less
error-prone.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 22:16:36 +08:00
Kefu Chai
f24a773457 pybind/mgr/mgr_module: flake8 fixes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 22:16:36 +08:00
Sebastian Wagner
70828cb1a8
Merge pull request #39040 from liewegas/cephadm-qa-targets
qa/suites/rados/cephadm: test against ubuntu 20.04 (and update ubuntu_latest)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-24 12:32:26 +01:00
Or Ozeri
35c1dfe409 librbd: do not share crypto image layers with ancestors
This commit creates a unique crypto image dispatch per image,
instead of sharing a single object with all ancestors.
This should fix a seg-fault caused when closing an encrypted cloned image.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-24 09:32:44 +02:00
Or Ozeri
4f438f0dc3 test/librbd: add luks encryption cli test
This commit adds a cli test for rbd encryption verifying LUKS compatbility with cryptsetup

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-24 09:11:50 +02:00
Sage Weil
7fe3853af9 qa/workunits/cephadm/test_repos: adjust target version
15.2.7 is available for 20.04; 15.1.1 is not

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-23 18:59:39 -06:00
Kefu Chai
24e858ec31
Merge pull request #39042 from tchaikov/wip-cram
test/run-cli-tests: use cram from github

Reviewed-by: Sage Weil <sage@redhat.com>
2021-01-24 03:30:29 +08:00
Kefu Chai
d8cbe4eec1 test/downloads: drop cram-0.5 tar ball
since we install cram from github, drop the tarball from repo

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 03:10:54 +08:00
Kefu Chai
c74e614813 test/run-cli-tests: use cram from github
to address the failure of

  WARNING: Built wheel for cram is invalid: Metadata 1.2 mandates PEP 440 version, but '0.5.0ceph.2011-01-14' is not
Failed to build cram

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 03:10:54 +08:00
Kefu Chai
97480142a6
Merge pull request #39035 from tchaikov/wip-cephadm-call
cephadm: rewrite call() with asyncio

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-01-24 02:23:38 +08:00
Kefu Chai
8ad7e4c028 cephadm: rewrite call() with asyncio
for better readability, also return 124 when subprocess times out

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-24 01:43:33 +08:00
Sage Weil
8304bf6510 qa/suites/rados/cephadm: add 20.04 targets
Add 20.04 targets.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-23 11:26:06 -06:00
Sage Weil
170dd14af4 qa/suites/rados/cephadm: add 20.04 target
Link directly to the distro version... no need to use _latest here since
it obscures the podman vs docker difference.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-23 11:23:13 -06:00
Sage Weil
a07ca4c5c0 qa/distros: change ubuntu_latest to 20.04
Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-23 11:21:58 -06:00
Sage Weil
24c2d2b124 qa/distros: add ubuntu 20.04 targets
Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-23 11:20:52 -06:00
Sage Weil
c6fba10f7f Merge PR #38974 into master
* refs/pull/38974/head:
	qa/suites/rados/cephadm/upgrade: avoid allow_ptrace with old cephadm

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-01-23 11:03:10 -06:00
Casey Bodley
334ecc04dd
Merge pull request #39032 from adamemerson/wip-layout-initialize
rgw: Initialize Layout Logs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-01-23 11:32:51 -05:00