Commit Graph

117153 Commits

Author SHA1 Message Date
Patrick Donnelly
f7c9ce50ae
Merge PR #38031 into master
* refs/pull/38031/head:
	client: check rdonly file handle on truncate
	test/libcephfs: test truncate on rdonly fd

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-11-25 13:28:25 -08:00
Patrick Donnelly
f7c056020e
Merge PR #38104 into master
* refs/pull/38104/head:
	client: dump which fs is used by client for multiple-fs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-25 13:27:19 -08:00
Mykola Golub
8dbb87cf5f
Merge pull request #38068 from dillaman/wip-librbd-migration-4
librbd: S3-based migration source 

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-11-25 17:30:54 +02:00
zdover23
6497da98d7
Merge pull request #38195 from anthonyeleven/anthonyeleven/docs-14730
doc/rados/troubleshooting: description of upgrade procedure - stopping/starting OSDs

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-26 01:27:06 +10:00
Kefu Chai
7924528b8c
Merge pull request #38276 from tchaikov/wip-labeler
github/labeler: add "rgw" label to rgw related changes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-11-25 23:03:02 +08:00
zdover23
2d25182893
Merge pull request #38091 from anthonyeleven/anthonyeleven/docs-46367
doc/dev/developer_guide: verb disagreement on "Testing: Unit Tests" page

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-26 00:19:38 +10:00
zdover23
ea57d5d7ee
Merge pull request #38143 from anthonyeleven/anthonyeleven/docs-46364
doc/dev/developer_guide: What Is Merged and When?" could be less colloquial

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-26 00:16:51 +10:00
zdover23
3cea1e5a2a
Merge pull request #38079 from anthonyeleven/anthonyeleven/docs-24143
doc/rados/configuration/common.rst:  enhance the running multiple clusters section

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-26 00:14:51 +10:00
zdover23
eefa5e33ff
Merge pull request #38088 from anthonyeleven/anthonyeleven/docs-47627
doc/mgr: document dashboard configuration unsetting

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-25 22:10:41 +10:00
Jan Fajerski
72c9cc7c6d Merge PR #38156 into master
* refs/pull/38156/head:
	ceph-volume batch: reject partitions in argparser

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-25 11:41:19 +01:00
Kefu Chai
3c0ab38aba github/labeler: label src/test/** with "tests"
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-25 18:14:42 +08:00
Kefu Chai
daee16839a github/labeler: add "rgw" label to rgw related changes
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-25 18:14:42 +08:00
Kefu Chai
71f36e8ad4
Merge pull request #38274 from dillaman/wip-github-labeler-rbd
github/labeler: add rbd label to src/tools/rbd_[ggate/wnbd]

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-25 14:46:16 +08:00
Jason Dillaman
048d0892b5 github/labeler: add rbd label to src/tools/rbd_[ggate/wnbd]
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-24 20:55:21 -05:00
Jason Dillaman
4f4e94c5be
Merge pull request #38213 from petrutlucian94/rbd_exit
rbd: minor rbd-wnbd improvements

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-11-24 20:52:02 -05:00
Anthony D'Atri
00d1948491
Merge branch 'master' into anthonyeleven/docs-46367 2020-11-24 12:27:31 -08:00
Yuri Weinstein
a01f710a90
Merge pull request #38254 from neha-ojha/wip-48299
doc/releases/nautilus.rst: add release notes for 14.2.15

Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2020-11-24 10:51:40 -08:00
zdover23
11311241c7
Merge pull request #38144 from anthonyeleven/anthonyeleven/docs-46365
doc/dev/developer_guide: prefer Latinate verbs to English phrasal verbs

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-25 04:51:13 +10:00
Ernesto Puerta
4ddace0bcd
Merge pull request #38260 from bk201/wip-48338
mgr/dashboard: fix tox py3 environment error

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-24 19:35:36 +01:00
Anthony D'Atri
3ea3975ad0 doc/mgr: document dashboard configuration unsetting
Fixes: https://tracker.ceph.com/issues/47627
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2020-11-24 10:03:34 -08:00
Casey Bodley
3fa037fc6d
Merge pull request #38253 from cbodley/wip-rgw-gc-disable-defer
rgw: temporarily disable calls to defer_gc() in RGWGetObj

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-11-24 12:14:22 -05:00
Neha Ojha
2d63f47878 doc/releases/nautilus.rst: add release notes for 14.2.15
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-11-24 16:40:45 +00:00
Ilya Dryomov
b4129a3e2f
Merge pull request #38142 from theanalyst/doc/releases/v14.2.14
doc: release notes for octopus & nautilus backports

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2020-11-24 17:23:00 +01:00
Mykola Golub
acf3b72b4d
Merge pull request #38247 from dillaman/wip-librbd-test-rename
test/librbd: rename tests to match proper namespaces

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-11-24 17:15:29 +02:00
Abhishek Lekshmanan
b04b1ba566 doc: releases: v15.2.6 Octopus release notes
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
2020-11-24 15:43:37 +01:00
Abhishek Lekshmanan
cf1a3d7a39 doc: releases: v14.2.14 Nautilus release notes
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
2020-11-24 15:43:33 +01:00
Kefu Chai
30a3261f1a
Merge pull request #38261 from tchaikov/wip-45919
pybind/mgr/dashboard/run-backend-api-tests: print more log from mgr

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2020-11-24 18:14:37 +08:00
Kefu Chai
c1ae6fa3fd
Merge pull request #38217 from ideepika/wip-fix-lua-failure
cmake: require Lua 5.3 and up

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-24 18:00:15 +08:00
Kefu Chai
58c708ba49
Merge pull request #38204 from lnsyyj/wip-iouring
blk/kernel/io_uring: do not guard liburing backend with __x86_64__ anymore

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-24 17:55:17 +08:00
Kefu Chai
2e690bba7e
Merge pull request #38256 from badone/wip-dont-build-io_uring.c-every-time
cmake: Stop 'configure' running every time

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-24 16:04:51 +08:00
Kefu Chai
6e431c09c6 pybind/mgr/dashboard/run-backend-api-tests: print more log from mgr
in hope to more context when debugging https://tracker.ceph.com/issues/45919

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-24 15:47:47 +08:00
Kiefer Chang
dc974fa953
mgr/dashboard: fix tox py3 environment error
Add `ceph_argparse` module to the base environment.

Fixes: https://tracker.ceph.com/issues/48338
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-11-24 15:44:43 +08:00
Brad Hubbard
e0240b9af4 cmake: Stop 'configure' running every time
Currently the configure script is run on 'liburing_ext' not only for the
initial build but for subsequent incremental builds.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-11-24 13:38:37 +10:00
Ilya Dryomov
4ee600b354
Merge pull request #38243 from idryomov/wip-relax-preauth-asserts-again
msg/async/ProtocolV2: allow rxbuf/txbuf get bigger in testing, again

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-11-24 02:48:48 +01:00
Kefu Chai
a0230e1e6f
Merge pull request #38252 from idryomov/wip-resurrect-log-compression
qa/tasks/ceph: resurrect log compression

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-24 09:46:40 +08:00
Ilya Dryomov
0e4bc27722 qa/tasks/ceph: resurrect log compression
Commit 9536625558 ("qa/tasks/ceph: use Cluster.sh() and Remote.sh()
when appropriate") dropped run.wait(), which waits for all given
processes to exit.  This resulted in errors like

  INFO:teuthology.orchestra.run.smithi107.stderr:tar: ./objectstore_tool..log: file changed as we read it
  INFO:teuthology.orchestra.run.smithi107.stderr:tar: ./ceph-client.admin.175125.log: File removed before we read it

as the task moved on to archiving semi-corrupted and uncompressed logs,
filling up the lab cluster.

Revert that hunk, as Cluster.sh() is useless here -- we don't need
stdout or stderr, but very much need parallel execution and wait for
the compression to finish.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2020-11-24 00:51:54 +01:00
Casey Bodley
94df9cd37a rgw: temporarily disable calls to defer_gc() in RGWGetObj
cls_rgw_gc_queue_update_entry() is known to cause data loss when called
on objects that have not actually been scheduled for garbage collection

RGWGetObj is the only caller, and uses defer_gc() when reads are taking
a long time compared to rgw_gc_obj_min_wait. if an object has since been
deleted and submitted for garbage collection, this allows RGWGetObj to
defer that gc until the entire read completes

by disabling these calls to defer_gc(), very long reads (longer than 1hr,
with default configuration) may fail if the object gets deleted, and a
retry will result in a 404 Not Found error as expected

Fixes: https://tracker.ceph.com/issues/47866

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-11-23 18:06:31 -05:00
Anthony D'Atri
d4b0215a84 doc/dev/developer_guide: prefer Latinate verbs to English phrasal verbs
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
Fixes: https://tracker.ceph.com/issues/46365
2020-11-23 13:07:19 -08:00
Anthony D'Atri
8a47edfced
Merge branch 'master' into anthonyeleven/docs-14730 2020-11-23 09:31:29 -08:00
Jason Dillaman
07d2ded966 test/librbd: rename tests to match proper namespaces
Some of the new crypto tests use a generic test name that
doesn't include the full namespace of the class under test.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 12:08:18 -05:00
Jason Dillaman
c569a302be
Merge pull request #37875 from orozery/librbd-luks-format
librbd: add LUKS support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 12:06:32 -05:00
Jason Dillaman
c21d4cbc83
Merge pull request #38201 from lixiaoy1/minor_fix_parent_cache
librbd/plugin: delete field m_image_ctx in ParentCache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 11:12:44 -05:00
Or Ozeri
ea3c1bfb9e librbd: add LUKS support
This commit introduces internal (not yet part of the api) librbd functions for:
1. formating an RBD image in LUKS format
2. parsing an RBD image in LUKS format

 The actual implementation of the LUKS format is done via libcryptsetup, which is added as a new dependency.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2020-11-23 10:32:14 -05:00
zdover23
8c6b533ee8
Merge pull request #38089 from anthonyeleven/anthonyeleven/docs-47976
doc/mgr: Fix msgr2 TCP port in a couple of examples

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-24 00:32:22 +10:00
zdover23
f6c3592916
Merge pull request #38180 from anthonyeleven/anthonyeleven/docs-23442
doc/rados/operations: Add description of new auth profiles

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-11-24 00:05:00 +10:00
Jason Dillaman
8ee92b87b9 librbd/api/Migration: fix potential double-close of image
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 08:45:50 -05:00
Jason Dillaman
b58bb486d6 librbd/migration: implement a basic S3 stream source
The new S3 stream requires a fully qualified URL to a bucket
(i.e. no virtual host names) along with the access and secret
keys to the bucket.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 08:45:50 -05:00
Jason Dillaman
393dddb79c librbd/migration: simplify common http result code handling
Move the processing of common HTTP result codes (not found, permission
denied, etc) higher up the stack so it can be re-used between all
request paths.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 08:45:50 -05:00
Jason Dillaman
7e3fa13ab3 librbd/migration: implement a callback hook for processing http requests
The S3 authentication scheme requires adding additional headers to the
outbound request as well as computing a secure signature for the request
using the secret key.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 08:45:50 -05:00
Jason Dillaman
fef41bf131 librbd/migration: move read processing into HttpClient
This will allow the logic to be re-used between the standard
HTTP stream and the future S3 stream.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-23 08:45:49 -05:00