Commit Graph

116591 Commits

Author SHA1 Message Date
Matt Benjamin
a559cd4cc0 rgw_file: fix some zipper flow for RGWLibContinuedReq
Some bits of the standard Zipper conversions were missed for
the RGWLibContinuedReq case, where the setup is encapsulated in
the request, but execution is broken up in to steps.   This
currently affects only RGWWriteRequest.

Fixes: https://tracker.ceph.com/issues/48136

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-11-05 12:17:10 -05:00
Kefu Chai
a86df246c6
Merge pull request #37919 from tchaikov/wip-cmake-liburing
cmake: enable LIBURING bluestore backend by default

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2020-11-05 23:32:32 +08:00
Casey Bodley
40bc302a54
Merge pull request #37929 from BryceCao/wip-modify-error-code-of-bucket-info-API
rgw : modify error message to NoSuchBucket when bucket doesn't exist in bucket info API

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-11-05 10:04:30 -05:00
Lenz Grimmer
c6330e71a2
Merge pull request #37913 from rhcs-dashboard/button-disable
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-11-05 14:54:16 +01:00
Kefu Chai
c5f8bb268e
Merge pull request #37832 from tchaikov/wip-doc-slow-ops
doc: reformat health-messages.rst and revise SLOW_OPS section

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-05 21:05:17 +08:00
Kefu Chai
c1fb08cdaf
Merge pull request #37951 from agayev/misc-zbd-changes
doc, os/bluestore: Zoned storage related cleanup.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-05 21:01:46 +08:00
Lenz Grimmer
0533281eb8
Merge pull request #37637 from p-se/dashboard-smart-logging
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-11-05 13:38:59 +01:00
Lenz Grimmer
fe5a213c50
Merge pull request #37454 from votdev/custom_container_docs
doc/mgr/orchestrator: Add hints related to custom containers to the docs

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-11-05 10:57:51 +01:00
Lenz Grimmer
0524d7f5f2
Merge pull request #37946 from bk201/wip-47373
mgr/dashboard: disable cluster selection in NFS export editing form

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-11-05 10:51:54 +01:00
Kefu Chai
5f96b311a8
Merge pull request #37955 from tchaikov/wip-cmake-crimson-cleanup
cmake: rename crimson tests named like foo_bar to foo-bar

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-11-05 16:55:30 +08:00
Kefu Chai
b0f9c6b598
Merge pull request #37953 from tchaikov/wip-crimson-recovery-cleanup
test/crimson: do not support variadic future

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-11-05 16:53:03 +08:00
Kefu Chai
6c1bd89b45 cmake: do not name demo-like tests with unittest-*
these tests cannot be exercised without human intervention.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-05 14:46:55 +08:00
Kefu Chai
943295221d cmake/test: remove duplicated target_link_libraries()
it is a copy-n-paste error.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-05 14:46:55 +08:00
Kefu Chai
3e384222cf cmake: rename crimson tests named like foo_bar to foo-bar
for two reasons:

* less typing: no need to press "shift" for inputting "_"
* more consistent with executable names like "ceph-conf"
* simpler to grep when compiling the tests. there is chance
  we need to kill the dead jobs on a jenkins worker node
  where it happens to be compiling the tests.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-05 14:46:55 +08:00
Kefu Chai
df43069c42
Merge pull request #37940 from yuvalif/remove_lua_submodule
rpm,deb,cmake: remove the lua submodule

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-05 13:07:25 +08:00
Patrick Donnelly
254253780d
Merge PR #37869 into master
* refs/pull/37869/head:
	qa: for the latest kclient it will also return EIO

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-04 21:03:02 -08:00
Patrick Donnelly
af0aab64d9
Merge PR #37713 into master
* refs/pull/37713/head:
	mds: fix wrong log text and allignment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-04 21:02:29 -08:00
Patrick Donnelly
d4e115c2c5
Merge PR #37664 into master
* refs/pull/37664/head:
	cephfs: release client dentry_lease before send caps release to mds

Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-11-04 21:01:35 -08:00
Patrick Donnelly
4b26dd797f
Merge PR #37654 into master
* refs/pull/37654/head:
	mds: print the METRIC_TYPE for debug info
	mds: add XXPayload:print support
	mds: only update the requesting metrics

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-11-04 21:00:50 -08:00
Patrick Donnelly
96ff34ca67
Merge PR #32479 into master
* refs/pull/32479/head:
	mds: add dentry lease metric payload support
	client: add _dentry_valid() helper support

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-11-04 20:59:44 -08:00
Kefu Chai
3e785bc16a crimson/common: refactor crimson::do_until()
* define `errorator_t` instead of `futurator` to simplify the code a
  little bit.
* use seastar helper function / types when appropriate

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-05 12:52:35 +08:00
Kefu Chai
c3340e7c08 test/crimson: do not support variadic future
also add "unittest-seastar-errorator" to unit test.

this should address the build failure introduced by
688b95798b

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-05 12:52:35 +08:00
Kefu Chai
edd6d38ee0
Merge pull request #37950 from tchaikov/wip-crimson-recovery-cleanup
crimson: cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-11-05 12:31:53 +08:00
Patrick Donnelly
d85e4dca6b
Merge PR #37899 into master
* refs/pull/37899/head:
	qa: avoid using ceph.dir.entries for empty check
	qa: do not test dir size equals rbytes
	qa: disable check for value kclient fails to get
	qa: leave one standby available to avoid warning
	qa: do not check access timestamps
	qa: ignore skip errors for kclient
	qa: add more clients for test_volume_client
	qa: accept client returning pool id for layout
	qa: ignore expected warning about client missing features
	qa: do not fail on skip
	qa: fix mount directory
	qa: rename distro link to distros
	qa: accept eio for mount failure
	qa: unmount volumes before removal
	qa: some sessionmap tests require fuse
	qa: remove obsolete test
	qa: do not fail on test skip
	qa: use more OSDs for workload tests
	qa: use 4 clients for volumes testing
	qa: relocate tasks
	qa: rename workload suite
	qa: relocate new tasks from fs:basic_functional
	qa: remove kcephfs suite
	qa: move kcephfs:mixed-clients to fs
	qa: remove dead kcephfs:cephfs suite
	qa: merge snap tests for kcephfs/fs
	qa: add new fs:shell sub-suite
	qa: remove task-less kcephfs:recovery suite
	qa: add new fs:volumes sub-suite
	qa: merge fs:basic_functional and kcephfs:recovery
	qa: move kcephfs:thrash to fs:thrash
	qa: reorganize thrash tasks
	qa: move multifs/failover tests
	qa: run norstats workunit for klient
	qa: merge workload tests from kcephfs with fs
	qa: use consistent syntax
	qa: remove inline testing
	qa: move log-rotate config to common yaml
	qa: remove fuse default permissions config
	qa: fix whitespace

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-11-04 11:04:17 -08:00
Abutalib Aghayev
2a3eb6e2bc os/bluestore: Cleanup. Move ZBD related functions into one place and add zoned
prefix to all of them.

Signed-off-by: Abutalib Aghayev <agayev@psu.edu>
2020-11-04 13:57:05 -05:00
Abutalib Aghayev
482e274556 doc: Update contact email address and documentation for zoned-storage.rst.
Signed-off-by: Abutalib Aghayev <agayev@psu.edu>
2020-11-04 13:56:22 -05:00
J. Eric Ivancich
4ee736e57a
Merge pull request #32394 from zhangsw/fix-appendobj-invalidposition
rgw: using strict_strtoll for append position. If not, any invalid value will be decoded as zero which is not correct.

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2020-11-04 10:03:51 -05:00
J. Eric Ivancich
9339e99937
Merge pull request #37703 from theanalyst/rgw/sal/init-quota-fixes
rgw: op: check for bucket existance as well in init_quota

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2020-11-04 10:03:20 -05:00
J. Eric Ivancich
5933fe06f8
Merge pull request #37160 from linuxbox2/wip-rgwlc-tagfilter
rgwlc: fix clause counting in LCFilter_S3::decode_xml()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-11-04 10:02:18 -05:00
Yuval Lifshitz
80223d8b3c rpm/deb/cmake: remove the lua submodule
instead use the official lua-dev/el packages

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2020-11-04 17:01:58 +02:00
J. Eric Ivancich
89d122643d
Merge pull request #37066 from smanjara/wip-metadata-reshard
rgw: don't allow overwrites to BucketInfo's layout field when sync fetches bucket metadata

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-11-04 10:00:48 -05:00
J. Eric Ivancich
fac6f46c6c
Merge pull request #37892 from IlsooByun/fix_bucket_sync_err
rgw: keep syncstopped flag when copying bucket shard headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-11-04 10:00:03 -05:00
Kefu Chai
4d9b9c5115 osd/PGLog: do not use unique_ptr explicitly
* do not use unique_ptr<> explicitly, use `seastar::do_with()` for
  better readability
* use seastar::do_until() instead of seastar::repeat() for better
  readability. plain boolean is simpler than
  `seastar::stop_iteration::yes`
* do not capture variables using FuturizedStoreLogReader if we could
  pass them by instant parameters.
* rename "start()" to "read()". as "read" is more specific in this
  context.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:58 +08:00
Kefu Chai
0494d786cd osd/PGLog: move PGLog::read_log_and_missing_crimson() into .cc
to speed up the compilation. and for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:58 +08:00
Kefu Chai
e9fd452b3e osd/PGLog: drop template parameter of FuturizedStoreLogReader
there is no need for the generic type at this moment.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:58 +08:00
Kefu Chai
011734d064 crimson/osd: use seastar:do_until() when appropriate
a plain boolean is easier to understand than
`seastar::stop_iteration::yes`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:58 +08:00
Kefu Chai
f891bf385d crimson/osd: mark CyanStore::CyanOmapIterator methods final
this class is not intended to be inherited by classes, and these methods
are "final", so mark them "final".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:58 +08:00
Kefu Chai
4b8ac1ba4b crimson/osd: mark methods pure virtual
if the default implementation does not make sense in any case, just mark
it pure virtual.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:58 +08:00
Kefu Chai
02334cd0e4 crimson/osd: let OmapIterator mutators return future<>
instead of returning future<int> where the int represents an errno which
is not checked by any callers, we should just return a future<>, we
could erroratorize it later on, but returning an int is not the right
way to handle this, and it is not consistent with how we handle other
errors elsewhere in crimson.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 22:48:52 +08:00
Volker Theile
1927809b0b doc/mgr/orchestrator: Add hints related to custom containers to the docs
Fixes: https://tracker.ceph.com/issues/48113

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-11-04 14:39:44 +01:00
zdover23
2f98e81708
Merge pull request #37867 from zdover23/wip-doc-cephadm-install-oct-2020
doc/cephadm: improve install guide

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-04 22:01:53 +10:00
Kefu Chai
fa9c9d1cef
Merge pull request #37944 from tchaikov/wip-crimson-recovery-cleanup
crimson/osd: extract read_omap_for_push_op()

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-11-04 18:01:08 +08:00
Kiefer Chang
d678d8076c
mgr/dashboard: disable cluster selection in NFS export editing form
We should not allow changing an export's cluster because an export ID
might live in one cluster but not in another one. Editing a non-existing
export in a cluster causes an error.

Fixes: https://tracker.ceph.com/issues/47373
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-11-04 17:42:31 +08:00
Xiubo Li
b4cc30617d mds: add dentry lease metric payload support
Fixes: https://tracker.ceph.com/issues/43423
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-11-04 02:59:30 -05:00
Xiubo Li
d646ffb3b5 client: add _dentry_valid() helper support
Fixes: https://tracker.ceph.com/issues/43423
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-11-04 02:59:30 -05:00
Kefu Chai
26e61a02ec crimson/osd: extract read_omap_for_push_op()
extract read_omap_for_push_op() out of build_push_op() for better
readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 13:30:10 +08:00
Kefu Chai
d302ffb8d3 crimson/osd: correct the function names in logging messages
it was a leftover of fbfc53c708

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-04 13:30:10 +08:00
Patrick Donnelly
f372aad22b
Merge PR #37881 into master
* refs/pull/37881/head:
	doc/mgr/orchestrator: Update about nfs add in rook
	mgr/rook: Refactor nfs add/apply methods
	mgr/rook: Udpate to mypy py3 type style in _create(), _update() and apply()
	mgr/rook: Remove unused 'current' argument from update_func method

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-03 16:22:32 -08:00
Patrick Donnelly
0939625530
qa: avoid using ceph.dir.entries for empty check
This avoids a bug [1] in the kernel client.

[1] https://tracker.ceph.com/issues/48104

Fixes: https://tracker.ceph.com/issues/23718
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-03 15:43:04 -08:00
Patrick Donnelly
718f1f9731
Merge PR #37825 into master
* refs/pull/37825/head:
	doc: document MDS cache configuration
	test: add DecayCounter test

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-11-03 13:06:12 -08:00