Commit Graph

116483 Commits

Author SHA1 Message Date
Xuehan Xu
cf07974c89 crimson/osd: make PglogBasedRecovery op take recovering objs triggered elsewhere into account
PGRecovery::start_recovery_ops() should wait for all inflight recovery ops, whether they are
started by BackgroundRecovery or not, otherwise there may be circumstances in which BackgroundRecovery
keep recursively invoking its do_recovery when start_recovery_ops returns recovery done while there are
still missing objects.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2020-11-10 15:18:01 +08:00
Xuehan Xu
5f495290a1 crimson/osd: reset pending_txn's all_committed after set its value
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2020-11-04 10:48:48 +08:00
Patrick Donnelly
f372aad22b
Merge PR #37881 into master
* refs/pull/37881/head:
	doc/mgr/orchestrator: Update about nfs add in rook
	mgr/rook: Refactor nfs add/apply methods
	mgr/rook: Udpate to mypy py3 type style in _create(), _update() and apply()
	mgr/rook: Remove unused 'current' argument from update_func method

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-03 16:22:32 -08:00
Patrick Donnelly
718f1f9731
Merge PR #37825 into master
* refs/pull/37825/head:
	doc: document MDS cache configuration
	test: add DecayCounter test

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-11-03 13:06:12 -08:00
Patrick Donnelly
4b8ac2cdaa
Merge PR #37923 into master
* refs/pull/37923/head:
	client: increment file position on _read_sync near eof

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-11-03 12:54:16 -08:00
Lenz Grimmer
799f65e77f
Merge pull request #37737 from votdev/issue_47926_orch_status
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-11-03 13:57:51 +01:00
Lenz Grimmer
30da5ce6c0
Merge pull request #37757 from votdev/issue_47884_display_time
mgr/dashboard: displaying time in human-readable format

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2020-11-03 09:30:52 +01:00
Mykola Golub
ff3c76a25f
Merge pull request #37827 from dillaman/wip-librbd-migration-2
librbd: added API and CLI support for source-spec import migrations

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-11-03 09:30:18 +02:00
Varsha Rao
a251cea714 doc/mgr/orchestrator: Update about nfs add in rook
"nfs add" is no longer supported in rook. Instead "apply nfs" should be used to
deploy nfs-ganesha daemons.

Signed-off-by: Varsha Rao <varao@redhat.com>
2020-11-03 12:13:10 +05:30
Varsha Rao
125d5fa422 mgr/rook: Refactor nfs add/apply methods
Fixes: https://tracker.ceph.com/issues/47923
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-11-03 12:11:34 +05:30
Varsha Rao
89176cab9b mgr/rook: Udpate to mypy py3 type style in _create(), _update() and apply()
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-11-03 12:11:34 +05:30
Varsha Rao
8c993bd933 mgr/rook: Remove unused 'current' argument from update_func method
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-11-03 12:11:34 +05:30
Kefu Chai
a663b3e65f
Merge pull request #37829 from liu-chunmei/seastore-extentmap-fix
crimson/seastore: add Extentmap node type in cache.cc

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-03 09:22:08 +08:00
Jason Dillaman
c4be773b27 doc/man: add new RBD migration prepare source-spec optionals
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-02 19:58:55 -05:00
Jason Dillaman
9ca496a7d1 qa/suite/rbd: simple migration import test case
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-02 19:58:55 -05:00
Jason Dillaman
81f9b91d62 qa/tasks: support explicit disk configuration for qemu task
The 'disks' key will now be treated as a dictionary where all previous
global settings can be individually applied. Additionally, a disk can be
pre-created and provided for use by qemu.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-02 19:58:55 -05:00
Jason Dillaman
4ab2a183ca rbd: add new 'migration prepare --import-only' CLI optional
The '--import-only' optional can be combined with either a
'--source-spec' or '--source-spec-path' optional to define
the source for the read-only import.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-02 19:57:58 -05:00
Josh Durgin
99a36731f0
Merge pull request #37872 from theanalyst/doc/releases/v14.2.13
doc: releases: v14.2.13 release notes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-11-02 15:58:56 -08:00
Patrick Donnelly
83f84f601a
client: increment file position on _read_sync near eof
Fixes: https://tracker.ceph.com/issues/48076
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-02 15:12:41 -08:00
chunmei-liu
b684961be2 crimson/seastore: add replay test for extent map tree
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2020-11-02 11:52:43 -08:00
Lenz Grimmer
161b7e118b
Merge pull request #37287 from pcuzner/dashboard-goals
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-11-02 19:27:40 +01:00
Patrick Donnelly
a8d31b0ed4
Merge PR #37878 into master
* refs/pull/37878/head:
	doc/mgr/orchestrator: Update about "{mds, rgw} add" status in rook

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-02 09:00:09 -08:00
Patrick Donnelly
788282ed75
Merge PR #37732 into master
* refs/pull/37732/head:
	osdc: restart read on truncate/discard

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-02 08:24:04 -08:00
Abhishek Lekshmanan
b79e4048c9 doc: releases/nautilus: add more changelog entries & fix date
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-11-02 17:16:34 +01:00
Mykola Golub
8f393c0fc1
Merge pull request #37898 from lxbsz/nbd_reexpandmeta
rbd-nbd: reexpand the conf meta in child process

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-11-02 14:53:26 +02:00
Igor Fedotov
27953f49e3
Merge pull request #37547 from ifed01/wip-ifed-fix-bmap-alloc
os/bluestore: fix segfault on out-of-bound offset provided to  claim_…

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2020-11-02 15:26:04 +03:00
Kefu Chai
ef12ea6bae
Merge pull request #37879 from tchaikov/wip-seastar-v6
crimson: use Seastar API v6

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2020-11-02 18:27:54 +08:00
Mykola Golub
903c798a29
Merge pull request #37894 from dillaman/wip-48035
librbd: global config overrides do not apply to in-use images

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-11-02 11:02:59 +02:00
Kefu Chai
59c170e7c6
Merge pull request #37907 from athanatos/sjust/wip-fix-init-weak
crimson/os/seastore/.../lba_btree_node_impl: hold reference to *this during lookup

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-02 16:38:23 +08:00
Samuel Just
7230aac40c crimson/os/seastore/.../lba_btree_node_impl: hold reference to *this during lookup
4f2f4f modified TransactionManager::mount() to use a weak_transaction
while calling init_cached_extents.  This is fine, but lookup() needs
to hold a reference to *this until the child lookup completes in order
to ensure residence in the lba pinning set.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-11-02 00:05:37 -08:00
Paul Cuzner
16c449280d doc/dev/dashboard: typo fix
Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-11-02 17:50:42 +13:00
Paul Cuzner
88346c790c doc/dev/dashboard: link designs to dev guide
Add a feature design section to the dashboard dev
guide, linking to the individual docs under dov/dev/dashboard

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-11-02 17:36:44 +13:00
Paul Cuzner
c0a49c8fd9 doc/dev/dashboard: Document key UI design principles
Document which defines the high level design principles
for the Ceph Dashboard UI

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-11-02 17:36:43 +13:00
Kefu Chai
ac33cea383
Merge pull request #37590 from pcuzner/prometheus-slow-ops
mgr/prometheus: Add SLOW_OPS healthcheck as a metric

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-02 11:32:06 +08:00
Paul Cuzner
fc1905b4a7 mgr: add doc to describe relationship to prometheus
The healthcheck for slow_ops is used by the mgr/prometheus
module so it's important the alert generated matches the
format expected in mgr/prometheus.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-11-02 15:33:46 +13:00
Paul Cuzner
2010432b50 mgr/prometheus: Add healthcheck metric for SLOW_OPS
SLOW_OPS is triggered by op tracker, and generates a health
alert but healthchecks do not create metrics for prometheus to
use as alert triggers. This change adds SLOW_OPS metric, and
provides a simple means to extend to other relevant health
checks in the future

If the extract of the value from the health check message fails
we log an error and remove the metric from the metric set. In
addition the metric description has changed to better reflect
the scenarios where SLOW_OPS can be triggered.

Signed-off-by: Paul Cuzner <pcuzner@redhat.com>
2020-11-02 15:30:49 +13:00
Jason Dillaman
87bc86e311 rbd: show migration source spec in image status
When using the non-legacy mode image migration, the 'rbd status'
command will now show the source-spec for the migration if the
source pool id is invalid.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:44:30 -05:00
Jason Dillaman
51478deab2 rbd: ensure the help printer doesn't print past the end of the line
When long command names and long optional names are combined,
it's possible for the help text to be printed beyond the 80
character limit.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:44:30 -05:00
Jason Dillaman
9edc30366b librbd: replace direct reads from ImageRequest with ImageDispatchSpec
Read requests will always need to go via the image dispatch layer to
ensure migrations are properly handled.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:44:29 -05:00
Jason Dillaman
8a9f6dde28 librbd/migration: allow native v2 images to be opened by name
When in import-only mode, allow the source image to be opened by
name in case it's a V2 image format w/o its image id specified in
the JSON.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
9304f93bc6 librbd/migration: close image if failed to open source
To remain consistent between the native which will automatically close
the ImageCtx if it fails to open the image, the raw format should also
close the ImageCtx.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
ecac0a7563 librbd/migration: initialize default source image layout
For backing source formats that don't care about the low-level
RBD layout details, provide safe default values for the layout
to ensure that striping logic will still function.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
bb38c391b6 librbd: fix potential recursive lock of ImageCtx::image_lock
If the "list_snaps" IO call returns immediately, it can result
in an attempted recursive lock starting from CopyupRequest.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
0ef9cec0bd librbd: migration execute should allow "EXECUTING" state
If the "execute" action was interrupted, we need to be able to
restart it.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
d63a564f22 librbd: read image size when opening migration raw format
The migration prepare operation will need to know the size of
the source image so that it can create the destination image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
9ad4d7936a librbd/migration: fix issue with prepare-import and native-format
When preparing an import, the native format can not expect to have
the child image opened since it hasn't been created yet.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
d6133f9bf2 librbd: added new 'migration_prepare_import' API methods
These related methods accept a JSON-encoded source-spec that
describes how to read from the source image. The migration is
a one-way operation, so children/groups/etc won't be updated
to point to the new destination image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
e5398d1429 librbd: store migration progress on both source+destination images
When migrating from an import-only source-spec like an external
file, it will be important to also store the progress on the
destination image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
5869d13fb5 librbd: switch to migration source-spec builder
Remove all the hard-coded references to the NativeFormat
handler and allow the RawFormat+FileStream handlers to be
instantiated.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00
Jason Dillaman
c7bbb676a8 librbd: switch migration native format to use source-spec JSON object
Remove the dependency on the MigrationInfo struct and allow the native
format to directly pull parameters from the JSON-encoded object
structure.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-01 09:22:38 -05:00