Commit Graph

124249 Commits

Author SHA1 Message Date
Patrick Donnelly
e3d771702d
pybind/mgr: add sqlite3 db module helpers
This creates a new '.mgr' pool for storing a default sqlite3 database
for each mgr module. Each module's database is stored in:

        file:///.mgr:<mgr module name>/main.db?vfs=ceph

The "main.db" is the only one used presently but perhaps a module may
want extra databases for some reason. The module name is used for the
RADOS namespace.

Databases are versioned in a common table called MgrModuleKV using the
"__version" key. A mechanism is in place (SCHEMA_VERSIONED) to allow
modules to upgrade their databases over time in a consistent way.

Fixes: https://tracker.ceph.com/issues/50278
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:16 -07:00
Patrick Donnelly
103bb6b6bb
doc: add missing note about restoring allow_standby_replay
Fixes: https://tracker.ceph.com/issues/51187
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:07:52 -07:00
Patrick Donnelly
dcb340f3d8
doc: clarify distributed ephemeral pins new mechanism
This documentation was for the old code, the new code (by Zheng)
fragments the directory and distributes those fragments.

Fixes: https://tracker.ceph.com/issues/51187
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:07:50 -07:00
Patrick Donnelly
570c7ffc01
doc: update mds_autoscaler documentation to reflect recent changes
Notably, Sage changed how this worked in [1].

[1] 6d8c0e4722

Fixes: https://tracker.ceph.com/issues/51187
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:07:42 -07:00
Kefu Chai
7513b24aa5
Merge pull request #40480 from kamoltat/wip-ksirivad-fix-bug-49988
pybind/mgr/progress: Disregard unreported pgs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-06-12 08:37:35 +08:00
Kefu Chai
16c663eb7a
Merge pull request #41772 from cyx1231st/wip-seastore-onode-tree-node-size
crimson/onode-staged-tree: extend tree node sizes to fit insert upper-bounds

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-12 08:11:45 +08:00
Patrick Donnelly
95f0e9c959
Merge PR #39505 into master
* refs/pull/39505/head:
	qa: test nowsync option in kernel client workflows
	qa: deep merge top level overrides for fuse/kclient

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-06-11 17:10:41 -07:00
Kefu Chai
689ba06bca
Merge pull request #40952 from tchaikov/wip-deb-build-profile
debian: use build-profile to manage build deps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-12 08:08:47 +08:00
Patrick Donnelly
07bafc1b23
Merge PR #41683 into master
* refs/pull/41683/head:
	qa: update RHEL to 8.4

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-11 17:07:06 -07:00
Patrick Donnelly
8eeb1455ee
qa: specify distro for fs:bugs
Fixes: https://tracker.ceph.com/issues/51184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 16:25:50 -07:00
Ernesto Puerta
0b9f06d63d
Merge pull request #41550 from aaryanporwal/navbar-fix
mgr/dashboard: fix for right sidebar nav icon not clickable

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-06-11 19:46:15 +02:00
Yingxin Cheng
25254533a9 crimson/onode-staged-tree: convert ghobject_t to key_hobj_t to print
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
266e54b318 crimson/onode-staged-tree: merge with single-value tail internal node
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
ec3b7828dc crimson/onode-staged-tree: switch to and test larger node sizes
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
b1730b0502 crimson/onode-staged-tree: support unaligned read to leaf_sub_items_t::num_keys
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
ba9e465f2d crimson/onode-staged-tree: add the missing retired validation
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
e11c177396 crimson/onode-staged-tree: encode/decode stage_size instead
end-offset may risk overflow the node_offset_t, but stage_size will not.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
d163f1f9d1 crimson/onode-staged-tree: use extent_len_t and node_offset_t correctly
extent_len_t represents a value that may include the node size, but
node_offset_t cannot and may overflow. Also add validations when
try to cast a larger type to node_offset_t.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
b2f1c7df9c crimson/onode-staged-tree: remove unused code
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
ee005111fb crimson/onode-staged-tree: validate node size where possible
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
38a6b41614 crimson/onode-staged-tree: remove compile-time NODE_BLOCK_SIZE
Switch to run-time node_size.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:18 +08:00
Yingxin Cheng
55605f6e34 crimson/onode-staged-tree: pass node_size to lower node stages
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:17 +08:00
Yingxin Cheng
ca86033b52 crimson/onode-staged-tree: remove compile-time node SIZE
Switch to run-time node_extent_t::node_size instead.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:59:06 +08:00
Yingxin Cheng
9ecaeadf61 crimson/onode-staged-tree: cleanup internal_fields_3_t template
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
2258730bee crimson/onode-staged-tree: pass node_size into node_extent_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
f7e77c9bee crimson/onode-staged-tree: extend num_keys_t to support larger nodes
Also adjust the accroding layout related unit tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
7e24c625ed crimson/onode-staged-tree: remove hard-coded node_stage_t::EXTENT_SIZE
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
70ced70d04 crimson/onode-staged-tree: check the maximum tree height
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
a28ad7ebb1 crimson/onode-staged-tree: validate safe-split from tree_conf_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
37d721cf69 crimson/onode-staged-tree: add and use node sizes in tree_conf_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
50d1afd7a2 crimson/onode-staged-tree: implement size upper-bounds to value
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
6fb8807678 crimson/onode-staged-tree: implement size upper-bounds to ns/oid
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
431d69ebd6 crimson/onode-staged-tree: introduce compile-time tree_conf_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
1cb6f09d3d crimson/onode-staged-tree: cleanup, decouple test_item_t from TestValue
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
8b6422b49a crimson/seastore: introduce TransactionManager::read_extent(t, offset)
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
88fd967b78 crimson/onode-staged-tree: correct the node size equation
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:58 +08:00
Yingxin Cheng
9e8bb747ca crimson/onode-staged-tree: try merge peer if node has one value
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:43:47 +08:00
Yingxin Cheng
2921f27b97 crimson/onode-staged-tree: cleanup and introduce has_single_value()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-11 22:42:25 +08:00
Kefu Chai
8273a96198
Merge pull request #41814 from tchaikov/wip-crimson-fmt-seq
crimson/osd: specialize fmt::formatter<> for OpSequencer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-11 19:59:46 +08:00
Ernesto Puerta
9dabec8485
Merge pull request #41759 from rhcs-dashboard/wip-api_label-master
.github/labeler: add api-change label

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-11 11:08:18 +02:00
Sebastian Wagner
de653833d1
Merge pull request #41242 from zdover23/wip-doc-cephadm-monitoring-deploying-monitoring-second-half-2021-05-10
doc/cephadm: rewrite "deploy. mon. w/cephadm" 2/2

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-11 10:45:47 +02:00
Sebastian Wagner
c64a6e9534
Merge pull request #41241 from zdover23/wip-doc-cephadm-monitoring-deploying-monitoring-first-half-2021-05-10
doc/cephadm: rewrite "deploy. mon. w/cephadm"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-11 10:45:04 +02:00
Kefu Chai
b0c5a6f097 crimson/osd: specialize fmt::formatter<> for OpSequencer
so we can print it like fmt::format("{}", sequencer)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-11 14:19:14 +08:00
Michael Fritch
db8d2a25c3
cephadm: raise Error() when unable to bind to an ip
Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-06-10 22:16:04 -06:00
Michael Fritch
9939573adb
cephadm: remove unit test inheritance
avoid running the `TestCephAdm` tests twice

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-06-10 21:23:52 -06:00
jianglong01
73dd266dc8 mgr/cephadm: When test osd active, it need to add "mock call" to make sure execute correctly
Signed-off-by: jianglong01 <jianglong01@qianxin.com>
2021-06-11 10:56:30 +08:00
Kefu Chai
3cbca12ef7
Merge pull request #41812 from agayev/zoned-fix-block-size
os/bluestore: Fix the size of the block in the Allocator base class t…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-06-11 08:41:36 +08:00
Kefu Chai
d53d5d0aab
Merge pull request #41744 from tchaikov/wip-cmake-no-python-bindings-on-wnin32
cmake: vstart do not depend on cython_rbd if WIN32

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 08:32:30 +08:00
Patrick Donnelly
9147350fb4
Merge PR #41442 into master
* refs/pull/41442/head:
	doc/cephfs/nfs: add ganesha configuration hierarchy

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-06-10 14:53:21 -07:00
Patrick Donnelly
3ed699d0f6
Merge PR #41797 into master
* refs/pull/41797/head:
	qa: update cephfs-shell distro to ubuntu 20.04

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-10 13:55:12 -07:00