Commit Graph

92923 Commits

Author SHA1 Message Date
Andrew Schoen
071e7ce41a ceph-volume: use Device.lvm_size in batch strategies
We should show the user what the size of the device will be after lvm
creates a pv out of it. This way there isn't a discrepency between the
sizes that are reported to the user and what is actually created.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:47 -06:00
Andrew Schoen
5197c16b44 ceph-volume: adds a lvm_size property to util.Device
This will give us the size lvm should report because it takes into
account the 1GB physical extent size we set.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:47 -06:00
Andrew Schoen
0023961397 ceph-volume: batch mixed type scenarios have no need to calulate data size
We know with a mixed type scenario the device used for data will be used
at 100% capacity. This means we do not need to be explict when asking
for the size of the data lvs, which avoids rounding errors with very
small device sizes.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:47 -06:00
Andrew Schoen
3a5b7b5fa9 ceph-volume: add /dev/nvme1n1 to filestore mixed type batch tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:47 -06:00
Andrew Schoen
fdfb79b91c ceph-volume: set use_large_block_db in validate, not compute
The self.use_large_block_db property was never getting set because
the block in compute was never called as block_db_size was reset in
validate if it was 0. We needed to set self.use_large_block_db in
validate instead of compute.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:47 -06:00
Andrew Schoen
4a1b97efc8 ceph-volume: set a 1G extent size when creating vgs
This allows us to create larger lvs than the default of 4m
and is easier to reason about when sizing the lvs as everythign is
reported as GBs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:47 -06:00
Andrew Schoen
eab962c6ef ceph-volume: set block_db_size to 9GB in centos7 mixed type test
This is to enable testing of the --block-db-size flag

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:46 -06:00
Andrew Schoen
934c32acb3 ceph-volume: use two NVME drives in centos7 mixed_type tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:46 -06:00
Andrew Schoen
add9f88dc8 ceph-volume: use extents instead of size when creating block.db lvs
In the batch command use extents instead of size when creating lvs. This
gives a more precise size and avoids rounding errors.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-12-06 12:29:46 -06:00
Andrew Schoen
893b61b256 ceph-volume: create two NVME drives for testing
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: bz#1650306
2018-12-06 12:29:46 -06:00
Lenz Grimmer
3370095dda
Merge pull request #25423 from p-na/wip-pna-fix-pool-edit
mgr/dashboard: Fix renaming of pools

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2018-12-06 17:16:21 +01:00
Igor Fedotov
7b2d092068 mon/PGMap: Partially unify 'ceph pg stat' output with 'ceph df'
Global stats parameter (like 'available' or 'total') are dumped under the
same names now).

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
242ba0cacf pybind/diskprediction: rename raw_bytes_used field reference.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
61cb798120 pybind/mgr/telegraf: rename raw_bytes_used field reference.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
1991b70d3a qa/mgr: rename raw_bytes_used field reference.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
f4ea669147 osd: dump legacy (calculated for now) params in osd_stat_t::dump.
This is to ensure backward compatibility with python modules that use
them.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
e932053be4 PendingReleaseNotes: update according to statistics refactoring
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
71fe9149ac mgr: rename ceph df stat columns for both influx and prometheus.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
79fd227639 qa: replace raw_bytes_used field access in QA test cases
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
587f076ff7 mon: rearrange columns in ceph df output
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
377726afac os/test: extend BluestoreStatFS test with pool_statfs verification.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
deabe939ef os/bluestore: add fsck/repair coverage for per-pool stats
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
bfe3cd408b librados: fix a way rados approximate per-pool net amount.
In fact that's just a workaround for now as we need proper raw_used_rate
present at librados.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
24944bdfb4 test/mon/PGMap: fix the test case
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
b5b8aa6cf3 osd: remove assert on inconsistent total/available values from osd_stat_t encode
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
7ca25dfd5a osd,os,mon: extend 'ceph df' report to provide both USED and RAW_USED
totals where USED is space allocated for object keeping while RAW_USED
also includes space allocated/reserved for internal purposes, e.g.
DB/WAL or journal.

Signed-off-by: Igor Fedotv <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
6a01f36853 os/bluestore: add debug option to disable per pool statistics collection
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
25664c4ccf test: add test coverage for rados df reports
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
650aef1958 mon/pgmap: make pool id type consistent for statistics operations
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
db5c5cce55 rados,mon: improve ceph and rados df reports.
Fixes: https://tracker.ceph.com/issues/22159
Fixes: https://tracker.ceph.com/issues/20870

This adds  more parameters to estimate allocation granularity overhead and compression
effectiveness. RAW_USED column has been removed. USED column reflects
amount of space allocated for a pool at all replicas. Including
allocaction granularity overhead and taking ompression savings and
object content gaps into account.

Following columns were appended:
* STORED - approximation of bytes users've actually stored in a
pool, i.e netto data amount without compression, allocation and
other overheads (not applicable for 'rados df').
* USED COMPR - amount of bytes allocated to store compressed data, i.e.
compressed data plus allocation/replication overhead.
* UNDER COMPR - amount of data(icluding repicated one) passed through
compression and stored in that form.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
63df406082 osd,mon: start using per-pool statistics reported from OSD.
Notable changes:
* pool statistics collection has been changed to track actual allocated
  store space instead of previous object logical sizes summation.
* 'ceph/rados df' report total space(over all replicas) allocated for a pool under "USED" column.
* "RAW USED" column removed from 'ceph df detail' report

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
ad3d2e542f os: extend ObjectStore interface with per-pool statistics access method
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Igor Fedotov
9d433ee648 os/bluestore: add per-pool statistics collection
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Alfredo Deza
59f6cec6dd ceph-volume lvm.zap initial take on zapping by OSD ID, FSID
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-12-06 10:54:11 -05:00
Matt Benjamin
6fd2b15f4e
Merge pull request #25208 from cbodley/wip-rgw-getobj-throttle
rgw: adapt AioThrottle for RGWGetObj
yay!
2018-12-06 10:46:03 -05:00
Sage Weil
9ee3ce1ecd Merge PR #25345 into master
* refs/pull/25345/head:
	qa/suites: fix bluestore links
	qa/objectstore: bluestore -> bluestore-{bitmap,stupid}

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-12-06 08:42:04 -06:00
Sage Weil
96ca7ed859 Merge PR #25378 into master
* refs/pull/25378/head:
	mgr/hello: add serve() method

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-12-06 08:41:44 -06:00
Sage Weil
fb4bf34370 Merge PR #25382 into master
* refs/pull/25382/head:
	mon/OSDMonitor: include host, container info for 'osd find' command

Reviewed-by: Dan Mick <dmick@redhat.com>
2018-12-06 08:41:27 -06:00
Radoslaw Zarzynski
3e1593e2fb common: hint bufferlist's buffer_track_c_str accordingly.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-12-06 15:40:03 +01:00
Jason Dillaman
c0b8dab209
Merge pull request #25301 from wjwithagen/wjw-fix-dtor-pair
rbd: workaround for llvm linker problem, avoid std:pair dtor

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-06 08:27:33 -05:00
Jason Dillaman
634c659134
Merge pull request #25265 from trociny/wip-rbd-fuse-namespace
rbd-fuse: namespace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-06 08:26:55 -05:00
Volker Theile
ec3cae8fed
Merge pull request #24968 from s0nea/wip-dashboard-osd-recv-speed-profiles
mgr/dashboard: add profiles to set cluster's rebuild performance

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
2018-12-06 13:00:30 +01:00
Patrick Nawracay
8a4936e934 mgr/dashboard: Fix renaming of pools
The validation fails to allow dots in pool names and as such, it wasn't
allowed to create or rename pools with dots in their names.

Fixes: https://tracker.ceph.com/issues/37534

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-12-06 12:59:20 +01:00
Li Wang
19bd21d3a7 tools/rados: rename misleading variables
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
2018-12-06 08:46:27 +00:00
Li Wang
efa26b41c0 tools/rados: remove misleading comments
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
2018-12-06 08:45:41 +00:00
Li Wang
415f2a4e09 tools/rados: fix memory leak in error path
Signed-off-by: Li Wang <laurence.liwang@gmail.com>
2018-12-06 08:38:03 +00:00
Kefu Chai
90ee8b8580
Merge pull request #25409 from tchaikov/wip-public-iterator-traits
buffer: mark the iterator traits "public"

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-12-06 13:49:17 +08:00
Jianpeng Ma
88c4554af3 common/buffer: remove repeated condtion-check.
In func push_back(), it already check buffe::ptr::length().

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-12-06 13:47:21 +08:00
Jason Dillaman
6db4c152a6
Merge pull request #25266 from trociny/wip-24608
rbd-ggate: support namespaces

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-05 20:25:54 -05:00
Kefu Chai
5c708ece39 buffer: mark the iterator traits "public"
the iterator traits should be accessible from non-dervative classes.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-06 08:59:35 +08:00