Commit Graph

519 Commits

Author SHA1 Message Date
Joao Eduardo Luis
85d5e29f66 mon: add osd info to obtain info on specific osd
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-07-15 17:13:00 +00:00
Sage Weil
e577581d45 qa/workunits/cephtool/test.sh: cache forward mode is no longer allowed
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-10 17:39:19 -05:00
Gu Zhongyan
73e88f7b72 qa/workunits: add auth caps allow command case
this case introduces multiple quotes in caps line
it will trigger the bug like http://tracker.ceph.com/issues/22227

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
2019-06-25 15:10:07 +08:00
xie xingguo
ec27a162de mgr, osd: 'ceph osd df' by pool
Our test admin has been asking for this for the past few years:-)
Besides, this is also useful for operating on large Ceph clusters with
mutliple storage pools possibly spanning over all osds.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-18 20:29:40 +08:00
xie xingguo
199d92aec0 qa: tolerate wildcard position
It does not make much sense to add this kind of restrictions
as long as user is aware of what is going on.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-05-30 10:52:38 +08:00
xie xingguo
5d695267ef osd: add no{out,down,in,out} flags by device class
This works as a good supplement of https://github.com/ceph/ceph/pull/27563.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-05-30 10:52:35 +08:00
xie xingguo
9a05a065b9 osd,mon: add 'ceph osd {set,unset}-group <flags> <who>' command
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-05-30 09:46:51 +08:00
Joao Eduardo Luis
28c70208d8
Merge pull request #26488 from mogeb/per-pool-scrub-status
mon: show no[deep-]scrub flags per pool in the status

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2019-05-07 17:50:52 +01:00
David Zafman
39cc14bdc1
Merge pull request #27503 from dzafman/wip-39099
osd: Give recovery for inactive PGs a higher priority

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-25 15:06:56 -07:00
David Zafman
444aa9f9fe osd, mon: New pool recovery priority range -10 to 10
Use OSD_POOL_PRIORITY_MAX and OSD_POOL_PRIORITY_MIN constants
Scale legacy priorities if exceeds maximum

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
xie xingguo
bae2231cc5 qa: add crush-node-flags test
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-23 14:20:17 +08:00
Sage Weil
3760e8f918 mon/OSDMonitor: add mon_debug_no_require_octopus
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-02 16:19:43 -05:00
Mohamad Gebai
aa9cb58390 test: add tests for per-pool scrub status
Signed-off-by: Mohamad Gebai <mgebai@suse.com>
2019-04-02 05:50:03 -04:00
xie xingguo
2f195b7072 mon: add set-weight command
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-20 11:23:32 +08:00
Sage Weil
ab0a652826 erasure-code: ensure m >= 1
Fixes: http://tracker.ceph.com/issues/38682
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-12 13:12:58 -05:00
Sage Weil
4c5ed29925 Merge PR #26764 into master
* refs/pull/26764/head:
	mgr: 'osd df' by specified class or (crush) name
	mon/OSDMonitor: add 'osd crush get-device-class' command
2019-03-07 08:52:56 -06:00
xie xingguo
af02d1031d mgr: 'osd df' by specified class or (crush) name
For large clusters, we use device classes to isolate storage pools.
The existing 'osd df' output turns out to be too nosiy, say, if
you care about only single storage pool with osds possibly spanning over
all hosts.

With this change you are now being able to do 'osd df' by class (or by pool,
if you simply use classes to separate different pools), or by a specified
crush bucket name you are currently interested in, which is much more
convenient.

Some examples:
```
$ bin/ceph osd df tree
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60 1.00   -        root default
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
-5       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
 1   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  61     up         osd.1
 2   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  51     up         osd.2
                    TOTAL 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree class aaa
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60 1.00   -        root default
-3       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
-5       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
                    TOTAL 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree name ceph11
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
                    TOTAL 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-06 11:10:56 +08:00
Xie Xingguo
ad8e7d33b1
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-03-05 20:16:18 +08:00
xie xingguo
733fff22bb mon/OSDMonitor: add boundary check for pool recovery_priority
See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 08:15:39 +08:00
Joao Eduardo Luis
dfafdf3dae mon: retire some osd set/unset flags
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-03-02 11:36:23 +00:00
xie xingguo
fa98f09674 mgr: add per pool force-recovery/backfill commands
For those with multiple storage pools sharing the same devices,
I think it would make much more sense to offer per-pool
commands to bring pools with high priority, e.g., because they
are hosting data of more importance than others, back to normal
quickly.

Fixes: http://tracker.ceph.com/issues/38456
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-02-23 12:18:24 +08:00
xie xingguo
97d55fd113 mgr: add per pool scrub commands
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-02-21 09:04:41 +08:00
Sage Weil
b84b8c9c66 qa/workunits/cephtool: fix blacklist test
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-21 15:31:32 -06:00
Sage Weil
87828e5cdd qa/workunits/cephtool/test.sh: pg_autoscale_mode=off while testing pg_num etc
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 13:30:54 -06:00
Sage Weil
4d3407ba93 mon/OSDMonitor: remove max_split_count configurable
This isn't really relevant or useful now that the mgr is throttling the
actual pg_num adjustment based on pg_num_target, % misplaced, etc.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-18 13:30:54 -06:00
Sage Weil
738789b057 mon/PGMap: GLOBAL -> RAW STORAGE in 'df' output
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-07 13:28:07 -06:00
Igor Fedotov
79fd227639 qa: replace raw_bytes_used field access in QA test cases
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Joao Eduardo Luis
256ced3586 qa/cephtool: test bounds on pool's hit_set_*
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-10-31 16:05:40 +00:00
Patrick Donnelly
c0ba741a6e
Merge PR #24158 into master
* refs/pull/24158/head:
	cephfs: remove vestiges of mds deactivate

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2018-09-24 14:39:45 -07:00
Kefu Chai
4b0e2c8ed4 qa: fix typos
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-21 12:41:42 +08:00
Patrick Donnelly
f113fa80a9
cephfs: remove vestiges of mds deactivate
Fixes: http://tracker.ceph.com/issues/24001

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-19 15:23:38 -07:00
Sage Weil
4591f134c8 Merge PR #24025 into master
* refs/pull/24025/head:
	mon/OSDMonitor: reject pg_num -> 0

Reviewed-by: huangjun <huangjun@xsky.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-09-12 11:51:58 -05:00
xie xingguo
6d8d20a55b mon/OSDMonitor: reject pg_num -> 0
Otherwise it will finally crash the mgr process:
```
-2> 2018-09-11 16:23:31.272 7ff521d59700 -1 /clove/xxG/ceph/src/osd/osd_types.cc: In function 'pg_t pg_t::get_parent() const'
 thread 7ff521d59700 time 2018-09-11 16:23:31.272096
/clove/xxG/ceph/src/osd/osd_types.cc: 587: FAILED ceph_assert(bits)
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-09-12 08:58:10 +08:00
Sage Weil
4fc02a7f48 osd/OSDMap: include age in up and in counts for ceph status
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-07 09:07:50 -05:00
Sage Weil
eaca033d17 qa/workunits/cephtool: remove auid tests
Signed-off-by: Sage Weil <sage@redhat.com>
2018-08-31 09:26:19 -05:00
Sage Weil
aa485c9b41 mon/PGMap: fix spacing around pretty-printed SI units
"1k foo" not "1kfoo"

Signed-off-by: Sage Weil <sage@redhat.com>
2018-08-08 10:55:18 -05:00
Sage Weil
95b2281f9d mon/OSDMonitor: add 'osd repeer <pgid>' command
Selecting force peering on a single PG.  In reality this probably induces
*2* interval changes.

Note that in the case of a single OSD cluster we can't actually force a
repeer on a single PG because the pg_temp code is pretty robust about
filtering out redundant or meaningless changes, so we can't pg_temp our
way into a new interval if there are no other OSDs to switch to and the
code also prevents an empty pg_temp.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-09 16:33:11 -05:00
Sage Weil
761f68dc0d mon: destroy-new -> purge-new
What we actually want is a purge, not a destroy.  Destroy leaves the OSD
ID in used and allows it to be recreated.  What ceph-volume wants is to
purge all trace of the failed OSD setup.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-15 14:11:01 -05:00
Sage Weil
547d96dc01 mon: add 'osd destroy-new' command that only destroys NEW osd slots
ceph-volume may run into a problem and want to clean up, but we do not
want to give it blanket access to the 'osd destroy' command.  Instead,
make an 'osd destroy-new' that can only create new OSDs (ones that are
in the process of being created but have never booted yet).

Signed-off-by: Sage Weil <sage@redhat.com>
2018-06-05 17:36:06 -05:00
Sage Weil
b18fdb6e10 qa/workunits: osd require-osd-release mimic -> nautilus
Signed-off-by: Sage Weil <sage@redhat.com>
2018-05-25 15:19:28 -05:00
Sage Weil
0b956ef37d qa/workunits/cephtool/test.sh: update unit formatting
Signed-off-by: Sage Weil <sage@redhat.com>
2018-05-18 13:53:04 -05:00
Kefu Chai
5910c2902e qa: fix test on "ceph fs set cephfs allow_new_snaps"
snapshot is now enabled for new fs in 81d0554e, so this test should be
updated accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-05 09:40:41 +08:00
Kefu Chai
106737323d qa/workunits/cephtool/test.sh: fix test_mon_mds()
"ceph fs set cephfs allow_multimds false" is deprecated, and multimds is
enabled by default, so "ceph fs set cephfs max_mds 4" won't fail with
the default settings.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 09:20:35 +08:00
Patrick Donnelly
bb95292213
Merge PR #16608 into master
* refs/pull/16608/head:
	qa: whitelist mds down wrn during cephfs testing
	mds: add config to disable fragmentation
	qa: add max_mds thrash test
	qa: mds_thrash updates for new max_mds behavior
	doc: update upgrade procedure and release notes
	qa: add test for cluster resizing
	qa: remove use of mds deactivate
	cephfs: add new down/joinable fs flags
	mds: evict all clients if last mds shutting down
	cephfs: deprecate ceph mds deactivate
	cephfs: kill allow_dirfrags
	cephfs: Kill allow_multimds
	cephfs: Change behavior of cluster_down flag
	mon/FSCommands: Set extra MDS to standby
	cephfs: Health check changes
	mon/MDSMonitor: Remove command support for legacy syntax

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-18 14:36:54 -07:00
Douglas Fuller
ae3992ce4f
cephfs: Kill allow_multimds
With multi-mds now declared stable, allow_multimds now defaults to 1.
Given the max_mds parameter, it is now redundant. Remove it, leaving a
comment placeholder in the features bitmap.

ceph fs set <fs> allow_multimds is now deprecated and prints a warning
message.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Kefu Chai
d4186fb541
Merge pull request #19117 from jan--f/jan-object-counts-decimal
include/types: format decimal numbers with decimal factor

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-15 22:49:50 +08:00
Jan Fajerski
61504f117c qa/workunits/cephtool/test.sh: fix SI unit test, add IEC unit test
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-04-13 18:07:33 +02:00
Sage Weil
497a63251b qa/workunits/cephtool: test with valid auth syntax
'allow' is not a valid mon cap; use 'allow *'.  Add some negative tests
to ensure the validation is working.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-10 07:39:06 -05:00
Jan Fajerski
d3cecebacd include/types: format decimal numbers with decimal factor
Until now bytes and objects were formatted using si_t which used 1024 as
the factor to pretty print large numbers. For object counts a factor of
1000 is preferred. This commit retires the si_t formatting (as well as
prettybyte_t, kb_t and pretty_si_t) completely and adds structs and
formatting for binary and decimal units, bin_u_t and dec_u_t respectively.
Fixes: http://tracker.ceph.com/issues/22095

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-04-03 09:23:24 +02:00
Kefu Chai
c1ba9ac544
Merge pull request #20711 from guzhongyan/mgr-nodels
mon: 'node ls' mgr support

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-08 23:14:50 +08:00