Commit Graph

563 Commits

Author SHA1 Message Date
Nathan Cutler
4f0cecbadd qa/cephtool: clean up after removal of cache-tier modes
The "proxy" and "forward" cache-tier modes have been completely removed,
so it's sufficient to test once that they cannot be set.

Fixes: a0a3ed324a
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-05-21 13:42:09 +02:00
Neha Ojha
61ad12e6ad
Merge pull request #34541 from neha-ojha/wip-balancer-on
mgr: turn on balancer in upmap mode by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-15 15:03:28 -07:00
Neha Ojha
bca3b3f6e0 qa/workunits/cephtool/test.sh: don't set-require-min-compat-client luminous
This should not be required since mon_osd_initial_require_min_compat_client
is now luminous.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-04-14 00:05:02 +00:00
Sage Weil
6833f89990 qa/workunits/cephtool/test.sh: fix require-osd-release test
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-08 08:09:41 -05:00
Deepika Upadhyay
21508bd9dd mon/OSDMonitor: add flag --yes-i-really-mean-it for setting pool size 1
Adds option `mon_allow_pool_size_one` which will be disabled by default
to ensure pools are not configured without replicas.
If the user still wants to use pool size 1, they will have to change the
value of `mon_allow_pool_size_one` to true and then have to pass flag
`--yes-i-really-mean-it` to cli command:

Example:
`ceph osd pool test set size 1 --yes-i-really-mean-it`

Fixes: https://tracker.ceph.com/issues/44025
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-03-09 23:27:36 +05:30
Sage Weil
3eb830e938 Merge PR #33188 into master
* refs/pull/33188/head:
	qa/workunits/cephtool/test.sh: delete test_erasure pool

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-02-11 08:28:27 -06:00
Sage Weil
4332eeb694 qa/workunits/cephtool/test.sh: delete test_erasure pool
Other parts of this script leave OSDs reweighted, which can make this test
fail to go fully clean.

 0   ssd 0.08789         osd.0          up  0.63213 1.00000
 1   ssd 0.08789         osd.1          up  0.63213 1.00000
 2   ssd 0.08789         osd.2          up  1.00000 1.00000

35.0 raw ([2,1,2147483647], p2) up ([2,1,2147483647], p2) acting ([2,1,2], p2)

Fix by just deleting this pool when we're done.

Fixes: https://tracker.ceph.com/issues/44067
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-10 11:25:51 -06:00
Sage Weil
013a5cc3f8 ceph.in: fix -h/help behavior for tell commands
- add "Tell $type commands" heading
- 'ceph tell mon.a -h' now works
- 'ceph tell mon.a prefix -h' also works

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-10 08:24:44 -06:00
Sage Weil
780b0c1205 mon/OSDMonitor: accept 'autoscale_mode' argument to 'osd pool create'
Allow the autoscale mode to be set atomically with pool creation.

Fixes: https://tracker.ceph.com/issues/42638
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-06 07:04:00 -06:00
Kefu Chai
acbd038522 mon: drop "mon compact" command
quote from Sage's reply

> This is a mon-specific command--it doesn't make sense as a CLI command
> for the entire cluster--it only makes sense as a command to tell a
> specific monitor.  Like ``ceph tell mon.a compact``.  Back when Joao
> did #4595 these commands were all mixed together and putting it under
> 'ceph mon ...' made sense, but now you're specifially sending it to a
> mon, so the 'mon' part of the command is redundant.

so let's drop "mon compact" in favor of "compact" command

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-20 15:56:04 +08:00
Kefu Chai
e9a5ce0897 mon: mark "scrub" commands obsolete
"scrub" was marked deprecated in
1814d7441b. this commit
was in turn included by v10.0.0. so it's long enough for its
retirement.

the test is updated accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-20 15:56:04 +08:00
Sage Weil
253a557f10 qa/workunits: python -> python3 throughout
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-19 12:38:47 -06:00
Kefu Chai
4594ab9aa7 qa/workunits/cephtool/test.sh: set min_{bytes,pgs}_per_osd for reweight
otherwise ceph-mgr will refuse to perform reweight

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 21:17:57 +08:00
Kefu Chai
abc237960a qa/workunits/cephtool/test.sh: create pool with power-of-2 pg num
otherwise wait_for_health() fails like:

wait_for_health:  ceph health detail
HEALTH_WARN 1 pool(s) have non-power-of-two pg_num
[WRN] POOL_PG_NUM_NOT_POWER_OF_TWO: 1 pool(s) have non-power-of-two pg_num
    pool 'rbd' pg_num 10 is not a power of two
../qa/workunits/cephtool/../../standalone/ceph-helpers.sh:1613: wait_for_health:  return 1

the failure was found when testing test_mon_pg().

this behavior was introduced by 6e46b1c0e5

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 21:17:57 +08:00
Kefu Chai
2977086d1d qa/workunits/cephtool/test.sh: use "ceph compact" in test_mon_deprecated_commands()
both "compact" and "scrub" are now cli command, so we cannot issue them
using "tell" anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 21:17:56 +08:00
Kefu Chai
e96df8c44b qa/workunits/cephtool/test.sh: use "sessions" in test_mon_tell()
"version" is not an asok command anymore in the sense that it's served
by registered asock hook. so in this change, we replace "version" with
"sessions", so we can verify that audit channel does not the dispatched
"sessions" command sent from ceph cli.

also, restructure the test as a loop for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 17:08:07 +08:00
Kefu Chai
6f7842080d qa/workunits/cephtool/test.sh: use tell instead of --admin-daemon for asok commands
ceph cli will be able to route the command to the target daemon when
handling "tell" commands.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 13:51:19 +08:00
Kefu Chai
5bec1b3f7d qa/workunits/cephtool/test.sh: remove heap cli tests
"heap" cli commands are removed in the big asok/tell cleanup, they are now
asock commands.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 13:51:19 +08:00
Kefu Chai
d28057a487 qa/workunits/cephtool/test.sh: fix iec/si tests
tests all IEC and SI units related test with a tier pool. as
`target_max_objects` and `target_size_bytes` only apply to tier
pools. so, for the sake of simplicity, tests all of them using
a tier pool.

introduced by 9095f67e

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-18 13:51:19 +08:00
Kefu Chai
5830813f2e
Merge pull request #31291 from liewegas/bug-42578
mgr/ActivePyModules: behave if a module queries a devid that does not exist

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-05 04:17:17 -08:00
Sage Weil
67e5139b02 mgr/ActivePyModules: behave if a module queries a devid that does not exist
The GIL reacquire was being handled in the lambda, but that was not
getting called if the device didn't exist, leading to a crash.

Add a trivial CLI test.

Fixes: https://tracker.ceph.com/issues/42578
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-31 10:44:16 -05:00
Sage Weil
9095f67e92 mon/OSDMonitor: expand iec_options for 'osd pool set'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-28 10:27:53 -05:00
Sage Weil
fab205d45a qa/workunits/cephtool/test.sh: fix 'quorum enter' caller
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-12 12:05:36 -05:00
Sage Weil
70367de903 qa: change mon_status calls to quorum_status or tell commands
The tests were doing logs of 'ceph mon_status'; change that to
quorum_status or tell.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-12 12:05:36 -05:00
Sage Weil
e3260a81d5 mon: consolidate 'sync force' commands
Replace the 'ceph [mon] sync force' commands and just use the asok
sync_force command instead.  This is a low-level command that nobody should
reasonsbly using except in an emergency, so do not bother with trying to
maintain compatibility; it's a bit rediculous that we had 3 variations of
this to being with!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-12 12:05:36 -05:00
Sage Weil
bbc7bb5a22 Merge PR #30217 into master
* refs/pull/30217/head:
	crimson: common/admin_socket kludge so that it builds
	mon/MonClient: fix sending mon command to a specific rank
	src/.gitignore: ignore .tox
	mon/MonClient: interpret numeric mon target name as rank
	mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
	qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
	common/ceph_context: make 'config unset' idempotent
	qa/tasks/dump_stuck: mon.a, not mon.0
	qa/suites/rados/singleton/all/admin-socket: fix test
	common/config: EPERM setting config option after startup
	qa/workunits/cephtool/test.sh: fix tell output error check
	common/admin_socket: pass Formatter from generic infrastructure
	common/admin_socket: pass ostream to call() for error output
	os/bluestore: fix asok hook return value
	rgw: fix asok return value
	common/ceph_context: return error code from asok commands
	test/pybind/test_rados: fix accidental mon tell test
	mon: print entity_name along with caps to debug log
	PendingReleaseNotes: notes about asok changes
	mgr/MgrClient: empty target string for 'tell' means active mgr
	common/admin_socket: report error code as part of output string
	osd: change trigger_[deep_]scrub tommands to a pg tell command
	osd: remove old command workqueue, threadpool
	osd: drop MMonCommand handling
	osdc/Objecter: resend OSD tell commands on EAGAIN
	osd: route tell commands to asok; migrate commands
	osd: use unique_ptr<Formatter> for asok_command
	common/ceph_context: add generic asok 'injectargs'
	common/admin_socket: allow dup prefixes
	common/admin_socket: refactor with sync and async execute_command variants
	common/admin_socket: pass input bufferlist
	osd: transition to call_async() for asok
	common/admin_socket: support alternative call_async()
	mon/MonClient: send tell commands out of band via MCommand
	mon: accept tell commands via MCommand and send them to asok handler
	common/admin_socket: return int from hook call()
	mgr/DaemonServer: route MCommand (for octopus+) to asok commands
	do not use 'ceph tell mgr'
	pybind/ceph_argparse: disambiguate mgr tell and CLI commands
	ceph: make 'ceph tell mgr.*' send to the active mgr
	ceph: send 'ceph tell mgr.X' to the right mgr
	librados: add rados_mgr_command_target
	mgr/MgrClient: add start_command variant that takes a target
	common/admin_socket: drop unregister_command(); use per-hook variant
	common/admin_socket: drop explicit prefix arg to register_command
	common/admin_socket: simplify command routing
	common/admin_socket: add ability to process MCommand via asok queue
	common/admin_socket: pass cmdvec to execute_command
	common/admin_socket: use pipe for general wakeup
	include/compat: add flags arg to pipe_cloexec
	common/admin_socket: drop unused args

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-06 09:08:28 -05:00
Sage Weil
99f940641f qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
Error output has changed slightly.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:03 -05:00
Sage Weil
05bc99f82e qa/workunits/cephtool/test.sh: fix tell output error check
It goes to stderr instead of stdout now.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:03 -05:00
Sage Weil
8128045511 do not use 'ceph tell mgr'
This is now for a daemon/asok command, not a module command.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-01 16:30:53 -05:00
Kefu Chai
4ecb5dccbd mon/OSDMonitor: do not handle "osd set/unset" full
this flag is not honored by client or cluster anymore

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-28 16:09:58 +08:00
Jeff Layton
9c406d0ab3 mon: deprecate CephFS inline_data support
The plan is to start deprecating this feature now so that we can remove
it in a future release. Change it to require the
--yes-i-really-really-mean-it flag, and to emit a custom
warning when that isn't specified.

For now, we leave the testing in place since we do want to be notified
if something breaks before we're ready to rip it out completely.

Fixes: https://tracker.ceph.com/issues/41311
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-19 09:15:13 -04:00
Joao Eduardo Luis
85d5e29f66 mon: add osd info to obtain info on specific osd
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-07-15 17:13:00 +00:00
Sage Weil
e577581d45 qa/workunits/cephtool/test.sh: cache forward mode is no longer allowed
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-10 17:39:19 -05:00
Gu Zhongyan
73e88f7b72 qa/workunits: add auth caps allow command case
this case introduces multiple quotes in caps line
it will trigger the bug like http://tracker.ceph.com/issues/22227

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
2019-06-25 15:10:07 +08:00
xie xingguo
ec27a162de mgr, osd: 'ceph osd df' by pool
Our test admin has been asking for this for the past few years:-)
Besides, this is also useful for operating on large Ceph clusters with
mutliple storage pools possibly spanning over all osds.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-18 20:29:40 +08:00
xie xingguo
199d92aec0 qa: tolerate wildcard position
It does not make much sense to add this kind of restrictions
as long as user is aware of what is going on.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-05-30 10:52:38 +08:00
xie xingguo
5d695267ef osd: add no{out,down,in,out} flags by device class
This works as a good supplement of https://github.com/ceph/ceph/pull/27563.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-05-30 10:52:35 +08:00
xie xingguo
9a05a065b9 osd,mon: add 'ceph osd {set,unset}-group <flags> <who>' command
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-05-30 09:46:51 +08:00
Joao Eduardo Luis
28c70208d8
Merge pull request #26488 from mogeb/per-pool-scrub-status
mon: show no[deep-]scrub flags per pool in the status

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2019-05-07 17:50:52 +01:00
David Zafman
39cc14bdc1
Merge pull request #27503 from dzafman/wip-39099
osd: Give recovery for inactive PGs a higher priority

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-25 15:06:56 -07:00
David Zafman
444aa9f9fe osd, mon: New pool recovery priority range -10 to 10
Use OSD_POOL_PRIORITY_MAX and OSD_POOL_PRIORITY_MIN constants
Scale legacy priorities if exceeds maximum

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-04-25 13:53:27 -07:00
xie xingguo
bae2231cc5 qa: add crush-node-flags test
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-23 14:20:17 +08:00
Sage Weil
3760e8f918 mon/OSDMonitor: add mon_debug_no_require_octopus
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-02 16:19:43 -05:00
Mohamad Gebai
aa9cb58390 test: add tests for per-pool scrub status
Signed-off-by: Mohamad Gebai <mgebai@suse.com>
2019-04-02 05:50:03 -04:00
xie xingguo
2f195b7072 mon: add set-weight command
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-20 11:23:32 +08:00
Sage Weil
ab0a652826 erasure-code: ensure m >= 1
Fixes: http://tracker.ceph.com/issues/38682
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-12 13:12:58 -05:00
Sage Weil
4c5ed29925 Merge PR #26764 into master
* refs/pull/26764/head:
	mgr: 'osd df' by specified class or (crush) name
	mon/OSDMonitor: add 'osd crush get-device-class' command
2019-03-07 08:52:56 -06:00
xie xingguo
af02d1031d mgr: 'osd df' by specified class or (crush) name
For large clusters, we use device classes to isolate storage pools.
The existing 'osd df' output turns out to be too nosiy, say, if
you care about only single storage pool with osds possibly spanning over
all hosts.

With this change you are now being able to do 'osd df' by class (or by pool,
if you simply use classes to separate different pools), or by a specified
crush bucket name you are currently interested in, which is much more
convenient.

Some examples:
```
$ bin/ceph osd df tree
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60 1.00   -        root default
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
-5       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
 1   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  61     up         osd.1
 2   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  51     up         osd.2
                    TOTAL 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree class aaa
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60 1.00   -        root default
-3       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
-5       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
                    TOTAL 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree name ceph11
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
                    TOTAL 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-06 11:10:56 +08:00
Xie Xingguo
ad8e7d33b1
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-03-05 20:16:18 +08:00
xie xingguo
733fff22bb mon/OSDMonitor: add boundary check for pool recovery_priority
See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 08:15:39 +08:00