Commit Graph

267 Commits

Author SHA1 Message Date
Mykola Golub
e6dcf1419a osd: store per pool scrub intervals in pool options
ceph osd pool set $POOL scrub_min_interval N
  ceph osd pool set $POOL scrub_max_interval N
  ceph osd pool set $POOL deep_scrub_interval N

If N > 0, this value is used for the pool instead of
the corresponding global parameter from the config
(osd_scrub_min_interval, osd_scrub_max_interval or
osd_deep_scrub_interval).

Fixes: #13077
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-11-27 13:35:51 +02:00
Sage Weil
d633b64566 Merge pull request #5734 from cxwshawn/mon-fix
auth: make keyring without mon entity type return -EACCES

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-11-26 17:24:59 -05:00
Sage Weil
6cbdd6750c mon/PGMonitor: remove map_pg_creates, send_pg_creates commands
These shouldn't be triggered manually.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-11-23 08:38:50 -05:00
Xiaowei Chen
1ace4d03b2 auth: keyring without mon entity type should return -EACCES
test:
                         see test.sh:test_mon_caps
                         before modify:
                         when we first exec ../qa/workunits/cephtool/test.sh -t mon_caps --asok-does-not-need-root , it stuck.
                         after modify:
                         exec again, return Permission denied.

Signed-off-by: Xiaowei Chen <chen.xiaowei@h3c.com>
2015-11-10 02:54:35 -05:00
Sage Weil
8d97771c3c Merge pull request #6084 from liewegas/wip-revert-scrub
Revert "osd: new pool settings: scrub intervals"

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2015-09-28 16:40:31 -04:00
Sage Weil
80b7237e5e qa/workunits/cephtool/test.sh: don't assume crash_replay_interval=45
e.g., it's 5 in teuthology's ceph.conf.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-28 13:13:40 -04:00
Sage Weil
216eef53e9 Revert "osd: new pool settings: scrub intervals"
This reverts commit 30810da4b5.

After some discussion we have decided it is better to build a generic
dictionary in pg_pool_t to store infrequently used per-pool properties.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-25 14:01:19 -04:00
Mykola Golub
30810da4b5 osd: new pool settings: scrub intervals
ceph osd pool set $POOL scrub_min_interval N
 ceph osd pool set $POOL scrub_max_interval N
 ceph osd pool set $POOL deep_scrub_interval N

If N > 0, this value is used for the pool instead of
the corresponding global parameter from the config
(osd_scrub_min_interval, osd_scrub_max_interval or
osd_deep_scrub_interval).

Fixes: #13077
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-09-21 09:08:16 +03:00
Mykola Golub
48db7b16d4 osd: new pool flags: noscrub, nodeep-scrub
This allows to disable periodic scrub on per pool basis.

Fixes: #13076
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-09-21 09:08:16 +03:00
Mykola Golub
b97ae7660e osd: make 'ceph osd pool get' work for all settable pool flags
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-09-21 09:08:16 +03:00
Mykola Golub
10235e3ff3 osd: refactor setting write_fadvise_dontneed pool flag
Handle together with other flags.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-09-21 09:08:16 +03:00
Loic Dachary
21a1e75d8a tests: update to match crushmap validation message
http://tracker.ceph.com/issues/13182 Fixes: #13182

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-21 00:13:08 +02:00
Sage Weil
4a5a5b3705 qa/workunits/cephtool/test.sh: make mds epoch check more tolerant
This can race with an actual mdsmap epoch update for some other
reason.  We just need to make sure the epoch *increased*, not that
it is exactly old + 1.

Fixes: #12991
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-12 13:51:13 -04:00
Kefu Chai
09464ec6b7 Merge pull request #5570 from dachary/wip-12673-proxy-read
osd: trigger the cache agent after a promotion

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-08-31 12:04:59 +08:00
Samuel Just
afb4db73cc Merge remote-tracking branch 'origin/wip-sam-proxy-write'
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Conflicts:
	src/include/ceph_features.h
	src/osd/ReplicatedPG.cc
	src/osd/ReplicatedPG.h
2015-08-19 16:07:30 -07:00
Zhiqiang Wang
61c86a1281 test: change the test_tiering test case accordingly for proxy write
When an object is first created, it's proxied to base tier, need to
change the behavior of the test_tiering test case accordingly.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-08-18 11:25:24 -07:00
Loic Dachary
7924231930 tests: tiering agent and proxy read
Verify that an object promoted to a cache tier because of a proxy read
is evicted as expected.

http://tracker.ceph.com/issues/12673 Refs: #12673

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-08-14 14:31:52 +02:00
Sage Weil
383185bfba mon/OSDMonitor: osd set/unset sortbitwise
Add monitor command to flip the switch on the OSD hobject_t sort
order.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-07 16:23:46 -04:00
Josh Durgin
7d11b6b9d1 Merge remote-tracking branch 'origin/next'
Conflicts:
	src/Makefile-client.am (link rbd to libglobal again)
2015-08-04 18:13:52 -07:00
John Spray
cb51b17043 mon: reject over-large values of max_mds
Fixes: #12222
Signed-off-by: John Spray <john.spray@redhat.com>
2015-07-30 07:12:50 +01:00
Sage Weil
de8a950506 qa/workunits/cephtool/test.sh: escape osd.* and mon.*
Otherwise we get filenames if the glob happens to match something in the
source dir.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-07-23 17:04:53 -04:00
Kefu Chai
017e206cf1 qa/workunits: cephtool: take EOPNOTSUPP as an alias of ENOTSUP
the proble breaks `test_mon_deprecated_commands` on ubuntu precise,
on the python shipped with ubuntu precise, errno.errorcode[95]
evalutes to `EOPNOTSUPP` but not `ENOTSUP`. but these two errnos
are equal in glibc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-07-19 00:21:23 +08:00
Sage Weil
2746f59da4 Merge remote-tracking branch 'gh/next' 2015-07-16 12:03:58 -04:00
Joao Eduardo Luis
038452e70f mon: Monitor: use 'ceph mon metadata' instead of 'ceph mon_metadata'
'ceph mon_metadata' was added still during this dev cycle, so there is
no need to deprecate it first.

Fixes: #11545

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2015-07-16 15:54:03 +01:00
Joao Eduardo Luis
66e7510cad qa/workunits: cephtool: test deprecated commands
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2015-07-16 12:20:02 +01:00
Kefu Chai
2d471d0def Merge pull request #5143 from trociny/fix-mds_metadata
mon: fix mds metadata

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-07-16 14:41:41 +08:00
Samuel Just
4946d103de OSDMonitor: allow addition of cache pool with non-empty snaps with config
We need to be able to allow the version of ceph_test_* from earlier
versions of ceph to continue to work.  This patch also adjusts the
work unit to use a single rados snap to test the condition without
--force-nonempty to ensure that we don't need to be careful about
the config value when running that script.

Signed-off-by: Samuel Just <sjust@redhat.com>
2015-07-09 13:56:29 -07:00
Loic Dachary
d791a722a7 tests: improve shell output readability
Modify the test traces to include the file name in addition to the
function and line name. It makes it easier to locate the faulty line
without going back to the test name.

Format the trace lines to be emacs friendly (filename:lineno) so that
C-x ` or C-c C-c jumps to the right file and the right line when running
the test with M-x compile.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-07-08 16:14:01 +02:00
Mykola Golub
b53e3e275f qa: add tests for 'ceph mds metadata' command
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2015-07-07 09:59:32 +03:00
Li Wang
c1bd02c978 Merge pull request #4792 from kylinstorage/wip-writeback-throttling-for-cache-tiering
Wip writeback throttling for cache tiering

This patch is to do write back throttling for cache tiering, which is similar to what the Linux kernel does for  page cache write back.  A paramter 'cache_target_dirty_high_ratio'  (default 0.6) is introduced as the high speed flushing threshold, while leave the 'cache_target_dirty_ratio' (default 0.4) to represent the low speed  threshold. The flush speed is controlled by limiting the parallelism of flushing. The maximum parallelism under low speed is half of the parallelism under high speed. If there is at least one PG such that the dirty ratio beyond the high threshold, full speed mode is entered; If there is no PG such that dirty ratio beyond the low threshold, idle mode is entered; In other cases, slow speed mode is entered.

Signed-off-by: Mingxin Liu <mingxinliu@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>
Suggested-by: Nick Fisk <nick@fisk.me.uk>
Tested-by: Kefu Chai <kchai@redhat.com>
2015-06-23 15:15:47 +08:00
Gregory Farnum
5a3da7b891 Merge pull request #4602 from ceph/wip-11504
mon: refine check_remove_tier checks

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-06-05 11:13:42 -07:00
liumingxin
7aba9476e6 Doc: add write back throttling stuff in document and test scripts
Signed-off-by: Mingxin Liu <mingxinliu@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>
Suggested-by: Nick Fisk <nick@fisk.me.uk>
2015-06-03 15:57:14 +08:00
David Zafman
17f73fafc4 Merge pull request #4816 from dachary/wip-11624-cephtools-test
tests: cephtool/test.sh wait_for_clean after PG (re)mapping 

Reviewed-by: David Zafman <dzafman@redhat.com>
2015-06-02 17:28:37 -07:00
Kefu Chai
c2b06ab372 Merge pull request #4643 from tchaikov/wip-fix-11569
fix "pg ls" with states of "recovering" and/or "repair"

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-06-02 20:43:26 +08:00
John Spray
f9b11b0abe qa: update cephtool test for CephFS tier cases
1. Creating a filesystem using a
   readonly tier on an EC pool (should be forbidden)
2. Removing a tier from a replicated base pool (should
   be permitted)

Signed-off-by: John Spray <john.spray@redhat.com>
2015-06-01 14:16:15 +01:00
Loic Dachary
7e2a7a8685 tests: cephtool/test.sh wait_for_clean after PG (re)mapping
A get/set command may fail with

   Error EBUSY: currently creating pgs, wait

if issued before the PGs are clean. Call wait_for_clean after the pool
is created or a pool setting is changed and remaps the PGs it
contains (size, pg_num...) to ensure the PGs are clean and the set/get
command that follow will succeed.

http://tracker.ceph.com/issues/11624 Fixes: #11624

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-06-01 01:13:44 +02:00
Loic Dachary
f6e4000589 tests: reduce pg_num in test_mon_osd_pool_set
No need for more than one pg to test get/set on pools: it also is faster
to become clean.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-06-01 01:13:44 +02:00
Loic Dachary
d2cc2b184b tests: use get_pg from ceph-helpers.sh
The semantic and interface of get_pg are the same, that avoids
duplication and the ceph-helpers.sh version is tested and documented.

Make the ceph-test package dependent on xmlstarlet because it is
needed by ceph-helpers.sh.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-06-01 01:11:28 +02:00
Samuel Just
8a95f13a69 Merge remote-tracking branch 'upstream/wip-11493'
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-05-29 14:17:42 -07:00
John Spray
38a319d515 qa/cephtool: add blacklist json output check
...not very elegantly because this is bash, but
at least check the expected value is somewhere
present in the JSON output.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-26 10:58:32 +01:00
Loic Dachary
5c69f5e15f tests: ceph create may consume more than one id
When CEPH_CLI_TEST_DUP_COMMAND=1 is set, ceph osd create will consume
two osd id and return the later. Fix the test to account for that and
not assume the osd id being allocated by osd create is always the
next available osd id.

The other osd create tests do not suffer from the same variation because
they provide a UUID argument that guarantees the same osd id is going to
be returned every time.

http://tracker.ceph.com/issues/11618 Fixes: #11618

Signed-off-by: Loic Dachary <ldachary@redhdat.com>
2015-05-22 10:16:24 +02:00
Samuel Just
11b7801bb5 OSDMonitor: disallow ec pools as tiers
Fixes: 11650
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-15 13:20:00 -07:00
Sage Weil
bbec53edf9 mon: prevent pool with snapshot state from being used as a tier
If we add a pool with snap state as a tier the snap state gets clobbered
by OSDMap::Incremental::propogate_snaps_to_tiers(), and may prevent OSDs
from starting.  Disallow this.

Include a test.

Fixes: #11493
Backport: hammer, giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-15 10:39:49 -07:00
Loic Dachary
f88275dd04 tests: tiering health report reworked
Instead of

* setting limit
* populate the cache
* check the health warnings

do the following

* populate the cache
* set limits below the content of the cache
* check the health warnings

The problem with the former approach is that the limits stored by the
OSD internally do not exactly match the one set by the user: they are
converted in ratios and there may be rounding errors.

Also replace the busy loop waiting for pg stats to flush with

  ceph tell osd.* flush_pg_stats || true

for simplicity.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 10:39:37 +02:00
Loic Dachary
5f252d6135 tests: no agent when testing tiering agent border case
On a machine slow enough, the tiering agent can be activated while
testing border cases where the cache is almost full. Prevent that
by deactivating the tiering agent.

http://tracker.ceph.com/issues/11359 Fixes: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 09:28:06 +02:00
Loic Dachary
b2c40d5e76 tests: uncomment tiering agent tests
http://tracker.ceph.com/issues/11359 Refs: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 09:25:57 +02:00
Samuel Just
05db2b0a25 Merge pull request #4452 from tchaikov/wip-interactive-ceph-cli
Wip interactive ceph cli

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-11 10:29:40 -07:00
Kefu Chai
87f37b6b1a Merge pull request #4227 from tchaikov/node-ls
mon,osd,mds: add "{mon,mds} metadata" and "node ls" commands

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2015-05-11 22:01:21 +08:00
Kefu Chai
89f89ca347 mon: s/recovery/recoverying/ in "pg ls*" commands' spec
* also translate "repair" if specified as "states"
* update test_mon_pg in cephtool-test-mon.sh

Fixes: #11569
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-11 19:40:03 +08:00
Kefu Chai
410175be89 Merge pull request #3743 from trociny/wip-osd_create
mon: osd create: add optional 'id' parameter

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2015-05-08 00:13:37 +08:00