Commit Graph

75875 Commits

Author SHA1 Message Date
Sage Weil
6c4992aeca qa/workunits/cephtool/test.sh: fix test to watch audit channel
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-21 11:40:48 +08:00
Dan Mick
47726f8b66 doc/mgr: add Prometheus plugin docs
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-07-20 20:25:08 -07:00
Dan Mick
3b1a8a78ac mgr/index.rst: clarify which toplevel items are plugins
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-07-20 20:09:39 -07:00
Dan Mick
c40c254cc6 doc/mgr/zabbix.rst: fix subheadings, add subsubheadings
(otherwise subheadings appear as top-level items in the TOC, because
they're marked just like the title)

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-07-20 20:09:05 -07:00
xie xingguo
396e2277d3 mgr: drop unnecessary write permission for test-reweight-by-* commands
dry run does not ask for it

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-07-21 10:50:14 +08:00
xie xingguo
3941bda996 mon/PGMap: fix unable to shut up pg-not-scrubbed[deep-scrubbed] warn
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-07-21 08:00:31 +08:00
Sage Weil
2e8413dede qa: remove workloadgen test
The CRUSH rule creation is busted (rules and buckets out of order), but
after I fix that it doesn't seem to run right anyway.  Remove it.
We get the mon thrasher coverage from rados/monthrash already; I don't
think this is adding meaningful coverage for the amount of effort it takes
to maintain.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 18:06:50 -04:00
Sage Weil
c39ebd62ab crush/CrushCompiler: enforce buckets before rules rule
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 18:04:44 -04:00
Sage Weil
91548a33ce osd: bail out if no map yet
If we get pings before finishing our startup we may not yet have an
osdmap.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 17:44:58 -04:00
Sage Weil
59e3827be7 qa/tasks/reg11184: import run
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 17:42:59 -04:00
Josh Durgin
2effa0fbe3 osd/PGLog: force write of may_include_deletes after split
When a collection is split this needs to be persisted again. Normally
this is only persisted when the missing set is rebuilt during a new
interval when it previous did not include deletes, but during split we
keep the in-memory missing set may_include_deletes flag, but do not
rebuild the missing set.

Fixes: http://tracker.ceph.com/issues/20704
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-07-20 16:27:27 -04:00
Josh Durgin
e36f4677f2 ceph_objectstore_tool: always write may_include_deletes if it's set
This prevents us from importing a missing set without also setting the
may_include_deletes_in_missing omap value if appropriate.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-07-20 15:44:39 -04:00
Sage Weil
8e6a15d01a msg: restore peer addr checks
This was dropped in bf49385679 but should
not have been.  Since we are advertising the addr and not the bind
addr there is no reason to disable this check.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 13:59:45 -04:00
Sage Weil
f159a093ec osd,mds,mgr: do not dereference null rotating_keys
Immediately after we bind to a port, but before we have set up our
auth infrastructure, we may get incoming connections.  Deny them.  Since
we are not yet advertising ourselves these are peers trying to connect
to old instances of daemons, not us.

This triggers now because of bf49385679.
Previously, the peer would see we were a different addr and drop the
connection.  Now, it continues.

Fixes: http://tracker.ceph.com/issues/20667
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 13:07:32 -04:00
Sage Weil
27e8d75f61 Merge pull request #16429 from liewegas/wip-jewel-x
qa/suites/upgrade/jewel-x: misc fixes for new health checks
2017-07-20 10:47:05 -05:00
Sage Weil
ed48ed1b74 Merge pull request #16374 from majianpeng/bluestore-double-count-kv-queue-lat
os/bluestore/BlueStore: Avoid double counting state_kv_queued_lat.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-07-20 10:44:40 -05:00
Sage Weil
3de9f22ce0 Merge pull request #16423 from liewegas/wip-ls
mon: '* list' -> '* ls'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-20 10:43:34 -05:00
Sage Weil
de856e7371 Merge pull request #16434 from badone/wip-fix-deep-age-reporting
mon: Fix deep_age copy paste error

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-20 10:43:15 -05:00
Kefu Chai
acc24bf0dc Merge pull request #16444 from tchaikov/wip-test-osd-stat
qa/workunits/cephtool/test.sh: "ceph osd stat" output changed, update accordingly

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2017-07-20 23:41:53 +08:00
Sage Weil
583a38bca2 qa/tasks/ceph_manager: wait for osd to start after objectstore-tool sequence
Fixes: http://tracker.ceph.com/issues/20705
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 11:41:36 -04:00
Yuri Weinstein
94b02a69d4 Merge pull request #14287 from oritwas/wip-rgw-usage
rgw: usage

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2017-07-20 08:39:21 -07:00
Yuri Weinstein
c31f25bedf Merge pull request #15673 from yehudasa/wip-20107
cls/refcount: store and use list of retired tags

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2017-07-20 08:35:07 -07:00
Kefu Chai
153d030e1d Merge pull request #16421 from tchaikov/wip-19939
messages/: always set header.version in encode_payload()

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-07-20 22:52:17 +08:00
Jos Collin
c6e90d93a3 Merge pull request #16418 from theanalyst/doc-rgw-tenant-names
doc: rgw clarify limitations when creating tenant names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-07-20 14:38:45 +00:00
Kefu Chai
3dfa9daeca Merge pull request #16443 from wjwithagen/bug-wjw-qa-test-reorder
cephtool/test.sh: Only delete a test pool when no longer needed.

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-07-20 22:13:37 +08:00
Jason Dillaman
b47eb210c8 Merge pull request #16451 from tchaikov/wip-qa-reg11184
qa/tasks/reg11184: use literal 'foo' instead pool_name

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-07-20 09:37:53 -04:00
Kefu Chai
a1d16185a2 qa/tasks/reg11184: use literal 'foo' instead pool_name
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-07-20 21:35:41 +08:00
Kefu Chai
ff039c082f Merge pull request #16412 from wido/zabbix-release-notes
doc: Add Zabbix ceph-mgr plugin to PendingReleaseNotes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-20 20:20:54 +08:00
Matt Benjamin
5f838c05c7 rgw_file: properly & |'d flags
Found by "Supriti Singh" <Supriti.Singh@suse.com>.

Fixes http://tracker.ceph.com/issues/20663

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-07-20 08:18:22 -04:00
Wido den Hollander
dacd41e043
release notes: Add Zabbix ceph-mgr module to PendingReleaseNotes
Signed-off-by: Wido den Hollander <wido@42on.com>
2017-07-20 14:08:45 +02:00
Abhishek Lekshmanan
878d8486be doc: rgw clarify limitations when creating tenant names
We only allow alphanumeric and underscore characters in tenant names
according to the validation in `RGWHandler_REST::validate_tenant_name`

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-07-20 14:01:07 +02:00
Kefu Chai
ba525a829c qa/workunits/cephtool/test.sh: "ceph osd stat" output changed, update test accordingly
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-07-20 19:34:53 +08:00
Willem Jan Withagen
e3760fa936 cephtool/test.sh: Only delete a test pool when no longer needed.
the pool_getset pool is deleted before all tests on it are complete

4: /home/jenkins/workspace/ceph-master/qa/workunits/cephtool/test.sh:1990: test_mon_osd_pool_set:  ceph osd pool delete pool_get
set pool_getset --yes-i-really-really-mean-it
4: pool 'pool_getset' removed
4: /home/jenkins/workspace/ceph-master/qa/workunits/cephtool/test.sh:1992: test_mon_osd_pool_set:  ceph osd pool get rbd crush_r
ule
4: /home/jenkins/workspace/ceph-master/qa/workunits/cephtool/test.sh:1992: test_mon_osd_pool_set:  grep 'crush_rule: '
4: crush_rule: replicated_rule
4: /home/jenkins/workspace/ceph-master/qa/workunits/cephtool/test.sh:1994: test_mon_osd_pool_set:  ceph -f json osd pool get poo
l_getset compression_mode
4: Error ENOENT: unrecognized pool 'pool_getset'

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-07-20 12:24:14 +02:00
Kefu Chai
aea471d73a Merge pull request #16403 from wjwithagen/bug-wjw-ceph-osd-stat
test: ceph osd stat out has changed, fix tests for that

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-20 18:06:47 +08:00
Ilya Dryomov
67db89f6c2 Merge pull request #16428 from idryomov/wip-krbd-luminous-thrash
qa: thrash tests for backoff and upmap

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
2017-07-20 11:28:22 +02:00
Piotr Dałek
ec7094cc8b PGMonitor: return -EPERM on pg force-* commands during upgrade
Return -EPERM on pg force-* commands during upgrade.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:56 +02:00
Piotr Dałek
b0134cc7a8 qa: add force/cancel recovery/backfill to QA testing
This randomly issues pg force-recovery/force-backfill and
pg cancel-force-recovery/cancel-force-backfill during QA
testing. Disabled for upgrades from hammer, jewel and kraken.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:55 +02:00
Piotr Dałek
3fea25e1f5 doc: document new force-recovery/force-backfill commands
Documentation for new pg force-recovery, pg force-backfill,
pg-cancel-force-recovery and pg-cancel-force-backfill.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:54 +02:00
Piotr Dałek
ff9a32d94b osd, pg: implement force_recovery
This commit implements the MOSDForceRecovery handler along with
all required code to have PGs processed in desired order
(PGs with force_recovery/force_backfill first).
Obviously it's not going to work cluster-wide and OSDs that are
not affected - but may affect affected OSDs - may cut into PG
recovery queue and cause PGs with force_* flags to get recovered
or backfilled later than expected, but still way earlier than
without it.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:54 +02:00
Piotr Dałek
50742c5cd0 OSD, PG: remove optional arg from queue_for_recovery
The optional arg ("front") was meant to control if PG was supposed
to be put in front or back (default) of awaiting_throttle. For some
reason, this was't used at all, so this commit removes it and
replaces with logic that checks whether the PG has forcecd
backfill or recovery set, and lets it in the front only in that
case.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:53 +02:00
Piotr Dałek
de0566f604 Mgr: implement force-recovery/backfill commands
Implement commands "pg force-recovery", "pg force-backfill", "pg
cancel-force-recovery" and "pg cancel-force-backfill" that accept
an one or more PG IDs and cause these PGs to be recovered or
backfilled first. "cancel-*" commands can be used to revert the
effect of "pg force-*" commands. 

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:53 +02:00
Piotr Dałek
8584836b9b messages: introduce MOSDForceRecovery
Introduce new message type (MOSDForceRecovery) that will be used to
force (or cancel forcing) PG recovery/backfill.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:52 +02:00
Piotr Dałek
14f66ba2e1 PG: introduce forced recovery/backfill
Reduce max automatically calculable recovery/backfill priority to 254
and reserve 255 for forced backfill/recovery, so recovery/backfill on
user-designated PGs can be requested before other currently backfilled
and/or recovered PGs. Clear PG_STATE_FORCED_BACKFILL and
PG_STATE_FORCED_RECOVERY once recovery/backfill is done.

Signed-off-by: Piotr Dałek <piotr.dalek@corp.ovh.com>
2017-07-20 09:35:52 +02:00
Kefu Chai
37a7035a8e Merge pull request #16424 from tchaikov/wip-assert-in-c
include/assert: test c++ before using static_cast<>

Reviewed-by: Sage Weil <sage@redhat.com>
2017-07-20 14:49:49 +08:00
Yang Honggang
671fdb8c96 osd: fix pg ref leaks when osd shutdown
Fixes: http://tracker.ceph.com/issues/20684
Signed-off-by: Yang Honggang <joseph.yang@xtaotech.com>
2017-07-20 14:01:16 +08:00
fang.yuxiang
957b9bdee2 rgw: fix rgw hang when do RGWRealmReloader::reload after go SIGHUP
Quota async processer reference count err when bucket has no explicit shard

Fixes: http://tracker.ceph.com/issues/20686

Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com
2017-07-20 13:27:53 +08:00
Sage Weil
25bbe36b98 Merge pull request #16400 from liewegas/wip-crush-rule-id
crush: s/ruleset/id/ in decompiled output; prevent compilation when ruleset != id

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2017-07-19 22:02:41 -05:00
Sage Weil
676ea6e2d7 Merge pull request #16402 from liewegas/wip-20105
ceph_test_rados_api_watch_notify: flush after unwatch

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-19 22:02:04 -05:00
Sage Weil
d6db84ca76 mon: hide 'osd crush rule list' (we prefer 'ls')
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-19 22:47:21 -04:00
Kefu Chai
c483e8b658 Merge pull request #16433 from dmick/wip-ccache
do_cmake.sh: fix syntax for /bin/sh (doesn't have +=)

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-20 10:09:28 +08:00