Commit Graph

101756 Commits

Author SHA1 Message Date
Kefu Chai
228916db34 osd: use get_req<> instead of static_cast<>(get_req())
less repeating, better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-30 14:28:35 +08:00
Kefu Chai
25d78984f7 mon: use get_req<> instead of static_cast<>(get_req())
less repeating, better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-30 14:27:25 +08:00
Yan Jun
ce49112bbd osd: drop useless get_possibly_deleted_pool_pg_num
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2019-08-30 13:55:16 +08:00
Yan Jun
db681a7ef2 osd: no need to dump empty waiting_for_split
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2019-08-30 13:51:16 +08:00
Kiefer Chang
4c1c9438c3
mgr/dashboard: Add frontend unit tests for orchestrator components
- Add unit tests for hosts, inventory, and services
- Refine permission of host-details component

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2019-08-30 13:36:21 +08:00
Kiefer Chang
aab092ba10
mgr/dashboard: Add Teuthology tests for orchestrator controllers
- `/api/host`: list hosts (with orchestrator hosts)
- `/api/orchestrator/inventory`: list inventory
- `/api/orchestrator/services`: list services

Depends on https://github.com/ceph/ceph/pull/29595

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2019-08-30 13:36:21 +08:00
Kiefer Chang
9f3a1dfd7e
mgr/dashboard: Add backend unit tests for orchestrator controllers
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2019-08-30 13:36:21 +08:00
Kiefer Chang
7e20b5c734
mgr/dashboard: orchestrator integration initial works
- Display hosts, inventory, and services from orchestrator
- Allow adding/removing hosts

Fixes: https://tracker.ceph.com/issues/40337
Fixes: https://tracker.ceph.com/issues/40336
Fixes: https://tracker.ceph.com/issues/38233

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2019-08-30 13:36:17 +08:00
Kefu Chai
f154153c5d
Merge pull request #29982 from falcon78921/wip-doc-41516
doc: updated ceph monitor config options

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-30 12:22:50 +08:00
James McClune
8d8cbabff4 doc: updated ceph monitor config options
Executed ceph-conf --dump-all on a freshly installed v14.2.2 (nautilus)
cluster. Compared the global defaults to the keys/values specified in
mon-config-ref.rst. Checked options.cc to make sure the obsolete keys
are no longer used.

Fixes: https://tracker.ceph.com/issues/41516
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2019-08-29 23:54:10 -04:00
xie xingguo
6afe4f16db osd: prime splits/merges for any potential fabricated split/merge participant
If we have to fabricate a merge target, we need to prime any future splits
it might have.  Otherwise a sequence like

- e100 1.f merge to 1.7
- e110 1.7 split to 1.f, 1.17, 1.1f

where we process all of the above in one go at, say, e120, will lead to
a crash in register_and_wake_split_child because 1.17 and/or 1.1f aren't
primed.

Fix by making identify_splits_and_merges do a recursive scan on any
merge/split participants detected too.

Fixes: http://tracker.ceph.com/issues/38483
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-30 08:24:30 +08:00
Neha Ojha
2758ba1fb9 mon/Monitor.cc: fix condition that checks for unrecognized auth mode
Introduced by 771682aad0

Fixes:https://tracker.ceph.com/issues/41429
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-08-29 10:37:24 -07:00
J. Eric Ivancich
a0fee94982 rgw: make rollback refcount tag match
When the refcount does a "get", the tag ends with a null character,
but during a rollback, when the tag is "put" it does not end with a
null character, so the same tag is not being manipulated and the
refcounts are therefore off, preventing future deletion of an rgw
object. This adds the null character to the "put" operation.

This change was originally submitted by lltlo <120680451@qq.com> but
wasn't signed-off.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-08-29 10:47:42 -04:00
J. Eric Ivancich
7d32b082f8 rgw: make sure object's idtag is updated when available
This change was originally submitted by lltlo <120680451@qq.com> but
had some formatting issues and wasn't signed-off.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-08-29 10:47:27 -04:00
Neha Ojha
1d5a1e0eb9
Merge pull request #29939 from majianpeng/bluestore-print-correctly-info
os/bluestore: print correctly info.

Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-08-29 07:36:02 -07:00
Kefu Chai
26eb51fbbc
Merge pull request #30005 from wjwithagen/wjw-fix-vstart-mkdir
vstart.sh: Make sure mkdir succeeds

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-29 22:35:22 +08:00
Volker Theile
a973cb3486 mgr/dashboard: Fix pool deletion e2e
Signed-off-by: Volker Theile <vtheile@suse.com>
2019-08-29 16:25:51 +02:00
Volker Theile
75e9d9a0f0 mgr/dashboard: A block-manager can not access the pool page
Fixes: https://tracker.ceph.com/issues/41573

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-08-29 15:45:54 +02:00
Willem Jan Withagen
4f1c7815af vstart.sh: Make sure mkdir succeeds
Even in the case the full path was not created, make sure mkdir
does create the full path.

And while there replaced some `test ....` with `[ ... ]`
like in the remainder of the file.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-08-29 15:20:46 +02:00
Nathan Cutler
deca13e83a script/ceph-backport.sh: wholesale refactor
This commit refactors the script to make it more user-friendly and
maintainable. Added features:

* script now determine release/milestone from backport issue instead of
  requiring user to provide it
* script now generates a more verbose PR description
* backport PR title is based on original PR instead of the tracker issue
* improved error handling
* new --debug option that triggers "set -x"
* new --verbose option
* errors and diagnostic messages are now printed to stderr

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-08-29 14:35:36 +02:00
Abhishek Lekshmanan
796dfce8f5 doc: release notes for v14.2.3 nautilus
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-29 13:12:06 +02:00
lei01.liu
da088211c6 mgr/prometheus: export standby mds metadata
Signed-off-by: lei01.liu <lei01.liu@horizon.ai>
2019-08-29 18:28:54 +08:00
Rishabh Dave
20d6264538 ceph-volume: test number of times LVs list was created
Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-08-29 15:49:45 +05:30
Kefu Chai
48d6afb9e8
Merge pull request #29917 from tchaikov/wip-mon-always-on-modules
mgr/MgrMonitor: print pending.always_on_modules before updating it

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-29 17:31:54 +08:00
Kefu Chai
94b883911e
Merge pull request #29857 from dzafman/wip-41255
backfill_toofull seen on cluster where the most full OSD is at 1%

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-29 17:30:58 +08:00
Kefu Chai
5a5d417620
Merge pull request #29604 from votdev/python_cleanup
mgr/dashboard: Cleanup Python code

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-29 17:29:40 +08:00
Ricardo Dias
ea4236550e
mgr/dashboard: access_control: add grafana scope read access to *-manager roles
Fixes: https://tracker.ceph.com/issues/41572

Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-08-29 10:28:36 +01:00
Volker Theile
7d490c05d3
mgr/dashboard: Extract E2E wait calls into PageHelper (#29789)
mgr/dashboard: Extract E2E wait calls into PageHelper

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-08-29 11:02:42 +02:00
Kefu Chai
fa4d8022bc
Merge pull request #29881 from tchaikov/wip-test-bufferlist
test/bufferlist.cc: encode/decode int64_t instead of long

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-08-29 09:24:56 +08:00
Brad Hubbard
bedd022fe2
Merge pull request #29916 from badone/wip-tracker-41518-grafana-server
qa/ceph-ansible: Disable dashboard

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-08-29 09:52:33 +10:00
Tiago Melo
89e1cba2a2 mgr/dashboard: Extract E2E wait calls into PageHelper
Signed-off-by: Tiago Melo <tmelo@suse.com>
2019-08-28 23:03:05 +00:00
Patrick Donnelly
229e3d5223
Merge PR #20160 into master
* refs/pull/20160/head:
	cephfs: Make mds-mds per message versioned

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-28 15:02:35 -07:00
Patrick Donnelly
cb51f3d817
Merge PR #29911 into master
* refs/pull/29911/head:
	qa: ignore RECENT_CRASH for multimds snapshot testing

Tested-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-28 13:44:33 -07:00
Ilya Dryomov
918fd28f88
Merge pull request #29966 from idryomov/wip-krbd-zero-after-overlap
qa: krbd_parent_overlap.t: fix read test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-08-28 20:34:04 +02:00
Abhishek L
6171399fde
Merge pull request #29967 from theanalyst/rgw/beast-endpoint-fix
rgw: asio: check the remote endpoint before processing requests

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2019-08-28 17:33:24 +02:00
Willem Jan Withagen
d411ee26fb bluestore/bdev: initialize size when creating object.
The other partss of the struct are initialized by their
ctors. Only for unint_64 there is no ctor.

Otherwise ceph-dencoder tests will fail in comparing
the exported output.
[~/master36] wjw@cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 1 encode export /tmp/typ-yFISvjvgj
[~/master36] wjw@cephdev.digiware.nl> hexdump -C !$
hexdump -C /tmp/typ-yFISvjvgj
00000000  62 6c 75 65 73 74 6f 72  65 20 62 6c 6f 63 6b 20 |bluestore block |
00000010  64 65 76 69 63 65 0a 30  30 30 30 30 30 30 30 2d |device.00000000-|
00000020  30 30 30 30 2d 30 30 30  30 2d 30 30 30 30 2d 30 |0000-0000-0000-0|
00000030  30 30 30 30 30 30 30 30  30 30 30 0a 02 01 28 00 |00000000000...(.|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000050  00 00 70 74 03 04 00 00  00 00 00 00 00 00 00 00 |..pt............|
00000060  00 00 00 00 00 00 00 00  00 00 |..........|
0000006a

[~/master36] wjw@cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 1 encode decode encode export /tmp/typ-MjWXdCpzJ
[~/master36] wjw@cephdev.digiware.nl> hexdump -C !$
hexdump -C /tmp/typ-MjWXdCpzJ
00000000  62 6c 75 65 73 74 6f 72  65 20 62 6c 6f 63 6b 20 |bluestore block |
00000010  64 65 76 69 63 65 0a 30  30 30 30 30 30 30 30 2d |device.00000000-|
00000020  30 30 30 30 2d 30 30 30  30 2d 30 30 30 30 2d 30 |0000-0000-0000-0|
00000030  30 30 30 30 30 30 30 30  30 30 30 0a 02 01 28 00 |00000000000...(.|
00000040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000050  00 00 73 64 00 00 00 00  00 00 00 00 00 00 00 00 |..sd............|
00000060  00 00 00 00 00 00 00 00  00 00 |..........|
0000006a

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-08-28 17:19:22 +02:00
Ilya Dryomov
f47e953420 qa: krbd_parent_overlap.t: fix read test
It turns out this test has not been triggering the right code path for
a while now (krbd_parent_overlap.t was added just over a year ago, but
the original test goes back a few years).

To do its job it needs an empty clone with the overlap set to somewhere
in the middle of an object.  The test relied on shrinking for this, but
shrinking no longer fits the bill because a copyup is now performed for
the object with the overlap -- we never go to the parent after that.

Let's size the parent so that it ends in the middle of an object and
then grow an empty clone instead.  Also, while reading from page cache
does seem to trigger a read that straddles the overlap, it's not very
reliable, so supplement it with a direct read.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-08-28 17:07:07 +02:00
Nathan Cutler
9ec83c094c
Merge pull request #29948 from smithfarm/wip-ceph-backport-sh-doc
doc: script/ceph-backport.sh: add "Troubleshooting notes"

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-08-28 16:34:11 +02:00
Nathan Cutler
5fa672d878
Merge pull request #29950 from smithfarm/wip-bri-deep-scan-of-pr-desc
script/backport-resolve-issue: handle tracker URLs better

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-28 16:33:37 +02:00
Abhishek Lekshmanan
caa6531968
rgw: asio: check the remote endpoint before processing requests
`socket.remote_endpoint()` can throw exceptions corresponding to errors in the
`getpeername` syscall, make sure these are handled.

Fixes: CVE-2019-10222, https://tracker.ceph.com/issues/40018
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-28 15:56:12 +02:00
Casey Bodley
9a0740b0d1
Merge pull request #29815 from smanjara/wip-fix-bucket-link
rgw: fix missing tenant prefix in bucket name during bucket link

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-08-28 08:10:11 -04:00
Nathan Cutler
2c8e3a3688 script/backport-resolve-issue: handle tracker URLs better
Before this commit, backport-resolve-issue would blindly grab the first tracker
URL it saw in the GitHub PR description. With this commit, it examines all the
tracker URLs it sees in the PR description and uses the first one that
corresponds to an issue in the Redmine Backport tracker.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-08-28 13:41:01 +02:00
Volker Theile
7eac5cb3ac
mgr/dashboard: migrate e2e tests to async await (#29758)
mgr/dashboard: migrate e2e tests to async await

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-08-28 12:26:17 +02:00
xie xingguo
93cf4f8bea osd/PrimaryLogPG: block obc for any inflight manifest ops
otherwise it is likely to race with any follow-up scrubbing activities.
Note that it's ok for us to requeue_scrub before re-executing the op
since scrubbing will be blocking for the (same) global pg lock we
are currently holding.

Fixes: https://tracker.ceph.com/issues/41427
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-28 18:08:07 +08:00
xie xingguo
7f74461bcc osd/PrimaryLogPG: cancel in-flight manifest ops on interval changing
We don't want to cache any object_contexts through the interval change.
We actually assert that all currently live references are dead by the
time the flush for the next interval completes.

On detecting a interval changing, let's cancel any in-flight manifest
ops, just as we do for any other cache-tier ops.

Fixes: https://tracker.ceph.com/issues/41514
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-28 18:03:20 +08:00
Kefu Chai
218fa12f0e
Merge pull request #29942 from tchaikov/wip-bump-seastar
seastar: pick up latest changes and cleanups

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2019-08-28 17:18:16 +08:00
Nathan Cutler
655d42bcc3 doc: script/ceph-backport.sh: add "Troubleshooting notes"
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-08-28 10:42:17 +02:00
Patrick Seidensal
70144d2119 mgr/dashboard: fix protractor.conf.js
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2019-08-28 09:40:41 +02:00
Patrick Seidensal
fc6605107e mgr/dashboard: set sensible implicitWaitTimeout
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2019-08-28 09:40:24 +02:00
Patrick Seidensal
982da4cff9 mgr/dashboard: fix issue in dashbboard test
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2019-08-28 09:38:49 +02:00