Commit Graph

97589 Commits

Author SHA1 Message Date
Sage Weil
0fe0ec3343 qa/suites/upgrade/*-x-singleton: enable bluestore debugging settings
We default to bluestore already; use the yaml explicitly so that we get
debugging turned up.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-25 08:33:56 -05:00
Casey Bodley
83ab6aac2a
Merge pull request #26107 from alimaredia/wip-dpp-rgw-put-obj-processor
rgw: log refactoring for putobj_processor

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-04-23 15:50:07 -04:00
Ali Maredia
f92a071e18 rgw: log refactoring for AtomicObjProcessor complete
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2019-04-23 15:07:37 -04:00
Sage Weil
760cb2222f Merge PR #27628 into master
* refs/pull/27628/head:
	common: optimize check_utf8

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-23 13:08:29 -05:00
Sage Weil
f7ceac90d1 Merge PR #27702 into master
* refs/pull/27702/head:
	osd/PG: fix last_complete re-calculation on splitting

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-23 13:06:52 -05:00
Sage Weil
fd3c323ffd Merge PR #27691 into master
* refs/pull/27691/head:
	os/bluestore: fix out-of-bound access in bmap allocator.
	os/bluestore: reproduce out-of-bound access for bmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-23 13:06:15 -05:00
Sage Weil
aa4adf8cc4 Merge PR #27707 into master
* refs/pull/27707/head:
	common/util: handle long lines in /proc/cpuinfo

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-04-23 13:05:44 -05:00
Patrick Donnelly
707ae12a29
Merge PR #27537 into master
* refs/pull/27537/head:
	mds: better output of 'ceph health detail'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-23 10:53:52 -07:00
Patrick Donnelly
97d6e9948e
Merge PR #27511 into master
* refs/pull/27511/head:
	mds: fix SnapRealm::resolve_snapname for long name

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-23 10:52:41 -07:00
Patrick Donnelly
f1defa179a
Merge PR #27077 into master
* refs/pull/27077/head:
	test: check listattr for snapshot btime entry
	test: extend LibCephFS.Xattrs test
	client: remove unused vxattr length helpers
	client: fix _listxattr() vxattr buffer length calculation
	test: add libcephfs snap.btime xattr test
	client: add ceph.snap.btime vxattr
	mds: carry snapshot creation time with InodeStat

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-04-23 10:50:49 -07:00
Casey Bodley
a487a86342
Merge pull request #27725 from theanalyst/perf-counter-names
rgw: sync counters: drop spaces from counter names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2019-04-23 12:48:55 -04:00
Abhishek Lekshmanan
97fb4eeae0 rgw: sync counters: drop spaces from counter names
Since this might break modules like prometheus and general json processing tools
aren't too happy with spaces.

Fixes: https://tracker.ceph.com/issues/39434
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-04-23 17:29:44 +02:00
Jason Dillaman
15c7294312
Merge pull request #27703 from tchaikov/wip-rbd-replay-denc
rbd_replay: call the member decode() explicitly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-23 10:07:26 -04:00
Lenz Grimmer
ce2c91f989
mgr/dashboard: Clean up TableComponent tests and code (#26784)
mgr/dashboard: Clean up TableComponent tests and code

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-04-23 15:54:23 +02:00
Kefu Chai
3ca9bba4ef rbd_replay: call the member decode() explicitly
otherwise, the one defined using WRITE_RAW_ENCODER is called instead.
so in this change, rename the the member function which happens to have
the same signature with

decode(type &v, ::ceph::bufferlist::const_iterator& p)

where `type` is `__u8`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-23 20:33:13 +08:00
Jason Dillaman
b7c7a4f39e
Merge pull request #27521 from trociny/wip-rbd-remove-clone_v2-parent
librbd: optionally move parent image to trash on remove

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-23 08:11:38 -04:00
Jason Dillaman
0834078e77
Merge pull request #27484 from majianpeng/rbd-nbd
rbd-nbd: sscanf return 0 mean not-match

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-04-23 08:11:07 -04:00
Casey Bodley
2542abc0bb
Merge pull request #27697 from cbodley/wip-rgw-bucket-list-unordered
rgw: cls_bucket_list_unordered lists a single shard

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-04-23 08:07:13 -04:00
Kefu Chai
f15f69d521
Merge pull request #27713 from tchaikov/wip-24842
doc/rbd/rbd-cloudstack: update disk offering URL to new docs

Reviewed-by: Wido den Hollander <wido@42on.com>
2019-04-23 19:14:55 +08:00
Kefu Chai
779eccd1fc doc/rbd/rbd-cloudstack: update disk offering URL to new docs
point hyperlinks to latest

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-23 13:26:47 +08:00
xie xingguo
aad5d47be6 osd/PG: fix last_complete re-calculation on splitting
We add hard-limit for pg_logs now, which means we might keep trimming
old log entries irrespective of pg's current missing_set. This as a
result can cause the last_complete pointer moving far ahead of the real
on-disk version (the oldest need of missing_set, for instance) the
corresponding pg should have on splitting:

```
2019-04-19 06:41:52.559247 7efd4725c700 10 osd.2 271 Splitting pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 les/251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 unknown NOTIFY m=518 mbc={}] into 5.16
2019-04-19 06:41:52.561413 7efd4725c700 10 osd.2 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=943 ec=223/223 lis/c 250/223 c/f 251/224/0 250/271/229) [5,2] r=1 lpr=271 pi=[223,271)/4 crt=270'943 lcod 0'0 unknown NOTIFY m=261 mbc={}] release_backoffs [MIN,MAX)
```

For the above example, parent's last_complete cursor changed from **0'0** to
**238'300** directly due to the effort of trying to catch up the oldest
log entry changing when splitting was done. However, back into v12.2.9 primary
would still reference shard's last_complete field when trying to figure out all
possible locations of a currently missing object (see PG::MissingLoc::add_source_info):

```c++
  if (oinfo.last_complete < need) {
    if (omissing.is_missing(soid)) {
      ldout(pg->cct, 10) << "search_for_missing " << soid << " " << need
                         << " also missing on osd." << fromosd << dendl;
      continue;
    }
  }
```

Hence a wrongly calculated last_complete could then make primary mis-consider
that a specific shard might have the authoritative object it currently
looking for:

```
2019-04-19 06:41:52.904163 7fd4cfb5a700 10 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}] proc_replica_log for osd.2: 5.6( v 270'943 lc 238'30
0 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/c/f 251/224/0 250/271/229) log((249'563,270'943], crt=270'943) missing(261 may_include_del
etes = 1)
2019-04-19 06:41:52.904645 7fd4cfb5a700 20 osd.5 pg_epoch: 271 pg[5.6( v 270'943 lc 238'300 (238'300,270'943] local-lis/les=250/251 n=471 ec=223/223 lis/c 250/223 les/
c/f 251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 peering m=16 mbc={}]  after missing 5:624c3a7a:::benchmark_data_smithi190
_39968_object1382:head need 226'110 have 0'0
2019-04-19 06:41:53.567820 7fd4d035b700 10 osd.5 pg_epoch: 272 pg[5.6( v 270'943 lc 0'0 (238'300,270'943] local-lis/les=271/272 n=471 ec=223/223 lis/c 250/223 les/c/f
251/224/0 250/271/229) [5,2] r=0 lpr=271 pi=[223,271)/4 crt=270'943 lcod 226'77 mlcod 0'0 unknown m=16 u=13 mbc={255={(1+0)=220,(2+0)=28}}] search_for_missing 5:624c3a
7a:::benchmark_data_smithi190_39968_object1382:head 226'110 is on osd.2
```

note that ```5:624c3a7a:::benchmark_data_smithi190_39968_object1382:head 226'110```
was actually missing on both primary and shard osd.2 whereas primary insisted that
object should exist on shard osd.2!

https://github.com/ceph/ceph/pull/26175 posted an indirect fix
for the above problem by ignoring last_complete when checking the missing set,
but it should generally make more sense to fill in the last_complete field correctly
whenever possible.
Hence coming this additional fix.

Fixes: http://tracker.ceph.com/issues/26958
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-23 08:57:23 +08:00
Sage Weil
24e7c39d11 Merge PR #27708 into master
* refs/pull/27708/head:
	doc/governance: add cbodey

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-04-22 17:07:27 -05:00
Sage Weil
9e5ee15d3e Merge PR #27693 into master
* refs/pull/27693/head:
	mgr/telemetry: default to reports every 24h; lower minimum
	mgr/telemetry: exclude hostname field in crash reports

Reviewed-by: Dan Mick <dmick@redhat.com>
2019-04-22 17:06:42 -05:00
Sage Weil
cf1328d959 doc/governance: add cbodey
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-22 15:37:11 -05:00
Sage Weil
b02e81935c common/util: handle long lines in /proc/cpuinfo
Fixes: http://tracker.ceph.com/issues/38296
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-22 12:51:41 -05:00
Ali Maredia
e972c80ecc rgw: thread DoutPrefixProvider into fetch_remote_obj
This is for the AtomicObjProcessor declared there

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2019-04-22 11:30:47 -04:00
Ali Maredia
0242d7400e rgw: log refactoring for putobj_processor
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2019-04-22 11:30:47 -04:00
Casey Bodley
d37d0339ff rgw: cls_bucket_list_unordered lists a single shard
CLSRGWIssueBucketList sends the request to every shard, but this loop
intended to list only the current_shard

Fixes: http://tracker.ceph.com/issues/39393

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-04-22 11:20:55 -04:00
Casey Bodley
cd1fc96c5c cls/rgw: expose cls_rgw_bucket_list_op for single shard
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-04-22 11:20:49 -04:00
Mykola Golub
34112907a7
Merge pull request #27694 from dillaman/wip-39386
qa/suites/rbd: added writearound cache test permutations

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-22 18:13:02 +03:00
Mykola Golub
5bc54e763d
Merge pull request #27682 from dillaman/wip-39031
librbd: async open/close should free ImageCtx before issuing callback

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-22 18:12:30 +03:00
Sage Weil
6b97d72ed9
Merge pull request #24744 from liewegas/wip-stale-prs
.github/stale.yml: warn at 60, close at 90; adjust message
2019-04-22 09:58:24 -05:00
Sage Weil
712987d533 mgr/telemetry: default to reports every 24h; lower minimum
Allow more frequent telemetry reports.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-22 09:51:11 -05:00
Sage Weil
4510f33656 mgr/telemetry: exclude hostname field in crash reports
On some systems the hostname is a fully-qualified domain name and
(even when not a fqdn) may inadvertantly allow the cluster to be
identified.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-22 09:51:11 -05:00
Sage Weil
cac90d0fd9 Merge PR #27605 into master
* refs/pull/27605/head:
	mon/OSDMonitor: osd add-no{up,down,in,out} - remove state checker

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-22 07:04:11 -05:00
Mykola Golub
b098cc9fd8
Merge pull request #27695 from dillaman/wip-39072
test/librbd: drop 'ceph_test_librbd_api' target

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-22 08:22:20 +03:00
Sage Weil
191ab33faf Merge PR #27288 into master
* refs/pull/27288/head:
	osd/PG: discover missing objects when an OSD peers and PG is degraded

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-21 21:28:02 -05:00
Sage Weil
762e2e9c5f Merge PR #27571 into master
* refs/pull/27571/head:
	mgr/telemetry: add report_timestamp to sent reports

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-21 21:26:04 -05:00
Sage Weil
de49f345d5 Merge PR #27572 into master
* refs/pull/27572/head:
	denc: fix build error by calling global snprintf

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-21 21:20:37 -05:00
Sage Weil
f9ab80499c Merge PR #27622 into master
* refs/pull/27622/head:
	mon: fix INCOMPAT_OCTOPUS feature number
	mon/Elector: format mon_release correctly

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-21 21:20:12 -05:00
Mykola Golub
47bf240939
Merge pull request #27577 from dillaman/wip-rbd-suite-distros
qa/suites/rbd: add random distro selection to librbd tests

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-21 20:19:15 +03:00
Kefu Chai
d912f0b461
Merge pull request #27690 from tchaikov/wip-crimson-one-safeguard-is-better-than-two
crimson: pass `Connection*` to Dispatch::ms_dispatch()

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-04-21 11:17:27 +08:00
Sage Weil
755e8c4ef2 Merge PR #27595 into master
* refs/pull/27595/head:
	osd: add 'ceph osd stop <osd.nnn>' command

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-20 08:52:01 -05:00
Sage Weil
df9d1b92a9 Merge PR #27619 into master
* refs/pull/27619/head:
	mgr/BaseMgrModule: run MonCommandCompletion on the finisher
	mgr/BaseMgrModule: fix leak

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2019-04-20 08:48:30 -05:00
Sage Weil
b2eae023bc Merge PR #27651 into master
* refs/pull/27651/head:
	test/osd/safe-to-destroy.sh: fix typo

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-20 08:48:14 -05:00
Sage Weil
542b4a21da Merge PR #27663 into master
* refs/pull/27663/head:
	osd/OSDMap: Replace get_out_osds with get_out_existing_osds

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-20 08:47:48 -05:00
Kefu Chai
176bfb9b33
Merge pull request #27631 from dmick/wip-telemetry-crasharray
mgr/telemetry: change crash dict to a list

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-20 14:18:28 +08:00
Kefu Chai
b5e5642c12
Merge pull request #27599 from gregsfortytwo/wip-stale-prs
Miscellaneous lost fixes

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-20 14:16:33 +08:00
Kefu Chai
a319182b84
Merge pull request #27615 from ifed01/wip-ifed-verbose-bluefs-replay
os/bluestore: be more verbose doing bluefs log replay.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-20 14:12:20 +08:00
Kefu Chai
ebc6db7fa9
Merge pull request #27509 from Aran85/fix-pg-dout
osd/PG: the warning seems more serious than what it wanna transmit

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-20 14:10:39 +08:00