Commit Graph

95742 Commits

Author SHA1 Message Date
Sage Weil
e030ba465d Merge PR #26740 into master
* refs/pull/26740/head:
	rpm: advance spec file copyright date

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2019-03-05 14:05:54 -06:00
Sage Weil
bf23c64eeb Merge PR #26767 into master
* refs/pull/26767/head:
	Revert "msg/async/Protocol*: send keep alive if existing wins"

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-05 11:08:03 -06:00
Sage Weil
11f986830e Merge PR #26719 into master
* refs/pull/26719/head:
	rpm: re-add pyOpenSSL as ceph-mgr runtime dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-05 09:12:43 -06:00
Sage Weil
bf615abbce Merge PR #25540 into master
* refs/pull/25540/head:
	mgr/ServiceMap: show service's id on summary

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-05 09:12:26 -06:00
Sage Weil
a603609a00 Merge PR #26567 into master
* refs/pull/26567/head:
	os/bluestore: fix bluefs inability to reclaim space

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-03-05 09:11:53 -06:00
Sage Weil
af9e76e29c Merge PR #26752 into master
* refs/pull/26752/head:
	msg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-05 09:10:33 -06:00
Sage Weil
9ad8946492 Merge PR #26742 into master
* refs/pull/26742/head:
	osd/PG: do not touch this->cct after PG is destroyed

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-05 09:09:29 -06:00
Patrick Donnelly
8a85bd2731
Merge PR #26460 into master
* refs/pull/26460/head:
	client: parameter "cap" is not used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-03-05 06:25:19 -08:00
Xie Xingguo
ad8e7d33b1
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-03-05 20:16:18 +08:00
xie xingguo
fd520a17db Revert "msg/async/Protocol*: send keep alive if existing wins"
This reverts commit 61b9432ef9.

If we are in the middle of replacing, we can not queue any further
write events into the old center because we may end up replacing
existing connection's center with a new one, and hence executing
the newly queued write events in the old thread.

See **transfer_existing** for a detailed description.

Also the patch does not make a lot of sense for the original issue
it tried to resolve, because **send_keepalive** is a pure noop if the
underlying connection is not ready, which is obviously true for the
case demonstrated in http://tracker.ceph.com/issues/38493..

Fixes: http://tracker.ceph.com/issues/38569
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 20:02:23 +08:00
huanwen ren
1b64815f13 client: parameter "cap" is not used
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2019-03-05 19:52:34 +08:00
Sebastian Wagner
e280c1a0df
Merge pull request #26715 from sebastian-philipp/script-pypy
src/script: add run_mypy to run static type checking on Python code

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-03-05 10:18:18 +01:00
xie xingguo
733fff22bb mon/OSDMonitor: add boundary check for pool recovery_priority
See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 08:15:39 +08:00
Sage Weil
bd52f35fd0 Merge PR #26444 into master
* refs/pull/26444/head:
	ceph_features: deprecate MON_SINGLE_PAXOS
	osdmap: map versions to releases
	mon: retire some `osd set/unset` flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-04 17:26:12 -06:00
Sage Weil
e6d6716ea0 Merge PR #26754 into master
* refs/pull/26754/head:
	osdc: remove lderr() hexdump from handle_osd_op_reply

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 17:15:41 -06:00
Sage Weil
db6201acaa Merge PR #26744 into master
* refs/pull/26744/head:
	mds: fix encoded version of mds_info_t

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 17:13:30 -06:00
Sage Weil
5c6d9d556f Merge PR #26694 into master
* refs/pull/26694/head:
	rpm: drop use of $FIRST_ARG

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2019-03-04 16:25:18 -06:00
Alfredo Deza
e5c1a5b423
Merge pull request #26753 from ceph/ceph-volume-osd-ports
ceph-volume: set number of osd ports in the tests

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 16:58:36 -05:00
Casey Bodley
19eafda6a7 osdc: remove lderr() hexdump from handle_osd_op_reply
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-03-04 16:29:51 -05:00
Sage Weil
d233010f9a Merge PR #26631 into master
* refs/pull/26631/head:
	msg/async, v2: fix unused variable warning in ::discard_out_queue().

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 15:00:09 -06:00
Andrew Schoen
1132cb3c25 ceph-volume: set number of osd ports in the tests
The number of ports the OSDs listen on depend on the version of ceph
being used, so we need to test for that number accordingly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2019-03-04 14:43:37 -06:00
Radoslaw Zarzynski
b498c5b503 msg/async, v2: round up segments to CRYPTO_BLOCK_SIZE in secure mode.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-04 20:58:47 +01:00
Sage Weil
94b825448f Merge PR #26745 into master
* refs/pull/26745/head:
	ceph_release: we're in rc phase

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 13:45:23 -06:00
Casey Bodley
97dbbf0cfd
Merge pull request #26741 from cbodley/wip-test-rgw-reshard-wait2
test/rgw: fix race in test_rgw_reshard_wait

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-04 12:30:48 -05:00
Sage Weil
8d26102b2a ceph_release: we're in rc phase
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-04 11:17:20 -06:00
Patrick Donnelly
fb594328e9
mds: fix encoded version of mds_info_t
Introduced-by: 1f5892f820
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-04 09:06:16 -08:00
Sebastian Wagner
881a14cb20
Merge pull request #26635 from noahdesu/cherrypy-ipv6
mgr/dashboard: fix for using '::' on hosts without ipv6

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-03-04 17:51:34 +01:00
Andrew Schoen
04c577429f
Merge pull request #26739 from alfredodeza/wip-rm38568
ceph-volume: replace testinfra command with py.test

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-03-04 10:29:33 -06:00
Patrick Donnelly
ff1c7adfd3
Merge PR #26734 into master
* refs/pull/26734/head:
	doc: correct vstart.sh file location

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-04 07:39:08 -08:00
Sage Weil
08d96fba1e osd/PG: do not touch this->cct after PG is destroyed
- thread A drops last second-to-last ref
- thread B drops last ref
- thread B deletes pg
- thread A does

  lgeneric_subdout(cct, refs, 1) << "PG::put " << this << " "
  ..

touching this->cct, a use-after-free that valgrind notices with

  <kind>InvalidRead</kind>
  <what>Invalid read of size 8</what>
  <stack>
    <frame>
      <ip>0x74ED34</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>PG::put(char const*)</fn>
      <dir>/usr/src/debug/ceph-14.1.0-283-g569f086/src/osd</dir>
      <file>PG.cc</file>
      <line>179</line>
    </frame>
    <frame>
      <ip>0x7C1E76</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>ContainerContext&lt;boost::intrusive_ptr&lt;PG&gt; &gt;::~ContainerContext()</fn>
      <dir>/usr/src/debug/ceph-14.1.0-283-g569f086/src/osd</dir>
      <file>PG.h</file>
      <line>566</line>
    </frame>
    <frame>
      <ip>0xC95675</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Finisher::finisher_thread_entry()</fn>
      <dir>/usr/src/debug/ceph-14.1.0-283-g569f086/src/common</dir>
      <file>Finisher.cc</file>
      <line>67</line>
    </frame>
    <frame>
      <ip>0xD656E24</ip>
      <obj>/usr/lib64/libpthread-2.17.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0xE5B4BAC</ip>
      <obj>/usr/lib64/libc-2.17.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
  <auxwhat>Address 0x470b29e8 is 152 bytes inside a block of size 11,632 free'd</auxwhat>
  <stack>
  ...

Fixes: http://tracker.ceph.com/issues/38484
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-04 09:13:55 -06:00
Casey Bodley
ee5f8ae082
Merge pull request #26682 from yuvalif/add_rgw_xml_unit_tests
Add rgw xml unit tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-04 10:04:17 -05:00
Casey Bodley
ab1b1a9fb9
Merge pull request #26680 from yuvalif/add_kill_9_to_mstop
mstop.sh allow kill -9 after failing to kill procs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-04 10:03:34 -05:00
Nathan Cutler
166ae7847c rpm: advance spec file copyright date
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-03-04 15:50:24 +01:00
Sage Weil
067f72f2a2 Merge PR #26704 into master
* refs/pull/26704/head:
	msg/async, v2: drop alloc_aligned_buffer().
	msg/async, v2: introduce frame late abort facility.
	Revert "msg/async, v2: move ceph_msg_header2 to last frame segment."
	msg, msg/async, v2: introduce late message abort facility.
	msg/async, v2: failure of msg decode doesn't block throtlles.
	msg/async, v2: move ceph_msg_header2 to last frame segment.
	fixup: use frame epilogue for crc32 integrity checking.
	msg/async, v2: epilogue size is variable in secure mode.
	msg/async, v2: drop support for the buggy rx_buffers mechanism.
	Revert "msg/async, v2: add flags field to frame's epilogue."
	msg/async, v2: add flags field to frame's epilogue.
	msg/async, v2: drop onwire_segment_t as epilogue had derogated it.
	msg, msg/async, v2: drop crc fields from ceph_msg_header2.
	msg/async, v2: use frame epilogue for crc32 integrity checking.
	msg/async, v2: clean the ProtocolV2::{front,middle,data} up.
	msg/async, v2: clean the ProtocolV2::epilogue up.
	msg/async, v2: move crypto processing to segment reader.
	msg/async, v2: handle epilogue separately from payload/data.
	msg/async, v2: dissect decryption from SignedEncryptedFrame.
	msg/async, v2: unify WAIT frames with other payload frames.
	msg/async, v2: implement epilogue handling in secure mode.
	msg/async, v2: message frames are pre-dispatched now.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-04 08:44:56 -06:00
Casey Bodley
3729d6a985 test/rgw: fix race in test_rgw_reshard_wait
start the timer before scheduling the wait

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-03-04 09:44:26 -05:00
Sage Weil
634780fe07 Merge PR #26696 into master
* refs/pull/26696/head:
	osdc/Objecter: preserve read-into-existing-buffer behavior
	osdc/Objecter: don't use post_rx_buffer or revoke_rx_buffer at all
	msg/Connection: disable {post,revoke}_rx_buffer
	msg/async/ProtocolV1: disable rx_buffers behavior
	msg/simple/Pipe: disable rx_buffer code

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-04 08:32:18 -06:00
Sage Weil
0040c9915d Merge PR #26727 into master
* refs/pull/26727/head:
	qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap
	osd: add 'send_beacon' tell/asok command(s)

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-04 08:13:34 -06:00
Alfredo Deza
e90ebc3264 ceph-volume tests.simple replace testinfra command for py.test
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 09:05:07 -05:00
Alfredo Deza
c4597fc631 ceph-volume tests.lvm replace testinfra command for py.test
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 09:04:51 -05:00
Alfredo Deza
dc7501af69 ceph-volume tests.batch replace testinfra command for py.test
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-03-04 09:04:36 -05:00
Sebastian Wagner
d206992bd0 src/script: add run_mypy to run static type checking on Python code
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-03-04 14:35:00 +01:00
Changcheng Liu
839c07822a doc: correct vstart.sh file location
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-03-04 16:55:31 +08:00
Mykola Golub
a277b5c1fe
Merge pull request #26683 from dillaman/wip-38500
librbd: race condition possible when validating RBD pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-03-04 10:43:35 +02:00
Radoslaw Zarzynski
fdcf7d3be8 msg/async, v2: drop alloc_aligned_buffer().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-04 01:51:57 +01:00
Radoslaw Zarzynski
2206a7f8e1 msg/async, v2: introduce frame late abort facility.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-04 01:50:14 +01:00
Sage Weil
533a9b6a07 Merge PR #26698 into master
* refs/pull/26698/head:
	common/str_map: better trim() impl using boost
	common/str_map: fix trim() on empty string

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-03 11:39:52 -06:00
Sage Weil
aae0e8c0c0 osdc/Objecter: preserve read-into-existing-buffer behavior
The libradosstriper::RadosStriperImpl::aio_read populates the target
outbl with a static buffer and relies on us reading into it.  This was
actually not reliable in the past (it could fail if the rx_buffers
optimization failed due to a retransmit or something else) but nevertheless
libradosstriper requires it to work *at all*.

Resolve this by modifying Objecter to copy the result into any provided
buffer at the lowest layer.  This should capture any other such user who
needed this behavior.

On the other hand, it will break any user who inadvertantly reads into a
non-empty bufferlist.  Given that any such user would already previously
have seen bad behavior due to the rx_buffers optimization, we expect
there to be 0 such instances.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-03 10:38:53 -06:00
Sage Weil
01316aa7bd qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap
This test introduces a map gap.  What *should* happen is that when there is
such a gap, we cannot import.  Previously, the test didn't reliably produce
a map gap at all, and didn't check that import failed--it verified that it
passed.

Fix the test so that it reliably produces a gap *and* reports
min_last_epoch_clean to the mon so we can trim.  Then verify we fail to
import, but can with --force.  But remove the pg again, because if we
force an import with a map gap the osd will refuse to start.

Fixes: http://tracker.ceph.com/issues/38525
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-03 10:23:27 -06:00
Sage Weil
6bcd631101 osd: add 'send_beacon' tell/asok command(s)
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-03 10:23:27 -06:00
J. Eric Ivancich
15c91a6eea
Merge pull request #25538 from mkogan1/wip-rgw-reshard-status
rgw: change the "rgw admin status" 'num_shards' output to signed int

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-03-03 08:58:20 -05:00