Commit Graph

93607 Commits

Author SHA1 Message Date
Patrick Donnelly
6873dfc06e
Merge PR #24172 into master
* refs/pull/24172/head:
	client: fix fuse client hang because its pipe to mds is not ok

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-08 08:20:59 -08:00
Patrick Donnelly
112d754650
Merge PR #25741 into master
* refs/pull/25741/head:
	mds: remove wrong assertion in Locker::snapflush_nudge

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-08 07:03:40 -08:00
Patrick Donnelly
5e073231f9
Merge PR #25479 into master
* refs/pull/25479/head:
	client: fix fuse client can't read or write data due its caps is invalid
	Revert "client: fix fuse client can't read or write data due its caps is invalid"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-08 07:00:26 -08:00
Lenz Grimmer
e0c8284ec5
Merge pull request #25812 from Exotelis/toc
mgr/dashboard: Add table of contents to HACKING.rst

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-01-08 15:34:53 +01:00
Lenz Grimmer
22947238f3
Merge pull request #25830 from rhcs-dashboard/grafana-500
mgr/dashboard: Fixes Grafana 500 error

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-01-08 15:33:41 +01:00
Sage Weil
587b548043 Merge PR #25801 into master
* refs/pull/25801/head:
	qa/tasks/radosbench: default to 64k writes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-08 08:30:25 -06:00
Matt Benjamin
87a1baa8a6
Merge pull request #24714 from pritha-srivastava/wip-rgw-role-doc
rgw: Adding documentation for Roles.
2019-01-08 09:11:26 -05:00
Sage Weil
d8207b916d Merge PR #25821 into master
* refs/pull/25821/head:
	doc/dev/msgr2: checksum frame header (tag + length)

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-01-08 07:23:48 -06:00
Alfredo Deza
4f345d4c9f
Merge pull request #25820 from alfredodeza/wip-rm37805
ceph-volume tests/functional declare ceph-ansible roles instead of importing them

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-01-08 07:49:21 -05:00
Kanika Murarka
609fe223d4 mgr/dashboard: Fixes Grafana 500 error
Fixes: http://tracker.ceph.com/issues/37809
Signed-off-by: Kanika Murarka <kmurarka@redhat.com>
2019-01-08 12:15:05 +05:30
YunfeiGuan
77bc12e21e client: fix fuse client can't read or write data due its caps is invalid
We can't do read or write if we don't have Fr or Fw. Similarly, if the
caps is invalid, we can't read or wirte either.

What's worse, the mds don't konw our cap gen is invalid and think client
inode's caps is right. Thus will lead client application hung all the time.

Fixes: http://tracker.ceph.com/issues/36189
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-01-08 11:42:53 +08:00
Yan, Zheng
c7f324b0ce Revert "client: fix fuse client can't read or write data due its caps is invalid"
This reverts commit 1d742aa51b.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-01-08 11:42:43 +08:00
Sage Weil
9badd953f4 doc/dev/msgr2: checksum frame header (tag + length)
Ensure we catch transport errors early (before we, say, try to allocate
a missized payload buffer).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-07 15:01:21 -06:00
Alfredo Deza
ad6b924e2b ceph-volume tests/functional declare ceph-ansible roles instead of importing them
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-01-07 15:15:21 -05:00
Sage Weil
d49e2e9075 Merge PR #25795 into master
* refs/pull/25795/head:
	ceph-objectstore-tool: fix split check on import

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-07 10:54:22 -06:00
Sage Weil
78070b96bf qa/tasks/radosbench: default to 64k writes
The default rados bench is 4MB writes, but this tends to fill up our
tiny test clusters.  Use 64KB instead.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-07 07:22:29 -06:00
Sebastian Krah
cb254e7fcd mgr/dashboard: Add table of contents
Adds a table of contents to the HACKING.rst to make the navigation easier

Signed-off-by: Sebastian Krah <skrah@suse.com>
2019-01-07 11:11:50 +01:00
Kefu Chai
fa24a0312f
Merge pull request #25646 from tone-zhang/nvme-hang
bluestore/NVMEDevice.cc: fix NVMEManager thread hang

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2019-01-07 16:00:11 +08:00
Kefu Chai
33a72c1537
Merge pull request #25797 from falcon78921/wip-doc-37793
doc: updated Ceph documentation links

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-07 15:29:08 +08:00
tone.zhang
4c0fb6c263 bluestore/NVMEDevice.cc: fix NVMEManager thread halt
When enable SPDK in Ceph and start up Ceph development cluster, met
NVMEManager thread halt.

On aarch64 platform, the log as below:

Starting SPDK v18.04.1 / DPDK 18.05.0 initialization...
[ DPDK EAL parameters: nvme-device-manager -c 0x1 -m 2048 --file-prefix=spdk_pid16987 ]
EAL: Detected 46 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/spdk_pid16987/mp_socket
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL:   probe driver: 8086:953 spdk_nvme
EAL:   using IOMMU type 1 (Type 1)
^C

The reason is that pthread_cond_destroy() cannot destroy the active
condition_variable parameter.

Also on x86 debug builds we get the following error messages due to
probe_queue_lock still being active during ~NVMEManager:

/home/ubuntu/ceph/src/common/mutex_debug.h: 114: FAILED ceph_assert(r == 0)
 ceph version 14.0.1-1862-g403622b (403622be72) nautilus (dev)

The change fixes the issue.

Fixes: http://tracker.ceph.com/issues/37720

Signed-off-by: tone.zhang <tone.zhang@arm.com>
Signed-off-by: Steve Capper <steve.capper@arm.com>
2019-01-07 11:14:43 +08:00
Sage Weil
ab517a8b5c Merge PR #25799 into master
* refs/pull/25799/head:
	msg: fix includes for ostringstream

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-06 09:12:15 -06:00
Sage Weil
c7b45ca63b Merge PR #25802 into master
* refs/pull/25802/head:
	qa/suites/rados/thrash-old-clients: no async-v2only

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-06 08:41:05 -06:00
Sage Weil
1688d8fd92 qa/suites/rados/thrash-old-clients: no async-v2only
Old clients don't support the v2 protocol.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-05 12:26:56 -06:00
Sage Weil
1e6f0a1250 Merge PR #25780 into master
* refs/pull/25780/head:
	mon/OSDMonitor: do not populate void pg_temp into nextmap

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-05 12:01:23 -06:00
Willem Jan Withagen
c68369d48c msg: fix includes for ostringstream
Clang complains...

```
/home/jenkins/workspace/ceph-master/src/msg/msg_types.h:420:19: error: implicit instantiation of undefined template 'std::__1::basic_ostringstream<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
    ostringstream ss;
                  ^
/usr/include/c++/v1/iosfwd:123:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_ostringstream;
                               ^
1 error generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-01-05 16:05:20 +01:00
Kefu Chai
43b6c32964
Merge pull request #25716 from cyx1231st/wip-crimson-msgr-errorleak
crimson/net: fix crimson msgr error leaks to caller

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-05 18:45:50 +08:00
Patrick Donnelly
4331a92ab7
Merge PR #25750 into master
* refs/pull/25750/head:
	common: expose internal StackStringStream via deref

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-04 14:27:34 -08:00
Sage Weil
a82db5978d Merge PR #25793 into master
* refs/pull/25793/head:
	Merge remote-tracking branch 'private/wip-mon-kv-fix' into wip-mimic-4
	Merge remote-tracking branch 'private/wip-rgw-max-keys' into wip-mimic-4
	doc/releases/mimic: v13.2.4
	rgw: fix issues with 'enforce bounds' patch
	rgw: enforce bounds on max-keys/max-uploads/max-parts
	mon/config-key: limit caps allowed to access the store

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-01-04 15:43:24 -06:00
James McClune
7ff2f215a3 doc: updated Ceph documentation links
Fixes include:

- Updating Doxygen link for reference (http://docs.ceph.com/docs/master/dev/documenting/)
- Adding :ref: label for User Management page (http://docs.ceph.com/docs/master/rados/operations/user-management/)
- Update User Management link for reference (http://docs.ceph.com/docs/master/rbd/rbd-snapshot/)

Fixes: https://tracker.ceph.com/issues/37793
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2019-01-04 16:32:21 -05:00
Sage Weil
c04f0672df ceph-objectstore-tool: fix split check on import
Consider pg 1.3:
- in epoch A it splits to 1.b
- in epoch B 1.b splits to 1.1b

We export 1.3 before A, and import into another OSD that already has 1.b.
The existing check looks for whether 1.3 splits into 1.1b from B-1 to B,
but it should really look look at whether 1.3 splits into 1.1b over the
*entire* range (the export epoch, which is something < A).

Fixes: http://tracker.ceph.com/issues/37671
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 15:25:01 -06:00
Sage Weil
e069c30cb3 Merge remote-tracking branch 'private/wip-mon-kv-fix' into wip-mimic-4 2019-01-04 14:03:56 -06:00
Sage Weil
768addfaaa Merge remote-tracking branch 'private/wip-rgw-max-keys' into wip-mimic-4 2019-01-04 14:03:38 -06:00
Sage Weil
6d0f4a6bc0 doc/releases/mimic: v13.2.4
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 14:01:11 -06:00
Sage Weil
251f667ef8 Merge PR #25009 into master
* refs/pull/25009/head:
	librbd: stringify locker name with get_legacy_str()
	osdc/Objecter: fix list_watchers addr rendering to match legacy
	test/crimson: disable unittest_seastar_messenger test
	msg/msg_types: encode entity_addr_t TYPE_ANY as TYPE_LEGACY for pre-nautilus
	client: make blacklist detection handle TYPE_ANY entries
	mon/OSDMonitor: maintain compat output for 'blacklist ls'
	client: maintain compat for {inst,addr}_str in status dump
	qa/tasks/ceph_manager: compare osd flush seq #'s as ints
	qa/suites/fs: make use of simple.yaml where appropriate
	qa/msgr: move msgr factet into generic re-usable dir
	crimson: fix monmap build for seastar
	doc/start/ceph.conf: trim the sample ceph.conf file
	doc/rados/operations: only describe --public-{addr,network} method for adding mons
	PendingReleaseNotes: deprecate 'mon addr'
	doc: fix some 'mon addr' references
	doc/rados/configuration: fix some 'mon addr' references
	doc/rados/configuration/network-config-ref: revise network docs somewhat
	doc/rados/configuration/network-config-ref: remove totally obsolete section
	qa/suites/rados: replace mon_seesaw.py task with a small bash script
	qa/suites/fs/upgrade: don't bind to v2 addrs
	qa/tasks/mon_thrash: avoid 'mon addr' in mon section
	mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set
	osd/OSDMap: maintain compat addr fields
	msg/msg_types: add get_legacy_str()
	mds/MDSMap.h: maintain compat addr field
	mon/MgrMap: maintain compat active_addr field
	mon/MonClient: reconnect to mon if it's addrvec appears to have changed
	qa/tasks/ceph.conf.template: increase mon_mgr_mkfs_grace
	msg/async/ProtocolV2: fill in IP for all peer_addrs
	msg/async: print all addrs on debug lines
	mon/MonMap: no noname- mon name prefix when for_mkfs
	ceph-monstore-tool: print initial monmap
	msg/async/ProtocolV2: advertise ourselves as a v2 addr when using v2 protocol
	msg/async: assert existing protocol matches current protocol
	msg/async: add missing modelines
	mon/MonMap: add missing modeline
	vstart.sh: put mon addrs in mon_host, not 'mon addr'
	msg/async: better debug around conn map lookups and updates
	mon/MonClient: dump initial monmap at debug level 10
	qa/standalone/osd/osd-fast-mark-down: use v1 addr w/ simplemessenger
	qa/tasks/ceph: set initial monmap features with using addrvec addrs
	monmaptool: add --enable-all-features option
	qa/tasks/ceph: only use monmaptool --addv if addr has [,:v]
	qa/tasks/ceph_manager: make get_mon_status use mon addr
	qa/tasks/ceph: keep mon addrs in ctx namespace
	mon/OSDMonitor: log all osd addrs on boot
	msg/simple: behave when v2 and v1 addrs are present at target
	mon/MonClient: warn if global_id changes
	msg/Connection: add warning/note on get_peer_global_id
	mds/MDSDaemon: clean up handle_mds_map debug output a bit
	qa/suites/rados/upgrade: debug mds
	mds/MDSRank: improve is_stale_message to handle addrvecs
	msg/async: make loopback detect when sending to one of our many addrs
	qa/suites/rados/upgrade: no aggressive pg num changes
	mon/OSDMonitor: require nautilus mons for require_osd_release=nautilus
	mon/OSDMonitor: require mimic mons for require_osd_release=mimic
	qa/suites/rados/thrash-old-clients: use legacy addr syntax in ceph.conf
	msg/async: preserve peer features when replacing a connection
	qa/tasks/ceph.py: move methods from teuthology.git into ceph.py directly; support mon bind * options
	mon/MonMap: adjust build_initial behavior for mkfs vs probe
	mon/MonMap: improve ambiguous addr behavior
	qa/suites/rados/upgrade: spread mons a bit
	qa/rados/thrash-old-clients: keep mons on separate hosts
	qa/standalone/mon/misc.sh: tweak test to be more robust
	qa/tasks/mon_seesaw: expect v1/v2 prefix in addr
	osd/OSDMap: fix is_blacklisted() check to assume type ANY
	mon/OSDMonitor: use ANY addr type for blacklisting
	mon/msg_types: TYPE_V1ORV2 -> TYPE_ANY
	qa/workunits/cephtool: fix blacklist test
	qa/suites/upgrade: install old version with only v1 addrs
	common/options: by default, bind to both msgr v1 and v2 addresses
	vstart.sh: add --msgr1, --msgr2, --msgr21 options
	msg/async/ProtocolV2: be flexible with server identity check
	msg/msg_types: fix entity_addrvec_t::parse() with null end arg
	qa/suites/rados/basic/msgr: no msgr2 addrs in initial monmaps
	qa/tasks/ceph: add 'mon_bind_addrvec' and 'mon_bind_msgr2' options
	monmaptool: add --addv argument to pass in addrvec directly
	qa/suites/rados/basic/msgr: do not use msgr2 with simplemessenger
	qa/suites/rados/basic/msgr: async is not experimental
	messages/MOSDBoot: fix compat with pre-nautilus
	mon/MonMap: allow v1 or v2 to be explicitly specified along with part
	msg/msg_types: allow parsing of IPs without assuming v1 vs v2
	msg/msg_types: default parse to v2 addrs
	msg: standarize on v1: and v2: prefixes for *all* entity_addr_t's
	vstart.sh: use msgr2 by default
	mon/MonMap: remove get_addr() methods
	ceph-mon: adjust startup/bind/join sequence to use addrs
	mon: use MonMap::get_addrs() (instead of get_addr())
	mon/MonClient: change pending_cons to addrvec-based map
	mon/MonMap: fix set_addr() caller, kill wrapper
	mon/MonMap: remove addr-based add()
	monmaptool: fix --add to do either legacy or msgr2+legacy
	monmaptool: clean up iterator use a bit
	mon/MonMap: handle ambiguous mon addrs by trying both legacy and msgr
	mon/MonMap: take addrvec for set_initial_members
	mon/MonMap: use addrvecs for test instances
	mon: pass addrvec via MMonJoin
	mon/MonmapMonitor: fix 'mon add' to populate addrvec
	mon/MonMap: addr -> addrvec
	msg/async/ProtocolV2: only update socket_addr if we learned our addr
	osd: go active even if mon only accepted our v1 addr
	test/msgr: add test for msgr2 protocol
	msg/async/ProtocolV2: share socket_addr and all addrs during handshake
	msg/async: print socket_addr for the connection
	msg/async: msgr2 protocol placeholder
	msg/async: move ProtocolV1 class to its own source file
	msg/async: keep listen addr in ServerSocket, pass to new connections
	msg/async/AsyncMessenger: fix set_addr_unknowns

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-04 13:42:09 -06:00
Sage Weil
d001dfac73 Merge PR #25790 into master
* refs/pull/25790/head:
	common/blkdev: Fix missing stub block_device_get_metrics() on other platforms

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-04 13:40:58 -06:00
Sage Weil
333836aa78 librbd: stringify locker name with get_legacy_str()
This avoids the new v1: v2: prefixes.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 13:00:11 -06:00
Sage Weil
7c09e2c7b9 osdc/Objecter: fix list_watchers addr rendering to match legacy
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 12:46:27 -06:00
Neha Ojha
645e465f6e
Merge pull request #25781 from neha-ojha/wip-fix-upgrade-rhel
qa/suites/upgrade/mimic-x: fix rhel runs

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2019-01-04 08:01:17 -08:00
Ricardo Marques
029ebe81ff
Merge pull request #25154 from rhcs-dashboard/dashboard-url
mgr/dashboard: check for existence of Grafana dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
2019-01-04 15:00:32 +00:00
Sage Weil
abac8c58b4 test/crimson: disable unittest_seastar_messenger test
This is currently failing, allegedly for reasons unrelated to this PR, so
I'm disabling for now!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 08:54:25 -06:00
Willem Jan Withagen
ee6972636e common/blkdev: Fix missing stub block_device_get_metrics() on other platforms
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-01-04 15:46:51 +01:00
Sage Weil
d7ff96e615 Merge PR #25637 into master
* refs/pull/25637/head:
	doc: release notes for 13.2.3 mimic

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2019-01-04 08:25:25 -06:00
Sage Weil
a6b3f53abd Merge PR #25455 into master
* refs/pull/25455/head:
	mgr/diskprediction Document guide add predictor accuracy

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-04 08:17:38 -06:00
Kanika Murarka
fe11e6d262 mgr/dashboard: check for existence of Grafana dashboard
Fixes: http://tracker.ceph.com/issues/36356
Signed-off-by: Kanika Murarka <kmurarka@redhat.com>
2019-01-04 16:34:04 +05:30
Ricardo Marques
695976705f
Merge pull request #25210 from dillaman/wip-dashboard-rbd-mirroring
dashboard: support configuring block mirroring pools and peers
2019-01-04 10:39:09 +00:00
Yingxin
fd8cec434f crimson/net: simplify logics and centralize fault handling in execute_open()
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2019-01-04 14:38:37 +08:00
Yingxin
b561b9615c crimson/net: don't schedule/execute send/keepalive when closing
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2019-01-04 14:06:21 +08:00
Yingxin
eb2775d469 crimson/net: add proper gating and fault handling for send/keepalive
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2019-01-04 14:06:20 +08:00
Yingxin
02e6cc4622 crimson/net: remove unnecessary empty buffer check
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2019-01-04 14:06:20 +08:00
Yingxin
4b4bc2f567 crimson/net: no exception forwarded from accetp/connect to send_ready
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2019-01-04 14:01:08 +08:00