Commit Graph

93570 Commits

Author SHA1 Message Date
Sage Weil
3d34c4ebd2 mon: adjust mon sync, extra_probe_peers to use addrvec
The peer addr stuff via asok is a bit fragile because the user must
provide an exact addrvec matching the mon to avoid some weirdness, but
it's rarely used, and the fix would be some robustness/tolerance in the
messenger that is a bigger project than this.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-08 13:08:31 -06:00
Patrick Donnelly
4331a92ab7
Merge PR #25750 into master
* refs/pull/25750/head:
	common: expose internal StackStringStream via deref

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-04 14:27:34 -08:00
Sage Weil
a82db5978d Merge PR #25793 into master
* refs/pull/25793/head:
	Merge remote-tracking branch 'private/wip-mon-kv-fix' into wip-mimic-4
	Merge remote-tracking branch 'private/wip-rgw-max-keys' into wip-mimic-4
	doc/releases/mimic: v13.2.4
	rgw: fix issues with 'enforce bounds' patch
	rgw: enforce bounds on max-keys/max-uploads/max-parts
	mon/config-key: limit caps allowed to access the store

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-01-04 15:43:24 -06:00
Sage Weil
e069c30cb3 Merge remote-tracking branch 'private/wip-mon-kv-fix' into wip-mimic-4 2019-01-04 14:03:56 -06:00
Sage Weil
768addfaaa Merge remote-tracking branch 'private/wip-rgw-max-keys' into wip-mimic-4 2019-01-04 14:03:38 -06:00
Sage Weil
6d0f4a6bc0 doc/releases/mimic: v13.2.4
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 14:01:11 -06:00
Sage Weil
251f667ef8 Merge PR #25009 into master
* refs/pull/25009/head:
	librbd: stringify locker name with get_legacy_str()
	osdc/Objecter: fix list_watchers addr rendering to match legacy
	test/crimson: disable unittest_seastar_messenger test
	msg/msg_types: encode entity_addr_t TYPE_ANY as TYPE_LEGACY for pre-nautilus
	client: make blacklist detection handle TYPE_ANY entries
	mon/OSDMonitor: maintain compat output for 'blacklist ls'
	client: maintain compat for {inst,addr}_str in status dump
	qa/tasks/ceph_manager: compare osd flush seq #'s as ints
	qa/suites/fs: make use of simple.yaml where appropriate
	qa/msgr: move msgr factet into generic re-usable dir
	crimson: fix monmap build for seastar
	doc/start/ceph.conf: trim the sample ceph.conf file
	doc/rados/operations: only describe --public-{addr,network} method for adding mons
	PendingReleaseNotes: deprecate 'mon addr'
	doc: fix some 'mon addr' references
	doc/rados/configuration: fix some 'mon addr' references
	doc/rados/configuration/network-config-ref: revise network docs somewhat
	doc/rados/configuration/network-config-ref: remove totally obsolete section
	qa/suites/rados: replace mon_seesaw.py task with a small bash script
	qa/suites/fs/upgrade: don't bind to v2 addrs
	qa/tasks/mon_thrash: avoid 'mon addr' in mon section
	mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set
	osd/OSDMap: maintain compat addr fields
	msg/msg_types: add get_legacy_str()
	mds/MDSMap.h: maintain compat addr field
	mon/MgrMap: maintain compat active_addr field
	mon/MonClient: reconnect to mon if it's addrvec appears to have changed
	qa/tasks/ceph.conf.template: increase mon_mgr_mkfs_grace
	msg/async/ProtocolV2: fill in IP for all peer_addrs
	msg/async: print all addrs on debug lines
	mon/MonMap: no noname- mon name prefix when for_mkfs
	ceph-monstore-tool: print initial monmap
	msg/async/ProtocolV2: advertise ourselves as a v2 addr when using v2 protocol
	msg/async: assert existing protocol matches current protocol
	msg/async: add missing modelines
	mon/MonMap: add missing modeline
	vstart.sh: put mon addrs in mon_host, not 'mon addr'
	msg/async: better debug around conn map lookups and updates
	mon/MonClient: dump initial monmap at debug level 10
	qa/standalone/osd/osd-fast-mark-down: use v1 addr w/ simplemessenger
	qa/tasks/ceph: set initial monmap features with using addrvec addrs
	monmaptool: add --enable-all-features option
	qa/tasks/ceph: only use monmaptool --addv if addr has [,:v]
	qa/tasks/ceph_manager: make get_mon_status use mon addr
	qa/tasks/ceph: keep mon addrs in ctx namespace
	mon/OSDMonitor: log all osd addrs on boot
	msg/simple: behave when v2 and v1 addrs are present at target
	mon/MonClient: warn if global_id changes
	msg/Connection: add warning/note on get_peer_global_id
	mds/MDSDaemon: clean up handle_mds_map debug output a bit
	qa/suites/rados/upgrade: debug mds
	mds/MDSRank: improve is_stale_message to handle addrvecs
	msg/async: make loopback detect when sending to one of our many addrs
	qa/suites/rados/upgrade: no aggressive pg num changes
	mon/OSDMonitor: require nautilus mons for require_osd_release=nautilus
	mon/OSDMonitor: require mimic mons for require_osd_release=mimic
	qa/suites/rados/thrash-old-clients: use legacy addr syntax in ceph.conf
	msg/async: preserve peer features when replacing a connection
	qa/tasks/ceph.py: move methods from teuthology.git into ceph.py directly; support mon bind * options
	mon/MonMap: adjust build_initial behavior for mkfs vs probe
	mon/MonMap: improve ambiguous addr behavior
	qa/suites/rados/upgrade: spread mons a bit
	qa/rados/thrash-old-clients: keep mons on separate hosts
	qa/standalone/mon/misc.sh: tweak test to be more robust
	qa/tasks/mon_seesaw: expect v1/v2 prefix in addr
	osd/OSDMap: fix is_blacklisted() check to assume type ANY
	mon/OSDMonitor: use ANY addr type for blacklisting
	mon/msg_types: TYPE_V1ORV2 -> TYPE_ANY
	qa/workunits/cephtool: fix blacklist test
	qa/suites/upgrade: install old version with only v1 addrs
	common/options: by default, bind to both msgr v1 and v2 addresses
	vstart.sh: add --msgr1, --msgr2, --msgr21 options
	msg/async/ProtocolV2: be flexible with server identity check
	msg/msg_types: fix entity_addrvec_t::parse() with null end arg
	qa/suites/rados/basic/msgr: no msgr2 addrs in initial monmaps
	qa/tasks/ceph: add 'mon_bind_addrvec' and 'mon_bind_msgr2' options
	monmaptool: add --addv argument to pass in addrvec directly
	qa/suites/rados/basic/msgr: do not use msgr2 with simplemessenger
	qa/suites/rados/basic/msgr: async is not experimental
	messages/MOSDBoot: fix compat with pre-nautilus
	mon/MonMap: allow v1 or v2 to be explicitly specified along with part
	msg/msg_types: allow parsing of IPs without assuming v1 vs v2
	msg/msg_types: default parse to v2 addrs
	msg: standarize on v1: and v2: prefixes for *all* entity_addr_t's
	vstart.sh: use msgr2 by default
	mon/MonMap: remove get_addr() methods
	ceph-mon: adjust startup/bind/join sequence to use addrs
	mon: use MonMap::get_addrs() (instead of get_addr())
	mon/MonClient: change pending_cons to addrvec-based map
	mon/MonMap: fix set_addr() caller, kill wrapper
	mon/MonMap: remove addr-based add()
	monmaptool: fix --add to do either legacy or msgr2+legacy
	monmaptool: clean up iterator use a bit
	mon/MonMap: handle ambiguous mon addrs by trying both legacy and msgr
	mon/MonMap: take addrvec for set_initial_members
	mon/MonMap: use addrvecs for test instances
	mon: pass addrvec via MMonJoin
	mon/MonmapMonitor: fix 'mon add' to populate addrvec
	mon/MonMap: addr -> addrvec
	msg/async/ProtocolV2: only update socket_addr if we learned our addr
	osd: go active even if mon only accepted our v1 addr
	test/msgr: add test for msgr2 protocol
	msg/async/ProtocolV2: share socket_addr and all addrs during handshake
	msg/async: print socket_addr for the connection
	msg/async: msgr2 protocol placeholder
	msg/async: move ProtocolV1 class to its own source file
	msg/async: keep listen addr in ServerSocket, pass to new connections
	msg/async/AsyncMessenger: fix set_addr_unknowns

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-04 13:42:09 -06:00
Sage Weil
d001dfac73 Merge PR #25790 into master
* refs/pull/25790/head:
	common/blkdev: Fix missing stub block_device_get_metrics() on other platforms

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-04 13:40:58 -06:00
Sage Weil
333836aa78 librbd: stringify locker name with get_legacy_str()
This avoids the new v1: v2: prefixes.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 13:00:11 -06:00
Sage Weil
7c09e2c7b9 osdc/Objecter: fix list_watchers addr rendering to match legacy
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 12:46:27 -06:00
Neha Ojha
645e465f6e
Merge pull request #25781 from neha-ojha/wip-fix-upgrade-rhel
qa/suites/upgrade/mimic-x: fix rhel runs

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2019-01-04 08:01:17 -08:00
Ricardo Marques
029ebe81ff
Merge pull request #25154 from rhcs-dashboard/dashboard-url
mgr/dashboard: check for existence of Grafana dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
2019-01-04 15:00:32 +00:00
Sage Weil
abac8c58b4 test/crimson: disable unittest_seastar_messenger test
This is currently failing, allegedly for reasons unrelated to this PR, so
I'm disabling for now!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-04 08:54:25 -06:00
Willem Jan Withagen
ee6972636e common/blkdev: Fix missing stub block_device_get_metrics() on other platforms
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-01-04 15:46:51 +01:00
Sage Weil
d7ff96e615 Merge PR #25637 into master
* refs/pull/25637/head:
	doc: release notes for 13.2.3 mimic

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2019-01-04 08:25:25 -06:00
Sage Weil
a6b3f53abd Merge PR #25455 into master
* refs/pull/25455/head:
	mgr/diskprediction Document guide add predictor accuracy

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-04 08:17:38 -06:00
Kanika Murarka
fe11e6d262 mgr/dashboard: check for existence of Grafana dashboard
Fixes: http://tracker.ceph.com/issues/36356
Signed-off-by: Kanika Murarka <kmurarka@redhat.com>
2019-01-04 16:34:04 +05:30
Ricardo Marques
695976705f
Merge pull request #25210 from dillaman/wip-dashboard-rbd-mirroring
dashboard: support configuring block mirroring pools and peers
2019-01-04 10:39:09 +00:00
Kefu Chai
41d266b9b9
Merge pull request #25770 from votdev/fix_telemetry_issues
mgr/telemetry: Fix various issues

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-04 13:23:47 +08:00
Kefu Chai
8c650f5b35
Merge pull request #25783 from neha-ojha/wip-25672
common/blkdev.h: use std::string

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Rick Chen <rick.chen@prophetstor.com>
2019-01-04 13:18:35 +08:00
Kefu Chai
1b17bf1ca8
Merge pull request #25758 from xiexingguo/wip-async-prefetch
msg/async: improve read-prefetch logic

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-04 11:16:22 +08:00
Kefu Chai
e506210477
Merge pull request #25760 from cxytt/bug-fix
mon: shutdown messenger early to avoid accessing deleted logger

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-04 11:13:28 +08:00
Neha Ojha
e03e65f5e2 common/blkdev.h: use std::string
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-03 18:57:28 -08:00
Sage Weil
9987923eef Merge PR #24159 into master
* refs/pull/24159/head:
	mon/PGMap: show pg state age instead of stamp

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-03 20:38:30 -06:00
Sage Weil
8917a153f4 Merge PR #25623 into master
* refs/pull/25623/head:
	common/ceph_time: 'mo' for month
	common/options: use new parse_timespan
	common/ceph_time: add parse_timespan
	common/config_proxy: pass err_ss through on set_val
	common/ceph_time: add exact_timespan_str

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-03 20:38:14 -06:00
Sage Weil
cb4d8a3158 Merge PR #25756 into master
* refs/pull/25756/head:
	vstart.sh: force setting dashboard options before first mgr starts
	mon/ConfigMonitor: add --force flag to 'config set'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-03 20:37:54 -06:00
Sage Weil
fbddc56abd Merge PR #25763 into master
* refs/pull/25763/head:
	ceph_argparse: make command ceph acceph SIGINT.

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-03 20:37:38 -06:00
Sage Weil
8c46f7b51f Merge PR #25672 into master
* refs/pull/25672/head:
	osd: OSD device smart data include additional nvme data
	common/blkdev: add missing get_device_id impl
	os/bluestore,filestore: use get_raw_devices
	osd: update metadata and smart code to report get_device_id errors
	mon: update metadata and smart commands to use get_raw_devices
	common/blkdev: add get_raw_devices helper
	common/blkdev: fix BlkDev::get_devid when we got a devname, not fd
	common/blkdev: return optional error string from get_device_id
	common/blkdev: refactor to add block_device_get_metrics returning json

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-03 20:11:11 -06:00
Patrick Donnelly
33a5f4b723
Merge PR #25593 into master
* refs/pull/25593/head:
	mds: create heartbeat grace config option

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-03 17:40:53 -08:00
hsiang41
2da890117c osd: OSD device smart data include additional nvme data
Add nvme addition data into the deveh health data. That use nvme tool
and command syntax "nvme <vendor> smart-log-add <dev> -json". The nvme
json output append in the dev smart "nvme_smart_health_information_add_log".

- made run_smartctl static/private
- changed get_metrics to take a const string, not c str

Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 18:10:26 -06:00
Sage Weil
9f3dff2012 common/blkdev: add missing get_device_id impl
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 18:10:26 -06:00
Sage Weil
b23fa4ca7a os/bluestore,filestore: use get_raw_devices
There is a minor change here: before the devices list would include
e.g. 'sda2' and 'sda', and now it will only include 'sda'.

Fixes: http://tracker.ceph.com/issues/37542
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 18:10:26 -06:00
Sage Weil
f879f88a27 osd: update metadata and smart code to report get_device_id errors
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 18:10:26 -06:00
Sage Weil
cfcb7bc1e8 mon: update metadata and smart commands to use get_raw_devices
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 18:10:26 -06:00
Sage Weil
2555233efc common/blkdev: add get_raw_devices helper
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 18:10:26 -06:00
Neha Ojha
a74129d26d qa/suites/upgrade/mimic-x: fix rhel runs
The following fragment was required for rhel on ovh
overrides:
    ansible.cephlab:
      skip_tags: entitlements,packages,repos

Since, this suite runs on smithi in our nightlies, we should not need
this.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-01-03 13:39:30 -08:00
Alfredo Deza
8cb6de178e
Merge pull request #25674 from alfredodeza/wip-rm37442
ceph-volume normalize comma to dot for string to int conversions 

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-01-03 13:44:06 -05:00
Patrick Donnelly
ee7d253494
Merge PR #24635 into master
* refs/pull/24635/head:
	test: add scrub control command tests
	mds: scrub abort/pause/resume/status control commands
	test: switch using "scrub start" tell interface to initiate scrub
	mds: introduce "scrub start" tell interface to initiate scrub
	mds: dump scrub formatted output when context completion
	mds: generate random scrub tag when empty
	mds: introduce C_ExecAndReply context completion class
	mds: use CInode::scrub_is_in_progress() wherever required

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-03 10:15:31 -08:00
Sage Weil
e708b729cf msg/msg_types: encode entity_addr_t TYPE_ANY as TYPE_LEGACY for pre-nautilus
This is primarily for OSDMap's blacklist, which, starting with nautilus,
is populated by ANY addrs instead of LEGACY addrs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
c378bd830a client: make blacklist detection handle TYPE_ANY entries
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
c4c511fd44 mon/OSDMonitor: maintain compat output for 'blacklist ls'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
e09ca7b80b client: maintain compat for {inst,addr}_str in status dump
Add 'addr' field too.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
0d4c4db3c0 qa/tasks/ceph_manager: compare osd flush seq #'s as ints
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
8a3d90199d qa/suites/fs: make use of simple.yaml where appropriate
There's more needed than just ms_type=simple now.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
d518eb6cac qa/msgr: move msgr factet into generic re-usable dir
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:38 -06:00
Sage Weil
3b43e2a7b0 crimson: fix monmap build for seastar
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
b4e292ca9a doc/start/ceph.conf: trim the sample ceph.conf file
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
cdcb05765f doc/rados/operations: only describe --public-{addr,network} method for adding mons
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
ad39c086a0 PendingReleaseNotes: deprecate 'mon addr'
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00
Sage Weil
8e2fcae16e doc: fix some 'mon addr' references
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-03 11:17:31 -06:00