Commit Graph

94582 Commits

Author SHA1 Message Date
Pritha Srivastava
463c5603bd rgw: Initial commit for AssumeRoleWithWebIdentity.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2019-01-29 14:21:12 +05:30
xie xingguo
5c4d241c7f crush: fix upmap overkill
It appears that OSDMap::maybe_remove_pg_upmaps's sanity checks
are overzealous. With some customized crush rules it is possible
for osdmaptool to generate valid upmaps, but maybe_remove_pg_upmaps
will cancel them.

Fixes: http://tracker.ceph.com/issues/37968
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-29 10:07:03 +08:00
Jason Dillaman
df4db9c5cb rbd: new 'perf image iostat/iotop' actions
This provides an 'iostat' and 'top'-like IO monitor for all
RBD images

Fixes: http://tracker.ceph.com/issues/37913
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 20:52:02 -05:00
David Zafman
a4b1574532 mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_INCONSISTENT does
Fixes: http://tracker.ceph.com/issues/38070

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-28 14:33:56 -08:00
Sage Weil
482cdca553 osd: record require_osd_release in objectstore meta
Record the require_osd_release value from the OSDMap in the 'meta' portion
of the osd's metadata that can be accessed without actually mounting the
OSD.  This will be useful as a safety gate to prevent you from mounting
an osd thet is too new that may make incompatible changes to the store.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-28 15:15:51 -06:00
Casey Bodley
1309e2d50e rgw: add RGWRados::repair_olh() to reconstruct olh attrs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
b8d7f0ad1f rgw: add RGWRados::bi_get_olh() to read/decode olh entry
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
a15e873634 rgw: bi_get() takes bucket_info
and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
c6c445f607 rgw: add BucketShard::init overload for bucket_info,obj
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Jason Dillaman
0474727294 rbd: simplify pool namespace positional argument handling
The pool and namespace can now be specified as in a
<pool-name>[/<namespace-name>] format as positional
arguments.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:36:44 -05:00
Jason Dillaman
88239791f6 mgr/rbd_support: support module for RBD features that require coordination
This initially adds mgr commands to collect RBD performance metrics
from the OSD dynamic perf collector:

  rbd perf image stats: collects IO rate statistics
  rbd perf image counters: collects raw IO counters

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:35:28 -05:00
David Zafman
3e6ff119e2
Merge pull request #25112 from dzafman/wip-scrub-warning
scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-28 10:46:18 -08:00
Casey Bodley
ee5abb4d17
Merge pull request #26081 from cbodley/wip-38007
rgw: bucket full sync handles delete markers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2019-01-28 12:07:18 -05:00
Yehuda Sadeh
a4e3959663
Merge pull request #25137 from yehudasa/wip-archive-zone-3
rgw: archive zone

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-28 09:05:23 -08:00
Igor Fedotov
ced308000a os/bluestore: cap reads issued during deep fsck
Fixes: https://tracker.ceph.com/issues/38065
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-01-28 19:19:21 +03:00
Jan Fajerski
bea35c63c8 ceph-volume: add explicit mixed-type functional tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-01-28 17:14:24 +01:00
Igor Fedotov
a4e55dd352 os/bluestore: do not buffer read data on deep fsck.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-01-28 17:49:32 +03:00
Casey Bodley
5ffd34c5a6 rgw: bucket full sync handles delete markers
full sync uses CLS_RGW_OP_LINK_OLH_DM for delete markers instead of
trying to fetch them from the source zone

Fixes: http://tracker.ceph.com/issues/38007

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:41:51 -05:00
Casey Bodley
8686b6429c test/rgw: add multisite test_suspended_delete_marker_full_sync()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:41:51 -05:00
Casey Bodley
fba4ed72fd test/rgw: add multisite test_delete_marker_full_sync()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:41:48 -05:00
Casey Bodley
4d2cf73927 test/rgw: use version_id from key directly
now that PutObj returns the x-amz-version-id, it's available through
Key.version_id without resorting to Bucket.list_versions()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:35:40 -05:00
Casey Bodley
305f6dd6e9 test/rgw: check_bucket_eq() supports delete markers
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:35:40 -05:00
Casey Bodley
22d1da7240 test/rgw: check_bucket_eq() uses bucket.list_versions()
get_all_versions() is documented as a lower-level api that doesn't
handle paging, ands suggests list_versions() instead. also caches the
results to avoid listing each bucket twice

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:35:40 -05:00
Casey Bodley
1c1d3d9642
Merge pull request #25794 from cbodley/wip-rgw-sync-olh-attrs
rgw: fetch_remote_obj filters out olh attrs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2019-01-28 09:33:52 -05:00
James P. Weaver
13a49683de common: Use OpenSSL for cryptographic hashing
The OpenSSL implementation of the hashing algorithms is faster than the NSS version, and
so should be preferred when available

Signed-off-by: James Weaver <james.barrett@bbc.co.uk>
2019-01-28 13:46:02 +00:00
Shyukri Shyukriev
28f0749eae qa:rbd/workunits : Replace rbd bench-write with rbd bench --io-type write
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
2019-01-28 14:24:54 +02:00
Sage Weil
40259dd3eb Merge PR #24546 into master
* refs/pull/24546/head:
	msg/async/ProtocolV2: clear dispatch throttle on connection stop
	msg/async: fix should_use_msgr2 behavior (including monc)
	msg/async/AsyncMessenger: clear need_addr *after* we set our new addr
	msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr
	ceph_test_msgr: do not connect_to on the client side
	msg/async: do not connect from server
	msg/async: do not use peer to addr detection; use getsockname()
	msg/async/ProtocolV2: always send non-empty addrvec for self
	msg/async: never fill out port in myaddr if we didn't bind
	ceph_test_msgr: use v2 addrs for simplemessenger
	msg/async: msgr2: don't force write event on every message received
	msg/async/ProtocolV2: be forgiving in server identity check
	msg/async/ProtocolV2: fault if we connect to the wrong peer
	msg/async: msgr2: clean cookie if connection failed in ACCEPT_SESSION
	msg/async/ProtocolV2: do not bump connect_seq for fault during ACCEPTING_SESSION
	msg/async: msgr2: don't send SESSION_RETRY_GLOBAL in handle_existing_connection
	msg/async: msgr2: organizing log messages
	msg/async: msgr2: fix connection fault when replacing
	msg/async: msgr2: fix replacing race handling
	msg/async: msgr2: fix connection race when existing connection is newer
	msg/async: msgr2: assign recv_stamp in handle_message
	msg/async: msgr2: fix peer_addrs discovery
	msg/async: msgr2: keep authorizer bufferlist across reconnects
	msg/async: msgr2: fix connection secret problems for WITH_SEASTAR builds
	msg/async: msgr2: send keepalive on connection race winner
	msg/async: msgr2: fix client address learning
	msg/async: msgr2: fix keepalive_ack message
	msg/async: msgr2: do not force updating rotating keys inline
	msg/async: msgr2: fix mark_down vs accept race
	msg/async: msgr2: unregister con from accept vs mark_down race
	auth/cephx/CephxSessionHandler: use connection_secret for encryption
	msg,cephx: establish a unique connection_secret for every connection
	msg/async: msgr2: use sha256_digest_t to print signature hex strings
	types.h,rgw: merge sha*_digest_t definitions
	msg/async: msgr2: close connection when no authorizer is given
	msg/async: msgr2: formatting fixes
	msg/async: msgr2: send client v2 address when only v1 address is defined
	msg/async: msgr2: add payload length to banner
	msg/async: msgr2: check protocol state after fast dispatch
	msg/async: msgr2: reduce log level for sending messages event
	msg/async: msgr2: call verify authorizer when CEPH_AUTH_NONE is used
	msg/async: msgr2: store peer entity name in the protocol
	msg/async: msgr2: apply sign/encrypt to messages data payload
	msg/async: msgr2: encryption/decryption of frames
	cephx: added encrypt/decrypt bufferlist method to session handler
	msg/async: msgr2: refactored the frame structures
	cephx: add sign bufferlist method
	options: msgr2 enable/disable signing and encrytion options
	msg/async: msgr2: cephx authentication
	msg/async: msgr2: implement reconnect
	msg/async: msgr2: fault handling
	msg/async: msgr2: messange exchange phase
	msg/async: msgr2: message flow handshake
	msg/async: msgr2: authentication phase
	msg/async: msgr2: exchange peer_type in banner phase
	test/msgr: cloned test_msgr test for testing msgr2 protocol
	msg/async: msgr2: banner exchange
	msg/async: asyncconnection: update the source address info
	msg/async: move base class Protocol its own source file

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-28 05:50:59 -06:00
Lenz Grimmer
ddd5a3b9e6
Merge pull request #25583 from rhcs-dashboard/timepicker
mgr/dashboard: Adds custom timepicker for grafana iframes

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-01-28 12:27:48 +01:00
Kefu Chai
8b175ee4cc
Merge pull request #25745 from socketpair/precision
Formatters: improve precision of double numbers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-28 17:29:50 +08:00
alfonsomthd
3ba8e20d7a mgr/dashboard: added 'env_build' to 'npm run e2e'
* 'e2e' & 'e2e:dev' need also to run 'env_build' to complete successfully.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-28 09:55:37 +01:00
Kefu Chai
8fe1edbdb7
Merge pull request #25725 from cfanz/wip-mgr-handle-monmap
mgr: update daemon_state when necessary

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-28 16:44:53 +08:00
Kefu Chai
6c72f8f0a3
Merge pull request #26155 from tchaikov/wip-osd-cleanup
osd/PG: remove unused functions

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-28 16:43:05 +08:00
Kefu Chai
1c54d28bed
Merge pull request #26156 from tchaikov/wip-use-latest-setuptools
install-deps.sh: use the latest setuptools

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-28 16:42:30 +08:00
runsisi
0ca5dbebd1 mon/LogMonitor: lower logging level for check_sub
Signed-off-by: runsisi <luo.runbing@zte.com.cn>
2019-01-28 15:52:54 +08:00
Kefu Chai
f52c22ebf5
Merge pull request #26138 from Songweibin/wip-rbd-py-cleanup
pybind/rbd: fix a typo in metadata_get comments

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-28 14:17:42 +08:00
Neha Ojha
9c1cd8c8f7
Merge pull request #26105 from neha-ojha/wip-37886
mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 12:43:43 -08:00
Patrick Donnelly
ec0b649d3d
Merge PR #26086 into master
* refs/pull/26086/head:
	include/xlist: prevent item moves
	include/xlist: privatize item members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 10:23:08 -08:00
Sage Weil
d1f4d30d53 msg/async/ProtocolV2: clear dispatch throttle on connection stop
This aligns us with the ProtocolV1 implementation.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-27 10:11:03 -06:00
Kefu Chai
d42daf378c
Merge pull request #25616 from s0nea/wip-dashboard-osd-recv-speed-default-vals
mgr/dashboard: consider config option default values

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2019-01-27 22:17:49 +08:00
Kefu Chai
11c9331263
Merge pull request #24813 from wjwithagen/wjw-fix-alpine-compat
common: Fix Alpine compatability for TEMP_FAILURE_RETRY and ACCESSPERMS

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-27 21:42:51 +08:00
Kefu Chai
77109d7582
Merge pull request #25916 from tchaikov/wip-debian-librados-dev
debian: librados-dev should replace librados2-dev

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-27 21:41:39 +08:00
Kefu Chai
fa6a69288c
Merge pull request #26099 from jdurgin/wip-remove-ms-async-affinity
common/options: remove unused ms async affinity options

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 21:41:10 +08:00
Kefu Chai
0cec1bdaad
Merge pull request #26026 from tchaikov/wip-memstore-ceph-mutex
os/memstore: use ceph::mutex and friends

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 21:40:30 +08:00
Sage Weil
1881d9abba msg/async: fix should_use_msgr2 behavior (including monc)
Be consistent about whether we should use v2 to connect or not, and fix
teh monclient check to use the same logic.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-27 07:33:51 -06:00
Kefu Chai
3ba9eaa890
Merge pull request #26159 from dzafman/wip-gitignore
git: Ignore tags anywhere

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-27 21:18:35 +08:00
Kefu Chai
94a84b6f5a test: listen on random port in tests which start ceph-mon
See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-27 21:16:54 +08:00
Ilya Dryomov
11ee8f02b0
Merge pull request #26150 from dillaman/wip-rados-lock-info
rados: return legacy address in 'lock info'

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2019-01-27 08:18:45 +01:00
Kefu Chai
a8d6b2170d cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV
to avoid conflict with other `make check` runs in the same host

See-also: http://tracker.ceph.com/issues/36737
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-27 14:34:39 +08:00
Patrick Donnelly
032fda563c
Merge PR #25599 into master
* refs/pull/25599/head:
	mds: do not log internal journal error
	mds: clean up log level 2 messages
	mds: log session open stats

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-26 21:20:06 -08:00
David Zafman
6eec939e6f git: Ignore tags anywhere
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-26 19:48:48 -08:00