Commit Graph

94544 Commits

Author SHA1 Message Date
Lenz Grimmer
602c875bd6
Merge pull request #26135 from Tinatinak/fix_permission_decorator
mgr/dashboard: Correct permission decorator

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-29 13:38:36 +01:00
Lenz Grimmer
d1ceffebf1
Merge pull request #26165 from rhcs-dashboard/ng-envbuild-e2e
mgr/dashboard: added 'env_build' to 'npm run e2e'

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-01-29 13:34:36 +01:00
Kefu Chai
05f09420f7 crimson/osd: fix boot process
* send my client addresses to mon
* always update osdmap with the latest one

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:54 +08:00
Kefu Chai
7b56e70385 crimson/osd: load_pgs()
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:54 +08:00
Kefu Chai
aa032af1eb crimson/os: add omap_get_values()
osd uses it for storing the pg_info.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:53 +08:00
Kefu Chai
f14f913fbf osd/HitSet: mark copy ctor of HitSet::Params noexcept
to be returned using seastar::future<...> the value type should satisfy
std::is_nothrow_constructible<T>.

with this change, pg_pool_t will be nothrow_constructible. and hence
can be returned using seastar::future<pg_pool_t>. otherwise
std::is_nothrow_constructible<pg_pool_t>::value would be false.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:53 +08:00
Kefu Chai
05a262d5aa crimson/osd: add operator<< to OSDState
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 19:32:59 +08:00
Kefu Chai
06e36ac991 crimson/os: note missing coll/oid in exception
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 19:32:59 +08:00
Kefu Chai
3b0db1db56 crimson/osd: load/store osdmap from/to store
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 19:32:58 +08:00
Mykola Golub
275bedc8d5
Merge pull request #26133 from dillaman/wip-37913
rbd: implement new 'rbd perf image iostat/iotop' commands

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-29 12:41:23 +02:00
Kefu Chai
a697025bc3 crimson/osd: add "--mkfs" support to crimson-osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 17:58:18 +08:00
Kefu Chai
004d088b29 crimson/os: move more features from MemStore to CyanStore
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 17:58:18 +08:00
Tina Kallio
ceb1fdc095 mgr/dashboard: Correct permission decorator
'CreatePermission' is required in order to restore an image from trash.

Signed-off-by: Tina Kallio <tina.kallio@gmail.com>
2019-01-29 10:07:36 +01:00
Pritha Srivastava
463c5603bd rgw: Initial commit for AssumeRoleWithWebIdentity.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2019-01-29 14:21:12 +05:30
xie xingguo
5c4d241c7f crush: fix upmap overkill
It appears that OSDMap::maybe_remove_pg_upmaps's sanity checks
are overzealous. With some customized crush rules it is possible
for osdmaptool to generate valid upmaps, but maybe_remove_pg_upmaps
will cancel them.

Fixes: http://tracker.ceph.com/issues/37968
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-29 10:07:03 +08:00
Jason Dillaman
df4db9c5cb rbd: new 'perf image iostat/iotop' actions
This provides an 'iostat' and 'top'-like IO monitor for all
RBD images

Fixes: http://tracker.ceph.com/issues/37913
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 20:52:02 -05:00
David Zafman
a4b1574532 mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_INCONSISTENT does
Fixes: http://tracker.ceph.com/issues/38070

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-28 14:33:56 -08:00
Casey Bodley
1309e2d50e rgw: add RGWRados::repair_olh() to reconstruct olh attrs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
b8d7f0ad1f rgw: add RGWRados::bi_get_olh() to read/decode olh entry
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
a15e873634 rgw: bi_get() takes bucket_info
and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
c6c445f607 rgw: add BucketShard::init overload for bucket_info,obj
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Jason Dillaman
0474727294 rbd: simplify pool namespace positional argument handling
The pool and namespace can now be specified as in a
<pool-name>[/<namespace-name>] format as positional
arguments.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:36:44 -05:00
Jason Dillaman
88239791f6 mgr/rbd_support: support module for RBD features that require coordination
This initially adds mgr commands to collect RBD performance metrics
from the OSD dynamic perf collector:

  rbd perf image stats: collects IO rate statistics
  rbd perf image counters: collects raw IO counters

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:35:28 -05:00
David Zafman
3e6ff119e2
Merge pull request #25112 from dzafman/wip-scrub-warning
scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-28 10:46:18 -08:00
Casey Bodley
ee5abb4d17
Merge pull request #26081 from cbodley/wip-38007
rgw: bucket full sync handles delete markers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2019-01-28 12:07:18 -05:00
Yehuda Sadeh
a4e3959663
Merge pull request #25137 from yehudasa/wip-archive-zone-3
rgw: archive zone

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-28 09:05:23 -08:00
Igor Fedotov
ced308000a os/bluestore: cap reads issued during deep fsck
Fixes: https://tracker.ceph.com/issues/38065
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-01-28 19:19:21 +03:00
Jan Fajerski
bea35c63c8 ceph-volume: add explicit mixed-type functional tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-01-28 17:14:24 +01:00
Igor Fedotov
a4e55dd352 os/bluestore: do not buffer read data on deep fsck.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-01-28 17:49:32 +03:00
Casey Bodley
5ffd34c5a6 rgw: bucket full sync handles delete markers
full sync uses CLS_RGW_OP_LINK_OLH_DM for delete markers instead of
trying to fetch them from the source zone

Fixes: http://tracker.ceph.com/issues/38007

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:41:51 -05:00
Casey Bodley
8686b6429c test/rgw: add multisite test_suspended_delete_marker_full_sync()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:41:51 -05:00
Casey Bodley
fba4ed72fd test/rgw: add multisite test_delete_marker_full_sync()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:41:48 -05:00
Casey Bodley
4d2cf73927 test/rgw: use version_id from key directly
now that PutObj returns the x-amz-version-id, it's available through
Key.version_id without resorting to Bucket.list_versions()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:35:40 -05:00
Casey Bodley
305f6dd6e9 test/rgw: check_bucket_eq() supports delete markers
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:35:40 -05:00
Casey Bodley
22d1da7240 test/rgw: check_bucket_eq() uses bucket.list_versions()
get_all_versions() is documented as a lower-level api that doesn't
handle paging, ands suggests list_versions() instead. also caches the
results to avoid listing each bucket twice

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 09:35:40 -05:00
Casey Bodley
1c1d3d9642
Merge pull request #25794 from cbodley/wip-rgw-sync-olh-attrs
rgw: fetch_remote_obj filters out olh attrs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2019-01-28 09:33:52 -05:00
James P. Weaver
13a49683de common: Use OpenSSL for cryptographic hashing
The OpenSSL implementation of the hashing algorithms is faster than the NSS version, and
so should be preferred when available

Signed-off-by: James Weaver <james.barrett@bbc.co.uk>
2019-01-28 13:46:02 +00:00
Shyukri Shyukriev
28f0749eae qa:rbd/workunits : Replace rbd bench-write with rbd bench --io-type write
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
2019-01-28 14:24:54 +02:00
Sage Weil
40259dd3eb Merge PR #24546 into master
* refs/pull/24546/head:
	msg/async/ProtocolV2: clear dispatch throttle on connection stop
	msg/async: fix should_use_msgr2 behavior (including monc)
	msg/async/AsyncMessenger: clear need_addr *after* we set our new addr
	msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr
	ceph_test_msgr: do not connect_to on the client side
	msg/async: do not connect from server
	msg/async: do not use peer to addr detection; use getsockname()
	msg/async/ProtocolV2: always send non-empty addrvec for self
	msg/async: never fill out port in myaddr if we didn't bind
	ceph_test_msgr: use v2 addrs for simplemessenger
	msg/async: msgr2: don't force write event on every message received
	msg/async/ProtocolV2: be forgiving in server identity check
	msg/async/ProtocolV2: fault if we connect to the wrong peer
	msg/async: msgr2: clean cookie if connection failed in ACCEPT_SESSION
	msg/async/ProtocolV2: do not bump connect_seq for fault during ACCEPTING_SESSION
	msg/async: msgr2: don't send SESSION_RETRY_GLOBAL in handle_existing_connection
	msg/async: msgr2: organizing log messages
	msg/async: msgr2: fix connection fault when replacing
	msg/async: msgr2: fix replacing race handling
	msg/async: msgr2: fix connection race when existing connection is newer
	msg/async: msgr2: assign recv_stamp in handle_message
	msg/async: msgr2: fix peer_addrs discovery
	msg/async: msgr2: keep authorizer bufferlist across reconnects
	msg/async: msgr2: fix connection secret problems for WITH_SEASTAR builds
	msg/async: msgr2: send keepalive on connection race winner
	msg/async: msgr2: fix client address learning
	msg/async: msgr2: fix keepalive_ack message
	msg/async: msgr2: do not force updating rotating keys inline
	msg/async: msgr2: fix mark_down vs accept race
	msg/async: msgr2: unregister con from accept vs mark_down race
	auth/cephx/CephxSessionHandler: use connection_secret for encryption
	msg,cephx: establish a unique connection_secret for every connection
	msg/async: msgr2: use sha256_digest_t to print signature hex strings
	types.h,rgw: merge sha*_digest_t definitions
	msg/async: msgr2: close connection when no authorizer is given
	msg/async: msgr2: formatting fixes
	msg/async: msgr2: send client v2 address when only v1 address is defined
	msg/async: msgr2: add payload length to banner
	msg/async: msgr2: check protocol state after fast dispatch
	msg/async: msgr2: reduce log level for sending messages event
	msg/async: msgr2: call verify authorizer when CEPH_AUTH_NONE is used
	msg/async: msgr2: store peer entity name in the protocol
	msg/async: msgr2: apply sign/encrypt to messages data payload
	msg/async: msgr2: encryption/decryption of frames
	cephx: added encrypt/decrypt bufferlist method to session handler
	msg/async: msgr2: refactored the frame structures
	cephx: add sign bufferlist method
	options: msgr2 enable/disable signing and encrytion options
	msg/async: msgr2: cephx authentication
	msg/async: msgr2: implement reconnect
	msg/async: msgr2: fault handling
	msg/async: msgr2: messange exchange phase
	msg/async: msgr2: message flow handshake
	msg/async: msgr2: authentication phase
	msg/async: msgr2: exchange peer_type in banner phase
	test/msgr: cloned test_msgr test for testing msgr2 protocol
	msg/async: msgr2: banner exchange
	msg/async: asyncconnection: update the source address info
	msg/async: move base class Protocol its own source file

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-28 05:50:59 -06:00
Lenz Grimmer
ddd5a3b9e6
Merge pull request #25583 from rhcs-dashboard/timepicker
mgr/dashboard: Adds custom timepicker for grafana iframes

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-01-28 12:27:48 +01:00
Kefu Chai
8b175ee4cc
Merge pull request #25745 from socketpair/precision
Formatters: improve precision of double numbers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-28 17:29:50 +08:00
alfonsomthd
3ba8e20d7a mgr/dashboard: added 'env_build' to 'npm run e2e'
* 'e2e' & 'e2e:dev' need also to run 'env_build' to complete successfully.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-28 09:55:37 +01:00
Kefu Chai
8fe1edbdb7
Merge pull request #25725 from cfanz/wip-mgr-handle-monmap
mgr: update daemon_state when necessary

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-28 16:44:53 +08:00
Kefu Chai
6c72f8f0a3
Merge pull request #26155 from tchaikov/wip-osd-cleanup
osd/PG: remove unused functions

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-28 16:43:05 +08:00
Kefu Chai
1c54d28bed
Merge pull request #26156 from tchaikov/wip-use-latest-setuptools
install-deps.sh: use the latest setuptools

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-28 16:42:30 +08:00
runsisi
0ca5dbebd1 mon/LogMonitor: lower logging level for check_sub
Signed-off-by: runsisi <luo.runbing@zte.com.cn>
2019-01-28 15:52:54 +08:00
Kefu Chai
f52c22ebf5
Merge pull request #26138 from Songweibin/wip-rbd-py-cleanup
pybind/rbd: fix a typo in metadata_get comments

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-28 14:17:42 +08:00
Neha Ojha
9c1cd8c8f7
Merge pull request #26105 from neha-ojha/wip-37886
mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 12:43:43 -08:00
Patrick Donnelly
ec0b649d3d
Merge PR #26086 into master
* refs/pull/26086/head:
	include/xlist: prevent item moves
	include/xlist: privatize item members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 10:23:08 -08:00
Sage Weil
d1f4d30d53 msg/async/ProtocolV2: clear dispatch throttle on connection stop
This aligns us with the ProtocolV1 implementation.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-27 10:11:03 -06:00