Commit Graph

94767 Commits

Author SHA1 Message Date
Patrick Donnelly
30aaa884bf
qa: test mds_max_caps_per_client conf
That the MDS will not let a client sit above mds_max_caps_per_client caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:31 -08:00
Patrick Donnelly
48ca097a9f
mds: limit maximum number of caps held by session
This is to prevent unsustainable situations where a client has so many
outstanding caps that a linear traversal/operation on the session's caps takes
unacceptable amounts of time.

Fixes: http://tracker.ceph.com/issues/38022
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:31 -08:00
Patrick Donnelly
7244cae4b8
mds: adapt drop cache for incremental recall
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:31 -08:00
Patrick Donnelly
ef46216d8d
mds: recall caps incrementally
As with trimming, use DecayCounters to throttle the number of caps we recall,
both globally and per-session.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Patrick Donnelly
b750b3bb25
mds: adapt drop cache for incremental trim
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Patrick Donnelly
7bf2f31abf
mds: add throttle for trimming MDCache
This is necessary when the MDS cache size decreases by a significant amount.
For example, when stopping a large MDS or when the operator makes a large cache
size reduction.

Fixes: http://tracker.ceph.com/issues/37723

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Patrick Donnelly
69efdaf646
mds: cleanup SessionMap init
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Patrick Donnelly
ce153b8112
mds: cleanup Session init
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Sage Weil
9f7713a905 osd: refuse to start if release > recorded min_osd_release + 2
If we try to start up the objectstore, we may make writeable changes to
(say) rocksdb that are not backwards compatible.  This happens, for
example, if you start a mimic osd.  Even if the compatset checks fail,
rocksdb may have written something that is not backwards compatible.

Fixes: http://tracker.ceph.com/issues/38076
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-29 16:36:03 -06:00
Sage Weil
141c305c12 doc/dev/cephx: document current implementation of cephx
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-29 12:28:36 -06:00
Casey Bodley
f25e85696c rgw: only update last_trim marker on ENODATA
if cls_log_trim() returns 0, it may have stopped after 1000 entries
before trimming all the way to to_marker. only update last_trim on
ENODATA, so we continue trimming until done

Fixes: http://tracker.ceph.com/issues/38075

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-29 12:26:23 -05:00
Casey Bodley
0a506c8ff4
Merge pull request #26157 from cbodley/wip-rgw-olh-repair
rgw: repair olh attributes that were broken by sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2019-01-29 12:04:39 -05:00
Matt Benjamin
e3b6c8356e
Merge pull request #26002 from pritha-srivastava/wip-sts-assumerole-webidentity
rgw: Initial commit for AssumeRoleWithWebIdentity.
2019-01-29 11:58:50 -05:00
Kefu Chai
c6edb27b3a
Merge pull request #26110 from yaozongyou/fix-statistics-bug
mon: fix pg_sum_old not copied correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-30 00:19:01 +08:00
Sebastian Wagner
cfc0141db1 doc/orchestrator: Aligned Documentaiton with specification
Also: add current implementation status
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-01-29 15:51:16 +01:00
Sebastian Krah
72fa86efa5 mgr/dashboard: Cleanup cluster and audit log
Applies the following changes:
- Makes the timestamp bold and not the message
- Colors the priority according to its level
- Indents the message correct if it's longer than one line
- Displays a message when the log is empty

Fixes: https://tracker.ceph.com/issues/37916
Signed-off-by: Sebastian Krah <skrah@suse.com>
2019-01-29 15:49:53 +01:00
Sebastian Krah
2a42aef5d5 mgr/dashboard: Refactor log pipe
The pipe returns a class name instead of an object now. This has the advantage,
that the layout can be modified directly in scss and keeps the code seperated
from the layout.

Fixes: https://tracker.ceph.com/issues/37916
Signed-off-by: Sebastian Krah <skrah@suse.com>
2019-01-29 14:59:14 +01:00
Sebastian Krah
0368346002 mgr/dashboard: Move and rename pipe
Moves file to the shared directory to make it available for every module.
Also renames the file, because it will be refactored.

Fixes: https://tracker.ceph.com/issues/37916
Signed-off-by: Sebastian Krah <skrah@suse.com>
2019-01-29 14:58:37 +01:00
Kefu Chai
60c3988601
Merge pull request #26090 from tchaikov/wip-crimson-osdmap
crimson: persist/load osdmap to/from store

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-01-29 21:31:51 +08:00
Patrick Nawracay
2f73d34b63 mgr/dashboard: Fix reloading of pool listing
Remove broken functionality that prevents pools from being reloaded after
deletion. The code also introduced a different problem; when clicking on a tab
shortly after the page has been loaded, the code will restore the tab to the
wrong one.

Introduced by 9e913bfd59

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-01-29 12:51:39 +00:00
Lenz Grimmer
602c875bd6
Merge pull request #26135 from Tinatinak/fix_permission_decorator
mgr/dashboard: Correct permission decorator

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-29 13:38:36 +01:00
Lenz Grimmer
d1ceffebf1
Merge pull request #26165 from rhcs-dashboard/ng-envbuild-e2e
mgr/dashboard: added 'env_build' to 'npm run e2e'

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-01-29 13:34:36 +01:00
Kefu Chai
05f09420f7 crimson/osd: fix boot process
* send my client addresses to mon
* always update osdmap with the latest one

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:54 +08:00
Kefu Chai
7b56e70385 crimson/osd: load_pgs()
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:54 +08:00
Kefu Chai
aa032af1eb crimson/os: add omap_get_values()
osd uses it for storing the pg_info.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:53 +08:00
Kefu Chai
f14f913fbf osd/HitSet: mark copy ctor of HitSet::Params noexcept
to be returned using seastar::future<...> the value type should satisfy
std::is_nothrow_constructible<T>.

with this change, pg_pool_t will be nothrow_constructible. and hence
can be returned using seastar::future<pg_pool_t>. otherwise
std::is_nothrow_constructible<pg_pool_t>::value would be false.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 20:04:53 +08:00
Kefu Chai
05a262d5aa crimson/osd: add operator<< to OSDState
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 19:32:59 +08:00
Kefu Chai
06e36ac991 crimson/os: note missing coll/oid in exception
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 19:32:59 +08:00
Kefu Chai
3b0db1db56 crimson/osd: load/store osdmap from/to store
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 19:32:58 +08:00
Mykola Golub
275bedc8d5
Merge pull request #26133 from dillaman/wip-37913
rbd: implement new 'rbd perf image iostat/iotop' commands

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-29 12:41:23 +02:00
Kefu Chai
a697025bc3 crimson/osd: add "--mkfs" support to crimson-osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 17:58:18 +08:00
Kefu Chai
004d088b29 crimson/os: move more features from MemStore to CyanStore
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-29 17:58:18 +08:00
Tina Kallio
ceb1fdc095 mgr/dashboard: Correct permission decorator
'CreatePermission' is required in order to restore an image from trash.

Signed-off-by: Tina Kallio <tina.kallio@gmail.com>
2019-01-29 10:07:36 +01:00
Pritha Srivastava
463c5603bd rgw: Initial commit for AssumeRoleWithWebIdentity.
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2019-01-29 14:21:12 +05:30
xie xingguo
5c4d241c7f crush: fix upmap overkill
It appears that OSDMap::maybe_remove_pg_upmaps's sanity checks
are overzealous. With some customized crush rules it is possible
for osdmaptool to generate valid upmaps, but maybe_remove_pg_upmaps
will cancel them.

Fixes: http://tracker.ceph.com/issues/37968
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-29 10:07:03 +08:00
Jason Dillaman
df4db9c5cb rbd: new 'perf image iostat/iotop' actions
This provides an 'iostat' and 'top'-like IO monitor for all
RBD images

Fixes: http://tracker.ceph.com/issues/37913
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 20:52:02 -05:00
David Zafman
a4b1574532 mon: A PG with PG_STATE_REPAIR doesn't mean damaged data, PG_STATE_INCONSISTENT does
Fixes: http://tracker.ceph.com/issues/38070

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-28 14:33:56 -08:00
Sage Weil
482cdca553 osd: record require_osd_release in objectstore meta
Record the require_osd_release value from the OSDMap in the 'meta' portion
of the osd's metadata that can be accessed without actually mounting the
OSD.  This will be useful as a safety gate to prevent you from mounting
an osd thet is too new that may make incompatible changes to the store.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-28 15:15:51 -06:00
Casey Bodley
1309e2d50e rgw: add RGWRados::repair_olh() to reconstruct olh attrs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
b8d7f0ad1f rgw: add RGWRados::bi_get_olh() to read/decode olh entry
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
a15e873634 rgw: bi_get() takes bucket_info
and calls the bucket_info overload of BucketShard::init() to avoid
reading the bucket info again

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Casey Bodley
c6c445f607 rgw: add BucketShard::init overload for bucket_info,obj
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-28 15:19:43 -05:00
Jason Dillaman
0474727294 rbd: simplify pool namespace positional argument handling
The pool and namespace can now be specified as in a
<pool-name>[/<namespace-name>] format as positional
arguments.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:36:44 -05:00
Jason Dillaman
88239791f6 mgr/rbd_support: support module for RBD features that require coordination
This initially adds mgr commands to collect RBD performance metrics
from the OSD dynamic perf collector:

  rbd perf image stats: collects IO rate statistics
  rbd perf image counters: collects raw IO counters

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:35:28 -05:00
David Zafman
3e6ff119e2
Merge pull request #25112 from dzafman/wip-scrub-warning
scrub warning check incorrectly uses mon scrub interval

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-28 10:46:18 -08:00
Casey Bodley
ee5abb4d17
Merge pull request #26081 from cbodley/wip-38007
rgw: bucket full sync handles delete markers

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2019-01-28 12:07:18 -05:00
Yehuda Sadeh
a4e3959663
Merge pull request #25137 from yehudasa/wip-archive-zone-3
rgw: archive zone

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-01-28 09:05:23 -08:00
Igor Fedotov
ced308000a os/bluestore: cap reads issued during deep fsck
Fixes: https://tracker.ceph.com/issues/38065
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-01-28 19:19:21 +03:00
Jan Fajerski
bea35c63c8 ceph-volume: add explicit mixed-type functional tests
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-01-28 17:14:24 +01:00
Igor Fedotov
a4e55dd352 os/bluestore: do not buffer read data on deep fsck.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-01-28 17:49:32 +03:00