Commit Graph

85198 Commits

Author SHA1 Message Date
Patrick Donnelly
95762d6375
qa: whitelist mds down wrn during cephfs testing
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 14:40:36 -07:00
Patrick Donnelly
c9a590574c
mds: add config to disable fragmentation
For disaster situations or testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
dde9ee1628
qa: add max_mds thrash test
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
1dc5b62557
qa: mds_thrash updates for new max_mds behavior
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
4e9ffffb95
doc: update upgrade procedure and release notes
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
88922255e9
qa: add test for cluster resizing
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
5503b2014c
qa: remove use of mds deactivate
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
ab4f33bd24
cephfs: add new down/joinable fs flags
Fixes: http://tracker.ceph.com/issues/20609

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:25:59 -07:00
Patrick Donnelly
283b3a6849
mds: evict all clients if last mds shutting down
This allows all ranks to stop. If the administrator shuts down the cluster then
we should immediately shut down all client's I/O.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:01:24 -07:00
Douglas Fuller
c7bd6f02c7
cephfs: deprecate ceph mds deactivate
MDS deactivation is now handled by the max_mds parameter. Deprecate
ceph mds deactivate and note it to be removed in a future release.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:23 -07:00
Douglas Fuller
7d97a6e6f8
cephfs: kill allow_dirfrags
As dirfrags are now standard in CephFS, remove the machinery for
tracking and enabling this feature.

ceph fs set <fs> allow_dirfrags is now deprecated and prints a warning
message.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Douglas Fuller
ae3992ce4f
cephfs: Kill allow_multimds
With multi-mds now declared stable, allow_multimds now defaults to 1.
Given the max_mds parameter, it is now redundant. Remove it, leaving a
comment placeholder in the features bitmap.

ceph fs set <fs> allow_multimds is now deprecated and prints a warning
message.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Douglas Fuller
a950bccfd1
cephfs: Change behavior of cluster_down flag
Setting the cluster_down flag will now set all active MDS
to standby and clearing it will restore the previous max_mds.
Changing max_mds when the cluster_down flag is set will clear the
flag automatically.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Douglas Fuller
7b2e24f543
mon/FSCommands: Set extra MDS to standby
When reducing max_mds, deactivate any MDS rank greater than the
new value of max_mds.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Douglas Fuller
0a59560954
cephfs: Health check changes
Issue the HEALTH_WARN MDS_UP_LESS_THAN_MAX whenever an MDS cluster
has fewer than its max_mds online.

Issue the HEALTH_ERR MDS_ALL_DOWN whenever an MDS cluster has no
active MDS.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Douglas Fuller
dc073a1c2b
mon/MDSMonitor: Remove command support for legacy syntax
No longer check for commands for deprecated legacy filesystems.
These commands are long since deprecated and this code is no longer
needed.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:01 -07:00
Patrick Donnelly
624efc6432
MDSMonitor: cleanup and protect fsmap access
This commit is designed to permanently protect the current epoch FSMap from
accidental changes. Modifications to the pending_fsmap are protected by
checking if the monitor is the leader. Additionally, code must explicitly
request a writeable reference.

Fixes http://tracker.ceph.com/issues/23762

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 10:56:52 -07:00
Patrick Donnelly
f1a733e683
Merge PR #21265 into master
* refs/pull/21265/head:
	doc: outline steps to take CephFS fs down

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2018-04-11 10:30:42 -07:00
Sage Weil
5d2f369b27
Merge pull request #21236 from jecluis/wip-mon-auth-bootstrap
mon/AuthMonitor: create bootstrap keys on create_initial()

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 10:57:20 -05:00
Sage Weil
68446cce8f
Merge pull request #21102 from trociny/wip-23487
mon: add  'ceph osd pool get erasure allow_ec_overwrites' command

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-04-11 10:49:09 -05:00
Mykola Golub
640171480e
Merge pull request #21300 from dillaman/wip-rbd-mirror-active-active
rbd-mirror: integrate image map policy as incremental step to active-active

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-11 18:15:11 +03:00
Kefu Chai
96aad4aed7
Merge pull request #21314 from badone/wip-ms_inject_socket_failures-SIGFPE
msg/async/AsyncConnection: Fix FPE in process_connection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-11 23:09:49 +08:00
Mykola Golub
7abc11538e
Merge pull request #21308 from dillaman/wip-23597
librbd: disable ENOENT tracking within the object cacher

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-11 18:09:01 +03:00
Kefu Chai
4cc3dab070
Merge pull request #21318 from badone/wip-qa-mon-misc-add-osdmap-prune-tests
qa/standalone/mon/misc.sh: Add osdmap-prune tests

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-11 23:08:33 +08:00
Kefu Chai
80667e8d71
Merge pull request #21306 from rzarzynski/wip-bs-bug23333-allow-eio
os/bluestore: alter the allow_eio policy regarding kernel's error list.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 23:05:46 +08:00
Kefu Chai
28e543f55f
Merge pull request #21288 from jcsp/wip-23584
mgr/prometheus: fix PG state names

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-04-11 23:03:06 +08:00
Kefu Chai
d0e6e0db86
Merge pull request #21281 from zealoussnow/wip-leozhang-0407
common: retry_sys_call no need take address of a function pointer

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-11 23:01:54 +08:00
Kefu Chai
dc4b525d3b
Merge pull request #21297 from tchaikov/wip-ceph-w-utf-8
ceph: print output of "status" as string not as bytes

Reviewed-by: Tim Serong <tserong@suse.com>
2018-04-11 23:01:07 +08:00
Kefu Chai
d9d568003e
Merge pull request #21275 from smithfarm/wip-23296
osdc/Objecter: add ignore overlay flag if got redirect reply

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-04-11 22:59:07 +08:00
Kefu Chai
3c31336f45
Merge pull request #21259 from rzarzynski/wip-common-hint-dout
common: hint the main branch of dout() accordingly to default verbosity.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-11 22:58:24 +08:00
Sage Weil
5ad70e086f
Merge pull request #20186 from jecluis/wip-codingstyle
CodingStyle: add python and javascript/typescript
2018-04-11 08:49:50 -05:00
Joao Eduardo Luis
7528728197 CodingStyle: add python and javascript/typescript
As we are rapidly expanding our code base beyond C++ and C, it is a good
idea to enforce coding styles for other languages.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-11 14:45:36 +01:00
Sage Weil
53588a41a1
Merge pull request #21334 from batrick/i23643
qa: fix typo in test_full

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 08:25:55 -05:00
Casey Bodley
e292ce22b0
Merge pull request #21349 from wjwithagen/wjw-freebsd-deps
FreeBSD: add new required packages to be installed

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-04-11 08:49:54 -04:00
Lenz Grimmer
37a3566040
Merge pull request #21258 from votdev/rgw_proxy
mgr/dashboard: Implement a RGW proxy

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-04-11 14:49:37 +02:00
John Spray
1b18aed576
Merge pull request #21347 from SUSE/wip-doc-mgr-plugin-limitations
doc/mgr: Add limitations section to plugin guide

Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-11 12:39:12 +01:00
Willem Jan Withagen
381d1ef56f FreeBSD: add new required packages to be installed
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-04-11 13:37:15 +02:00
Tim Serong
f3aadbf628 doc/mgr: Add limitations section to plugin guide
Signed-off-by: Tim Serong <tserong@suse.com>
2018-04-11 20:33:11 +10:00
Volker Theile
a18ffe8018 mgr/dashboard: Takeover PR #20799 and adapt comments.
Signed-off-by: Volker Theile <vtheile@suse.com>
2018-04-11 12:26:42 +02:00
Patrick Nawracay
e4a0b12abf mgr/dashboard: Implement RGW proxy
This implementation is basically a Rados Gateway reverse proxy.  It
additionally takes care of the authentication to the Rados Gateway, but to use
it you will have to be authenticated against the dashboards RESTful API.

The corresponding credentials can be configured using the following commands:

    dashboard set-rgw-api-secret-key <secret-key>

    dashboard set-rgw-api-access-key <access-key>

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-04-11 12:26:42 +02:00
Patrick Nawracay
460882543a mgr/dashboard: RGW proxy: Include python-requests-aws
This commit will include the `python-requests-aws` [1] library.  This library
extends the `requests` library by an authentication mechanism to S3 gateways.

[1] https://github.com/tax/python-requests-aws

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-04-11 12:26:15 +02:00
Lenz Grimmer
34232b347b
Merge pull request #21022 from sebastian-philipp/dashboard_get_range
mgr/dashboard: Refactor multiple duplicates of `get_rate()`

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-04-11 12:22:00 +02:00
Lenz Grimmer
f0614a8d4e
Merge pull request #20920 from sebastian-philipp/dashboard_v2-erasure-code-profile
mgr/dashboard: Add ErasureCodeProfile controller

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2018-04-11 12:19:40 +02:00
Lenz Grimmer
0909266072
Merge pull request #21086 from ricardoasmarques/fix-table-without-fetch-data
mgr/dashboard: Fix table without fetchData

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile vtheile@suse.com
2018-04-11 11:55:32 +02:00
Lenz Grimmer
6c9c545169
Merge pull request #21041 from votdev/custom_validators
mgr/dashboard: Add custom validators

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-04-11 11:11:09 +02:00
Lenz Grimmer
7fdf9ae7f2
Merge pull request #21078 from tspmelo/wip-notifications
mgr/dashboard: Add notification service/component

Reviewed-by: Volker Theile vtheile@suse.com
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-04-11 11:08:04 +02:00
Ricardo Marques
ccace858de mgr/dashboard: Loading indicator only if fechData provided
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-04-11 09:50:11 +01:00
Ricardo Marques
b4188367f4 mgr/dashboard: Hide refresh button if no fetchData provided
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-04-11 09:42:20 +01:00
Volker Theile
20a2ec5e94 mgr/dashboard: Add custom validators.
Signed-off-by: Volker Theile <vtheile@suse.com>
2018-04-11 08:26:40 +02:00
Patrick Donnelly
aaac83abb5
Merge PR #19220 into master
* refs/pull/19220/head:
	mds: limit run time of load balancer
	mds: add list to track recently used sub-directories
	mds: calculate other mds' last_epoch_under locally
	mds: cleanup mds_load map access/update
	mds: check export pin when choosing dirfrags for exporting
	mds: optimize MDBalancer::find_exports()
	mds: avoid creating unnecessary subtrees during load balance
	mds: mds: optimize MDBalancer::try_rebalance()
	mds: don't try exporting subdir if dirfrag is already being exported
	mds: don't try exporting dirfrags under mds's own mdsdir
	mds: cleanup MDBalancer::try_rebalance()
	mds: always pass current time to MDBalancer::{hit_inode,hit_dir}
	mds: remove unused MDBalancer::export_empties()
	mds: don't decay load counters when printing load
	mds: adjust pop_nested when importing subtree
	mds: adjust subtree popularity when import fails
	mds: adjust subtree popularity after rename
	mds: fix request rate calculation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-10 22:13:29 -07:00