Commit Graph

85272 Commits

Author SHA1 Message Date
Patrick Donnelly
f1a733e683
Merge PR #21265 into master
* refs/pull/21265/head:
	doc: outline steps to take CephFS fs down

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2018-04-11 10:30:42 -07:00
Guillaume Abrioux
a991778341 specs: require of e2fsprogs
in ceph/ceph-container we've realized that `e2fsprogs` isn't installed in
centos container image because ceph hasn't a dependency for it.
It has for consequence to fail when deploying a containerized cluster
with dmcrypt when using centos image.

Typical error encountered:

typical error:
```
......
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sda5 uuid path is /sys/dev/block/8:5/dm/uuid
populate: Creating lockbox fs on %s: mkfs -t ext4 /dev/sda5
command_check_call: Running command: /usr/sbin/mkfs -t ext4 /dev/sda5
mkfs.ext4: No such file or directory
Traceback (most recent call last):
  File "/usr/sbin/ceph-disk", line 9, in <module>
    load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
......
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-04-11 19:22:10 +02:00
J. Eric Ivancich
6da5a5888c rgw: ability to list bucket contents in unsorted order for efficiency
Add the ability to list the contents of a bucket without the results
being sorted. This can have performance benefits since all bucket
shards do not need to be queried. This is done by adding a parameter
to the REST query string (i.e., "allow_unordered" [note underscore]
for swift or "allow-unordered" [note hyphen] for s3) to true.

This works with other request parameters such as marker, end_marker,
and prefix. But it does not work when delimeter is used. Also, because
the results are not in order, a subsequent marker may precede an
earlier marker.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2018-04-11 13:01:03 -04:00
Sage Weil
5d2f369b27
Merge pull request #21236 from jecluis/wip-mon-auth-bootstrap
mon/AuthMonitor: create bootstrap keys on create_initial()

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 10:57:20 -05:00
Sage Weil
68446cce8f
Merge pull request #21102 from trociny/wip-23487
mon: add  'ceph osd pool get erasure allow_ec_overwrites' command

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-04-11 10:49:09 -05:00
Mykola Golub
640171480e
Merge pull request #21300 from dillaman/wip-rbd-mirror-active-active
rbd-mirror: integrate image map policy as incremental step to active-active

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-11 18:15:11 +03:00
Kefu Chai
96aad4aed7
Merge pull request #21314 from badone/wip-ms_inject_socket_failures-SIGFPE
msg/async/AsyncConnection: Fix FPE in process_connection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-11 23:09:49 +08:00
Mykola Golub
7abc11538e
Merge pull request #21308 from dillaman/wip-23597
librbd: disable ENOENT tracking within the object cacher

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-11 18:09:01 +03:00
Kefu Chai
4cc3dab070
Merge pull request #21318 from badone/wip-qa-mon-misc-add-osdmap-prune-tests
qa/standalone/mon/misc.sh: Add osdmap-prune tests

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-11 23:08:33 +08:00
Kefu Chai
80667e8d71
Merge pull request #21306 from rzarzynski/wip-bs-bug23333-allow-eio
os/bluestore: alter the allow_eio policy regarding kernel's error list.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 23:05:46 +08:00
Kefu Chai
28e543f55f
Merge pull request #21288 from jcsp/wip-23584
mgr/prometheus: fix PG state names

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-04-11 23:03:06 +08:00
Kefu Chai
d0e6e0db86
Merge pull request #21281 from zealoussnow/wip-leozhang-0407
common: retry_sys_call no need take address of a function pointer

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-11 23:01:54 +08:00
Kefu Chai
dc4b525d3b
Merge pull request #21297 from tchaikov/wip-ceph-w-utf-8
ceph: print output of "status" as string not as bytes

Reviewed-by: Tim Serong <tserong@suse.com>
2018-04-11 23:01:07 +08:00
Kefu Chai
d9d568003e
Merge pull request #21275 from smithfarm/wip-23296
osdc/Objecter: add ignore overlay flag if got redirect reply

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-04-11 22:59:07 +08:00
Kefu Chai
3c31336f45
Merge pull request #21259 from rzarzynski/wip-common-hint-dout
common: hint the main branch of dout() accordingly to default verbosity.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-11 22:58:24 +08:00
Kefu Chai
c5a004d7c2 pybind/mgr/dashboard: silence lint warnings
this silences following errors:

9: ./awsauth.py:42:1: E722 do not use bare except'
9: ./awsauth.py:116:13: E722 do not use bare except'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-11 22:41:03 +08:00
Kefu Chai
b41e049cca cmake: no need to setup venv for dashboard
unlike ceph-disk and ceph-detect-init, dashboard is not a standalone
python application, it is a python application hosted by ceph-mgr,
so no need to create a venv in which it is deployed. the python env
created by tox would suffice.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-11 22:28:46 +08:00
Kefu Chai
213652aa6d test/dashboard: only test enabled python bindings
we should test the python version of MGR_PYTHON_VERSION_MAJOR, if we are
testing both py2.7 and py3, we are preparing for distributing dashboard
as a separated package independent of ceph-mgr.

restructure the dashboard tests as 2 matrices, so we have 2*2 tests for
coverage and lint respectively.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-11 22:25:12 +08:00
Sage Weil
5ad70e086f
Merge pull request #20186 from jecluis/wip-codingstyle
CodingStyle: add python and javascript/typescript
2018-04-11 08:49:50 -05:00
Joao Eduardo Luis
7528728197 CodingStyle: add python and javascript/typescript
As we are rapidly expanding our code base beyond C++ and C, it is a good
idea to enforce coding styles for other languages.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-11 14:45:36 +01:00
Sage Weil
53588a41a1
Merge pull request #21334 from batrick/i23643
qa: fix typo in test_full

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 08:25:55 -05:00
Casey Bodley
e292ce22b0
Merge pull request #21349 from wjwithagen/wjw-freebsd-deps
FreeBSD: add new required packages to be installed

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-04-11 08:49:54 -04:00
Lenz Grimmer
37a3566040
Merge pull request #21258 from votdev/rgw_proxy
mgr/dashboard: Implement a RGW proxy

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-04-11 14:49:37 +02:00
John Spray
1b18aed576
Merge pull request #21347 from SUSE/wip-doc-mgr-plugin-limitations
doc/mgr: Add limitations section to plugin guide

Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-11 12:39:12 +01:00
Willem Jan Withagen
381d1ef56f FreeBSD: add new required packages to be installed
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-04-11 13:37:15 +02:00
Ricardo Marques
9d316c34d2 mgr/dashboard: Update selected items on table refresh
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-04-11 12:10:43 +01:00
Tim Serong
f3aadbf628 doc/mgr: Add limitations section to plugin guide
Signed-off-by: Tim Serong <tserong@suse.com>
2018-04-11 20:33:11 +10:00
Volker Theile
a18ffe8018 mgr/dashboard: Takeover PR #20799 and adapt comments.
Signed-off-by: Volker Theile <vtheile@suse.com>
2018-04-11 12:26:42 +02:00
Patrick Nawracay
e4a0b12abf mgr/dashboard: Implement RGW proxy
This implementation is basically a Rados Gateway reverse proxy.  It
additionally takes care of the authentication to the Rados Gateway, but to use
it you will have to be authenticated against the dashboards RESTful API.

The corresponding credentials can be configured using the following commands:

    dashboard set-rgw-api-secret-key <secret-key>

    dashboard set-rgw-api-access-key <access-key>

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-04-11 12:26:42 +02:00
Patrick Nawracay
460882543a mgr/dashboard: RGW proxy: Include python-requests-aws
This commit will include the `python-requests-aws` [1] library.  This library
extends the `requests` library by an authentication mechanism to S3 gateways.

[1] https://github.com/tax/python-requests-aws

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-04-11 12:26:15 +02:00
Lenz Grimmer
34232b347b
Merge pull request #21022 from sebastian-philipp/dashboard_get_range
mgr/dashboard: Refactor multiple duplicates of `get_rate()`

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-04-11 12:22:00 +02:00
Lenz Grimmer
f0614a8d4e
Merge pull request #20920 from sebastian-philipp/dashboard_v2-erasure-code-profile
mgr/dashboard: Add ErasureCodeProfile controller

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2018-04-11 12:19:40 +02:00
Lenz Grimmer
0909266072
Merge pull request #21086 from ricardoasmarques/fix-table-without-fetch-data
mgr/dashboard: Fix table without fetchData

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile vtheile@suse.com
2018-04-11 11:55:32 +02:00
Lenz Grimmer
6c9c545169
Merge pull request #21041 from votdev/custom_validators
mgr/dashboard: Add custom validators

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-04-11 11:11:09 +02:00
Lenz Grimmer
7fdf9ae7f2
Merge pull request #21078 from tspmelo/wip-notifications
mgr/dashboard: Add notification service/component

Reviewed-by: Volker Theile vtheile@suse.com
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-04-11 11:08:04 +02:00
Ricardo Marques
ccace858de mgr/dashboard: Loading indicator only if fechData provided
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-04-11 09:50:11 +01:00
Ricardo Marques
b4188367f4 mgr/dashboard: Hide refresh button if no fetchData provided
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2018-04-11 09:42:20 +01:00
Volker Theile
20a2ec5e94 mgr/dashboard: Add custom validators.
Signed-off-by: Volker Theile <vtheile@suse.com>
2018-04-11 08:26:40 +02:00
Tim Serong
f8a8b7ef2b pybind/ceph_argparse: Fix UnboundLocalError if command doesn't validate
The error case in validate_command() where the command is found but the
command itself is somehow invalid raises an UnboundLocalError, because it
tries to refer to a variable named 'e', but it should be 'ex'.  For example:

  # ceph balancer mode
  Traceback (most recent call last):
    File "/usr/bin/ceph", line 1178, in <module>
      retval = main()
    File "/usr/bin/ceph", line 1109, in main
      verbose)
    File "/usr/bin/ceph", line 535, in new_style_command
      valid_dict = validate_command(sigdict, cmdargs, verbose)
    File "/usr/lib/python3.6/site-packages/ceph_argparse.py", line 1090, in validate_command
      print("Invalid command:", e, file=sys.stderr)
  UnboundLocalError: local variable 'e' referenced before assignment

Replacing 'e' with 'ex' gives the desired output:

  # ceph balancer mode
  Invalid command: missing required parameter mode(none|crush-compat|upmap)
  balancer mode none|crush-compat|upmap :  Set balancer mode
  Error EINVAL: invalid command

Signed-off-by: Tim Serong <tserong@suse.com>
2018-04-11 16:09:20 +10:00
Patrick Donnelly
aaac83abb5
Merge PR #19220 into master
* refs/pull/19220/head:
	mds: limit run time of load balancer
	mds: add list to track recently used sub-directories
	mds: calculate other mds' last_epoch_under locally
	mds: cleanup mds_load map access/update
	mds: check export pin when choosing dirfrags for exporting
	mds: optimize MDBalancer::find_exports()
	mds: avoid creating unnecessary subtrees during load balance
	mds: mds: optimize MDBalancer::try_rebalance()
	mds: don't try exporting subdir if dirfrag is already being exported
	mds: don't try exporting dirfrags under mds's own mdsdir
	mds: cleanup MDBalancer::try_rebalance()
	mds: always pass current time to MDBalancer::{hit_inode,hit_dir}
	mds: remove unused MDBalancer::export_empties()
	mds: don't decay load counters when printing load
	mds: adjust pop_nested when importing subtree
	mds: adjust subtree popularity when import fails
	mds: adjust subtree popularity after rename
	mds: fix request rate calculation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-10 22:13:29 -07:00
Kefu Chai
723b92d9cc
Merge pull request #21310 from tchaikov/wip-23610
osd: resend osd_pgtemp if it's not acked

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 13:00:01 +08:00
David Zafman
80137ca39f
Merge pull request #20947 from dzafman/wip-23364
Special scrub handling of hinfo_key errors

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-10 16:32:09 -07:00
Radoslaw Zarzynski
bebf35a61d common: hint the main branch of dout() accordingly to default verbosity.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-04-11 01:27:57 +02:00
Yuri Weinstein
6adac01923 qa/tests: Changed rhel7.4 to rhel7.5
Note: This is a temp fix till we get $ implementaion

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-04-10 15:02:24 -07:00
David Zafman
da0f5887a4 doc: Add release notes for the output format changes
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-10 13:52:10 -07:00
David Zafman
9c5ef19f93 test: Be smarter about when jsonschema can be used
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-10 13:52:10 -07:00
David Zafman
60ae2b8eb3 osd rados command: Show snapset in list-inconsistent-snapset
Add SnapSet bufferlist to inconsistent_snapset_t

Partial fix for http://tracker.ceph.com/issues/23428

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-10 13:51:48 -07:00
Jason Dillaman
10522f4b49 rbd-mirror: image map shut down should reset task context
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-04-10 16:32:13 -04:00
Jason Dillaman
08fdb71599 rbd-mirror: wire image map between pool watcher and instance watcher
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-04-10 16:32:13 -04:00
Jason Dillaman
e01c6c4bcf rbd-mirror: wire instance notifications through to pool replayer
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-04-10 16:32:13 -04:00