Commit Graph

92204 Commits

Author SHA1 Message Date
Patrick Donnelly
0aa5566c81
Merge PR #24490 into master
* refs/pull/24490/head:
	mds: flush dirty dirfrags that weren't logged when deactivating mds
	mds: use MDlog::trim_all() to trim log when deactivating mds
	mds: don't cap log when there are replicated objects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-13 13:03:02 -08:00
Mark Nelson
9e2d1a52ab doc/rados: Add bluestore memory autotuning docs.
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2018-11-13 13:19:24 -06:00
Nathan Cutler
0185abfeb4 tests: make ceph-admin-commands.sh log what it does
Fixes: http://tracker.ceph.com/issues/37089
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-13 19:46:24 +01:00
Casey Bodley
d444c6dc9f
Merge pull request #25057 from joke-lee/wip-ceph-dencoder-support
ceph-dencoder: add RGWRealm and RGWPeriod  support.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-13 08:56:26 -05:00
Kefu Chai
e0e977f4a0
Merge pull request #25071 from tchaikov/wip-install-dep-for-python-saml
install-deps.sh: install libtool-ltdl-devel for building python-saml

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2018-11-13 18:17:54 +08:00
yuliyang
17378991c0 osd: add --dump-journal option in ceph-osd help info
Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
2018-11-13 17:38:46 +08:00
Kefu Chai
c8a89df744 install-deps.sh: install libtool-ltdl-devel for building python-saml
python-saml depends on dm.xmlsec.binding. which links against ltdl. so
without libtool-ltdl-devel, we will have

/opt/rh/devtoolset-7/root/usr/libexec/gcc/aarch64-redhat-linux/7/ld:
cannot find -lltdl

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-13 15:28:45 +08:00
yuliyang
6fb5e7b243 ceph-dencoder: add RGWRealm support.
Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
2018-11-13 08:24:24 +08:00
Kefu Chai
c89e0715cb
Merge pull request #25037 from tchaikov/wip-qa-upgrade
qa: add librados3 to exclude_packages for ugprade tests

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-13 06:38:21 +08:00
Alfredo Deza
debfa76540
Merge pull request #25063 from alfredodeza/wip-rm36768
ceph-volume: patch Device when testing 

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-11-12 14:59:14 -05:00
Alfredo Deza
82d96f8764 ceph-volume tests patch Device() by splitting parametrized method
This was causing failures on systems where there is no LVM or where the
device names don't match. Patching is always recommended to avoid
conflicts with the system testing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-11-12 14:04:23 -05:00
Alfredo Deza
7ee7a59910 ceph-volume tests inventory should patch before using .get_devices()
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-11-12 14:04:23 -05:00
Kefu Chai
b4aa2af446
Merge pull request #25039 from tchaikov/wip-install-setuptools
install-deps: install setuptools before upgrading virtualenv

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-11-12 20:05:29 +08:00
alfonsomthd
4047f091e7 mgr/dashboard: add pytest plugin: faulthandler
- Plugin for pytest that automatically enables the faulthandler module during tests.
- Very useful when you run tests and face segmentation faults.

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-11-12 11:51:02 +01:00
Sage Weil
aadb68cc84 Merge PR #25051 into master
* refs/pull/25051/head:
	doc/governance: reference the new Ceph Foundation

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
2018-11-12 02:50:23 -06:00
Sage Weil
d578d4af1b doc/governance: reference the new Ceph Foundation
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-12 01:32:45 -06:00
Jerry Lee
b09aefc2d6 ceph-mgr: hold lock while accessing the request list and submitting request
The request creation can fire up the notify event early and it can cause
a race condition where the actual request was not yet added to the
self.requests list which makes the submit_request() function waits
forever without accepting new requests.

https://marc.info/?l=ceph-devel&m=154104291714160&w=2

Fixes: https://tracker.ceph.com/issues/36764

Signed-off-by: Jerry Lee <leisurelysw24@gmail.com>
2018-11-12 14:27:53 +08:00
Kefu Chai
95386752ab install-deps: install setuptools before upgrading virtualenv
this should address the failures when running install-deps.sh, like

Downloading/unpacking virtualenv
  Running setup.py egg_info for package virtualenv
    /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'
      warnings.warn(msg)
    error in virtualenv setup command: 'extras_require' must be a
dictionary whose values are strings or lists of strings containing valid
project/version requirement specifiers.
    Complete output from command python setup.py egg_info:
    /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown
distribution option: 'python_requires'

  warnings.warn(msg)

error in virtualenv setup command: 'extras_require' must be a dictionary
whose values are strings or lists of strings containing valid
project/version requirement specifiers.

this only happens on very old virtualenv shipped with RHEL7.4

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-10 13:47:13 -08:00
Sage Weil
3b4970053c Merge PR #25033 into master
* refs/pull/25033/head:
	doc/governance: update CLT membership

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-11-10 14:07:11 -06:00
Kefu Chai
fa5012af5c
Merge pull request #25038 from yuriw/wip-yuriw-crontab-master
qa/tests: added "-n 7" to make sure mimic-x runs on built master branch

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-10 11:46:19 -08:00
Yuri Weinstein
5127d0166e qa/tests: added "-n 7" to make sure mimic-x runs on built master branch
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-10 10:18:49 -08:00
Sage Weil
2b38ce9a36 doc/governance: update CLT membership
- Neha is the new RADOS lead, but Josh is sticking around on the CLT
- John is leaving Red Hat and the Ceph project (we will miss him!)
- alphabetize

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-10 10:28:47 -06:00
Kefu Chai
f10fc004bc qa: add librados3 to exclude_packages for ugprade tests
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-10 07:17:24 -08:00
Sage Weil
d3aea056d3 Merge PR #25023 into master
* refs/pull/25023/head:
	doc/governance: Add Sebastian to CLT

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-10 07:46:53 -06:00
Nathan Cutler
6fb4132235 doc: Luminous v12.2.9 release notes
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-10 14:46:29 +01:00
Sebastian Wagner
e0eb2dbd98 doc/governance: Add Sebastian to CLT
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-11-10 11:59:46 +01:00
Mykola Golub
56412bf6d2
Merge pull request #24830 from dillaman/wip-rbd-pool-stats
rbd: expose pool stats summary tool

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-11-10 12:23:40 +02:00
Jason Dillaman
43840db329
Merge pull request #24265 from trociny/wip-36091
osd: collect client perf stats when query is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 19:56:11 -05:00
Jason Dillaman
6ac41ccbaf
Merge pull request #24214 from iridescent-rsy/bursting_io
librbd: reduce the TokenBucket fill cycle and support bursting io configuration

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 19:53:55 -05:00
Josh Durgin
fd2a4c5733
Merge pull request #22476 from dzafman/wip-23875
Removal of snapshot with corrupt replica crashes osd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-09 15:15:01 -08:00
Sage Weil
919aa37f82 Merge PR #24806 into master
* refs/pull/24806/head:
	os/filestore: fix merge_collection replay guards

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-09 14:53:56 -06:00
Sage Weil
130e6625eb Merge PR #24843 into master
* refs/pull/24843/head:
	osd/osd_types: fix notify-ack string rendering

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-11-09 14:53:16 -06:00
Jason Dillaman
1a14963aeb rbd: added new 'pool stats' action
This provides a quick summary of provisioned pool usage.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 12:40:41 -05:00
Jason Dillaman
f07fb350af librbd: new pool init/stat API methods
The init method is a stub for handling new pool initialization. It
currently only handles setting the application tag. The stats method
will quickly calculate the number of images and provisioned space for
those images within the pool. Querying the pool stats on a pool with
10,000 images only required approximately 2 seconds as compared to
over 2 minutes for a "rbd ls -l" scan.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-11-09 12:40:39 -05:00
Ranjitha G
1fb39dbe6f mgr/dashboard: Add unit test case for controller/erasure_code_profile.py
Signed-off-by: Ranjitha G <ranjitha.kmg@gmail.com>
2018-11-09 22:45:26 +05:30
Andrew Schoen
91dd98112b
Merge pull request #24998 from alfredodeza/wip-rm36470
ceph-volume enable  --no-systemd flag for simple sub-command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-11-09 09:44:44 -06:00
Lenz Grimmer
5b0a957961
Merge pull request #24627 from Devp00l/wip-manage-ec-profiles
mgr/dashboard: Adds ECP management to the frontend

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 16:29:43 +01:00
Sage Weil
e6556f4662 Merge PR #24459 into master
* refs/pull/24459/head:
	os: Transaction uses append_hole() to minimize bl:_buffers inflation.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-09 09:24:57 -06:00
J. Eric Ivancich
b713bb77a1 rgw: reshard improvements
Improve error log message when an expired reshard lock is renewed.

Add two new configurable options to manage resharding:
* rgw_reshard_batch_size : number of reshard entries to batch together
  before sending the operations to the CLS back-end.
* rgw_reshard_max_aio : maximum number of outstanding asynchronous i/o
  operations to allow at a time.

Alter rgw_reshard_bucket_lock duration default from 2 minutes to 6
minutes.

Add documentation, minimum values, tags, and service to a few rgw
reshard configuration options. Change some rgw_reshard_* options from
LEVEL_DEV to LEVEL_ADVANCED.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2018-11-09 10:21:56 -05:00
Sage Weil
0b14f926ec mgr/DeviceState: fix DeviceState initial refcount
Should be 0

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-09 08:52:07 -06:00
Sage Weil
03908113b4 qa/suites: valgrind ceph-mgr too
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-09 08:52:07 -06:00
Sage Weil
f21d56aa9c Merge PR #24890 into master
* refs/pull/24890/head:
	mon: fix 'log last' missing out latest event

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-09 08:38:33 -06:00
Sage Weil
644f91f5e4 Merge PR #24904 into master
* refs/pull/24904/head:
	common: point of this may be called after released

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-09 08:38:14 -06:00
Sage Weil
8f3d7a6d56 Merge PR #24925 into master
* refs/pull/24925/head:
	Avoid import _strptime failed
	Avoid exception if remote plugin not enabled
	Separate diskprediction local plugin from the diskprediction plugin

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-09 08:37:10 -06:00
Alfredo Deza
974bd43461
Merge pull request #24859 from jan--f/ceph-volume-inventory
ceph-volume: add inventory command

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-11-09 08:49:55 -05:00
Jan Fajerski
57adfc6bb8 ceph-volume: add inventory command
The inventory command provides information about a nodes disk inventory.
Existing logical volumes on a disk or one of its partitions are scanned
and reported.
The output can be formatted as plain text or json.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-11-09 13:49:35 +01:00
Lenz Grimmer
b8e4f4eb75
Merge pull request #24934 from votdev/improve_crush_map_viewer
mgr/dashboard: Improve CRUSH map viewer

Reviewed-by: Stephan Müller <smueller@suse.com>
2018-11-09 13:27:46 +01:00
Lenz Grimmer
3ba874004e
Merge pull request #24489 from ricardoasmarques/wip-saml2
mgr/dashboard: SSO - SAML 2.0 support

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-11-09 13:11:22 +01:00
Alexey Stupnikov
bff5a2122f doc: put command template into literal block
``ceph-deploy osd create --data {device} {ceph-node}`` command
is not displayed properly because of missing double colons.

Signed-off-by: Alexey Stupnikov <aleksey.stupnikov@gmail.com>
2018-11-09 10:48:40 +01:00
Matthew Vernon
dd24ddcbf7 debian: correct ceph-common relationship with older radosgw package
Fixes: https://tracker.ceph.com/issues/36741

9fd30b93f7 moved
/etc/bash_completion.d/radosgw-admin from radosgw to ceph-common. This
means that if you try and install a newer ceph-common over an older
radosgw, there's a conflict, and the install fails:

```
Unpacking ceph-common (12.2.8-1xenial) over (10.2.9-0ubuntu0.16.04.1) ...
dpkg: error processing archive ceph-common_12.2.8-1xenial_amd64.deb (--install):
 trying to overwrite '/etc/bash_completion.d/radosgw-admin', which is also in package radosgw 10.2.9-0ubuntu0.16.04.1
```

Per Debian policy (
https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-in-other-packages
) the correct way to handle a package taking over a file is for a
versioned Replaces and Breaks.

The change went into 12.0.3, so this commit adds Replaces and Breaks
against radosgw less than that version. It should be backported to
Luminous to avoid issues with upgrades from older versions (Jewel and
Kraken).

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
2018-11-09 09:35:35 +00:00