Commit Graph

82494 Commits

Author SHA1 Message Date
Jason Dillaman
6253490445
Merge pull request #19996 from Songweibin/wip-rbd-deep-cp
rbd: add deep cp CLI method

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-25 21:59:31 -05:00
Patrick Donnelly
19b56486db
Merge PR #20060 into master
* refs/pull/20060/head:
	ceph-debug-docker.sh: allow specifying branch sha1

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2018-01-25 14:52:17 -08:00
Kefu Chai
52bf90bb10
Merge pull request #20047 from jcsp/wip-prometheus-qa
qa: add new prometheus test to rados/mgr suite

Reviewed-By: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-25 23:43:07 +08:00
Casey Bodley
c42f7632b3
Merge pull request #19596 from qrGitHub/wip-rgw-lc-scheduleNextStartTime
rgw: optimize next start time for lifecycle

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2018-01-25 10:19:21 -05:00
Kefu Chai
50136330fc
Merge pull request #20077 from smithfarm/wip-spdk-fast-forward
spdk: update submodule to more recent upstream

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-25 23:02:21 +08:00
John Spray
474828ddfc
Merge pull request #19235 from jcsp/wip-mgr-can-run
mgr: improved module loading for error reporting etc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-25 11:23:36 +00:00
Nathan Cutler
34cc708c0d
Merge pull request #19122 from smithfarm/wip-lowmem-builder
build/ops: rpm: set build parallelism based on available memory

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
2018-01-25 09:36:39 +01:00
Nathan Cutler
e020ec3343 build/ops: cmake: do not guard certain tools with WITH_TESTS
As a follow-up to d7b493a710 we need to stop guarding

    ceph-osdomap-tool
    ceph-monstore-tool

with WITH_TESTS because they have been moved out of the ceph-test package.

(N.B. ceph-kvstore-tool was also moved out of ceph-test, but apparently never
had the guard.)

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-24 23:20:57 +01:00
Nathan Cutler
d048cb2cd8 build/ops: rpm: let ctest benefit from memory calculation
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-24 23:20:57 +01:00
Nathan Cutler
0cf627b4e3 build/ops: rpm: override %_smp_mflags if not enough memory
Sometimes the build machine has lots of processor cores and not enough
memory to successfully build Ceph on all of them at once. Calculate
how many parallel build processes we can sustain with the memory we
have and set a lower build parallelism if necessary. Never exceed
the value set by %_smp_mflags even if memory is aplenty.

Credits to Tomáš Chvátal for the original idea and implementation.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-24 23:20:56 +01:00
Nathan Cutler
353ff67780 build/ops: rpm: fix Group for rados-objclass-devel subpackage
Fixes RPMLINT warning "non-standard-group Development/Libraries"

Also, the Group: line is only needed for SUSE so put it in an appropriate
distro conditional.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-24 23:20:50 +01:00
Casey Bodley
15e0630b42
Merge pull request #19384 from ivancich/wip-rgw-lazy-cache-put-on-excl-create
rgw: do not update all gateway caches upon creation of system obj w/ exclusive flag

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-01-24 15:43:58 -05:00
John Spray
b3e6b8e1e2 qa: always dump corrupt messages in rados/upgrade
Previously only a service with debug_ms>=1 would dump
corrupt messages: in an upgrade test we're *alway*
interested in a corrupt message.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:21 -05:00
John Spray
9999ddf67b mgr: refactor get_modules/list_modules
list_modules is really about searching for them
on disk, so it's now probe_modules and private.

Both methods now return values instead of populating
an argument, since when called they were always writing into
a newly constructed container.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:21 -05:00
John Spray
191cce74e1 doc: note new mgr module error codes
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:21 -05:00
John Spray
6eb5c636dd mon: expose can_run flag in module ls + mgr dump
Enable people to see can_run failures and the explanatory
messages (telling them about a missing dependency)
before trying to enable a module.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:21 -05:00
John Spray
d9a47181c4 mgr: add health checks for failed modules
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
John Spray
f95b079c21 qa/mgr: add test for command execution errors
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
John Spray
834bc27940 mgr: load command definitions earlier
...and for all modules, not just the active ones.

This enables us to give better feedback to the user
when they try and use a command from a disabled module,
and also fixes the race between enabling a module and
trying to use its commands.

Fixes: http://tracker.ceph.com/issues/21683
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
John Spray
712ad57d09 mgr: evaluate can_run method on modules
...and transmit the result to the monitor in
our beacon.

Fixes: http://tracker.ceph.com/issues/21502
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
John Spray
6a8da7ca73 mgr: load all modules (not just active ones)
This is to enable us to learn more about the module
before it is enabled, such as whether its can_run method
return true.

We can also use this to enable loading a module's
commands before it is enabled, to give the user
a better response when they try to use a command
belong to a module that is not loaded.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-24 13:08:20 -05:00
Andrew Schoen
f852f640bf
Merge pull request #20054 from ceph/wip-rm22619
ceph-volume: dmcrypt support for lvm

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-01-24 18:22:56 +01:00
Nathan Cutler
290d9094d6 build/ops: rpm: introduce _remote_tarball_prefix macro
With this macro, we can use a single Source0 line for all supported distros.
RH/CentOS/Fedora needs the prefix, while SUSE builds in the OBS use a local
tarball.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-24 17:16:26 +01:00
sysrich
2305bf06f8 build/ops: rpm: use %_fillupdir macro in openSUSE/SLES
Replace references to /var/adm/fillup-templates with new %_fillupdir macro

Fixes: https://bugzilla.opensuse.org/show_bug.cgi?id=1069468
Signed-off-by: Richard Brown <rbrown@suse.com>
2018-01-24 17:16:26 +01:00
Nathan Cutler
dd08d17fdf build/ops: rpm: reconstruct SUSE bconds
Disable java build completely. Enable lttng build on SLES only, and only for
certain architectures.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-24 17:16:26 +01:00
Sage Weil
1c2ac30803
Merge pull request #20083 from liewegas/wip-msgr2-goals
doc/dev/msgr2.rst: a few notes on protocol goals

Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-01-24 04:22:38 -06:00
songweibin
0551029242 rbd: add deep cp CLI method
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-01-24 09:04:33 +08:00
Sage Weil
73934df3b0 doc/dev/msgr2.rst: a few notes on protocol goals
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-23 17:40:43 -06:00
Alfredo Deza
1ed0062eb6
Merge pull request #20010 from ceph/wip-rm22653
ceph-volume: adds a --destroy flag to ceph-volume lvm zap

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 14:24:55 -05:00
Alfredo Deza
163ee6a612 doc/ceph-volume add lvm/encryption to the index
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 13:32:13 -05:00
Alfredo Deza
babcdf2fa5 doc/man/ceph-volume add dmcrypt support
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 13:32:13 -05:00
Alfredo Deza
ad73be2c2a doc/man/ceph-volume add docs for zap sub-command
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 13:32:13 -05:00
Alfredo Deza
498d7a06a1 doc/man/ceph-volume add docs for list sub-command
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 13:32:13 -05:00
Alfredo Deza
46babe80b6 doc/dev/ceph-volume update internal LVM metadata information
Includes dmcrypt information, and updated key names to reflect current
code

Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 13:32:13 -05:00
Alfredo Deza
f0f0c1f8c4 ceph-volume tests.functional add xenial filestore tests for dmcrypt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-01-23 13:32:13 -05:00
Jason Dillaman
ccd16d2215
Merge pull request #20020 from mikechristie/wip-doc-mnc-update-iscsi-client
doc: update ceph iscsi kernel and package info

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-23 09:11:32 -05:00
Jason Dillaman
4dbcd4dd89
Merge pull request #18323 from thmour/thmour_rbdtrash
rbd: replace trash delay option, add rbd trash purge command

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-23 08:35:51 -05:00
Jason Dillaman
80f6c5cc4b
Merge pull request #20014 from liewegas/wip-rbd-validate
common/options,librbd/Utils: refactor RBD feature validation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-01-23 08:34:58 -05:00
Nathan Cutler
fe5e1eb102 spdk: update submodule to more recent upstream
This commit fast-forwards the SPDK submodule to upstream commit
93bd60e0de6b161567182d965239735bf467e85c to enable the py3-only build.
(Without said commit, SPDK configure script fails to find the python
interpreter in py3-only environment.)

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-23 12:30:57 +01:00
Theofilos Mouratidis
b9e4aa94b7 rbd trash: replace cli delay option, add rbd trash purge command
Replaced the delay argument for the trash move
command with a string acceptable by /bin/date, e.g.:

$ rbd trash move --pool foo --image bar --expires-in "2 weeks"

Added a "rbd trash purge" command that deletes any expired
image from the trash, has also a command to alter the current
expiration date with the "--older-than" argument which accepts
again a valid argument for /bin/date, e.g.:

$rbd trash purge mypool --older-than "2017-08-20"

There is also the "threshold" argument which tries to remove the
oldest trashed images (by deferment end time) until the pool space
is freed up to a percentage point, e.g.:

$ rbd trash purge mypool --threshold 0.9

If mypool uses 1GB it will try to remove trashed images until the
pool usage becomes equal to or lower than 900MB.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>
2018-01-23 11:41:54 +01:00
John Spray
e2c68d5e25 qa: assign prometheus ports during selftest
This was throwing IOError("Port 9283 not free on '::'",)
when trying to serve, since merging https://github.com/ceph/ceph/pull/19744

It's because the standbys (on the same node as the active) are
now trying to listen too.

Fixes: https://tracker.ceph.com/issues/22755
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-23 10:23:39 +00:00
John Spray
dd4f322592 qa: add new prometheus test to rados/mgr suite
Added in https://github.com/ceph/ceph/pull/19744

Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-23 10:23:39 +00:00
John Spray
cd70aa2cf3
Merge pull request #20005 from jcsp/wip-mgr-minor
mgr: remove a few junk lines

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-23 10:17:50 +00:00
John Spray
e1af667314 mgr: remove a few junk lines
Signed-off-by: John Spray <john.spray@redhat.com>
2018-01-23 10:17:15 +00:00
Kefu Chai
04d90a7d2c
Merge pull request #19988 from liewegas/wip-22673
osd: only exit if *latest* map(s) say we are destroyed

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-23 15:48:11 +08:00
Sage Weil
3911a2ae49
Merge pull request #20059 from ceph/wip-rm22427
doc/ceph-volume OSD use the fsid file, not the osd_fsid
2018-01-22 18:16:10 -06:00
Sage Weil
d431de74de
Merge pull request #19979 from k0ste/prometheus_metadata
mgr: prometheus: set metadata metrics value to '1' (#22717).

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-01-22 18:15:34 -06:00
Sage Weil
eafa6e0ef6
Merge pull request #19980 from k0ste/prometheus_latency
mgr: prometheus: added osd commit/apply latency metrics (#22718).

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-01-22 18:14:12 -06:00
Sage Weil
f9e02d51d7
Merge pull request #19986 from tchaikov/wip-denc-size-t-featured
denc: should check element's type not 'size_t'

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2018-01-22 17:51:34 -06:00
Sage Weil
6f7e54dbf7
Merge pull request #20028 from mogeb/steady-clock-rados
rados: use WaitInterval()'s return value instead of manual timing

Reviewed-by: Sage Weil <sage@redhat.com>
2018-01-22 17:51:02 -06:00