Commit Graph

88432 Commits

Author SHA1 Message Date
John Spray
d24e6cb32a mgr: generic self test command
Avoid need for each module to expose a self-test
command: they can just implement the method,
and then get it called via the selftest module.

As well as fewer LOC, this means that the self
test commands are not cluttering the interface
for end users, as they've invisible until
the selftest module is loaded.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-07-20 13:09:19 -04:00
John Spray
6ca387f275 doc: update mgr plugin page for remote calls
Signed-off-by: John Spray <john.spray@redhat.com>
2018-07-20 13:07:17 -04:00
John Spray
f02316adb4 mgr: enable inter-module calls
This is being done by passing native CPython objects
back and forth.  It's safe because sub-interpreters in CPython
share memory allocation infrastructure and share the GIL.

With a view to PEP554, we limit inter-interpreter calls
to pickleable objects, so that this may be implemented
using byte-arrays in future.

This infrastructure should enable:
 - the dashboard to display the status of other modules, for
   example the set of progress indicators from `progress`
 - dashboard and restful to share an underlying long running
   job mechanism.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-07-20 13:07:17 -04:00
Casey Bodley
0b5bed7ecf
Merge pull request #22919 from liewegas/wip-rgw-cls-test
qa/suites/rgw/verify/tasks/cls_rgw: test cls_rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-07-20 10:01:36 -04:00
Kefu Chai
abb3e2bedf
Merge pull request #22975 from tone-zhang/vstart-spdk
vstart.sh: Support SPDK in Ceph development deployment

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-20 22:00:26 +08:00
Casey Bodley
a52d430bff
Merge pull request #22790 from joke-lee/WIP-admin-rest-api-user-info-with-access-key
rgw: support admin rest api get user info through user's access-key

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-07-20 09:59:24 -04:00
Casey Bodley
db59ee3bd0
Merge pull request #23099 from adamemerson/wip-all-your-fault
rgw: All Your Fault

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2018-07-20 09:57:10 -04:00
Casey Bodley
6a65439b38
Merge pull request #23058 from mkogan1/cont-19282-fix-rgw-multisite-datasync_v2
rgw: add curl_low_speed_limit and curl_low_speed_time config to avoid

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-07-20 09:52:31 -04:00
Casey Bodley
1c1e766cdc
Merge pull request #23119 from cbodley/wip-rgw-link-tenant
rgw: RGWBucket::link supports tenant

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2018-07-20 09:51:01 -04:00
Kefu Chai
a119734502
Merge pull request #23010 from noahdesu/mgr-expose-ec-profiles
mgr: expose ec profiles through manager

Reveiwed-by: John Spray <john.spray@redhat.com>
2018-07-20 20:54:13 +08:00
Kefu Chai
066b3f8cf3
Merge pull request #23016 from rzarzynski/wip-osd-cleanup-log_op_stats
osd: clean up and avoid extra ref-counting in PrimaryLogPG::log_op_stats.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-20 20:53:35 +08:00
Kefu Chai
b7d5506916
Merge pull request #22993 from jcsp/wip-progress
mgr/progress: introduce the `progress` module

Reviewed-by: Noah Watkins <nwatkins@redhat.com>
2018-07-20 20:49:38 +08:00
tone.zhang
71639aee00 vstart.sh: Support SPDK in Ceph development deployment
Support SPDK in vstart.sh, the developer can debug SPDK within
development deployment with the option
"--bluestore-spdk <vendor>:<device>".

Signed-off-by: tone.zhang <tone.zhang@arm.com>
2018-07-20 08:44:19 +08:00
Dan Mick
2365d597bf
Merge pull request #23121 from noahdesu/mgr-crash-history
mgr/crash: add hour granularity crash summary
2018-07-19 15:21:25 -07:00
Noah Watkins
caa121b5f0 mgr/crash: json report of recent crashes
reports crashes per entity type with hour granularity. primarily for
consumption by the insights module.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-07-19 09:38:17 -07:00
Noah Watkins
4dda7cb5dd mgr/crash: add timestamp filter helper
Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-07-19 09:38:07 -07:00
Alfredo Deza
12f1ffe784
Merge pull request #23129 from alfredodeza/wip-rm24952
ceph-volume lvm.listing only include devices if they exist

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-07-19 12:36:40 -04:00
Patrick Donnelly
fa4c0309c1
Merge PR #23097 into master
* refs/pull/23097/head:
	vstart_runner: pass ctx to FuseMount and load mount info

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2018-07-19 08:55:49 -07:00
Nathan Cutler
9b53c0d4fc
Merge pull request #23077 from smithfarm/wip-etc-sudoers
build/ops: rpm: make sudo a build dependency

Reviewed-by: Tim Serong <tserong@suse.com>
2018-07-19 16:26:20 +02:00
Nathan Cutler
9f54233104 build/ops: rpm: make sudo a build dependency
ede36df3253bd67bfe206abc0446f1f156feae64 added sudo as a runtime dependency,
but that commit also causes OBS builds to fail with:

[10666s] ... checking filelist
[10667s] ceph-osd-14.0.0-lp150.1332.1.x86_64.rpm: directories not owned by a package:
[10667s]  - /etc/sudoers.d

Since the "sudo" package is a runtime dependency of ceph-osd, it is guaranteed
to be installed on the target system and there is no reason for ceph-osd to own
this directory.

Having sudo as a build dependency makes OBS happy, and does no harm.

Follow-on fix for https://github.com/ceph/ceph/pull/22745

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-19 14:15:05 +02:00
Nathan Cutler
78a4f681de
Merge pull request #23089 from smithfarm/wip-seastar-build-deps
build/ops: rpm: fix seastar build dependencies for SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-19 14:10:31 +02:00
Nathan Cutler
aeb974b913 build/ops: rpm: fix seastar build dependencies for SUSE
cryptopp-devel is called libcryptopp-devel on SUSE.

/usr/bin/protoc is included in protobuf-devel on SUSE.

numactl-devel is called libnuma-devel on SUSE

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-19 10:08:41 +02:00
Alfredo Deza
37565f8163 ceph-volume lvm.listing only include devices if they exist
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-07-18 16:50:25 -04:00
Casey Bodley
1fd5320485 rgw: RGWBucket::link supports tenant
use tenant-aware helper functions to generate metadata keys/oids

this allows 'radosgw-admin bucket link' to succeed for a tenanted user,
provided that the bucket belongs to the same tenant

Fixes: http://tracker.ceph.com/issues/22666

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-07-18 16:35:57 -04:00
Alfredo Deza
158ed6105d
Merge pull request #23117 from ceph/wip-rm24796
ceph-volume: unmount lvs correctly before zapping

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-07-18 16:03:49 -04:00
Josh Durgin
1f780ef736
Merge pull request #23098 from neha-ojha/wip-limit-pglog
osd: limit pg log length under all circumstances

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-18 11:10:00 -07:00
Casey Bodley
4a00c16c4a
Merge pull request #22347 from cbodley/wip-24367
rgw: fixes for sync of versioned objects

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2018-07-18 13:25:58 -04:00
Casey Bodley
541d72d568
Merge pull request #23032 from cbodley/wip-24873
radosgw-admin: 'sync error trim' loops until complete

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2018-07-18 13:25:01 -04:00
Andrew Schoen
7bfc29d1ef
Merge pull request #23116 from alfredodeza/wip-rm24961
ceph-volume tests/functional run lvm list after OSD provisioning

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-07-18 17:11:20 +00:00
Alfredo Deza
e01fb206d6 ceph-volume tests/functional run lvm list after OSD provisioning
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-07-18 12:39:01 -04:00
Adam C. Emerson
7df69486f4 rgw: Cache notify fault injection
Allow a configured probability of dropping a notify message on the
floor without processing or acking it.

Fixes: http://tracker.ceph.com/issues/24962

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2018-07-18 12:20:18 -04:00
Andrew Schoen
5b175a5bb6 ceph-volume: unmount lvs correctly before zapping
If the lv found does not include the meatadata to construct an osd path,
then we should try to unmount it's path instead.

Fixes: http://tracker.ceph.com/issues/24796

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-07-18 11:08:15 -05:00
Alfredo Deza
ffa86bc07d
Merge pull request #23075 from alfredodeza/wip-24492
ceph-volume batch command

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-07-18 12:04:11 -04:00
David Zafman
48ae3d0844
Merge pull request #23083 from dzafman/wip-disable-skip-dd
osd: Change osd_skip_data_digest default to false and make it LEVEL_DEV

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-18 11:23:07 -04:00
Boris Ranto
2e4c8896b8
Merge pull request #22732 from b-ranto/wip-metric-reset
prometheus: Fix metric resets

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-07-18 16:20:57 +02:00
Patrick Donnelly
f973f9cac5
Merge PR #23040 into master
* refs/pull/23040/head:
	ceph-debug-docker: install python3 cephfs/rados packages

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2018-07-18 06:43:13 -07:00
Patrick Donnelly
8620c4caf5
vstart_runner: pass ctx to FuseMount and load mount info
Credit to Venky for finding the problem.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-18 06:42:16 -07:00
Jason Dillaman
67c8b6e0d0
Merge pull request #23008 from tchaikov/wip-rbd-silence-gcc-warning
tools/rbd: assert(g_ceph_context) not g_conf

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-07-18 08:10:07 -04:00
Lenz Grimmer
2c783ebe8b
Merge pull request #22902 from alfonsomthd/24778-dashboard-info-cards-refactor
mgr/dashboard: Dashboard info cards refactoring

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kanika Murarka <kmurarka@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-07-18 12:33:19 +02:00
Kefu Chai
3817e30d68
Merge pull request #23108 from tchaikov/wip-install-deps-seastar
run-make-check: set WITH_SEASTAR with a non-empty string

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2018-07-18 11:53:07 +08:00
Kefu Chai
4ca636f9b6 install-deps: set DEBIAN_FRONTEND for apt-get
so it won't complain if the tty is not an interactive one.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-18 10:28:08 +08:00
Kefu Chai
45bca777aa run-make-check: set WITH_SEASTAR with a non-empty string
otherwise "test -n $WITH_SEASTAR" fails, and the dependencies won't be
installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-18 10:25:21 +08:00
Patrick Donnelly
3e18ac7217
Merge PR #22966 into master
* refs/pull/22966/head:
	mds: clarify doc string for mds_beacon_grace

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-07-17 18:47:35 -07:00
Neha Ojha
a5329ba8dd osd/PGLog: allow pg log trim when complete_to is less than trim_to
Signed-off-by: Neha Ojha <nojha@redhat.com>
2018-07-17 14:06:19 -07:00
Boris Ranto
9130ab40b3 prometheus: Set the response header for cached response
Signed-off-by: Boris Ranto <branto@redhat.com>
2018-07-17 22:29:00 +02:00
Boris Ranto
83565c3483 prometheus: Reset the time the data was captured
This was forgotten during the previous patches.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-07-17 22:29:00 +02:00
Boris Ranto
2583672dba prometheus: Format metrics in the collect function
This allows us to clear metrics upon formatting in the collect function.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-07-17 22:28:59 +02:00
Boris Ranto
c00dd55f40 prometheus: Remove the Metrics class
The class was introduced to account for the 'pending' mechanism but that
is no longer being used by the code so we no longer need it since it
serves a purpose of a simple dictionary, now.

This simplifies the code and brings a bit of performance improvements
since the Metric classes in the collect method are created only if they
did not exist before.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-07-17 22:28:59 +02:00
Boris Ranto
d395868d9e prometheus: Optimize metrics formatting
It is faster to use join for join multiple strings than the regular
string concatenation since it does not create a new string on each
iteration.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-07-17 22:28:59 +02:00
Boris Ranto
0869351892 prometheus: Use instance instead of inst variable
Signed-off-by: Boris Ranto <branto@redhat.com>
2018-07-17 22:28:59 +02:00