Commit Graph

84137 Commits

Author SHA1 Message Date
Kefu Chai
6e0e5b5939
Merge pull request #20740 from Songweibin/wip-skip-logical-devices
osd: skip dumping logical devices

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-08 23:19:11 +08:00
Kefu Chai
cd7ece464a
Merge pull request #20743 from rjfd/wip-dashboard-settings
mgr/dashboard_v2: Configuration settings support

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-08 23:18:16 +08:00
Kefu Chai
0d5a33c6ce
Merge pull request #20727 from linuxbox2/wip-clang-5
bluestore: define default value of LoglevelV only once (3 templates)

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-08 23:15:53 +08:00
Kefu Chai
c1ba9ac544
Merge pull request #20711 from guzhongyan/mgr-nodels
mon: 'node ls' mgr support

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-08 23:14:50 +08:00
Kefu Chai
ecc64b034d
Merge pull request #20706 from xiexingguo/wip-balancer-03
pybind/mgr/balancer: fix pool-deletion vs auto-optimization race

Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-08 23:13:37 +08:00
Kefu Chai
34516b80ae
Merge pull request #20704 from xiexingguo/wip-balancer-02
mon/OSDMonitor: kill pending upmap changes too if pool is gone

Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-08 23:13:01 +08:00
Kefu Chai
a6c271e2be
Merge pull request #20703 from xiexingguo/wip-balancer-01
osd/OSDMap: ignore PGs from pools of failure-domain OSD

Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-08 23:12:27 +08:00
Kefu Chai
6e420093b0
Merge pull request #20644 from jcsp/wip-23166
mgr/prometheus: skip OSD output if missing from CRUSH devices

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-08 23:11:47 +08:00
Jason Dillaman
26df7ff329
Merge pull request #20773 from idryomov/wip-krbd-include-sysmacros
krbd: include sys/sysmacros.h for major, minor and makedev

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-08 08:48:55 -05:00
Ilya Dryomov
a048b5e612
Merge pull request #20750 from idryomov/wip-krbd-whole-object-discard-test
qa: krbd whole-object-discard test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-08 12:52:56 +01:00
John Spray
a0a001a8df
Merge pull request #20755 from LenzGr/mgr-dashboard-documentation
doc: Updated dashboard v2 feature list

Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-08 10:56:51 +00:00
Ilya Dryomov
070ce41f0b
Merge pull request #20792 from yangdongsheng/krbd_osd_request_timeout
rbd: support osd_request_timeout in rbd map command

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2018-03-08 11:31:49 +01:00
Dongsheng Yang
f37ae96fa7 rbd: support osd_request_timeout in rbd map command
Default of osd_request_timeout in krbd is 0 means no timeout,
but we sometimes need to set this value by rbd command. then
pass osd_request_timeout to kernel rbd in rbd mapping.

Fixes: http://tracker.ceph.com/issues/23073
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2018-03-08 04:12:41 -05:00
songweibin
e6bb0a9578 rbd: pool_percent_used should not divided by 100
The value of 'percent_used' in `ceph df --format=json`
has already been divided by 100.

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2018-03-08 10:29:22 +08:00
xie xingguo
1e3498ad91 pybind/mgr/balancer: sanity check against empty roots
To avoid crashes as below:
```
File "/usr/lib/ceph/mgr/balancer/module.py", line 181, in calc_stats
    score += target[k] * (math.erf(((adjusted - avg)/avg) / math.sqrt(2.0)))
ZeroDivisionError: float division by zero
```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-08 09:05:58 +08:00
xie xingguo
193c3af4fb pybind/mgr/balancer: s/iteritems/iterkeys/
We don't care about the values here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-08 09:05:43 +08:00
Mykola Golub
9ffbac6fc5
Merge pull request #20729 from dillaman/wip-22738
qa/suites/rbd: segregated v1 image format tests

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-03-07 22:53:12 +02:00
Patrick Donnelly
a1c5a552b8
Merge PR #20705 into master
* refs/pull/20705/head:
	doc: Fix -d description in ceph-fuse

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-03-07 11:27:58 -08:00
Jason Dillaman
4e59ea9568 librbd: discard from cache after committed to disk
This prevents a possible race condition with readahead racing with a
discard and re-populating the cache with non-zeroed data.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
475f0fd9ef librbd: moved skip partial discard logic to object request
This allows the journal object dispatch layer to properly complete
skipped extents and commit the associated event.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
59802c5255 librbd: avoid potential for lock inversion in journal IO replay
If an IO replay immediately fails due to a image shrink applied after
the IO, the lock order will be incorrect.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
e980bd74d4 librbd: reduce lock scope on copy-on-read IO path
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
1ea789c247 librbd: removed legacy object cacher hooks
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
c9e35cfaad rbd-mirror: corrected implicitly included dout header
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
898223eea6 librbd: removed legacy journal event tracking hooks from IO path
The new journal object dispatch layer handles delaying writes to the
OSD until the journal event is safe. This avoid the need for two
different code paths for cache and non-cache case.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
1231e33d25 librbd: removed object cacher hooks from IO image request
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
9440e5a1da librbd: removed object cacher hooks from IO object request
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
4f866c6f7f librbd: integrate object cacher writeback handler with dispatcher
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
dcdc224144 librbd: switch to new object dispatcher-based cache hooks
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
9aed97c043 librbd: pass image flush requests to object dispatcher
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
b000324a08 librbd: include flush source in image dispatch spec
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:42 -05:00
Jason Dillaman
425e579a3d librbd: wrap a object dispatcher facade around object cacher
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:41 -05:00
Jason Dillaman
623294202a librbd: added object dispatch handler for journal IO
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:45:00 -05:00
Jason Dillaman
7c3ca2b528 librbd: switch to object IO requests to dispatcher interface
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-03-07 12:44:59 -05:00
Sebastian Wagner
f3d6067658 mgr/dashboard_v2: removed unused code that broke teuthology
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-07 18:18:31 +01:00
Ricardo Dias
17cfac2b68
mgr/dashboard: Added instructions on how to add config settings to HACKING
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-03-07 16:29:55 +00:00
Ricardo Dias
f681b23280
mgr/dashboard: added settings module
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-03-07 16:29:54 +00:00
Sebastian Wagner
0ad029f2ea common/config: Fix dumping of md_config_t::config_options.
Caused by 49a5e775aa92a8218bb659909de20637d5aa660a

This fixes the dashboard_v2 `cluster_conf` endpoint.

Fixes: http://tracker.ceph.com/issues/23265

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-07 16:27:36 +01:00
Casey Bodley
b497d9fdd3 qa/rgw: combine swift, s3tests, ragweed into single verify task
these tests don't need to be split across the job matrix the same way
that we split features like frontend, ssl, objectstore, etc. by
combining them, we can still test the whole matrix of features, but with
only 1/3 of the total jobs

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-03-07 09:21:55 -05:00
Casey Bodley
cf52fc5ab9
Merge pull request #20693 from cbodley/wip-rgw-s3website-redirect
rgw: s3website error handler uses original object name

Reviewed-by: Robin H. Johnson <robin.johnson@dreamhost.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2018-03-07 07:57:57 -05:00
Patrick Nawracay
45b9174191 mgr/dashboard: Fix backend tests for newer CherryPy versions
The backend test run by `run-backend-api-tests.sh` fails for CherryPy
versions 13.x and 14.x.

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-03-07 13:45:52 +01:00
Tiago Melo
64a9e554a3 mgr/dashboard_v2: add mocked service provider for TcmuIscsiService
This removes the warnings shown when running the tests with PhantomJS.

Signed-off-by: Tiago Melo <tmelo@suse.com>
2018-03-07 12:25:36 +00:00
Ilya Dryomov
8d8390ff95 qa: krbd whole-object-discard test
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-03-07 12:06:33 +01:00
Ilya Dryomov
fbc97f5565 krbd: include sys/sysmacros.h for major, minor and makedev
Previously we got these through sys/types.h, but that's now deprecated:

  warning: In the GNU C Library, "major" is defined
   by <sys/sysmacros.h>. For historical compatibility, it is
   currently defined by <sys/types.h> as well, but we plan to
   remove this soon. To use "major", include <sys/sysmacros.h>
   directly. If you did not intend to use a system-defined macro
   "major", you should undefine it after including <sys/types.h>.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2018-03-07 11:42:38 +01:00
Kefu Chai
480f078c14
Merge pull request #20768 from tchaikov/wip-cmake-dashboard
cmake,test/mgr: restructure dashboard tests and cmake related fixes

Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-03-07 18:39:22 +08:00
Orit Wasserman
f9395d63f8 rgw: fix crash with rgw_run_sync_thread false
Fixes: http://tracker.ceph.com/issues/20448
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2018-03-07 12:01:21 +02:00
Kefu Chai
59221274cb test/mgr: restructure dashboard tests
* restructure dashboard tests so they follow the model of other tests
  based on ceph-helpers.sh. i.e. extract the TEST_* out of run()
  function.
* expose mon-host using CEPH_ARGS env variable.
* mgr-dashboard_v2-smoke.sh: do not export CEPH_ARGS in a subshell, so
  the run_mgr() and ceph cli can also have access to the updated
  CEPH_ARGS, and hence they will be able to locate the mon. and because
  auth-supported=none is populated using CEPH_ARGS, this test won't fail
  because of missing keyring.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-03-07 17:27:36 +08:00
Kefu Chai
aedbbe791f test: run 'npm' command in a directory specified by absolute path
in case developer wants to run the test in "./build" or some random
directory.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-03-07 17:27:36 +08:00
Kefu Chai
ad0d7c89a1 cmake: check for "npm" before using it
and allows use to specify the path in which npm is installed. in my
case, if $HOME/.npmrc points "prefix" to ${HOME}/.npm-packages, i will
be able to run

NPM_ROOT=${HOME}/.npm-packages cmake ..

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-03-07 17:23:49 +08:00
xie xingguo
020a031b62 osd/OSD: batch-list objects to reduce memory consumption
For PG with a huge amount of objects, it wouldn't be an ideal
way to list all of them at a time. Split them into small batches
which we can handle individually efficiently should instead be
the preferred option.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-03-07 17:20:36 +08:00