Commit Graph

85873 Commits

Author SHA1 Message Date
John Spray
6e447fd101 mgr: clean up ActivePyModules logging
This was running the prefix and message together.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-04-23 07:29:46 -04:00
John Spray
3bc9850e6f mgr: fix PyModuleConfig::set_config
Commands were slightly wrong.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-04-23 07:29:45 -04:00
John Spray
48c9071d63 mgr: remove dead update_perf_schema fn definition
Signed-off-by: John Spray <john.spray@redhat.com>
2018-04-23 07:29:45 -04:00
John Spray
37484af0b8 mgr: rework kv store load path
The locking and blocking around this was a bit
tricky.  Do the simple thing, and pull the
load_store out to Mgr so that it can be safely
done as part of the background_init process (just drop
Mgr::lock across blocking actions).

Signed-off-by: John Spray <john.spray@redhat.com>
2018-04-23 07:29:45 -04:00
John Spray
bd80cf9398 mgr: update python interface for store vs. config
Signed-off-by: John Spray <john.spray@redhat.com>
2018-04-23 07:29:45 -04:00
John Spray
306e4854b5 mgr: separate out module_config and store_cache
Signed-off-by: John Spray <john.spray@redhat.com>
2018-04-23 07:29:45 -04:00
Rubab-Syed
3193d40d0d mgr: centralized setting/getting of mgr configs
Signed-off-by: Rubab Syed<rubab.syed21@gmail.com>
2018-04-23 07:29:44 -04:00
Lenz Grimmer
a2639b6072 PendingReleaseNotes: Added note about Dashboard v2, fixed typo
Added a bullet point about the new Ceph mgr dashboard, fixed typo in the
LZ4 compression bullet point.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
2018-04-23 13:28:53 +02:00
Kefu Chai
baa095efa3
Merge pull request #21588 from xiexingguo/wip-fix-ec-ar-order
osd/PG: perfer EC async_recovery_targets in reverse order of cost

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-23 18:52:48 +08:00
Ricardo Dias
6b29e8abc7
Merge pull request #21562 from LenzGr/mgr-dashboard-documentation
doc: updated dashboard feature list (added new RGW details, Pools)

Reviewed-by: Volker Theile <vtheile@suse.com>
2018-04-23 11:36:46 +01:00
Sebastian Wagner
3a795a42d5 mgr/dashboard: Fix data race and use-before-assignment
The race happes, if a task raises an exception very early.
Then, `self.getter_thread.event` fails as `getter_thread` is
already `None`

Also fix use-before-assignment, as `t0` and `t1` are only defined,
if no exception was raised.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-04-23 10:13:28 +02:00
Mykola Golub
7ca8e70471
Merge pull request #21564 from dillaman/wip-23809
rbd: cleanup handling of IEC byte units

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-04-23 09:07:19 +03:00
Kefu Chai
9f35318ee6
Merge pull request #21408 from hsepeng/oisizemismatch-osd-bugfix
osd/PrimaryLogPG: fix the oi size mismatch with real object size

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-23 13:59:15 +08:00
Sage Weil
bee466cd1c mgr/DaemonServer: use helper to get DaemonKey
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-22 22:07:06 -05:00
Sage Weil
082a700ac8 mgr/MgrClient: close mgr session on daemon MgrClient shutdown
If we are a service daemon, send a MMgrClose on shutdown to clear ourselves
out of the ServiceMap.

Note that this is a best-effort attempt; we don't block shutdown if the
mgr isn't currently available.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-22 22:07:06 -05:00
Gu Zhongyan
a7677c76a5 mon,mgr: improve 'mgr module disable/enable' cmds
when running those cmds, check if the module do exist
or is disabled already.
also check if the module is enabled already.

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
2018-04-23 09:35:06 +08:00
xie xingguo
441876f4d2 osd/PG: perfer EC async_recovery_targets in reverse order of cost
This is a follow-up fix of https://github.com/ceph/ceph/pull/21578,
in which I forget that erasure-coded-pools share the same logic
when determining the async_recovery_targets..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-04-23 09:08:09 +08:00
Kefu Chai
4f73c6035a
Merge pull request #21549 from rjfd/wip-dashboard-fix-qa-whitelist
qa/suites/rados/mgr/tasks/dashboard: add MDS_ALL_DOWN to whitelist

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-21 23:15:53 +08:00
Ricardo Dias
7309be145b
qa/suites/rados/mgr/tasks/dashboard: add MDS_(ALL_DOWN|UP_LESS_THAN_MAX) to whitelist
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-04-21 15:58:23 +01:00
Kefu Chai
c477b199af
Merge pull request #21560 from jan--f/fix_strtoll_interface
common/strtoll: remove superfluous const modifier

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-21 22:33:36 +08:00
Kefu Chai
bfce48aeeb
Merge pull request #21550 from wjwithagen/wjw-kevent-fix
EventKqueue: Clang want realloc return to be typed

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-21 22:32:43 +08:00
Kefu Chai
586ea5cb29
Merge pull request #21579 from tchaikov/wip-test.sh-with-multi-mds
qa/workunits/cephtool/test.sh: fix test_mon_mds()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-21 22:31:43 +08:00
Kefu Chai
adfb9503b9
Merge pull request #21578 from xiexingguo/wip-fix-async-recovery-peer
osd/PG: perfer async_recovery_targets in reverse order of cost

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-21 22:31:07 +08:00
Kefu Chai
6f834d5da6
Merge pull request #21552 from smithfarm/wip-23805
qa/standalone/ceph-helpers.sh: provide argument to dirname

Reviewed-by: David Zafman <dzafman@redhat.com>
2018-04-21 22:30:33 +08:00
Kefu Chai
096668ec5d objclass-sdk: expose __cls_init() to the world
the __cls_init() function should be visible. otherwise we cannot call it
to register the cls methods

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 17:15:59 +08:00
xie xingguo
8a395f2586 osd/PGLog: assert out on performing overflowed log trimming
Performing overflowed log-trim can be a sign of big trouble, e.g.,
the **complete_to** iterator will now point to an invalid position
of the original pg-log list when the trimming is done, and hence
randomly trigger **Segmentation fault**s as below:
```
2018-03-07 17:38:46.109018 7f274a4ed700 -1 *** Caught signal (Segmentation fault) **
1: (()+0xa51f31) [0x7f278290bf31]
2: (()+0xf370) [0x7f277fb4f370]
3: (PrimaryLogPG::recover_got(hobject_t, eversion_t)+0x266) [0x7f2782512786]
4: (PrimaryLogPG::on_local_recover(hobject_t const&, ObjectRecoveryInfo const&, std::shared_ptr<ObjectContext>, bool, ObjectStore::Tran
saction*)+0x2a4) [0x7f278251f3b4]
5: (ReplicatedBackend::handle_push(pg_shard_t, PushOp const&, PushReplyOp*, ObjectStore::Transaction*)+0x2e2) [0x7f2782690f82]
6: (ReplicatedBackend::_do_push(boost::intrusive_ptr<OpRequest>)+0x194) [0x7f2782691224]
7: (ReplicatedBackend::_handle_message(boost::intrusive_ptr<OpRequest>)+0x2f1) [0x7f278269fd41]
8: (PGBackend::handle_message(boost::intrusive_ptr<OpRequest>)+0x50) [0x7f27825c2470]
```

The root cause of why PGs are starting to trim more log entries than
we expect is still lost to me, but setting the trap here should generally
do no harm and hopefully expose the above problem a little bit more offen.
We'll see.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-04-21 11:03:16 +08:00
Kefu Chai
735d5fd414
Merge pull request #21507 from liewegas/wip-rados-test
qa/workunits/rados/test.sh: ensure tee output is valid filename

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-21 09:30:02 +08:00
Kefu Chai
b464c539cd
Merge pull request #20921 from jcsp/wip-23385
mgr: improve reporting on unloadable modules

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-21 09:29:25 +08:00
Kefu Chai
6b83d99e09
Merge pull request #20343 from trociny/wip-rados-ordersnap
librados: add OPERATION_ORDERSNAP flag and yet another aio_operate method

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-21 09:28:05 +08:00
Kefu Chai
717b941e7a
Merge pull request #21521 from tchaikov/wip-stoll
common/strtol: fix strict_strtoll() so it accepts hex starting with 0x

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-04-21 09:24:55 +08:00
Kefu Chai
106737323d qa/workunits/cephtool/test.sh: fix test_mon_mds()
"ceph fs set cephfs allow_multimds false" is deprecated, and multimds is
enabled by default, so "ceph fs set cephfs max_mds 4" won't fail with
the default settings.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 09:20:35 +08:00
Kefu Chai
53a1500a8f cmake,make-dist: bump up boost version to 1.67
* to pick up the fix of https://svn.boost.org/trac10/ticket/11622
* also the boost::python's library name now includes the version suffix
  of python version, so update BuildBoost.cmake accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 09:02:28 +08:00
xie xingguo
1118b44361 osd/PG: perfer async_recovery_targets in reverse order of cost
Theoretically peers which have a longer list of objects to recover
shall equivalently take a longer time to recover and hence have a
bigger chance to block client ops.

Also, to minimize the risk of data loss, we want to bring those broken
(inconsistent) peers back to normal as soon as possible. Putting them
into the async_recovery_targets queue, however, did quite the oppsite.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-04-21 08:52:52 +08:00
Casey Bodley
307b5182c9
Merge pull request #21358 from cbodley/wip-23663
rgw: fix use of libcurl with empty header values

http://pulpito.ceph.com/cbodley-2018-04-20_19:15:33-rgw-wip-cbodley-testing-distro-basic-smithi/
2018-04-20 20:35:09 -04:00
Casey Bodley
66a22702c7
Merge pull request #21380 from cbodley/wip-23680
rgw: fix radosgw linkage with WITH_RADOSGW_BEAST_FRONTEND=OFF

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-04-20 20:33:47 -04:00
Casey Bodley
2e1b7f971c
Merge pull request #21567 from cbodley/wip-23778
random: revert change from boost::optional to std::optional

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-20 20:32:32 -04:00
Alfredo Deza
6f60a995de
Merge pull request #21455 from dmick/wip-ceph-volume
ceph-volume: Nits noticed while studying code
2018-04-20 20:10:31 -04:00
Jason Dillaman
7491123493
Merge pull request #21571 from tchaikov/wip-23517
cmake: hide symbols import from other libraries in libcls_*

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-04-20 16:53:03 -04:00
Kefu Chai
94271de7ff
Merge pull request #21573 from tchaikov/wip-cmake
cmake: s/boost_256/boost_sha256/

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-04-21 02:01:30 +08:00
Andrew Schoen
e5dfec58ae
Merge pull request #21570 from alfredodeza/wip-cv-doc-updates
ceph-volume update man page for prepare/activate flags

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-04-20 17:40:11 +00:00
Kefu Chai
112f79f8de cmake: s/boost_256/boost_sha256/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:33:24 +08:00
Kefu Chai
82cb956ec3
Merge pull request #21484 from tchaikov/wip-cmake-common-cleanup
cmake, common: silence cmake and gcc warnings

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-04-21 01:21:32 +08:00
Kefu Chai
3178bb9638 cmake: disable gflags support in rocksdb
to silence the warnings like

CMake Warning at CMakeLists.txt:73 (find_package):
  By not providing "Findgflags.cmake" in CMAKE_MODULE_PATH this project
has
  asked CMake to find a package configuration file provided by "gflags",
but
  CMake did not find one.

  Could not find a package configuration file provided by "gflags" with
any
  of the following names:

    gflagsConfig.cmake
    gflags-config.cmake

  Add the installation prefix of "gflags" to CMAKE_PREFIX_PATH or set
  "gflags_DIR" to a directory containing one of the above files.  If
"gflags"
  provides a separate development package or SDK, be sure it has been
  installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:17:37 +08:00
Kefu Chai
19aee456c0 cmake,make-dist: use sha256 instead of md5 for checksum
the boost's download page offers the SHA256 hash, so it'd be easier to
verify the hash this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:17:35 +08:00
Kefu Chai
3f24a7da92 cmake: hide symbols import from other libraries in libcls_*
so they will not be involved when resolving symbols. ld tries to
keep a shared library around even if it fails to load it if it offers
some unique symbols. in that case, the library will not be properly
unloaded, and even worse it will interfere with following dlopen()
calls, because it is marked with NODELETE by dlopen(). if it has some
unresolved symbol and does offer some "unique" symbols required by
the library to be loaded, the library will fail to load, despite the
fact that the "unique" symbol is also offered by the executable.

for more details, see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60731 and
https://sourceware.org/bugzilla/show_bug.cgi?id=14577

Fixes: http://tracker.ceph.com/issues/23517
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-21 01:02:21 +08:00
Alfredo Deza
a454abc863 doc/man/ceph-volume update flags for activate
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-20 12:48:09 -04:00
Alfredo Deza
ae96b9f0a2 doc/man/ceph-volume update flags for prepare
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-20 12:48:01 -04:00
Casey Bodley
d505d77b7e random: revert change from boost::optional to std::optional
somehow this was breaking the seeding of thread-local engines on gcc.
we'll have to investigate this further, but for now i'm reverting this
piece to get messengers working again

Fixes: http://tracker.ceph.com/issues/23778

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-04-20 12:24:06 -04:00
Lenz Grimmer
580852e229 doc: updated dashboard feature list (added new RGW details, Pools)
Updated the dashboard feature list in the documentation to mention
the possibility to list Ceph pools and the RGW users and their buckets.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>
2018-04-20 18:11:07 +02:00
Ricardo Dias
2f0984ae06
Merge pull request #21516 from rjfd/wip-dashboard-rest-improvements
mgr/dashboard: RESTController improvements

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-04-20 16:45:26 +01:00