Commit Graph

73122 Commits

Author SHA1 Message Date
Sage Weil
22ddc2e64a qa/suites/upgrade/kraken-x: enable experimental for bluestore
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-30 09:28:13 -04:00
Sage Weil
99d96c3c09 Merge pull request #15294 from liewegas/wip-osd-tree
osd: 'osd tree in|out|up|down' to filter tree results

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-30 08:22:35 -05:00
Kefu Chai
9fd19500bf Merge pull request #15225 from smithfarm/wip-s390-rgw
build/ops: conditionalize rgw Beast frontend so it isn't built on s390x architecture

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-30 20:39:12 +08:00
Sage Weil
0027a3b9be mon: add up|down|in|out filters to 'osd tree'
For example, 'ceph osd tree down' will show *just* down OSDs and their
ancestors.  \o/

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-30 08:21:16 -04:00
Nathan Cutler
3dfffa5996 cmake: link radosgw with boost libraries
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-05-30 13:01:16 +02:00
Dongsheng Yang
8f3ef393f3 bluestore: fix a typo about bleustore
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2017-05-30 18:32:49 +08:00
Jos Collin
80145b4c9d test/msgr: silence warnings from -Wsign-compare
The following warning appears during make:

ceph/src/test/msgr/test_msgr.cc: In member function ‘virtual void MessengerTest_MarkdownTest_Test::TestBody()’:
ceph/src/test/msgr/test_msgr.cc:1474:46: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     CHECK_AND_WAIT_TRUE(srv_dispatcher.count > last + 1);
                         ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
ceph/src/test/msgr/test_msgr.cc:54:9: note: in definition of macro ‘CHECK_AND_WAIT_TRUE’
     if (expr)                           \
         ^~~~
ceph/src/test/msgr/test_msgr.cc:1475:30: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (srv_dispatcher.count == last) {
         ~~~~~~~~~~~~~~~~~~~~~^~~~~~~

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-05-30 10:49:23 +05:30
Kefu Chai
592d95e58e Merge pull request #15355 from joscollin/wip-cleanup-testdenc-warning
test: test_denc.cc: silence warning from -Wsign-compare

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-30 12:28:12 +08:00
Jos Collin
e6a40e9e0e test: test_denc.cc: silence warning from -Wsign-compare
The following warning appears during make:
ceph/src/test/test_denc.cc:669:5:   required from here
ceph/src/googletest/googletest/include/gtest/gtest.h:1530:28: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
ceph/src/googletest/googletest/include/gtest/gtest.h:1510:7:
   if (val1 op val2) {\

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-05-30 08:36:01 +05:30
Sage Weil
f53f56350b os/bluestore: deep decode onode value
In particular, we want the attrs (map<string,bufferptr>) to be a deep
decode so that we do not pin this buffer, and so that any changed attr
will free the previous memory.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:19 -04:00
Sage Weil
d334d2d3cb os/bluestore: fsck: use btree_set to replace set<uint64_t>
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:18 -04:00
Sage Weil
7a51a34292 include/cpp-btree/btree_set: add btree_set
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:18 -04:00
Sage Weil
0fae9bf434 ceph-bluestore-tool: init deep = false
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:18 -04:00
Sage Weil
8761624574 os/bluestore: rebuild Buffer buffers with too much waste
Avoid pinning extra memory by rebuilding Buffer buffers when we waste too
much.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:18 -04:00
Sage Weil
cf98e9d5bf buffer: make wasted() const
Remove useless assert (we'll segv on the next line anyway).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:18 -04:00
Sage Weil
ecb3167f1b os/bluestore: wait for kv thread to start before stopping it
Otherwise we can assert out when we try to join a thread that
hasn't started.

- move everything into _kv_start() and _kv_stop()
- separate stop bools for each thread
- wait until thread starts before signalling stop (and potentially calling
join()).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:17 -04:00
Sage Weil
811c38ed6b os/bluestore: fix fsck deferred_replay
_deferred_replay needs the kv_sync_thread to complete IOs; start them
just for that, but then shut them down again.  (We might revisit that
later if/when fsck does any sort of repair.)

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:56:17 -04:00
Sage Weil
d79c7723eb Merge pull request #15058 from yuyuyu101/wip-rocksdb-opt
kv/rocksdb: supports SliceParts interface

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-29 20:47:52 -05:00
Sage Weil
5988651e8c Merge pull request #15068 from liewegas/wip-osdmap-require
osd/OSDMap: replace require_*_osds flags with a single require_osd_release field

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-29 20:47:06 -05:00
Sage Weil
43a58ec2a2 Merge pull request #15313 from Linbing/wip-pg-scrub-code-optimize
mon: it's no need to get pg action_primary osd twice in pg scrub

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-29 20:42:50 -05:00
Sage Weil
3d6b6d72f7 mon/OSDMonitor: fix osd metadata update ordering
Do not do both.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:34:29 -04:00
Sage Weil
e570005e2c osd: replace all require_*_osds flag checks with a require_osd_release comparison
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:34:29 -04:00
Sage Weil
3118d9a154 osd: replace require_*_osds flags with require_osd_release field
- OSDMap encode and decode translate between the flags and int
representations.
- OSDMap::Incremental only does decode; we do not expect to ever encode
an incremental osdmap for an old osd that sets any of these flags.
- the 'osd set' command still lets you set the jewel and kraken flags,
but not luminous.
- OSDMap::apply_incremental handles the conversion of legacy require flags
to the new field if the jewel or kraken flags have to be set before
starting the osd upgrade.
- clear out the legacy flags when we make the luminous transition only;
until then we keep using the old flag in the encoded and decoded version
(although the require_osd_release field will be accurate in memory in all
cases).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:33:17 -04:00
Sage Weil
0b517992e0 mon: new 'osd require-osd-release' command
Note that we only allow setting this to 'luminous'.  If the 'jewel' or 'kraken'
flags need to be set that can be done through the legacy interface.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:33:17 -04:00
Sage Weil
3d4c4d9d9d osd/OSDMap: add require_osd_release numeric field, CEPH_RELEASE_*
Instead of eating up a flag for each release, which is super-awkward and
annoying, just keep a numeric release version min.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-29 21:33:17 -04:00
Willem Jan Withagen
9b1afe4aca cmake: delete excidentail reintroduced line.
- This target got reintroduced by accident after the creation
  of ceph-common in commit:
    https://github.com/ceph/ceph/commit/046b2bde6d93992c36d4943c4f7859993f034ca

  Detection was trigger by refering to libresolv, and results in a
  linking error.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-05-29 22:47:47 +02:00
Nathan Cutler
25d794d98a build/ops: deb: no rgw Beast frontend on s390x
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-05-29 22:47:46 +02:00
Nathan Cutler
12625e1e92 build/ops: rpm: no rgw Beast frontend on s390x
Since the Beast frontend uses boost::context which is not supported on
s390x.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-29 22:47:46 +02:00
Nathan Cutler
7d81a4b710 build/ops: rpm: reverse s390 bcond conditional block
I'm not a fan of "if NOT x - then - else" blocks.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-05-29 22:47:46 +02:00
Nathan Cutler
b22cde8e88 cmake: build boost::context and coroutine only with rgw Beast frontend
boost::context is currently (1.63) unsupported for s390x and anyway
it makes sense to conditionalize Boost components so they are only
built with the Ceph components that need them (like is already being
done for mgr).

Fixes: http://tracker.ceph.com/issues/20048
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-29 22:47:44 +02:00
Sage Weil
676fc82ced Merge pull request #15324 from yuyuyu101/wip-20093
msg/async: avoid requeue racing with handle_write

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-29 14:49:57 -05:00
Sage Weil
8554158574 Merge pull request #15325 from liewegas/wip-redirect
osd,librados: add manifest, redirect

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-29 14:48:33 -05:00
Sage Weil
10b32aa501 Merge pull request #15326 from xiexingguo/fix-kvtool-tip
tools/ceph_kvstore_tool: add "bluestore-kv" to usage

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-29 14:48:10 -05:00
Sage Weil
0f803c5ac8 Merge pull request #15337 from wjwithagen/wip-wjw-freebsd-fix-includes
common: include: Redo some includes for FreeBSD
2017-05-29 14:47:50 -05:00
Logan Blyth
89e36def22 Added libradosstriper example Signed-off-by: Logan Blyth <mrbojangles3@gmail.com> 2017-05-29 15:26:15 -04:00
Kefu Chai
7a5a0a893e Merge pull request #15338 from trociny/wip-get_process_name
common: get_process_name: use getprogname on bsd systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2017-05-29 20:55:30 +08:00
Brad Hubbard
24b0e2bb9b Merge pull request #15345 from badone/wip-tracker-19900-wait_auth_rotating-crash-2
osd: Return early on shutdown

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-29 18:25:36 +10:00
Brad Hubbard
ae6e32dc4b osd: Return early on shutdown
eb5c02d was missing a return statement, rectify that.

Fixes: http://tracker.ceph.com/issues/19900

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-05-29 11:29:23 +10:00
Willem Jan Withagen
dd558967d3 cmake: delete excidentail reintroduced line.
- This target got reintroduced by accident after the creation
  of ceph-common in commit:
    https://github.com/ceph/ceph/commit/046b2bde6d93992c36d4943c4f7859993f034ca

  Detection was trigger by refering to libresolv, and results in a
  linking error.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-05-28 19:43:30 +02:00
Kefu Chai
0d3e6a9db4 Merge pull request #15336 from wjwithagen/wjw-wip-iso_8601
common/iso_8601.cc: Make return expression Clang compatible

Reviewed-By: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-29 00:44:46 +08:00
Willem Jan Withagen
2f1b0c1863 erasure-code/isa/ErasureCodePluginIsa.cc: fixup includes
- Buffer.h is needed to prevent Clang seriously complaining
   about missing and wrong forward declarations of
	ceph::buffer:ptr
   including buffer_fwd.h does not help.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-05-28 14:57:06 +02:00
Willem Jan Withagen
973a4a0101 build/include: Redo som includes for FreeBSD
- During include cleanup just a too bit much was removed for FreeBSD
  to get things compiled.
  This redoes some of the includes.

Tracker: http://tracker.ceph.com/issues/19883
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-05-28 14:56:04 +02:00
Jesse Williamson
de4d111305 Adds C++ warning flag for C Variable-Length Arrays.
C VLAs are not supported in C++. However, the GNU compiler allows
them as an extension, which it does not warn about when not in
pedantic mode. Unfortunately, it's easy to accidentally write
a VLA, even unintentionally-- adding this warning will help us
catch that.

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2017-05-28 05:24:26 -07:00
Kefu Chai
ef9d93b8ea Merge pull request #15055 from mogeb/wip-with-instrument-functions
cmake: Add -finstrument-functions flag to OSD code

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-28 14:45:57 +08:00
Sage Weil
420dd7ff02 Merge pull request #15319 from liewegas/wip-upmap
mon,osd/OSDMap: a couple pg-upmap fixes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-27 22:17:14 -05:00
Sage Weil
d8f3ee1623 Merge pull request #15292 from liewegas/wip-map-cache
osd: reduce map cache size

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-27 22:11:57 -05:00
Sage Weil
a9a728fe4d Merge pull request #15296 from liewegas/wip-fix-at-end
qa/tasks/repair_test: unset flags we set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-27 22:11:31 -05:00
Sage Weil
040fd8ce0d Merge pull request #15297 from liewegas/wip-19595
mgr/MgrStandby: prevent use-after-free on just-shut-down Mgr

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-27 22:10:12 -05:00
Kefu Chai
c6f1c06de9 Merge pull request #14284 from tchaikov/wip-19439
qa/tasks/rebuild_mondb: grant "mgr:allow *" to client.admin

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-28 10:38:26 +08:00
Sage Weil
0b0c7e56a1 Merge pull request #15284 from majianpeng/bluestore-batch-throotle
os/bluestore: batch throttle

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-27 21:02:49 -05:00