Commit Graph

71633 Commits

Author SHA1 Message Date
Sage Weil
5f1c03e536 Merge pull request #14330 from ivancich/wip-bring-in-dmclock
dmclock: initial commit of dmclock QoS library
2017-05-01 20:29:43 -05:00
Sage Weil
0bded75235 Merge pull request #14896 from liewegas/wip-19638
osd: stop MgrClient callbacks on shutdown

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-01 20:25:29 -05:00
Sage Weil
2d6be11c2f Merge pull request #14899 from liewegas/wip-19433
ceph_test_rados_api_watch_notify: make LibRadosWatchNotify.Watch3Timeout tolerate thrashing

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-01 20:24:48 -05:00
Sage Weil
5376b83b39 Merge pull request #14901 from liewegas/wip-19552
osdc/Objecter: fix pool dne corner case

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-01 20:23:52 -05:00
Kefu Chai
6ce6acf526 Merge pull request #14881 from tchaikov/wip-crosscompiling-armhf
cmake: enable cross-compilation of boost 

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-02 09:19:51 +08:00
Sage Weil
e4874b4091 Merge pull request #14788 from liewegas/wip-jewel-x-rgw
qa/suites/jewel-x/point-to-point: don't scane for keys on second s3tests either

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-01 17:12:39 -05:00
Sage Weil
d60c1c40ce osdc/Objecter: fix pool dne corner case
We have a request that is queued before a pool exists, there is one
epoch where it exists, and then the pool disappears again.  The two maps
are processed at the same time.  For the first we set needs_resend, map to
an osd, and remove from the homeless sessin.  For the second, the pool
dne, we set osd back to -1, and send a map check request.  Finally,
handle_osd_maps scans need_resend, sees the pool dne, and removes from
need_resend.  The difference from the "usual" case is that we are neither
on the need_resend list nor on the homeless session.

Fix this by concluding immediately that the pool existed (briefly) and
then no longer exists.

Fixes: http://tracker.ceph.com/issues/19552
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-01 15:10:44 -05:00
Sage Weil
a28282b157 ceph_test_rados_api_watch_notify: make LibRadosWatchNotify.Watch3Timeout tolerate thrashing
If the PG moves we will reconnect and fail to time out.
Wait for longer so that we mask the effects of osd thrashing.

Fixes: http://tracker.ceph.com/issues/19433
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-01 13:21:30 -04:00
Radoslaw Zarzynski
56baf955ad Merge pull request #14775 from Jing-Scott/wip-con-obj-bulkupload
rgw: add support container and object levels of swift bulkupload

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-05-01 17:44:36 +02:00
Kefu Chai
ab77b1dd6a Merge pull request #14892 from tchaikov/wip-kill-warnings
test: kill warnings

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-01 23:43:51 +08:00
Radoslaw Zarzynski
b4099a9a61 Merge pull request #14750 from theanalyst/rgw/fix/frontend_array
rgw: civetweb don't go past the array index while calling mg_start

Reviewed-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2017-05-01 17:17:45 +02:00
Casey Bodley
cb041c9b16 Merge pull request #14359 from Jing-Scott/remove-zg-dup-post
rgw multisite: remove the redundant post in OPT_ZONEGROUP_MODIFY

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-01 10:54:41 -04:00
Casey Bodley
6ac2dbf89e Merge pull request #14390 from shashalu/drop-unused-param
rgw: drop unused param "bucket" from select_bucket_placement

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-01 10:54:01 -04:00
Casey Bodley
1e0b919e51 Merge pull request #14839 from chardan/jfw-wip-halflife_atomic_t-ebirah
rgw: migrate atomic_t to std::atomic<> (ebirah)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-01 10:53:00 -04:00
Casey Bodley
0e30e3ef01 Merge pull request #14845 from cbodley/wip-rgw-qa-s3tests
qa/rgw: add cluster name to path when s3tests scans rgw log

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2017-05-01 10:49:12 -04:00
Sage Weil
83c45ad6a4 osd: stop MgrClient callbacks on shutdown
Stop the MgrClient callbacks to report PG stats at the
start of shutdown() so that we don't get a callback
during/after we are done shutting down.

Protect the cb update with the MgrClient's lock so that
we don't race with MgrClient::send_report() itself.

Fixes: http://tracker.ceph.com/issues/19638
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-01 10:33:36 -04:00
Kefu Chai
8b8646f7ac test/librados: silence -Woverloaded-virtual warning
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-01 22:19:55 +08:00
Kefu Chai
c83c62ea9b test/librbd: silence warning from -Wsign-compare
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-01 22:13:40 +08:00
Matt Benjamin
1c9a48cec8 Merge pull request #14770 from rzarzynski/wip-rgw-19754
rgw: fix RadosGW hang during multi-chunk upload of AWSv4.
2017-05-01 10:09:10 -04:00
Sage Weil
1e0162fecc Merge pull request #14887 from liewegas/wip-kraken-x
qa/suites/upgrade/kraken-x: misc fixes
2017-05-01 06:55:30 -05:00
Sage Weil
feb532c6d6 Merge pull request #14890 from xiexingguo/wip-fix-bluestore-typo
os/bluestore: fix typo(s/trasnaction/transaction/)
2017-05-01 06:43:53 -05:00
Kefu Chai
e5a8d89d1c cmake: pass compiler and linker down to zstd
so it can respect the setting specfied by user.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-01 15:49:59 +08:00
Kefu Chai
5126f55319 test/librbd: print uint64_t in a platform neutral way
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-01 15:45:42 +08:00
Kefu Chai
c8ca98ed02 crush: pass by const reference to kill compiling error
i have following error when compiling with gcc-6 on armhf

ceph/src/crush/CrushCompiler.cc: In member function 'int
CrushCompiler::decompile(std::ostream&)':
ceph/src/crush/CrushCompiler.cc:462:45: error: invalid initialization of
non-const reference of type 'std::pair<const long unsigned int,
crush_choo
se_arg_map>&' from an rvalue of type 'std::pair<const long unsigned int,
crush_choose_arg_map>'
       int ret = decompile_choose_args(i, out);

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-01 15:45:42 +08:00
Kefu Chai
4e0f7b48b0 cls/rbd: print uint64_t in a platform neutral way
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-01 15:45:42 +08:00
xie xingguo
42a9930aef os/bluestore: fix typo(s/trasnaction/transaction/)
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-05-01 15:32:59 +08:00
Kefu Chai
61a87c2c31 Merge pull request #14883 from tchaikov/wip-mgr-misc
mgr: Misc. bug fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-01 09:15:18 +08:00
Kefu Chai
45d03d8b55 Merge pull request #14878 from tchaikov/wip-19801
msg/simple: use my addr when setting sock priority

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-05-01 00:37:41 +08:00
Kefu Chai
f0aab98567 Merge pull request #14884 from tchaikov/wip-singleton-bluestore-tests
qa/suites/rados/singleton-bluestore: concat settings

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2017-05-01 00:13:38 +08:00
Kefu Chai
86c9137fa0 qa/suites/rados/singleton-bluestore: concat settings
otherwise the bluestore tests will fail with failures like

qa/workunits/cephtool/test.sh:1343: test_mon_osd_pool:  ceph osd pool set ec_test allow_ec_overwrites true
Error EINVAL: pool must only be stored on bluestore for scrubbing to work: osd.1 uses filestore
qa/workunits/cephtool/test.sh:1343: test_mon_osd_pool:  return 1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-30 23:19:07 +08:00
Sage Weil
1a0cfc4dcb Merge pull request #14876 from liewegas/wip-bluestore-alloc
os/bluestore: roundoff bluefs allocs to bluefs_alloc_size

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-30 10:02:44 -05:00
Mykola Golub
8c2631c6f7 Merge pull request #14846 from dillaman/wip-rbd-cli
qa/workunits/rbd: increased trash deferment period

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-04-30 09:59:52 +03:00
John Spray
cd69c0416a mon: tweak mgr status text
Previously at startup we saw contradictory status
that had a "no active mgr" health message and then
a line that said "active: x".

Mitigate that a bit by indicating that a !available
active daemon is starting up.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:30:03 +08:00
John Spray
f79ae2504a mgr: fix message refcounting
Let's be consistent and push the m->put down into
the handle_(M<class>* m) functions the way
we do in the rest of the codebase.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:30:03 +08:00
John Spray
b1e5d492d6 mgr: fix deadlock in dispatch
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:30:03 +08:00
John Spray
e7060410f7 mgr: get fresh maps on second load of Mgr
I think we got away with this when Client::init was
re-initialising the MonClient, but now that (bogus)
stuff isn't happening any more as Client doesn't
init/teardown monc/objecter itself.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:23:14 +08:00
John Spray
b4ed832ad3 vstart: fix mgr caps for MDS daemons
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:21:30 +08:00
John Spray
9988a564d8 mgr: improve handle_command
Run the python module calls in a finisher so that
they don't block the daemonserver lock and so that
they can call back into mgr stuff if they need to.

Fix passing through commands to python modules, this
was giving EINVAL because only things with a MgrCommand
were getting let in.

Also fix get_command_descriptions, which was not
including the output of the formatter in the response.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:21:30 +08:00
John Spray
f68ad16e86 mon: don't enqueue periodic digest if one is already waiting
This was causing mons to send far more digest messages than
should have been sent.  Could have been responsible for
reports of very high CPU consumption on the mgr daemon.

Fixes: http://tracker.ceph.com/issues/18994
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:21:29 +08:00
John Spray
64e36c5f80 common/finisher: use distinct lock names
Two Finishers should not be considered equivalent
for lockdep purposes: for example in mgr we have
our in-mgr Finisher, and then any finishers that
might be in libraries called by modules.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-30 10:21:29 +08:00
Kefu Chai
61e2ff4840 cmake: enable crosscompile of boost
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-30 02:01:33 +08:00
Kefu Chai
2b9c9f0a09 Merge pull request #14837 from wjwithagen/wip-wjw-freebsd-ceph-disk-mpath
ceph-disk: ceph-disk on FreeBSD should not use mpath-code

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-29 20:28:23 +08:00
Willem Jan Withagen
6727618e20 ceph-disk: ceph-disk on FreeBSD should not use mpath-code
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-04-29 12:48:50 +02:00
Kefu Chai
9658ff21f0 msg/async: refactor NetHandler::set_priority() to sync with msg/simple
so it's sync'ed with Pipe::set_socket_options()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-29 13:38:57 +08:00
Kefu Chai
256b5b8069 msg/simple: use my addr when setting sock priority
actually we always fails to set ToS of listening
socket. 6f1037e22c reveals this issue.

Fixes: http://tracker.ceph.com/issues/19801
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-29 13:38:56 +08:00
Kefu Chai
6b34e18e34 Merge pull request #14872 from tchaikov/wip-19770
qa/erasure-code: override min_size to 2

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-29 12:39:23 +08:00
Kefu Chai
7424345c77 qa/erasure-code: override min_size to 2
so isa(k=2,m=1) can survive with 1 down OSD.

Fixes: http://tracker.ceph.com/issues/19770
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-29 10:43:17 +08:00
Kefu Chai
5f50298025 qa/tasks/rados: add optional setting of "min_size"
this setting only affects the newly created pool

Fixes: http://tracker.ceph.com/issues/19770
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-04-29 10:39:02 +08:00
Sage Weil
a9bd266df0 Merge pull request #14758 from xiexingguo/fix-can_mark_out-output
mon/OSDMonitor: fix output func name in can_mark_out

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-04-28 19:40:44 -05:00
Sage Weil
4c41537556 Merge pull request #14847 from smithfarm/wip-19797
tests: rados: move cephtool.yaml to new singleton/bluestore subsuite

Reviewed-by: Sage Weil <sage@redhat.com>
2017-04-28 19:39:12 -05:00