Commit Graph

56879 Commits

Author SHA1 Message Date
Sage Weil
9181c3d0a5 Merge pull request #10717 from liewegas/wip-bluefs-async-compaction
os/bluestore/BlueFS: async compaction

Reviewed-by: Sage Weil <sage@redhat.com>
2016-08-18 13:21:40 -05:00
Sage Weil
a37357873d Merge pull request #10698 from liewegas/wip-bluestore-bluefs-allocation
bluestore: do not use freelist to track bluefs_extents
2016-08-18 11:27:53 -05:00
Sage Weil
8c39bb08c1 Merge pull request #10550 from tanghaodong25/in-memory-db
kv: fix some bugs in memdb

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-08-18 11:14:13 -05:00
Sage Weil
cae06afbf0 Merge pull request #10775 from xiexingguo/xxg-wip-clean-storetest
test/store_test: tidy-up SyntheticWorkloadState class 

Reviewed-by: Sage Weil <sage@redhat.com>
2016-08-18 11:11:24 -05:00
xie xingguo
df949d53bb test/store_test: avoid dynamic allocation of trans
For the sake of performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-08-18 15:50:10 +08:00
xie xingguo
e080e35c9f test/store_test: drop redundant transaction of callbacks
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-08-18 15:50:10 +08:00
xie xingguo
dd57e9ee18 test/store_test: fix incorrect prompt given by stash callback
"clone finish" belongs to C_SyntheticOnClone instead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-08-18 15:50:10 +08:00
haodong
f5a320c218 kv: fix iterator bug in prev
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2016-08-18 15:17:41 +08:00
haodong
7e72a090c5 kv: fill 'm_key_value' after 'seek_to_first' or 'seek_to_last'
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2016-08-18 15:17:41 +08:00
haodong
00289b2a00 kv: fix iterator invalidation in memdb.
Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2016-08-18 15:17:40 +08:00
haodong
cbca389d82 kv: delete store after pg destructor is called in OSD shutdown.
Using memdb for bluestore kvbackend, we will hit segfault when we use
'kill' command to shut down osd process. After destructing pg, some
reference to bluestore will be release, but bluestore has been deleted
at this time.

Signed-off-by: Haodong Tang <haodong.tang@intel.com>
2016-08-18 15:17:40 +08:00
Ilya Dryomov
0c9ada8ec6 Merge pull request #10481 from dillaman/wip-16171
librbd: request exclusive lock if current owner cannot execute op

Reviewed-off-by: Ilya Dryomov <idryomov@gmail.com>
2016-08-17 21:03:45 +02:00
Casey Bodley
c63a467d67 Merge pull request #10737 from weiqiaomiao/wqm-wip-rgw-core
rgw: fix radosgw daemon core when reopen logs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-08-17 14:39:47 -04:00
Sage Weil
2634bcf853 Merge pull request #10728 from xiexingguo/xxg-wip-kill-bs-warn
bluestore: initialize csum_order properly

Reviewed-by: Sage Weil <sage@redhat.com>
2016-08-17 12:13:50 -04:00
Mykola Golub
f7ae584db5 Merge pull request #10703 from dillaman/wip-16855
rbd-mirror: improve split-brain detection logic

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-08-17 18:16:17 +03:00
Orit Wasserman
8f535baace Merge pull request #10756 from linuxbox2/fix_gc_interval
rgw_file: restore local definition of RGWLibFS gc interval
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-08-17 17:03:44 +02:00
Sage Weil
2611cdca8a Merge pull request #10686 from liewegas/wip-bluefs-remove
os/bluestore/BlueFS: do not op_file_update deleted files

Tested-by: Somnath Roy <somnath.roy@sandisk.com>
2016-08-17 10:33:05 -04:00
Matt Benjamin
31936caa09 rgw_file: restore local definition of RGWLibFS gc interval
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-08-17 10:28:48 -04:00
Kefu Chai
810f660a7b Merge pull request #10755 from SUSE/wip-cmake-use-pie
cmake: Add -pie to CMAKE_EXE_LINKER_FLAGS

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-08-17 21:46:19 +08:00
Kefu Chai
aaacd3ef90 Merge pull request #10754 from SUSE/wip-cmake-fix-rgw-soversion
cmake: Fix mismatched librgw VERSION / SOVERSION

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-08-17 21:42:43 +08:00
Tim Serong
9e705be686 cmake: Add -pie to CMAKE_EXE_LINKER_FLAGS
Without this, rpmlint (on openSUSE Tumbleweed) fails with:

ceph-radosgw.x86_64: E: non-position-independent-executable
(Badness: 10000) /usr/bin/radosgw
This executable must be position independent.  Check that it
is built with -fPIE/-fpie in compiler flags and -pie in linker
flags.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-08-17 22:27:19 +10:00
Tim Serong
c8885d4a9e cmake: Fix mismatched librgw VERSION / SOVERSION
Without this, rpmlint (on openSUSE Tumbleweed) fails with:

librgw2.x86_64: E: shlib-policy-name-error (Badness: 10000) librgw1
Your package contains a single shared library but is not named
after its SONAME.

It seems that the VERSION/SOVERSION mismatch results in the
creation of librgw.so.1 and librgw.so.2.0.0, whereas it should
be librgw.so.2 and librgw.so.2.0.0.

Signed-off-by: Tim Serong <tserong@suse.com>
2016-08-17 22:21:59 +10:00
Orit Wasserman
80d914fc1d Merge pull request #10749 from mikulely/fix-rgw-client-cb
rgw: fix flush_read_list() error msg
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-08-17 09:12:20 +02:00
Brad Hubbard
f8fcef7670 Merge pull request #10747 from oliveiradan/master
gcc 6.1.1 complains about missing include: <random>. 4.8.3 does not c…
2016-08-17 16:53:58 +10:00
Jiaying Ren
befac18a3f rgw: fix flush_read_list() error msg
Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>
2016-08-17 14:19:48 +08:00
oliveiradan
f1ad760366 test: gcc 6.1.1 complains about missing include: <random>. 4.8.3 does not complain about it
Signed-off-by: Daniel Oliveira <doliveira@suse.com>
2016-08-16 23:22:49 -06:00
Kefu Chai
e354918ec8 Merge pull request #10264 from yuyuyu101/wip-async-abstract-backend
msg/async: allow other async backend implementations

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-08-17 02:18:39 +08:00
Haomai Wang
f6c3fd9994 test_msgr: ensure wait for ms_dispatch executed
Otherwise in slow runner, ms_dispatch may not executed when assert

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
9bfd11e7f9 msg/async/Event: remove global_centers assert
In tests we allow to reset EventCenter instance in the same CephContext,
so it may let global_centers->centers to set the same position multi times.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
6d3e495de2 test_async_networkstack: add networkstack tests
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
3a43a0bb34 msg/async/AsyncConnection: fix _conn_prefix racing when stopped
When the connection is lossy and enter fault, it will dispatch reset event.
If cleanup handler is executed as well as ms_handle_reset call mark_down,
it may exists racing for "cs". cleanup handler will reset "cs" but
_conn_prefix in mark_down will access "cs".

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
1d4ff2a30a msg/async: readd set_priority call
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
154376b211 msg/async: allow EventCenter::set_owner reentry
If daemonlize we need to respawn event threads, it need to allow set_owner again

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
8d63e2c709 msg/async/Stack: register fork watcher
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
524553ff31 common/ceph_context: add prefork/postfork wathcher support
Because daemon() will termniate all existing threads, it will make something go
wrong.

So we want to add hook at CephContext, do something before/after fork.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:25 +08:00
Haomai Wang
31833e8b48 msg/async/Stack: disable smart thread spawn now
New async msgr runtime need to spawn threads when binding, but ceph-osd will
call daemon() after binding port. So we need to respawn threads if forked.

Then thread spawn delay will increase complexity for this change and it's
really a simple strategy which help less, we disable auto spawn now.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:19:23 +08:00
Haomai Wang
4e57181b54 msg/async/PosixStack: increase log level
AsyncMessenger will try to loop the bind port range, so it will produce
some addr inuse errors which is not abnormal.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:17:55 +08:00
Haomai Wang
f62f28ceea msg/async/Event: remove event wakeup flag
Now only dispatch external event will wakeup event thread(previously
delete_time_event will call wakeup), we only need to use
"external_num_events" to indicate whether we have extra events.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:17:55 +08:00
Haomai Wang
fca0de1f07 msg/async/Event: debug event address when dispatching and executing
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:17:55 +08:00
Haomai Wang
d0cd88b3ee msg/async/AsyncConnection: support NetworkStack api instead of posix
1. replace sd to ConnectedSocket
2. Replace WorkerPool with Stack
3. Use Stack worker

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:17:55 +08:00
Haomai Wang
57629c6700 msg/async/net_handler: let set_socket_options return value
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:17:55 +08:00
Haomai Wang
31bbe39ac2 msg/async: add PosixStack support
Add default posix backend support, it should be the full replacement for
the original AsyncMessenger IO logics.

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:17:55 +08:00
Haomai Wang
0ba42237e2 msg/async/Stack: add abstract Stack
Stack is a network IO framework which encapsulates all necessary basic network
interface, then it manages threads to work.

Different network backend like posix, dpdk even RDMA need to inherit Stack
class to implement necessary interfaces. So it will make ease for other
network backend to integrated into ceph. Otherwise, each backend need to
implement the whole Messenger logics like reconnect, policy handle, session
maintain...

Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-08-16 23:16:07 +08:00
Jason Dillaman
3545d9ed19 test: fixed unused function warnings in unittest_rbd_mirror
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:11:07 -04:00
Jason Dillaman
beaef377d6 rbd-mirror: prevent syncing to non-primary image after failover
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:11:07 -04:00
Jason Dillaman
a6901ca1a0 rbd-mirror: demote/promote in same cluster results in split-brain
Fixes: http://tracker.ceph.com/issues/16855
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:11:07 -04:00
Jason Dillaman
ac590e85a4 librbd: normalize journal promotion/demotion events
A non-primary image's commit possition won't accurately reflect
the current demotion/promotion chain. Therefore, directly specify
the predecessor for promotion events.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:11:07 -04:00
Jason Dillaman
8b195e1fc8 librbd: block RPC requests when demoting the image
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:11:07 -04:00
Jason Dillaman
718befdae7 librbd: separate journal::TagData predecessor data into new struct
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:11:06 -04:00
Jason Dillaman
47d1e62500 rbd-mirror: include tag tid in bootstrap debug log messages
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-08-16 09:09:52 -04:00