Commit Graph

67667 Commits

Author SHA1 Message Date
Kefu Chai
7b8eca5be7 Merge pull request #12091 from wjwithagen/wip-wjw-nosetests
nosetests: use /usr/bin/env to find nosetests

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-11 00:43:24 +08:00
Adam C. Emerson
ea52d39be2 denc: Support boost::optional
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-01-10 11:40:34 -05:00
Adam C. Emerson
cf044f7aef denc: Support tuples
We could support zero-length tuples, but contiguous appender doesn't
like zero-length writes, so I've ruled them out for now.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-01-10 11:40:34 -05:00
Adam C. Emerson
26d622c48f denc: Add support for std::array
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-01-10 11:40:34 -05:00
Adam C. Emerson
c92cc9fe51 denc: Support boost::container::flat_set
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-01-10 11:40:34 -05:00
Adam C. Emerson
b7df89b519 denc: Scrap the container boilerplate!
Using template templates, kill off the reduplication in
container encoding.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-01-10 11:40:34 -05:00
Willem Jan Withagen
89a61ae2cd nosetests: start using /usr/bin/env to find noestests
- Option nocapture is not really required so no problem
   with the fact that FreeBSD env does not work for params
   for nosetests

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2017-01-10 16:07:40 +01:00
Gaurav Kumar Garg
6ab73e5f42 rbd: bench-write should return error if io-size >= 4G
Currently if user perform bench-write with io-size > 4G
then its crashing because currently during memory allocation
bufferptr taking size of buffer as a unsigned and io-size > 4G
will overflow with unsigned. so during memset operation it will
try to set io_size size of memory area pointed by bufferptr,
(bufferptr area is:  (4G - io_size)), so it will cause
segmentation fault.

Fix is to return error if io-size >= 4G

Fixes: http://tracker.ceph.com/issues/18422

Reported-by:  Jason Dillaman <dillaman@redhat.com>
Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>
2017-01-10 16:02:01 +01:00
Jason Dillaman
60b79473fe Merge pull request #12821 from SUSE/wip-drop-rbd_cli_tests
tests: drop rbd_cli_tests.pl and RbdLib.pm

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-01-10 09:29:11 -05:00
Jason Dillaman
5f2689b62a librbd: resolve static analyser warnings
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-01-10 08:34:38 -05:00
Sage Weil
c96cb2d02e Merge pull request #12850 from xiexingguo/xxg-wip-bluestore-20170110
os/bluestore: shrink buffer_map key into uint32_t

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-01-10 07:33:18 -06:00
Jason Dillaman
0dd25f240a Merge pull request #12815 from rjfd/wip-journal-unittest-build-error
tests: fix template specialization of PromoteRequest class

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-01-10 08:32:53 -05:00
Sage Weil
a7da1eb092 Merge pull request #12853 from xiexingguo/xxg-wip-kill-orphan
os/bluestore: kill orphan declaration of do_write_check_depth()

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-01-10 07:31:43 -06:00
Sage Weil
533311e999 Merge pull request #12856 from xiexingguo/xxg-wip-clean-kd
os/bluestore/KernelDevice: kill zeros

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
2017-01-10 07:27:57 -06:00
Sage Weil
f70b6fdec6 Merge pull request #12858 from yangdongsheng/rbd_cleanup
rbd: cleanup: fix the typo in namespace comment
2017-01-10 07:26:12 -06:00
optimistyzy
5ffee830f6 NVMEdevice: fix the unrelease segs issue
When the read or flush command fails, we need to
release segs. Or we can abort it like what we do
for write.

Signed-off-by: optimistyzy <optimistyzy@gmail.com>
2017-01-10 21:14:18 +08:00
Kefu Chai
7ac89f3725 cmake: s/snappy/${SNAPPY_LIBRARIES}/
also s/z/${ZLIB_LIBARIES}/

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-01-10 21:10:53 +08:00
Kefu Chai
8f7643792c cmake: turn libcommon into a shared library
prior to this change, libcommon is a convenient library which gets
linked into librados, librbd and libcephfs and all ceph executables.
this incurs some problems:
 - double dose of libcommon in memory space and HDD: waste of memory
   and disk space.
 - if an application links to two libraries including libcommon at the
   same time. take librados and libcephfs as an example, they could
   interfere with each other by changing the other guy's status.
after this change, libcommon is tuned into a shared library and
renamed to libceph-common. it will be installed into $prefix/lib/ceph,
and packaged in librados2.

ceph.spec.in,debian/librados2.install: package libceph-common in
  librados2.
CMakeLists.txt:
  - do not link against libboost-* if not necessary.
  - s/common/ceph-common/g
  - install libceph-common into $prefix/lib/ceph
  - set rpath to $prefix/lib/ceph
  - link against ceph-common if an executable needs access to non public
    symbols in ceph.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-01-10 21:10:53 +08:00
xie xingguo
842098d480 os/bluestore: shrink buffer_map key into uint32_t
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 20:45:02 +08:00
Pan Liu
7630df21be async: fixed the error "Cause: Cannot create lock on '/var/run/.rte_config'.",
when invoke more than one osds in same server and enable dpdk.

Signed-off-by: Pan Liu <pan.liu@istuary.com>
2017-01-10 18:49:20 +08:00
Yan, Zheng
525c52fd49 client: fix Client::handle_cap_flushsnap_ack() crash
Struct CapSnap holds a reference to its parent inode. So erasing
struct CapSnap from Inode::cap_snaps may drop inode's last reference.
The inode gets freed in the middle of erasing struct CapSnap

Fixes: http://tracker.ceph.com/issues/18460
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 17:41:50 +08:00
Dongsheng Yang
50e088deeb rbd: cleanup: fix the typo in namespace comment
Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2017-01-10 04:26:43 -05:00
Brad Hubbard
fbad77c727 Merge pull request #12855 from badone/wip-md_config_t-_apply_changes-unchangeable
config: Improve warning for unobserved value

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-10 18:17:52 +10:00
xie xingguo
12366280cc os/bluestore/KernelDevice: reuse r
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 15:53:25 +08:00
xie xingguo
c8123e50c1 os/bluestore/KernelDevice: drop redundant assignment statements
The memset() call has already done it.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 15:50:23 +08:00
xie xingguo
e353d8e0b1 os/bluestore/NVMeDevice: kill zeros
See 2cd63d2209
The aio_zero() interface is deprecated.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 15:46:32 +08:00
xie xingguo
b9c1c0fe7c os/bluestore/KernelDevice: kill zeros
This is not needed anymore and can save us 1MB memory.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 15:39:37 +08:00
Yan, Zheng
9b97554f0a mds: don't abort request that has already started slave operation
Access check and fragment size check can fail after start slave
operation. This causes mds to crash.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:35 +08:00
Yan, Zheng
5b832aa5b6 mds: use locked dentry trace to compose slave rmdir/rename request
{rmdir/rename}_prepare_witness() use full path to compose slave
requests. But they do not lock all dentries in the path. So someone
else changes the unlocked dentry and causes path travsese of slave
request to fail.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
da1e2614ab mds: add replicas of destdn to rename witness list
it's possible that null dentry is replicated

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
276d12547f mds: properly put unexpired dentry back to lru
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
c94a2c429e mds: cleanup Server::rdlock_path_xlock_dentry
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
530bd6e67a mds: add -ESTALE recovery code for unlink/rmdir
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
dbc579778f mds: limit number of pending exports when thash exports is enabled
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
6ae33aace1 mds: make MDCache::show_subtree less verbose when thrash exports is enabled
Subtree map can be very large when thrash exports is enabled.
Printing subtree map causes size log to increase quickly.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:34 +08:00
Yan, Zheng
a4e5ad2ff4 mds: don't journal inode when merging subtrees
When doing subtree migration or cross subtree rename, we rdlock
fragtreelock of subtree boundaries' inodes. rdlock fragtreelock
has already journaled dirfragtree updates. I can't see why we
need to journal it again when merging subtrees.

The resaon I want to remove this code is that it can journal
unconnected dirfrag in the Migrator::export_reverse() case.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:23:30 +08:00
Yan, Zheng
2b7dd50667 mds: fix MDSMap::is_rejoining()
the function checks if all recovering mds reach rejoin state. It
need to consider damaged mds set.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:19:23 +08:00
Yan, Zheng
3eb1b7bc04 mds: remove stale warning in log replay code
we don't re-fill prealloc_inos each time a prealloc ino get used

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:19:23 +08:00
Yan, Zheng
60b22247c6 mds: don't modify session->info before ESession get journaled
Server::journal_close_session() modifies session->info before
corresponding ESession get journaled. It's possible the updated
session gets saved to sessionmap before Server::_session_logged()
increases sessionmap's version. This confuses the journal replay
code.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 15:19:23 +08:00
Brad Hubbard
1e8e754007 config: Improve warning for unobserved value
When a variable is not being observed we currently mark it
"unchangable". This can be misleading so try something hopefully a
little more informative.

Fixes: http://tracker.ceph.com/issues/18424
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-01-10 15:35:19 +10:00
Haomai Wang
0bf9c6242e Merge pull request #12854 from liupan1111/wip-fix-async-dpdk-issue
async: fixed coredump when enable dpdk.

Reviewed-by: Haomai Wang <haomai@xsky.com>
2017-01-10 13:09:40 +08:00
Pan Liu
96046f53e9 async: fixed coredump when enable dpdk.
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2017-01-10 11:58:23 +08:00
xie xingguo
01ff6bf51a os/bluestore: kill orphan declaration of do_write_check_depth()
The gc logic is gone and this is not relevant anymore.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 11:47:36 +08:00
Yan, Zheng
80dae314ee mds: finish clientreplay requests before requesting active state
All clientreplay requests' finish contexts should be executed
before MDCache::export_remaining_imported_caps(). Otherwise
MDCache::try_reconnect_cap() may fail to reconnect client caps.

Fixes: http://tracker.ceph.com/issues/18461
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2017-01-10 11:37:14 +08:00
Mingxin Liu
57274488c0 mon: do not send duplicated osdmap msg to not sync'ed osd
prior to this change:
a peon may forward the pgstats to leader, and record it locally, but leader will
check if osd has the latest map before process, if not, will use a route op to
indicate peon to send it, then poen will delete routed op when fininaly send
out which make peon cannot send pgstatack when leader has processed the
pgstat update. so osd will always track it util reach a threshold block pgstats
sending, at worst, reopen mon session.
also, both leader and peon will send out the osdmap message to the osd.

after this change:
only the peon will send out the osdmap message. and the pgstatack message
will be routed to the osd as expected. so the osd will not keep track of the
"acked" pg stats in its queue forever before times out.

Fixes: http://tracker.ceph.com/issues/18458
Signed-off-by: Mingxin Liu <mingxin@xsky.com>
2017-01-10 11:14:02 +08:00
Samuel Just
dd48b972af PrimaryLogPG::failed_push: update missing as well
This way, pick_newest_available should do the right thing.

Fixes: http://tracker.ceph.com/issues/18165
Signed-off-by: Samuel Just <sjust@redhat.com>
2017-01-09 18:48:31 -08:00
Jason Dillaman
59c0108241 Merge pull request #12849 from xiexingguo/xxg-wip-fix-compile-crash
librados: fix compile errors from simplified aio completions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-01-09 21:45:41 -05:00
Mingxin Liu
ecb967a8ac mon: we may track a self-initiated message, recv_stamp remain unset
so using ceph_clock_now() instead to make optracker happy

Signed-off-by: Mingxin Liu <mingxin@xsky.com>
2017-01-10 10:39:07 +08:00
Kefu Chai
327791feac Merge pull request #9032 from tchaikov/wip-15786
doc: document repair/scrub features

Reviewed-by: David Zafman <dzafman@redhat.com>
2017-01-10 10:23:39 +08:00
xie xingguo
8b64515d1c librados: fix building crash
See https://github.com/ceph/ceph/pull/12607

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-01-10 10:01:54 +08:00