RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2024-12-17 08:57:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
55,575 Commits 211 Branches 394 Tags 2 GiB
b7ba134782
Commit Graph

55575 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yan Jun
c41ff0eb8c rgw: fix the missing return value 
If `get_bucket_index_object` returns an error code (-ENOTSUP)
with ivalid `hash_type`, we should return this negative error code
that resulting in `BucketShard::init` failed as expected,
oherwise it will go wrong.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
 2016-06-30 10:49:41 +08:00
Yan Jun
9d92d0d30b rgw: remove uneccessary temporary variable 
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
 2016-06-30 10:29:01 +08:00
xie xingguo
9cc550e03e os/bluestore: make trim() of 2Q cache more fine-grained 
1. count total bytes as per buffer types respectively,
   so we know the exact size of each kind of buffer.

2. evict by list actual size against expected size,
   which is more accurate.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-06-30 08:34:21 +08:00
Yehuda Sadeh
492bb60873 rgw: finish error_repo cr in stop_spawned_services() 
Fixes: http://tracker.ceph.com/issues/16530

Need to call finish, otherwise drain_all() wouldn't be able to
drain it.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
 2016-06-29 14:50:12 -07:00
Matt Benjamin
13c13c7106 Merge pull request #9993 from weiqiaomiao/wqm-wip-clean-bufferlist 
rgw: remove unused bufferlist

(build verified, provably unused/not aliased)
 2016-06-29 16:52:14 -04:00
Ali Maredia
3e0aee8b20 Merge pull request #10025 from tchaikov/wip-cmake 
cmake: install ceph_test_cls_rgw

Reviewed-by: Ali Maredia <amaredia@redhat.com>
 2016-06-29 16:34:57 -04:00
John Coyle
b3b3185008 rgw-ldap: add ldap lib to rgw lib deps based on build config 
Fixes autotools and Cmake linking on Alpine.

Signed-off-by: John Coyle <dx9err@gmail.com>
 2016-06-29 16:34:43 -04:00
Willem Jan Withagen
1b5b34e791 net_handler.cc: make it more compatible with BSDs 
According the Linux manual page:
POSIX.1-2001 does not require the inclusion of <sys/types.h>, and this header file is not required on Linux.  However, some historical (BSD) implementations required this header file, and portable applications are probably wise to include it.

Submitted-by: Willem Jan Withagen <wjw@digiware.nl>
 2016-06-29 22:34:38 +02:00
Patrick Donnelly
208e3b6fd5
client: add missing client_lock for get_root 
A segmentation fault was observed in a recent test failure
(test_client_cache_size) [1]. An analysis of the core dump revealed that the
client->root inode was NULL. The client log [2] showed that the root inode was
deleted prior to the segfault:

    2016-06-27 14:24:13.358500 7fe75cff9700 20 client.4161 _ll_put 0x7fe76000ba40 100000003e9 1 -> 0
    2016-06-27 14:24:13.358502 7fe75cff9700 10 client.4161 put_inode on 100000003e9.head(faked_ino=0 ref=1 ll_ref=0 cap_refs={} open={} mode=41777 size=0/0 mtime=2016-06-27 14:24:10.700562 caps=pAsLsXsFs(0=pAsLsXsFs) 0x7fe76000ba40)
    2016-06-27 14:24:13.358513 7fe75cff9700 15 inode.put on 0x7fe76000ba40 100000003e9.head now 0
    2016-06-27 14:24:13.358515 7fe75cff9700 10 client.4161 remove_cap mds.0 on 100000003e9.head(faked_ino=0 ref=0 ll_ref=0 cap_refs={} open={} mode=41777 size=0/0 mtime=2016-06-27 14:24:10.700562 caps=pAsLsXsFs(0=pAsLsXsFs) 0x7fe76000ba40)
    2016-06-27 14:24:13.358538 7fe75cff9700 15 client.4161 remove_cap last one, closing snaprealm 0x7fe76000c1b0
    2016-06-27 14:24:13.358544 7fe75cff9700 20 client.4161 put_snap_realm 1 0x7fe76000c1b0 1002 -> 1001
    2016-06-27 14:24:13.358546 7fe75cff9700 10 client.4161 put_inode deleting 100000003e9.head(faked_ino=0 ref=0 ll_ref=0 cap_refs={} open={} mode=41777 size=0/0 mtime=2016-06-27 14:24:10.700562 caps=- 0x7fe76000ba40)

After looking through the reference count inc/dec, I noticed this mismatched inc/dec:

    2016-06-27 14:24:13.352622 7fe75dffb700  3 client.4161 ll_forget 100000003e9 1
    2016-06-27 14:24:13.352601 7fe7767fc700  1 -- 172.21.15.58:0/3762258427 >> 172.21.15.57:6804/11121 conn(0x7fe789bf3b60 sd=20 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7 cs=1 l=0). == rx == mds.0 seq 1473 0x7fe764037a80 client_reply(???:470 = 0 (0) Success safe) v1
    2016-06-27 14:24:13.352665 7fe75dffb700 20 client.4161 _ll_put 0x7fe76000ba40 100000003e9 1 -> 1
    2016-06-27 14:24:13.352687 7fe75d7fa700  3 client.4161 ll_lookup 0x7fe76000ba40 testdir
    2016-06-27 14:24:13.352699 7fe75d7fa700 20 client.4161 _lookup have dn testdir mds.-1 ttl 0.000000 seq 0
    2016-06-27 14:24:13.352705 7fe75d7fa700 15 inode.get on 0x7fe760011da0 100000003ea.head now 537
    2016-06-27 14:24:13.352693 7fe76effd700  1 -- 172.21.15.58:0/3762258427 <== mds.0 172.21.15.57:6804/11121 1473 ==== client_reply(???:470 = 0 (0) Success safe) v1 ==== 27+0+0 (3458149698 0 0) 0x7fe764037a80 con 0x7fe789bf3b60
    2016-06-27 14:24:13.352708 7fe75d7fa700 10 client.4161 _lookup 100000003e9.head(faked_ino=0 ref=3 ll_ref=1 cap_refs={} open={} mode=41777 size=0/0 mtime=2016-06-27 14:24:10.700562 caps=pAsLsXsFs(0=pAsLsXsFs) 0x7fe76000ba40) testdir = 100000003ea.head(faked_ino=0 ref=537 ll_ref=999 cap_refs={} open={} mode=40755 size=0/0 mtime=2016-06-27 14:24:13.325297 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE parents=0x7fe750001f50 0x7fe760011da0)

This sequence of ll_forget/ll_lookup is repeated in this test several hundred times. This is the prior sequence (in one thread):

    2016-06-27 14:24:13.324896 7fe75dffb700  3 client.4161 ll_forget 100000003e9 1
    2016-06-27 14:24:13.324904 7fe75dffb700 20 client.4161 _ll_put 0x7fe76000ba40 100000003e9 1 -> 1
    2016-06-27 14:24:13.324915 7fe75dffb700  3 client.4161 ll_lookup 0x7fe76000ba40 testdir
    2016-06-27 14:24:13.324921 7fe75dffb700 20 client.4161 _lookup have dn testdir mds.-1 ttl 0.000000 seq 0
    2016-06-27 14:24:13.324931 7fe75dffb700 15 inode.get on 0x7fe760011da0 100000003ea.head now 621
    2016-06-27 14:24:13.324904 7fe7767fc700  1 -- 172.21.15.58:0/3762258427 >> 172.21.15.57:6804/11121 conn(0x7fe789bf3b60 sd=20 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=7 cs=1 l=0). == rx == mds.0 seq 1387 0x7fe764026720 client_reply(???:385 = 0 (0) Success safe) v1
    2016-06-27 14:24:13.324934 7fe75dffb700 10 client.4161 _lookup 100000003e9.head(faked_ino=0 ref=3 ll_ref=2 cap_refs={} open={} mode=41777 size=0/0 mtime=2016-06-27 14:24:10.700562 caps=pAsLsXsFs(0=pAsLsXsFs) 0x7fe76000ba40) testdir = 100000003ea.head(faked_ino=0 ref=621 ll_ref=998 cap_refs={} open={} mode=40755 size=0/0 mtime=2016-06-27 14:24:13.321021 caps=pAsLsXsFsx(0=pAsLsXsFsx) COMPLETE parents=0x7fe750001f50 0x7fe760011da0)

The key line is `_lookup 100000003e9.head(...)`. ll_ref=2 is the expected count
after this sequence but, as we see at 14:24:13.352708, ll_ref=1.

This racing behavior is not serialized by client_lock because
Client::get_root() does not lock the client_lock. Additionally, this race is
not easy to identify in the logs because Client::get_root() does not use
Client::_ll_get to increment the reference, which hides the increments in the
logs. Instead it directly increments the reference using Inode:ll_get().

This commit adds the client_lock for Client::get_root() and
Client::get_root_ino() (which may not strictly be necessary but can't hurt).

[1] http://pulpito.ceph.com/pdonnell-2016-06-27_06:54:23-fs-wip-pdonnell-i16164---basic-smithi/280361/
[2] http://qa-proxy.ceph.com/teuthology/pdonnell-2016-06-27_06:54:23-fs-wip-pdonnell-i16164---basic-smithi/280361/remote/smithi058/log/ceph-client.0.23448.log.gz

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2016-06-29 16:03:57 -04:00
Matt Benjamin
f7303ce61a Merge pull request #9745 from weiqiaomiao/wqm-wip-rgw-usage 
rgw:add missing "--staging" to radosgw-admin usage

lgtm (build verified f23)
 2016-06-29 15:58:24 -04:00
Michal Jarzabek
d288cdb78f msg/AsyncMessenger.cc: remove code duplication 
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>
 2016-06-29 20:57:38 +01:00
Kefu Chai
45c05d4205 debian: package etc/init.d/ceph in ceph-base 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-30 02:20:28 +08:00
Gregory Farnum
be55cfc331 Merge pull request #9606 from gregsfortytwo/wip-16002 
ObjectCacher: fix bh_read_finish offset logic

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2016-06-29 10:59:14 -07:00
Kefu Chai
c8ac2e5c8e cmake: install ceph_test_cls_rgw 
it is used by cls/test_cls_rgw.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-30 01:53:32 +08:00
Matt Benjamin
4eb7caf9d6 Merge pull request #9754 from dx9/wip-fix-rgw-include 
rgw/rgw_http_client: add compat.h include for TEMP_FAILURE_RETRY

lgtm (build verified, f23)
 2016-06-29 13:43:24 -04:00
Gregory Farnum
a69a264f39 Merge pull request #9994 from xiexingguo/xxg-wip-kill-client-warn 
client: kill compiling warning

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
 2016-06-29 10:41:13 -07:00
Kefu Chai
dc1721baf9 Merge pull request #9831 from linuxbox2/wip-ssl3 
cmake: rework NSS and SSL

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2016-06-30 01:34:09 +08:00
Matt Benjamin
b9190ed961 Merge pull request #9807 from Yan-waller/yj-wip-rgwtoolsadderrnostr 
rgw: ext_mime_map_init add string describing for error number

lgtm  (build verified)
 2016-06-29 13:29:17 -04:00
Matt Benjamin
1aff646335 automake, ssl: real openssl detection, fix ssl linkage w/NSS 
This change mainly intends to prevent linking with libssl when the
crypto provider is NSS, which provides an SSL implementation (the
implementation we must use and prefer when NSS is selected).

Secondarily, actually detect openssl when it is selected, which
happens by default and co-selects with cryptopp.  To do this, we
import the ax_check_openssl.m4 from the automake archive.

Since upstream builds now prefer NSS crypto, make this the default.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
 2016-06-29 12:37:02 -04:00
Matt Benjamin
83e66809e9 cmake: rework NSS and SSL 
The cmake build should select NSS or CryptoPP consistently with
current autoconf logic, as used in our RHEL builds (-DWITH_NSS),
and the SSL linkage must match (for NSS, -lssl3).

With this change, there is no direct linkage with libssl when NSS
is selected (I still see indirect linkage via libcurl, in turn
via libssh2, looking into that).

With guidance from upstream and downstream, NSS builds are now
selected by default.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
 2016-06-29 12:36:46 -04:00
Matt Benjamin
ee7949f1e5 Merge pull request #9851 from cbodley/wip-rgw-tombstone-pgver 
rgw: add pg_ver to tombstone_cache
 2016-06-29 11:50:50 -04:00
Kefu Chai
266d17d05d Merge pull request #9995 from tchaikov/wip-cmake 
cmake changes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2016-06-29 22:52:48 +08:00
Kefu Chai
87a6147522 cmake: build install libcls_numops 
test_cls_numops uses it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 22:50:30 +08:00
Kefu Chai
79f0b30d4c cmake: install ceph_test_filestore_idempotent_sequence 
run_seed_to.sh in task "filestore_idempotent" uses it

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 22:50:30 +08:00
Kefu Chai
11e426bf8b cmake: use full path for substitution and definitions 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 22:50:30 +08:00
Kefu Chai
6b12a395eb cmake: use CMAKE_POSITION_INDEPENDENT_CODE property instead of -fPIC 
use the global setting of CMAKE_POSITION_INDEPENDENT_CODE instead of
hardwiring the CMAKE_C_FLAGS with -fPIC. this helps to reduce the
overhead of symbol resolution when user is using static libraries.
see
https://cmake.org/cmake/help/v2.8.11/cmake.html#variable:CMAKE_POSITION_INDEPENDENT_CODE

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 22:26:25 +08:00
Kefu Chai
c8fe1782a6 cmake: move librados.cc out of libradosstriper 
NObjectIterator::__EndObjectIterator is a static member variable
whose destructor deletes a pointer. librados.cc is the source file that
defines __EndObjectIterator. before this change rados cli is linked
against libradosstriper and librados. both of them include librados.cc.
that's why the dtor is called twice, hence double free when the dso
is closed and the static variables are destructed. in this change,
the librados.cc is moved out of libradosstriper, RadosClient.cc,
RadosXattrIter.cc and IoCtxImpl.cc kept as the OBJECT library, as
libradostriper is accessing the non-public symbols defined by them.

Fixes: http://tracker.ceph.com/issues/16504
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 22:26:25 +08:00
Dan Mick
bf5ccce9ad run-make-check.sh: modprobe won't work on containers; not needed 
Signed-off-by: Dan Mick <dan.mick@redhat.com>
 2016-06-29 22:26:24 +08:00
Sage Weil
97a9a7eb84 debian/rules: specify sysconf dir explicitly 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-06-29 22:26:24 +08:00
Sage Weil
74ec4f94b1 CMakeLists: nss only, ignore cryptopp 
Signed-off-by: Sage Weil <sage@redhat.com>
 2016-06-29 22:26:24 +08:00
Kefu Chai
a6e9379231 debian/rules: build deb with cmake 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 22:26:24 +08:00
Sage Weil
3ad5b63211 Merge branch 'kraken' 2016-06-29 10:01:01 -04:00
Igor Fedotov
6ad65ce749 os/bluestore: no need to use SIGNED varint encoding for bluestore_extent_ref_map_t offsets 
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
 2016-06-29 16:52:21 +03:00
Igor Fedotov
9ef149435e os/bluestore: apply small encoding for bluestore_onode_t:extents 
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
 2016-06-29 16:41:58 +03:00
Kefu Chai
457ddea9c8 debian/rules: package in a more debhelper way 
this helps to keep the build rule simpler, and easier to customize

* -j$(NUMJOBS) is taken care of by 'dh --parallel'
* use 'autoreconf' dh add-on to autoconf cleanup
* add dh-autoreconf to Build-Depends
* bump debhelper compatibility level to 8, as jewel's supported
  debian based distro is jessie (with debhelper 9.20150101),
  ubuntu trusty (with debhelper 9.20131227ubuntu1)

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 16:12:24 +08:00
Yan Jun
34a29866a7 rgw: remove uneccessary temporary variable 'r' 
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
 2016-06-29 16:08:35 +08:00
weiqiaomiao
2e2b9f36bb rgw: add "radosgw-admin user list" cli 
add "radosgw-admin user list" command  in order to more intuitive list users than "radosgw-admin metadata list user" command

Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>
 2016-06-29 16:07:31 +08:00
xie xingguo
034917055e client: kill compiling warning 
The 'done' lable is not appliable any more.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-06-29 15:41:16 +08:00
weiqiaomiao
5de363ace3 rgw: remove unused bufferlist 
Signed-off-by: weiqiaomiao <wei.qiaomiao@zte.com.cn>
 2016-06-29 14:48:45 +08:00
Kefu Chai
3d5fa5e470 Merge pull request #9992 from yuyuyu101/wip-fix-rebase 
msg/async: add "queue_reset" argument

this fixes the broken build

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2016-06-29 14:21:03 +08:00
Haomai Wang
464c07709c msg/async: add "queue_reset" argument 
Signed-off-by: Haomai Wang <haomai@xsky.com>
 2016-06-29 13:56:39 +08:00
Haomai Wang
060b492012 Merge pull request #9991 from stiopaa1/msg_asyncmess_moveCProcessorAccept_class 
msg/AsyncMessenger: move C_processor_accept class

Reviewed-by: Haomai Wang <haomai@xsky.com>
 2016-06-29 13:52:52 +08:00
Haomai Wang
593008e9c7 Merge pull request #9783 from yuyuyu101/wip-16366 
msg/async: Support close idle connection feature
 2016-06-29 13:47:28 +08:00
xie xingguo
f35b05b517 os/bluestore: promote output to derr if out of memory 
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-06-29 12:43:32 +08:00
xie xingguo
1af00b15e1 os/bluestore: make block array of test_bmap_alloc self-adapting 
So we are be able change total_blocks or zone_size flexibly and
thus append more test cases.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-06-29 12:43:32 +08:00
xie xingguo
06c48e410a os/bluestore: fix memory leak during bit alloc ut 
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2016-06-29 12:43:32 +08:00
Haomai Wang
0001a07354 Revert "msg/AsyncMessenger: move Worker class to cc file" 
This reverts commit 23d4488e37.
 2016-06-29 12:14:29 +08:00
Haomai Wang
ada84e7c7d msg/async/Event: change to use pthread_t to indicate whether the same thread 
thread_local has problem in TmapMigratePP.DataScan case that it will be
assigned to zero unexpectedly.

pthread_t is much cheaper than gettid since it's a library implementation.

Signed-off-by: Haomai Wang <haomai@xsky.com>
 2016-06-29 12:14:29 +08:00
Haomai Wang
be8dec5597 msg/async/AsyncConnection: avoid dup RETRYGLOBAL sending 
if two accept all going to dispatch RETRYGLOBAL sending, it will cause
invalid connect side receiver.

Signed-off-by: Haomai Wang <haomai@xsky.com>
 2016-06-29 12:14:29 +08:00
Haomai Wang
1f65a355be msg/async/Event: don't execute inline if nonwait 
Signed-off-by: Haomai Wang <haomai@xsky.com>
 2016-06-29 12:14:29 +08:00