Commit Graph

72320 Commits

Author SHA1 Message Date
Sage Weil
bc6e9a304c Merge pull request #15056 from liewegas/wip-fix-osdmap-comment
osd/OSDMap: fix feature commit comment
2017-05-11 17:30:27 -05:00
Sage Weil
44ad0acf78 osd/OSDMap: fix feature commit comment
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-11 17:29:26 -05:00
Casey Bodley
74ddaa5eae Merge pull request #10260 from weiqiaomiao/wqm-wip-clean-opstate
rgw: clean unuse code in cls_statelog_check_state

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-11 12:01:26 -04:00
Casey Bodley
6e8d337151 Merge pull request #14411 from mikulely/rgw-cleanup-rados
rgw: rgw_rados drop deprecated global var

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-11 11:59:23 -04:00
Casey Bodley
9fbea01c96 Merge pull request #14906 from mikulely/lc-cleanup-2
rgw: cleanup lc continuation

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2017-05-11 11:58:38 -04:00
Casey Bodley
84fb0cd505 Merge pull request #15010 from shashalu/fix-bucket-create
rgw: when create_bucket use the same num_shards with info.num_shards

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-11 11:56:02 -04:00
Sage Weil
381f465839 Merge pull request #15017 from dillaman/wip-19863
common/bit_vector: utilize deep-copy during data decode

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-11 10:17:15 -05:00
Patrick Donnelly
bdf1a7a943
mds: use debug_mds for most subsys
DOUT_COND was obsoleted by f41887e38d.

Balancer and Mantle use a dout hack to continue working since we will probably
specifically want very verbose balancer output in the future.

Fixes: http://tracker.ceph.com/issues/19734

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-05-11 10:49:54 -04:00
lisali
2c91e91b8a test/fio_ceph_objectstore: fix fio plugin build failure by engine_data
The commit in fio 710bf9c592
changed engine_data to a union which leads build of fio_ceph_objectstore failed.

Signed-off-by: Lisa Li xiaoyan.li@intel.com
2017-05-11 10:44:43 -04:00
runsisi
1fdc92716e osd/PrimayLogPG: update modified range to include the whole object size for write_full op
Signed-off-by: runsisi <runsisi@zte.com.cn>
2017-05-11 21:54:30 +08:00
runsisi
bb6d2de31a osd/PrimaryLogPG: drop parameter count_bytes for write_update_size_and_usage
Signed-off-by: runsisi <runsisi@zte.com.cn>
2017-05-11 21:45:08 +08:00
Yan, Zheng
4a09e9431d osd/OSDMap: bump encoding version for require_min_compat_client
Otherwise, OSDMap::decode() throws expcetion when decoding old
version osdmap

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-05-11 21:37:50 +08:00
John Spray
745902aec6 Merge pull request #14743 from ukernel/wip-19635
mds: issue new caps when sending reply to client

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-11 10:46:41 +01:00
John Spray
1b1fc05506 Merge pull request #14704 from batrick/i19589
mds: fix mantle script to not fail for last rank

Reviewed-by: Michael Sevilla <mikesevilla3@gmail.com>
2017-05-11 10:12:22 +01:00
Yan, Zheng
9f751d6e21 mds: fix use-after-free in Locker::file_update_finish()
The capability may have already been freed when executing
Locker::file_update_finish()

Fixes: http://tracker.ceph.com/issues/19828
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-05-11 14:14:47 +08:00
Yan, Zheng
80b7672bf8 Merge pull request #14921 from batrick/ptrwaiter-balance
mds: relocate PTRWAITER put near get
2017-05-11 14:10:12 +08:00
Kefu Chai
f3c44a0dfc osd/PrimaryLogPG: do not call on_shutdown() if (pg.deleting)
when a callback is called, it could be facing a PG already shut down by
OSD. but if that callback wants to shut that PG down. it should check
the PG's status first.

Fixes: http://tracker.ceph.com/issues/19902
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-11 13:16:31 +08:00
Yan, Zheng
b67a599ebe Merge pull request #14598 from batrick/mds-balancer-pin
mds: support export pinning on directories
2017-05-11 11:56:34 +08:00
Yan, Zheng
3977c30337 client: fix UserPerm::gid_in_group()
Fixes: http://tracker.ceph.com/issues/19903
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-05-11 11:42:26 +08:00
Yan, Zheng
bbb3369b50 qa/suites/fs: fix write size calculation in full tests
'max_avail' has already taken full_ratio into account

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-05-11 11:18:22 +08:00
Brad Hubbard
2f3cd60855 common: Remove redundant includes
Fixes: http://tracker.ceph.com/issues/19883 (Partially)

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-05-11 11:59:08 +10:00
Patrick Donnelly
b6ba30ba57
mds: delay export until dir is stable
This catches an assertion failure due to exports on unjournaled directories:

    2017-05-10 12:45:18.193640 7eff65a04700  7 mds.0.migrator export_dir [dir 10000000003 /a/ [2,head] auth pv=3 v=1 cv=0/0 ap=1+2+2 state=1073742082|complete f()->f(v0 m2017-05-10 12:45:14.977151 1=0+1) n()->n(v0 rc2017-05-10 12:45:14.977151 1=0+1) hs=0+1,ss=0+0 | child=1 authpin=1 0x55ef655de000] to 1
    2017-05-10 12:45:18.193654 7eff65a04700 15 mds.0.18 hit export target 10 @ 2017-05-10 12:45:18.193653
    2017-05-10 12:45:18.193661 7eff65a04700 10 mds.0.cache.dir(10000000003) auth_pin by 0x55ef650cc130 on [dir 10000000003 /a/ [2,head] auth pv=3 v=1 cv=0/0 ap=2+2+2 state=1073742082|complete f()->f(v0 m2017-05-10 12:45:14.977151 1=0+1) n()->n(v0 rc2017-05-10 12:45:14.977151 1=0+1) hs=0+1,ss=0+0 | child=1 authpin=1 0x55ef655de000] count now 2 + 2
    2017-05-10 12:45:18.193701 7eff65a04700  7 mds.0.cache request_start_internal request(mds.0:4) op 5377
    2017-05-10 12:45:18.193708 7eff65a04700  7 mds.0.migrator dispatch_export_dir request(mds.0:4)
    2017-05-10 12:45:18.198281 7eff66205700 20 mgrc operator() 234 counters, of which 0 new
    2017-05-10 12:45:18.198294 7eff66205700 20 mgrc send_report encoded 2006 bytes
    2017-05-10 12:45:18.198298 7eff66205700  1 -- 127.0.0.1:6825/693985647 --> 127.0.0.1:6826/1025 -- mgrreport(+0-0 packed 2006) v2 -- 0x55ef655abe00 con 0
    2017-05-10 12:45:18.198364 7eff6a9ab700 10 _calc_signature seq 14 front_crc_ = 2388339344 middle_crc = 0 data_crc = 0 sig = 5026349537430007662
    2017-05-10 12:45:18.198376 7eff6a9ab700 20 Putting signature in client message(seq # 14): sig = 5026349537430007662
    2017-05-10 12:45:18.198411 7eff65a04700 -1 /home/pdonnell/ceph/src/mds/MDCache.cc: In function 'void MDCache::make_trace(std::vector<CDentry*>&, CInode*)' thread 7eff65a04700 time 2017-05-10 12:45:18.194136
    /home/pdonnell/ceph/src/mds/MDCache.cc: 8225: FAILED assert(parent)

     ceph version 12.0.1-2198-ge757c02 (e757c025fa3270b12fb2fca17cf159fa1bd72747)
     1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55ef5b780f02]
     2: (MDCache::make_trace(std::vector<CDentry*, std::allocator<CDentry*> >&, CInode*)+0x1cb) [0x55ef5b4fa12b]
     3: (Migrator::get_export_lock_set(CDir*, std::set<SimpleLock*, std::less<SimpleLock*>, std::allocator<SimpleLock*> >&)+0x55) [0x55ef5b5e8215]
     4: (Migrator::dispatch_export_dir(boost::intrusive_ptr<MDRequestImpl>&, int)+0xa74) [0x55ef5b5f72d4]
     5: (Migrator::export_dir(CDir*, int)+0x9ca) [0x55ef5b5ea75a]
     6: (MDBalancer::handle_export_pins()+0x7b4) [0x55ef5b61ab24]
     7: (MDBalancer::tick()+0x1e8) [0x55ef5b61b748]
     8: (MDSRankDispatcher::tick()+0x5f1) [0x55ef5b44bdb1]
     9: (Context::complete(int)+0x9) [0x55ef5b43bcc9]
     10: (SafeTimer::timer_thread()+0x452) [0x55ef5b77dd52]
     11: (SafeTimerThread::entry()+0xd) [0x55ef5b77f15d]
     12: (()+0x76ba) [0x7eff6d4bb6ba]
     13: (clone()+0x6d) [0x7eff6c52782d]

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-05-10 17:35:44 -04:00
Kefu Chai
5e3f837222 byteorder: refactor ceph_le{16,32,64} using template
to improve the readablity and easier for debugging.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:12:30 +08:00
Kefu Chai
b7a194eb65 byteorder: use gcc intrinsics for byteswap
* use gcc intrinsics for byteswap
* use template to wrap them.
* add the modeline for emacs/vim
* update the caller of the mswab/swab accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:12:30 +08:00
Kefu Chai
c44cdc27d5 radosstriper: do not use GCC extension to initialize struct members
it's obsolete since GCC 2.5 and is not portable. see
https://gcc.gnu.org/onlinedocs/gcc/Designated-Inits.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:12:30 +08:00
Kefu Chai
ba2a94c1dc byteorder: remove the cruft to detect endianness
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:12:30 +08:00
Kefu Chai
833705e885 common/sctp_crc32: include acconfig.h for the detecting endianness
so byteorder.h can be a pure c++ header.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:12:30 +08:00
Kefu Chai
e0347b39fb cmake: use cmake module to detect endianness
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:11:13 +08:00
Kefu Chai
2fc8d873b9 cmdparse: more constness
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 23:07:21 +08:00
Patrick Donnelly
02c41f683d
qa: add health warning test for insufficient standbys
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-05-10 11:05:09 -04:00
Patrick Donnelly
a4cb10900d
qa: turn off spurious standby health warning
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-05-10 10:21:28 -04:00
Jason Dillaman
2a616ddbc3 Merge pull request #15030 from chardan/jfw-wip-halflife_atomic_t-fix-compare_exchange_weak
tools: change compare_exchange_weak to compare_exchange_strong

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-10 10:04:45 -04:00
Jason Dillaman
fd0fdcfde2 Merge pull request #14975 from runsisi/wip-io-logging
librbd: cleanup logging code under librbd/io

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-10 10:00:56 -04:00
Sage Weil
bd71ca396a Merge pull request #14905 from johnzzpcrystal/aio
test/fsx:  Remove the dead code associated with aio backend

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-10 08:16:13 -05:00
Sage Weil
511e86636e Merge pull request #14911 from ovh/bp-small-enc-eradication
os/bluestore: use denc for varint encoding

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-10 08:15:16 -05:00
Sage Weil
99aeb527f8 Merge pull request #14957 from ovh/bp-crc-cache-miss-counter
buffer, osd: add missing crc cache miss perf counter

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-10 08:14:27 -05:00
Sage Weil
58f20ff595 Merge pull request #14996 from xiexingguo/wip-fix-dump_basic
mon/PGMap.cc: fix "osd_epochs" section of dump_basic

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-10 08:14:09 -05:00
Sage Weil
696f061670 Merge pull request #15003 from badone/wip-redundant-headers
common: Remove redundant includes

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-10 08:13:50 -05:00
Sage Weil
1fc5690bcd Merge pull request #15013 from TsaiJin/wip-pg-lock-cleanup
osd/PG.cc: unify the call of checking whether lock is held

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-10 08:12:48 -05:00
Sage Weil
9af1a1eba1 Merge pull request #14937 from SpandanKumarSahu/bug#17453
mgr: remove non-existent MDS daemons from FSMap

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-10 07:21:02 -05:00
runsisi
0bb74024f1 librbd: cleanup logging code under librbd/io
Signed-off-by: runsisi <runsisi@zte.com.cn>
2017-05-10 19:48:37 +08:00
Jesse Williamson
e573245cf4 tools: change compare_exchange_weak to compare_exchange_strong
On non x86 platforms without a dedicated compare-exchange instruction, std::compare_exchange_weak()
could fail.

Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2017-05-10 04:04:25 -07:00
xie xingguo
27e546a32f os/bluestore: pre-calculate number of ghost buffers to evict
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-05-10 19:01:49 +08:00
Yan, Zheng
9778b81bb8 qa/suites/fs: reserve more space for mds in full tests
Fixes: http://tracker.ceph.com/issues/19891
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-05-10 17:04:18 +08:00
Loic Dachary
accceddb17 Merge pull request #15016 from dachary/wip-19888-ceph-disk
ceph-disk: separate ceph-osd --check-needs-* logs

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
2017-05-10 10:04:17 +02:00
lu.shasha
4ce64a190b rgw: when create_bucket use the same num_shards with info.num_shards
pr #14388 only fix the num_shards in BucketInfo, "init_bucket_index" function still use local num_shards

Fixes: http://tracker.ceph.com/issues/19745

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
2017-05-10 10:42:20 +08:00
Kefu Chai
8bbafd2859 libradosstriper: do not assign garbage to returned value
if `rc` is not zero, the returned `ssize` is either garbage or
undefined. this silences warning from clang analyzer:

Assigned value is garbage or undefined

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 10:31:57 +08:00
Kefu Chai
f314bce374 test/messenger/simple*: throw when fails to malloc
silences the warning of "Null pointer passed as an argument to a
'nonnull' parameter" from clang analyzer.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-10 10:31:57 +08:00
Yan, Zheng
2c25c99cb4 pybind: fix open flags calculation
(O_WRONLY | O_RDWR) is invaild open flags

Fixes: http://tracker.ceph.com/issues/19890
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-05-10 09:46:59 +08:00
Sage Weil
bfcd2fffe1 Merge pull request #14854 from ceph/wip-reduce-smoke
qa/suites: Reduce fs combination tests for smoke, use bluestore

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-09 12:38:21 -05:00