Commit Graph

66987 Commits

Author SHA1 Message Date
Sage Weil
2a7013cd5a qa/tasks/workunit: clear clone dir before retrying checkout
If we checkout ceph-ci.git, and don't find a branch,
we'll try again from ceph.git. But the checkout will
already exist and the clone will fail, so we'll still
fail to find the branch.

The same can happen if a previous workunit task already
checked out the repo.

Fix by removing the repo before checkout (the first and
second times).  Note that this may break if there are
multiple workunit tasks running in parallel on the same
role.  That is already racy, so if it's happening, we'll
want to switch to using a truly unique clonedir for each
instantiation.

Fixes: http://tracker.ceph.com/issues/18336
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-22 13:05:22 -05:00
Jianpeng Ma
ca209949be os/bluestore/bluefs: make bluefs_preextend_wal_files default is false.
This make rocksdb log corruption. Before fix, disable this function.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2016-12-22 09:47:35 -05:00
Joao Eduardo Luis
36ecfbd8bf mon: MonMap: clear 'mon_info' during decode
It so happens that it's not safe to assume the monmap will be in an
empty state upon decoding.

Turns out the MonClient will reuse the MonMap instance when decoding
the just received map from the monitors. Should the monitors be on an
older version that do not support 'mon_info', this field will not be
decoded (after all, there's no field to decode from); but by this time,
the MonClient would already have a built monmap, which could have
populated 'mon_info' with temporary mon names from 'mon initial
members'.

Given the existing entries in 'mon_info', and the conflicting entries in
'mon_addr', we would end up asserting in 'sanitize_mons()'. This becomes
a non-issue if 'mon_info' is empty, as was unfortunately presumed.

Fixes: http://tracker.ceph.com/issues/18265
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2016-12-22 09:13:37 -05:00
Sage Weil
3824ad3fec Merge branch 'kraken' into wip-18314 2016-12-22 07:53:23 -06:00
Sage Weil
600b52c76a Merge pull request #12603 from liewegas/wip-fix-rwordered
osd: fix CEPH_OSD_FLAG_RWORDERED

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-12-21 21:03:43 -06:00
Sage Weil
f169b6b888 osd: fix CEPH_OSD_FLAG_RWORDERED
Broke this in 3a2ed3b136de68192f661b6992cbb96ce6aafd20.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-21 14:24:37 -05:00
Yan, Zheng
08066eb92d osd/PrimaryLogPG: enlarge omap request bytes limitation
For cephfs, 4M buffer can only encode about 5k dentries. It's too
small for directory.

Fixes: http://tracker.ceph.com/issues/18314
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2016-12-21 16:44:31 +08:00
Jason Dillaman
12706d7622 Merge pull request #12587 from yuriw/wip-fix-subset-script-master
qa: fixed script to schedule rados and other suites with --subset option

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-12-20 18:47:39 -05:00
Yuri Weinstein
56975c039b changed script name
added kraken
parameterized suite name to be able to use for rados, rbd etc.

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2016-12-20 15:34:56 -08:00
Sage Weil
d877b27883 Merge pull request #12590 from athanatos/wip-ping-write-ordered
osd: force watch PING to be write ordered

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-20 17:13:31 -06:00
Sage Weil
c0b64fb070 Merge pull request #12593 from wjwithagen/wip-wjw-freebssd-SO_reuseaddr
msg/async/net_handler.cc: Do not apply SO_REUSEADDR for FreeBSD

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-20 14:07:23 -06:00
Jason Dillaman
5563ff343f Merge pull request #12584 from mkoderer/openstack_nova_doc
doc: mention corresponding libvirt section in nova.conf

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-12-20 14:00:36 -05:00
Willem Jan Withagen
0825f170e4 msg/async/net_handler.cc: Do not apply SO_REUSEADDR for FreeBSD
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-12-20 19:43:47 +01:00
Samuel Just
3a2ed3b136 osd/: treat PINGs as RWORDERED
89fd030bf9 switched them to show up
as reads to avoid logging them, but we still pipeline them with
reconnects.  Thus, also force them to be rwordered.

Fixes: http://tracker.ceph.com/issues/18310
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-12-20 09:47:41 -08:00
Sage Weil
e0903c9e08 Merge pull request #12540 from dmick/wip-18297
ceph.in: allow 'flags' to not be present in cmddescs
2016-12-20 09:56:32 -06:00
Josh Durgin
a84de42792 Merge pull request #12577 from liewegas/wip-osd-backfill
qa/tasks/osd_backfill.py: wait for osd.[12] to start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-12-20 07:51:24 -08:00
Sage Weil
d5d864189a Merge pull request #12574 from liewegas/wip-bluestore-split
os/bluestore: preserve source collection cache during split

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-12-20 08:43:29 -06:00
Marc Koderer
6229edfec4 DOC: Mention corresponding section in nova.conf
rbd_user and rbd_secret_uuid are part of the libvirt configuration.

Signed-off-by: Marc Koderer <marc@koderer.com>
2016-12-20 14:52:34 +01:00
Mykola Golub
47d2662998 Merge pull request #12472 from dillaman/wip-18244
librbd: remove image header lock assertions

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-12-20 13:10:55 +02:00
Sage Weil
6a786a8306 Merge pull request #12532 from ifed01/wip-bluestore-large-onode-test
os/bluestore: add UT for an estimation of Onode in-memory size

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 21:26:41 -06:00
Sage Weil
34ab000053 Merge pull request #12505 from ceph/wip-jlayton-submount
client: set metadata["root"] from mount method when it's called with …

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-19 21:18:19 -06:00
Sage Weil
73fa83866e Merge pull request #12570 from wjwithagen/patch-9
common/str_list.h: fix clang warning about std::move

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-12-19 21:17:19 -06:00
Sage Weil
f674fa24fb Merge pull request #12541 from liewegas/wip-bluestore-omap-flush
os/bluestore: include modified objects in flush list even if onode unchanged

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-12-19 21:15:37 -06:00
Sage Weil
c922404a03 qa/tasks/osd_backfill.py: wait for osd.[12] to start
...before sending a tell command.  Otherwise osd.2 might
start without 1, the io unblocks, and the tell fails
because osd.1 is still down.

Fixes: http://tracker.ceph.com/issues/18303
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-19 21:56:11 -05:00
Sage Weil
ec5ba4e8cf os/bluestore: preserve source collection cache during split
OSD split transactions look something like

 mkcoll new
 split old
 ...
 omap_rmkey_range old
 omap_setkeys old
 omap_setkeys new

The last part splits the log into two pieces.  The
problem is that the rmkey_range needs to wait on old
omap transactions to flush, and those are linked to the
old onode, and split clears the cache.  The result is
that we don't wait, rmkeyrange leaves some recent pg log
keys behind, and on OSD restart we get an error because
the object doesn't belong to the (old) collection.

Fix this by preserving objects in the old collection and
only clear out objects that are moving to the newly
split collections.  This will include the pgmeta object
that we care about.

(Note that we are one step closer to preserving the
cache contents across the split, but not quite there
yet: at this point we don't have all of the destination
collections.  A change in the ObjectStore interface is
probably needed to make that not be extremely awkward.)

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-19 17:05:00 -05:00
Samuel Just
dc639fd9e5 Merge pull request #12571 from athanatos/wip-15943
osd: Fix map gaps again (bug 15943)

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 13:50:15 -08:00
vasukulkarni
c3afbbf30b Merge pull request #12572 from liewegas/wip-python-rados-test
qa/workunits: include extension for nose tests
2016-12-19 12:25:38 -08:00
Sage Weil
e82291342a Merge pull request #12568 from ifed01/wip-bluestore-onode-diet2
os/bluestore: reduce Onode in-memory footprint

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 14:15:03 -06:00
Sage Weil
3079dfc840 Merge pull request #12555 from socketpair/usr_bin
systemd: Fix startup of ceph-mgr on Debian 8

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 14:13:32 -06:00
Sage Weil
3649d1fc3c Merge pull request #12355 from chhabaramesh/bit_alloc_mempool
os/bluestore: simplified allocator interfaces to single apis

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 14:13:10 -06:00
Sage Weil
5666fd61d6 qa/workunits: include extension for nose tests
When you have a relative path you have to include the extension.
Weird.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-19 14:08:11 -05:00
jtlayton
20f936e841 Merge pull request #12412 from ceph/wip-jlayton-suid
client/mds: Clear setuid bits when writing or truncating
2016-12-19 13:26:27 -05:00
Willem Jan Withagen
d9acbf587b Clang/str_list.h: fix clang warning about std::move
/home/jenkins/workspace/ceph-master/src/include/str_list.h:99:10: warning: moving a local object in a return statement prevents copy elision [-Wpessimizing-move]
  return std::move(str_vec);
         ^
/home/jenkins/workspace/ceph-master/src/include/str_list.h:99:10: note: remove std::move call here
  return std::move(str_vec);
         ^~~~~~~~~~       ~
1 warning generated.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-12-19 17:29:38 +01:00
Jason Dillaman
2d982ae1df Merge pull request #12565 from trociny/wip-18247-fixup
common: fix clang compilation error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-12-19 10:33:24 -05:00
Igor Fedotov
7d1f46b2b7 os/bluestore: remove duplicate sbid member for Blob, use one from shared_blob member
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-12-19 15:06:47 +00:00
Igor Fedotov
a6158d5cce os/bluestore: declare a member for Blob's cached bufferlist if explicitly enabled by CACHE_BLOG_BL macro only
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-12-19 15:06:39 +00:00
Igor Fedotov
b07f3ffebb os/bluestore: recalculate shared blob key instead of keeping it in memory
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-12-19 15:06:24 +00:00
Sage Weil
1b319f2e8f os/bluestore: include modified objects in flush list even if onode unchanged
We use the onode flush list so that we can ->flush() as
a barrier before doing any read/modify/write.  For
example, omap_rmkeyrange will flush before reading to
see what keys to erase in order to ensure that any
previous inserts are applied to the db and we see them
and remove them.

However, some omap operations don't update the onode
itself, which means write_onode() doesn't get called and
we aren't put on this list.

Add a note_modified_object() helper that can be called
instead of write_onode() for those cases.  That way we
get on the list and flush() works as expected.

We could have resolved this by just putting ourselves on
the dirty onode list, but in practice every OSD op is
writing omap keys to the pgmeta object and there is no
need to touch the onode key in this case, so doing so
would be a big regression.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-19 10:03:44 -05:00
Sage Weil
991141c34f Merge pull request #12548 from Liuchang0812/add-document-rgw
doc: explain rgw_fcgi_socket_backlog in rgw/config-ref.rst

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 08:44:27 -06:00
Sage Weil
071cdbe35f Merge pull request #12561 from kylinstorage/wip-fix-osdmaptool-zero
osdmaptool: fix divide by zero error

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-19 08:38:43 -06:00
Mykola Golub
8da0f348d3 Merge pull request #12497 from dillaman/wip-18243
journal: prevent repetitive error messages after being blacklisted

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-12-19 15:58:59 +02:00
Ramesh Chander
9ce95bb1fd remove unused wait argument and related function versions
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-12-19 00:17:40 -08:00
Mykola Golub
bd584fe797 common: fix clang compilation
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-12-19 08:44:22 +02:00
Yunchuan Wen
45a0286e86 osdmaptool: fix divide by zero error
Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>
2016-12-19 12:56:37 +08:00
Haomai Wang
e293d40e6a Merge pull request #12558 from Adirl/fix_bad_len_msg
msg/async/rdma: fix bad message that went to the user

Reviewed-by: Haomai Wang <haomai@xsky.com>
2016-12-19 09:59:51 +08:00
Ramesh Chander
f1e9aee716 zone block offset bug fix and related unit test case
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

zone block offset related test case

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-12-18 10:40:40 -08:00
Ramesh Chander
86b6ca3dc2 allocat_extent api rename and tests for new api
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-12-18 10:40:40 -08:00
Ramesh Chander
78b490252d remove cont alloc alloc_blocks calls from allocator code and test
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-12-18 10:40:40 -08:00
Ramesh Chander
2942bc627a consolidate allocator calls to single alloc_extent api
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-12-18 10:40:40 -08:00
Ramesh Chander
8ed5ec0277 allocator min_alloc arguments in call stack and tests
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-12-18 10:40:40 -08:00