Commit Graph

66871 Commits

Author SHA1 Message Date
Sage Weil
0c061c14a6 Revert "tasks/workunit.py: depth 1 clone"
This reverts commit 4faf77a649.
2016-12-14 12:28:23 -05:00
Mykola Golub
4828ab63fa Merge pull request #12420 from dillaman/wip-16176
librbd: properly order concurrent updates to the object map

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2016-12-14 19:23:12 +02:00
Zack Cerza
124712a4d5 Merge pull request #1307 from ceph/wip-workunit-repo
tasks/workunit: use correct git mirror url
2016-12-14 10:21:05 -07:00
Sage Weil
4faf77a649 tasks/workunit.py: depth 1 clone
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-14 12:19:44 -05:00
Sage Weil
159c455a03 tasks/workunit: remove kludge to use git.ceph.com
This was hard-coded to ceph.git (almost) and breaks when
you specify --ceph-repo.  Remove it entirely.  We'll see if
github.com is better at handling our load than it used to
be!

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-14 12:18:29 -05:00
Jason Dillaman
52e46fdf21 Merge pull request #12483 from trociny/wip-18242
rbd-nbd: invalid error code for "failed to read nbd request" messages

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-12-14 11:40:09 -05:00
Sage Weil
bd83719e6e Merge pull request #1309 from ceph/wip-16800-kefu
tasks/ceph: restore context of osd mount path before mkfs
2016-12-14 09:56:25 -06:00
Sage Weil
49698d6e2b Merge pull request #12476 from liewegas/wip-vstart-rados
CMakeLists: add vstart-base target

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-12-14 09:23:52 -06:00
Matt Benjamin
04bf0f8ca1 rgw: add 'rgw log http headers' gloss to config-ref.rst
Explain the option for upstream doc.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-12-14 09:59:20 -05:00
Sage Weil
2b94702139 Merge pull request #12393 from dachary/wip-18036-throttle
tests: unittest_throttle avoid ASSERT_DEATH

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-14 08:54:34 -06:00
Mykola Golub
a1a2f3a0ce Merge pull request #12473 from dillaman/wip-18243
journal: avoid logging an error when a watch is blacklisted

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-12-14 16:51:04 +02:00
John Spray
2871374c38 Merge pull request #1156 from ceph/wip-kclient-python
Enable cephfs python tests on kclient
2016-12-14 14:09:13 +00:00
John Spray
63ae8579bf Merge pull request #11542 from batrick/systemd-ceph-fuse
systemd: add ceph-fuse service file

Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-14 13:55:33 +00:00
John Spray
3c7c65c669 Merge pull request #12337 from vishalkanaujia/wip_12132
tools: cleanup phase of cephfs-data-scan

Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-14 13:52:35 +00:00
John Spray
69e03da184 Merge pull request #12358 from tchaikov/wip-18157
ceph-fuse: start up log on parent process before shutdown

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-14 13:49:33 +00:00
John Spray
f0f552ca13 Merge pull request #12384 from renhwztetecs/renhw-wip-libcephfs-addreadlink
libcephfs: add readlink function in cephfs.pyx

Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-14 12:57:21 +00:00
John Spray
cdbad50eca Merge pull request #12395 from jcsp/wip-18166
mon/MDSMonitor: fix iterating over mutated map

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2016-12-14 12:56:35 +00:00
John Spray
e25f060a73 Merge pull request #12401 from kylinstorage/wip-typos
client: fix mutex name typos

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-12-14 12:56:11 +00:00
John Spray
8d4a95d50b Merge pull request #12443 from jcsp/wip-18225
mds: release pool allocator memory after exceeding size limit

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-12-14 12:52:23 +00:00
Kefu Chai
53225d5272 tasks/ceph: restore context of osd mount path before mkfs
all newly created files and directories under the mount dir inherit the
SELinux type of their parent directory. so we need to set it before
mkfs.

Fixes: http://tracker.ceph.com/issues/16800
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-12-14 19:15:16 +08:00
Mykola Golub
42c2514fe2 rbd-nbd: invalid error code for "failed to read nbd request" messages
Fixes: http://tracker.ceph.com/issues/18242
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-12-14 09:53:05 +02:00
xianxiaxiao
b26555a042 src/kv/RocksDBStore.cc:fix typos
Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
2016-12-14 07:32:23 +00:00
xianxiaxiao
679292b957 src/erasure-code/shec/ErasureCodeShec.cc:fix typos
Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
2016-12-14 07:30:23 +00:00
xianxiaxiao
46e79f4828 src/client/SyntheticClient.cc: fix typos
Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
2016-12-14 07:28:49 +00:00
xianxiaxiao
54d505d60a src/rgw/rgw_cache.cc:fix typos
Signed-off-by: xianxiaxiao <xianxia.xiao@kylin-cloud.com>
2016-12-14 07:27:25 +00:00
Sage Weil
4e23550bb2 Merge pull request #12333 from xiexingguo/xxg-wip-bluestore-1206
os/bluestore: fix potential csum_order overflow

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-14 01:07:37 -06:00
Sage Weil
6b6af6cfd4 Merge pull request #12382 from dachary/wip-18092-gf-complete
erasure-code: synchronize with upstream gf-complete
2016-12-14 01:06:47 -06:00
Sage Weil
3faf8b8466 CMakeLists: add vstart-base target
No mds or radosgw.  Add rados CLI.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-14 00:51:25 -06:00
Brad Hubbard
8d9d84bce9 librados: Memory leaks in object_list_begin and object_list_end
We allocate a cursor in the constructor but simply reassign it in these
functions without cleaning up the original. We have a utility setter that
handles this exact case so we should use it.

Fixes: http://tracker.ceph.com/issues/18252

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-12-14 16:29:08 +10:00
Sage Weil
80ac7b124b Merge pull request #12459 from runsisi/wip-ec-minor-cleanup
osd/ECBackend: minor cleanup to remove duplicate code

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-13 20:08:55 -06:00
Sage Weil
581a8c6310 Merge remote-tracking branch 'gh/kraken' 2016-12-13 17:48:47 -06:00
Matt Benjamin
4092c97ee5 rgw: custom headers: use boost::container::flat_map
Use this occasion of serialized map to implement encoders
for boost::container::flat_map, which is optimized for
in-order insertion.

After some discussion, I'm proposing to just add the new template
forms, rather than (e.g.) adding either a large amount of
specialization machinery, or alternatively using C-style
macros.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-12-13 18:14:13 -05:00
Matt Benjamin
b82919a152 rgw: add rgw_log_http_headers option
Tracks headers that should be handled conditionally (currently,
can only log, so using minimal structure to represent the
mapping).

Adds map of custom headers to rgw_log_entry, and populate it with
headers pre-selected for custom logging in RGWREST.  Added to encoder
and Formatter output.

Some additional optimization possible.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-12-13 18:13:42 -05:00
Jason Dillaman
7d743bfce6 librbd: block concurrent in-flight object map updates for the same object
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-13 16:57:59 -05:00
Jason Dillaman
b1d624b43e librbd: new block guard helper to prevent concurrent IO to blocks
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-13 16:57:59 -05:00
Jason Dillaman
ea7b30a4fb librbd: convert ObjectMap to template for unit testing
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-13 16:57:59 -05:00
Jason Dillaman
477ae54a56 librbd: clean up object map update interface
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-13 16:57:59 -05:00
Jason Dillaman
378b810cba librbd: update in-memory object map after on-disk update committed
Concurrent IO to the same object would previously result in the first
IO pausing to update the object map while the other IO would proceed
to directly update the object before the object map state was properly
updated.

Fixes: http://tracker.ceph.com/issues/16176
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-13 16:57:59 -05:00
Jason Dillaman
b82a1c1d1f journal: avoid logging an error when a watch is blacklisted
Fixes: http://tracker.ceph.com/issues/18243
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-13 14:40:23 -05:00
Sage Weil
c6198c3e7e Merge pull request #12447 from ifed01/wip-bluestore-fix-reshard2
os/bluestore: avoid resharding if the last shard size fall below shar…

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-13 11:47:42 -06:00
Kefu Chai
0be3f5f72e Merge pull request #12289 from liewegas/wip-pg-down
osd/PG: add "down" pg state (distinct from down+peering)

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-12-13 22:46:16 +08:00
Kefu Chai
c0e6171c95 Merge pull request #12389 from ovh/wip-rework-recovery-priorities
osd: Increase priority for inactive PGs backfill

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-13 22:44:52 +08:00
Kefu Chai
11733614b8 Merge pull request #12352 from kylinstorage/wip-cleanup-mark-queued
osd: mark queued flag for op

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-13 22:44:22 +08:00
Xinze Chi
0d1e9aabf6 common/blkdev: use realpath instead of readlink to resolve the recursion symbol link
Signed-off-by: Xinze Chi <xinze@xsky.com>
2016-12-13 22:14:46 +08:00
Igor Fedotov
353060638c os/bluestore: avoid resharding if the last shard size fall below shard min size
Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>
2016-12-13 13:41:41 +00:00
Willem Jan Withagen
517a77f55c FreeBSD/OSD.cc: add client_messenger to the avoid_ports set.
Observed "feature":
  During rebind due to a "wrongly marked down" log message, FreeBSD is
  able to bind to the port used by client_messenger.
  The Linux variant avoids that port because it is already in use.

Result:
  In FreeBSD there would be 2 listeners on the port, and due to the rebind
  they have different nonces. (This is written in the logfile)
  But they also will expect different protocols on that same port.

This is likely due to an interpretation difference in the SO_REUSEADDR
socket option:

Linux:
  SO_REUSEADDR
        Indicates that the rules used in validating addresses supplied
        in a bind(2)  call  should  allow  reuse  of  local addresses.
        For AF_INET sockets this means that a socket may bind,
        except when there is an active listening socket bound to the address.
        When the listening socket is bound to INADDR_ANY with a specific port
        then it is not possible to bind to this port for any local address.
        Argument is an integer boolean flag.

FreeBSD:
  SO_REUSEADDR
        Enables local address reuse
        indicates that the rules used in validating addresses supplied in a
        bind(2) system call should allow reuse of local addresses.

So FreeBSD doesn't guarantee that the connection is refused when there is already
a connection. So it is best avoided during rebinding otherwise any of the
cluster_messengers will attach to the port.

FreeBSD log with wrong connect:
bb98d80  0 log_channel(cluster) log [WRN] : map e18 wrongly marked me down
bb98d80  1 -- 127.0.0.1:6801/17881 rebind rebind avoid 6801,6802,6803
bb98d80  1 -- 127.0.0.1:6801/17881 shutdown_connections
bb98d80  1 -- 127.0.0.1:6800/1017881 _finish_bind bind my_inst.addr is 127.0.0.1:6800/1017881
bb98d80  1  Processor -- start
bb98d80  1 -- 127.0.0.1:6802/17881 rebind rebind avoid 6801,6802,6803
bb98d80  1 -- 127.0.0.1:6802/17881 shutdown_connections
bb98d80  1 -- 127.0.0.1:0/17881 learned_addr learned my addr 127.0.0.1:0/17881
bb98d80  1 -- 127.0.0.1:6804/1017881 _finish_bind bind my_inst.addr is 127.0.0.1:6804/1017881

FreeBSD with the correct behaviour:
bb98d80  0 log_channel(cluster) log [WRN] : map e17 wrongly marked me down
bb98d80  1 -- 127.0.0.1:6802/15296 rebind rebind avoid 6801,6802,6803,6812
bb98d80  1 -- 127.0.0.1:6802/15296 shutdown_connections
bb98d80  1 -- 127.0.0.1:6806/1015296 _finish_bind bind my_inst.addr is 127.0.0.1:6806/1015296
bb98d80  1  Processor -- start
bb98d80  1 -- 127.0.0.1:6803/15296 rebind rebind avoid 6801,6802,6803,6812
bb98d80  1 -- 127.0.0.1:6803/15296 shutdown_connections
bb98d80  1 -- 127.0.0.1:0/15296 learned_addr learned my addr 127.0.0.1:0/15296
bb98d80  1 -- 127.0.0.1:6807/1015296 _finish_bind bind my_inst.addr is 127.0.0.1:6807/1015296

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-12-13 12:02:21 +01:00
runsisi
fe7d068180 ECBackend: minor cleanup to remove duplicate code
Signed-off-by: runsisi <runsisi@zte.com.cn>
2016-12-13 16:27:16 +08:00
Pan Liu
ff718c8b09 osd: remove the lock heartbeat_update_lock, and change heatbeat_need_update to atomic_bool
Signed-off-by: Pan Liu <pan.liu@istuary.com>
2016-12-13 16:27:14 +08:00
Kefu Chai
a3bf341876 Merge pull request #12294 from dx9/test-ecp-uname
test: use uname -m vs. uname -p in test-erasure-code-plugins.sh

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-12-13 10:56:14 +08:00
Xiaoxi Chen
a88ceff276 librbd/diff_iterator: use proper snap to query parent overlap
If no snapshot has been made, we will use CEPH_NOSNAP instead of 0,
to query the parent overlap.

Fixes bug http://tracker.ceph.com/issues/18200

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
2016-12-13 10:53:37 +08:00