Commit Graph

55575 Commits

Author SHA1 Message Date
Kefu Chai
146d2b15ff run-make-check.sh: use "make check" for running tests
"make all" does not offer "ceph-disk" and "ceph-detect-init" for
testing. as they are solely used for testing purpose. instead, these two
python command line packages are installed by the "install" target. so
we need to use "make check" to 1) prepare the test dependencies 2)
launch ctest to perform the test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:41:57 +08:00
Kefu Chai
152b7b6c90 Revert "cmake: add ceph-{disk,detect-init} to ALL"
This reverts commit 19c0731751.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Kefu Chai
a38aa4a2e1 test//osd/osd-copy-from.sh: use absolute path for "rados put"
"rados -p rbd put foo rados" does not work if "rados" is not in current
path. so change it to "rados -p rbd put foo $(which rados)"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Kefu Chai
3a1dea7c74 cmake: pass more env vars to osd-copy-from.sh
we should use the cmake function of add_ceph_test() to add
osd-copy-from.sh as a test. then we won't miss any env variables.
w/o this change, $CEPH_BUILD_VIRTUALENV is not passed to
osd-copy-from.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Kefu Chai
06195b3476 qa/workunits/cephtool/test.sh: respect $TMPDIR
create temp directory and files in $TMPDIR. the $TMPDIR is hard-wired to
/tmp before this change, we'd better respect the env variable $TMPDIR,
so it would be more consistent, and easier to do the cleanup if any.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Kefu Chai
f3c8c28b60 test: vstart_wrapper.sh run tests in $TMPDIR
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Kefu Chai
444d37f2b0 run-make-check.sh: run "make check" with a unique $TMPDIR
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Sage Weil
96f6fc0643 run-make-check.sh: run make check with cmake if do_cmake.sh is available
Signed-off-by: Sage Weil <sage@redhat.com>
2016-07-06 23:48:45 +08:00
Jason Dillaman
b342bf8ea1 Merge pull request #10136 from runsisi/sisi-wip-fix-missing-return
librbd: fix missing return statement if failed to get mirror image state

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-07-05 21:44:32 -04:00
Samuel Just
2daf3ef220 Merge pull request #10094 from athanatos/wip-throttle-test
test/common/Throttle.cc: fix race in shutdown

Reviewed-by: David Zafman <dzafman@redhat.com>
2016-07-05 10:50:50 -07:00
Casey Bodley
953fdecf15 Merge pull request #10088 from pritha-srivastava/wip_s3_signedurl_errors
rgw: Fix for using port 443 with pre-signed urls.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2016-07-05 13:37:15 -04:00
Ali Maredia
59a46b96a5 Merge pull request #10112 from tchaikov/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-05 11:29:24 -04:00
Jason Dillaman
a700116ded Merge pull request #10134 from trociny/wip-run-rbd-tests-fixup
test: run-rbd-tests test cmake fixup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-07-05 08:27:42 -04:00
Kefu Chai
0c13676126 Merge pull request #10133 from tchaikov/wip-fixc-cephtool-test.sh
test: cephtool/test.sh: fix expect_false() calls

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-07-05 17:46:39 +08:00
Mykola Golub
8a1fa024a4 Merge pull request #10072 from dillaman/wip-16561
librbd: failed assertion after shrinking a clone image twice

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-07-05 11:44:45 +03:00
runsisi
ea775178b6 librbd: fix missing return statement if failed to get mirror image state
Signed-off-by: runsisi <runsisi@zte.com.cn>
2016-07-05 16:08:40 +08:00
Kefu Chai
5e16f2379d test: cephtool/test.sh: fix expect_false with pipe
this fixes failures like,

/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/cephtool/test.sh:
line 32: ceph osd blacklist ls | grep 192.168.0.1: command not found

where the failure is not the "failure" we are expecting.

in our tests, following command

expect_false "ceph osd blacklist ls | grep 192.168.0.1"

is designed to to verify that "ceph osd blacklist ls | grep 192.168.0.1"
fails with non-zero return code. but expect_false() evaluates the command
line using plain "$@", which will send the arguments direct to the shell,
and $0 is "ceph auth get client.xx | grep caps | grep mon", which does
not exist and is not built-in command. so we need to check the grep
command instead.

for multiple piped command line, use

expect_false sh <<< "echo foo  | grep bar | grep baz"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-05 15:26:43 +08:00
Mykola Golub
acb4d67b0e test: run-rbd-tests test cmake fixup
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-07-05 09:27:00 +03:00
Mykola Golub
5485296a3d Merge pull request #10076 from dillaman/wip-rbd-mirror-cmake
test: fix rbd-mirror workunit test cases for cmake

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-07-05 08:28:43 +03:00
Kefu Chai
38afdbe590 Merge pull request #9731 from badone/wip-doc-ceph-authtool-print-fix
doc: ceph-authtool man page option is --print-key not --print

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-05 13:03:46 +08:00
Kefu Chai
161e44554a Merge pull request #10124 from trociny/wip-cmake-tests
test: cmake related fixups for rbd tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-05 11:26:46 +08:00
Mykola Golub
3f66da7239 test: fix run-rbd-tests test for cmake
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-07-04 22:32:29 +03:00
Mykola Golub
8d1959ccfc test: fix run-rbd-valgrind-unit-tests when built with cmake
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-07-04 22:32:20 +03:00
Mykola Golub
0c4063d6b7 test: ceph_test_librbd does not need rados_test_stub
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2016-07-04 22:32:11 +03:00
Sage Weil
fa8107a0c5 Merge pull request #10117 from xiexingguo/xxg-wip-bluestore-2016-07-04
os/bluestore: fix incorrect pool decoding of bnode

Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-04 10:55:29 -04:00
Kefu Chai
c6286c231c Merge pull request #9755 from dx9/wip-ns-name-uncompress
common/dns_resolve: use ns_name_uncompress instead of ns_name_ntop

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 21:53:20 +08:00
Kefu Chai
873aba7dd9 Merge pull request #9766 from dx9/wip-autotools-arch
configure.ac: Use uname instead of arch.

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-04 21:49:41 +08:00
Kefu Chai
ed473638e5 Merge pull request #9772 from tchaikov/wip-close-on-exec
msgr: set close on exec flag

Reviewed-by: Haomai Wang <haomai@xsky.com>
2016-07-04 21:47:42 +08:00
Kefu Chai
0efb391a40 Merge pull request #9820 from dx9/wip-fix-res-query-detect
configure.ac: fix res_query detection

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 21:46:55 +08:00
Kefu Chai
81ca1cbdd7 Merge pull request #9845 from dx9/wip-config-basename
common: remove basename() dependency

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 21:46:14 +08:00
Kefu Chai
c5d50caa67 Merge pull request #9969 from liewegas/wip-assert
include/assert: clean up ceph assertion macros

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-07-04 21:44:17 +08:00
Kefu Chai
a758a028b6 Merge pull request #9983 from SUSE/wip-cmake-fix-fcgi-include-dir
cmake: Fix FCGI include directory

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 21:43:24 +08:00
Kefu Chai
e0f15d9946 Merge pull request #10093 from tchaikov/wip-16525
crush: reset bucket->h.items[i] when removing tree item

Reviewed-by: Sage Weil <sage@redhat.com>
2016-07-04 21:42:34 +08:00
Kefu Chai
042c0aa957 Merge pull request #9330 from dachary/wip-15176-partprobe
ceph-disk: partprobe should block udev induced BLKRRPART

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 20:29:16 +08:00
Kefu Chai
8b433e13c2 Merge pull request #9394 from xiexingguo/xxg-wip-batchpgstates
mon/PGMonitor: batch filter pg states; add sanity check

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 20:19:20 +08:00
xie xingguo
29d2931d81 os/bluestore: fix incorrect pool decoding of bnode
"pool" is a pointer, and we shall alter its value instead.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-04 17:08:59 +08:00
Kefu Chai
19c0731751 cmake: add ceph-{disk,detect-init} to ALL
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 17:02:00 +08:00
Loic Dachary
8519481b72 ceph-disk: partprobe should block udev induced BLKRRPART
Wrap partprobe with flock to stop udev from issuing BLKRRPART because
this is racy and frequently fails with a message like:

    Error: Error informing the kernel about modifications to partition
    /dev/vdc1 -- Device or resource busy.  This means Linux won't know about
    any changes you made to /dev/vdc1 until you reboot -- so you shouldn't
    mount it or use it in any way before rebooting.

Opening a device (/dev/vdc for instance) in write mode indirectly
triggers a BLKRRPART ioctl from udev (starting version 214 and up)
when the device is closed (see below for the udev release note).

However, if udev fails to acquire an exclusive lock (with
flock(fd, LOCK_EX|LOCK_NB); ) the BLKRRPART ioctl is not issued.

045e00cf16/src/udev/udevd.c (L1042)

Acquiring an exclusive lock before running the process that opens the
device in write mode is therefore an effective way to control this
behavior.

git clone git://anonscm.debian.org/pkg-systemd/systemd.git
systemd/NEWS:
CHANGES WITH 214:

  * As an experimental feature, udev now tries to lock the
       disk device node (flock(LOCK_SH|LOCK_NB)) while it
       executes events for the disk or any of its partitions.
       Applications like partitioning programs can lock the
       disk device node (flock(LOCK_EX)) and claim temporary
       device ownership that way; udev will entirely skip all event
       handling for this disk and its partitions. If the disk
       was opened for writing, the close will trigger a partition
       table rescan in udev's "watch" facility, and if needed
       synthesize "change" events for the disk and all its partitions.
       This is now unconditionally enabled, and if it turns out to
       cause major problems, we might turn it on only for specific
       devices, or might need to disable it entirely. Device Mapper
       devices are excluded from this logic.

Fixes: http://tracker.ceph.com/issues/15176

Signed-off-by: Marius Vollmer <marius.vollmer@redhat com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-07-04 11:01:32 +02:00
Orit Wasserman
76705d87ad Merge pull request #10077 from yehudasa/wip-16494
rgw: fix for issue 16494
2016-07-04 09:37:34 +02:00
Kefu Chai
b6a1e50ae8 Merge pull request #10064 from BlaXpirit/brag-py3
Port ceph-brag to Python 3 (+ small fixes)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 15:00:50 +08:00
Kefu Chai
4ca9152512 Merge pull request #10115 from runsisi/ss-wip-fix-rpm-spec
ceph.spec.in: fix rpm package building error

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-04 14:08:35 +08:00
Kefu Chai
e1f67bdea8 cmake: do not depend on sse jerasure plugins if not avaiable
should not depend on them if SSE3 and/or SSE4 is not around.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 13:55:44 +08:00
Kefu Chai
e8fa03d42a cmake: do not link libglobal with lttng-ust
only link tracing libraries with lttng-ust, this fixes selinux denial
errors at run-time:

SELinux denials found on ubuntu@mira054.front.sepia.ceph.com: ['type=AVC
msg=audit(1467543917.770:131781): avc: denied { open } for pid=17511
comm="ceph-mon" path="/dev/shm/lttng-ust-wait-5" dev="tmpfs" ino=75199
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:tmpfs_t:s0 tclass=file', 'type=AVC
msg=audit(1467543914.518:131756): avc: denied { open } for pid=17418
comm="ceph-osd" path="/dev/shm/lttng-ust-wait-5" dev="tmpfs" ino=75199
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:tmpfs_t:s0 tclass=file', 'type=AVC
msg=audit(1467543917.770:131781): avc: denied { read } for pid=17511
comm="ceph-mon" name="lttng-ust-wait-5" dev="tmpfs" ino=75199
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:tmpfs_t:s0 tclass=file', 'type=AVC
msg=audit(1467543914.518:131756): avc: denied { read } for pid=17418
comm="ceph-osd" name="lttng-ust-wait-5" dev="tmpfs" ino=75199
scontext=system_u:system_r:ceph_t:s0
tcontext=system_u:object_r:tmpfs_t:s0 tclass=file']

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 13:55:44 +08:00
runsisi
49dfad1f53 ceph.spec.in: fix rpm package building error as follows:
error: Installed (but unpackaged) file(s) found:
      /usr/lib/udev/rules.d/60-ceph-by-parttypeuuid.rules

Signed-off-by: runsisi <runsisi@zte.com.cn>
2016-07-04 13:52:03 +08:00
Kefu Chai
0c5fb822dd cmake: refactor distutils_install_module to follow the pattern of distutils_install_cython_module
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 11:49:26 +08:00
Kefu Chai
8c9ff9c5cb cmake: install python packages into $CMAKE_INSTALL_PREFIX
* Distutils.cmake:
  set --prefix=${CMAKE_INSTALL_PREFIX} for python packages installed using
  setuptools. it was set to --prefix=/user only when $DESTDIR is set. so
  if user installs ceph using -DCMAKE_INSTALL_PREFIX, these python
  packages still go to /usr, which is unexpected.
* ceph-disk/CMakeLists.txt:
  install script into ${CMAKE_INSTALL_SBINDIR} instead of /usr/sbin

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 11:49:26 +08:00
Kefu Chai
313ab1b51a cmake: do not install unpackaged stuff
* doc/start/ceph.conf: it was installed as /etc/ceph.conf.example.
  but it's unexpected, and ceph.spec does not packages it.
* vstart.sh: this is for development usage. no need to package it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-04 11:17:02 +08:00
Loic Dachary
b8a4ca7e4c Merge pull request #9885 from dachary/wip-16351-by-parttypeuuid
udev: always populate /dev/disk/by-parttypeuuid

Reviewed-by: Alexandre Derumier <aderumier@odiso.com>
2016-07-03 13:31:46 +02:00
Sage Weil
fcf8d9ca78 Merge pull request #10095 from liewegas/wip-bluestore-trim
os/bluestore: trim cache on reads

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-02 18:53:40 -04:00
Sage Weil
7cff281e7d Merge pull request #10080 from xiexingguo/xxg-wip-add-compress-ratio
os/bluestore: improve required compression threshold

Reviewed-by: Sage Weil <sage@
2016-07-02 07:32:38 -04:00