Commit Graph

61 Commits

Author SHA1 Message Date
Kefu Chai
79b55473ef run-make-check,do_cmake: set BOOST_J if !WITH_SYSTEM_BOOS
it's also more readable if we can set the number of make jobs
in a single place.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-11 22:22:47 +08:00
Kefu Chai
f0d7616b37 install-deps,run-make-check: use ceph-libboost repo
ceph-libboost1.67 repo is setup to satisfy the libboost dependencies
used when building ceph. now it offers arm64 and amd64 builds for xenial.

* install-deps.sh: add ceph-libboost1.67 repo hosted on chacra if the
  building host does not have any of the listed ceph-libboost1.67
  packages installed
* run-make-check.sh: add "-DWITH_SYSTEM_BOOST=ON
  -DBOOST_ROOT=/opt/ceph" if include/boost/config.hpp is found
  under /opt/ceph

See-also: http://tracker.ceph.com/issues/25186
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-11 22:22:47 +08:00
Kefu Chai
b2ef5ddb9a run-make-check: should use sudo for running sysctl
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-23 19:47:11 +08:00
Kefu Chai
b928c7777c run-make-check: increase fs.aio-max-nr to 1048576
the solution was suggested by Yingxin Cheng. otherwise we could have
EAGAIN returned by io_setup(2).

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-22 19:52:46 +08:00
Erwan Velu
d1a807a277 run-make-check: Showing configuration before the build
The actual code is resetting the statistics before doing the actual compilation and prints them after the build.

That is nice to understand how much the cache was used but doesn't help understanding how much it _could_ have been used.

This patch is adding a reporting (-s) when cleaning the statistics so we can estimate :
- the actual number of files in cache
- the actual size of the cache

With this two missing information, its now possible estimate if there is some miss-usage of the cache.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-08-17 10:16:58 +02:00
Kefu Chai
8bec1035ba run-make-check: drop CMAKE_PYTHON_OPTS variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-02 17:23:11 +08:00
Kefu Chai
cee0122780 cmake,run-make-check: always enable WITH_GTEST_PARALLEL
now that https://github.com/google/gtest-parallel/pull/63 has been
merged, we can now use gtest-parallel with py2 and also py3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-02 17:13:14 +08:00
Kefu Chai
45bca777aa run-make-check: set WITH_SEASTAR with a non-empty string
otherwise "test -n $WITH_SEASTAR" fails, and the dependencies won't be
installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-18 10:25:21 +08:00
Kefu Chai
f1f1526166 run-make-check.sh: enable WITH_SEASTAR
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-12 17:51:22 +08:00
Nathan Cutler
693beaa8f3 run-make-check.sh: fill in missing DRY_RUN and add doc hint
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:05 +02:00
Nathan Cutler
458a0a6e62 run-make-check.sh: reset trap after sourcing install-deps.sh
Since we source install-deps.sh, it replaces our trap with its own,
so we have to reinstate our trap after it finishes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
b79b463f90 cleanup: run-make-check.sh: no space between -D and name of cmake option
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
3d6909b0f3 run-make-check.sh: gtest-parallel only with Python 2
Fixes: http://tracker.ceph.com/issues/24817
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
f9c0691f84 run-make-check.sh: inform user of make -j setting
This can help the user figure out why the build is so slow.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
43fe00e90c doc: cleanup: wordsmith a comment in run-make-check.sh
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
2315928acd run-make-check.sh: ccache goodness for everyone
Since run-make-check.sh already ensures that ccache is installed,
it makes sense to let everyone benefit from the ccache
tweaks introduced by 4cb5a59053

Note 1: The previous solution using "date" would cause build tools to reset
their timestamps after 24 hours, on subsequent runs of run-make-check.sh.
In order to maximize ccache effectiveness, this commit sets SOURCE_DATE_EPOCH
to a fixed value: the number of seconds elapsed since the Unix epoch as at
January 1, 2000 (chosen to commemorate Y2K armageddon).

Note 2: this commit introduces "set -e". This was actually in effect
before, via "source install-deps.sh". Better to make it explicit.

Fixes: http://tracker.ceph.com/issues/24777
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:01 +02:00
Nathan Cutler
c8691cd9a3 run-make-check.sh: drop workaround for undeclared jq build dependency
jq build dependency is properly declared now.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:25:40 +02:00
Erwan Velu
4cb5a59053 run-make-check.sh: Adding ccache tuning for the CI
When run-make-check is run by the CI, some tuning can be performed to
speedup the build.

This commit :
- Detect jenkins by searching JENKINS_HOME env variable
- Defines the SOURCE_DATE_EPOCH to enforce a stable date across builds
- Ask cmake not to use the git versioning which adds useless entropy for a temporary build (ENABLE_GIT_VERSION=OFF)
- Define the ccache slopiness to increase efficiency
- Increase the ccache size to save multiple builds to maximise cache hit between PRs
- Print ccache statistics to evaluate ccache efficiency

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-07-04 11:55:43 +02:00
Kefu Chai
eb615410d3 run-make-check: enable WITH_GTEST_PARALLEL
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-15 15:55:27 +02:00
Nathan Cutler
a7908a9147 build/ops: run-make-check.sh: handle Python 2 absence
Building Ceph without Python 2 requires passing certain options to cmake.

Fixes: http://tracker.ceph.com/issues/23035
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-04-23 10:50:01 +02:00
Chang Liu
1d3a67cf16 test: delete asok directories correctly
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2018-03-23 20:26:03 +08:00
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Igor Fedotov
939805165e test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-03-05 20:35:08 +03:00
Nathan Cutler
5200405bf0 build/ops: run-make-check.sh: fix SUSE support
Get /usr/bin/which from the right RPM.

Fixes: http://tracker.ceph.com/issues/22875
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-02-27 13:58:42 +01:00
Kefu Chai
3f8647a7e8 install-deps.sh: use DTS on centos if GCC is too old
please note, run-make-check.sh sources install-deps.sh here to import
the $PATH and other environmental variables, which could be changed by
the the DTS "enable" script.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-08 19:36:58 +08:00
yang.wang
4e16380ab7 run-make-check.sh: run ulimit without sudo
ulimit is a shell builtin like cd, not a separate program.
sudo looks for a binary to run, but there is no ulimit binary,
then you will get an error message like:
          "sudo: ulimit: command not found"
ulimit can only be run with root if you want raise the ulimit -n

Change-Id: I59ab09240cb15cabe5b0a7342c3f562cd50c75e9
Signed-off-by: yang.wang <yang.wang@easystack.cn>
2017-09-15 10:11:03 +08:00
Kefu Chai
30b5b4627c Merge pull request #16494 from asomers/bin_bash
misc: Fix bash path in shebangs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-27 10:14:14 +08:00
Dan Mick
9f3b1cf5ea run-make-check.sh: set ulimit -n (because no one else does)
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-02 21:12:13 -07:00
Alan Somers
3aae5ca6fd scripts: fix bash path in shebangs
/bin/bash is a Linuxism.  Other operating systems install bash to
different paths.  Use /usr/bin/env in shebangs to find bash.

Signed-off-by: Alan Somers <asomers@gmail.com>
2017-07-27 13:24:26 -06:00
Kefu Chai
10b88b5d82 test: create asok files in a temp directory under $TMPDIR
to shorten the pathname of unix domain socket created for admin socket,
so it does not exceed the limit of 107 on GNU/Linux:

* ceph-helper.sh: the temp directory is named ${TMPDIR:-/tmp}/ceph-asok.$$
* vstart.sh: the temp directory is named `mktemp -u -d "${TMPDIR:-/tmp}/ceph-asok.XXXXXX"`

Fixes: http://tracker.ceph.com/issues/16895
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-07-22 01:05:29 +08:00
Nathan Cutler
ec01d242cb build/ops: make sure which is installed in run-make-check.sh
Fixes: http://tracker.ceph.com/issues/20127
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-06-14 12:15:47 +02:00
Sage Weil
15e80fa2d8 Merge pull request #9853 from dx9/wip-alpine-dev-support
alpine: add alpine linux dev support
2017-04-21 18:54:25 -05:00
Nathan Cutler
71c6c8ebac build/ops: add sanity checks to run-make-check.sh
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-12-27 12:59:29 +01:00
John Coyle
3d0c778b54 run-make-check.sh: Init defaults after deps are installed.
get_processors() depends on coreutils nproc. Alpine does not install coreutils by default.

Signed-off-by: John Coyle <dx9err@gmail.com>
2016-12-20 20:14:16 -05:00
Nathan Cutler
dcfff6127c tests: check hostname --fqdn sanity before running make check
Fixes: http://tracker.ceph.com/issues/18134
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-12-03 23:37:08 +01:00
Sage Weil
b424a35f3a run-make-check.sh: use cmake exclusively
..and drop the separate cmake script

Signed-off-by: Sage Weil <sage@redhat.com>
2016-10-03 09:46:50 -04:00
Kefu Chai
c79ca73d17 Merge pull request #11105 from badone/wip-dnf-enable-build-scripts
build scripts: Enable dnf for Fedora >= 22

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-09-19 10:15:32 +08:00
Brad Hubbard
07a5a98982 build scripts: Enable dnf for Fedora >= 22
The use of yum triggers a warning and reference to yum2dnf on Fedora >= 22.
Use either yum or dnf based on the Fedora version.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-09-16 18:59:59 +10:00
Sage Weil
d61cfb5c71 Merge pull request #11092 from badone/wip-dry-run-run-make-check
run-make-check.sh: Make DRY_RUN actually do a dry run
2016-09-15 09:07:59 -05:00
Brad Hubbard
2d5326e16e run-make-check.sh: Make DRY_RUN actually do a dry run
The "cd build" command will cause an error during a dry run if the directory
does not exist.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-09-15 20:08:52 +10:00
Brad Hubbard
abe914467a run-*make-check.sh: Make DRY_RUN actually mean a dry run
DRY_RUN does not currently do an actual dry run as it actually runs an install
command.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-09-14 17:15:44 +10:00
John Coyle
a40815a4d5 run-make-check.sh: pass args to do_cmake.sh
Signed-off-by: John Coyle <dx9err@gmail.com>
2016-08-11 20:11:40 -04:00
Kefu Chai
1727fdd593 Revert "run-make-check.sh: run tests in two steps"
This reverts commit cd0dfc305d.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00
Kefu Chai
cd0dfc305d run-make-check.sh: run tests in two steps
this is a workaround of the timeout found in jenkins. currently three
tests are found timeout, and they are labeld with "Racing" and
"LongRunning". so, to workaround this issue, we run the tests in two
phases:

1. run the racing tests with -j1
2. run the non-racing tests with -jN

if we all all tests with -j1, the total test time is 2683.57 sec

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 12:00:46 +08:00
Kefu Chai
ce5724effb cmake: add a "tests" target to build tests
please note "make test" is used by cmake to run tests, so we cannot just
repurpose it to *build* them.

* AddCephTest.cmake: depends on "tests"
* CMakeLists.txt: let "check" depend on "tests"
* src/CMakeLists.txt: update the run-tox tests
* run-make-check.sh: use "make tests" and "ctest" instead of "make check"
* ceph-detect-init/CMakeLists.txt: let "tests" depend on
    "ceph-detect-init"
* ceph-disk/CMakeLists.txt: let "tests" depend on "ceph-disk"

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-16 10:57:12 +08:00
Kefu Chai
146d2b15ff run-make-check.sh: use "make check" for running tests
"make all" does not offer "ceph-disk" and "ceph-detect-init" for
testing. as they are solely used for testing purpose. instead, these two
python command line packages are installed by the "install" target. so
we need to use "make check" to 1) prepare the test dependencies 2)
launch ctest to perform the test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:41:57 +08:00
Kefu Chai
444d37f2b0 run-make-check.sh: run "make check" with a unique $TMPDIR
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-07 12:11:05 +08:00
Sage Weil
96f6fc0643 run-make-check.sh: run make check with cmake if do_cmake.sh is available
Signed-off-by: Sage Weil <sage@redhat.com>
2016-07-06 23:48:45 +08:00
Dan Mick
bf5ccce9ad run-make-check.sh: modprobe won't work on containers; not needed
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2016-06-29 22:26:24 +08:00
Loic Dachary
8f0eefba00 tests: rm -fr /tmp/*virtualenv*
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-05-20 15:06:45 +02:00