Commit Graph

68 Commits

Author SHA1 Message Date
Kefu Chai
5fc657b40d cmake: drop WITH_PYTHON2 option
* drop WITH_PYTHON2 option
* print warning message if WITH_PYTHON3 is disabled
* drop MGR_PYTHON_VERSION option, as we don't support use different
  python version for python binding and ceph-mgr embedded python
  interpreter anymore. as before switching to python3-only build,
  we can build python3 and python2 python bindings, and ceph-mgr
  can use either of them. but after switching to python3-only
  build, ceph-mgr has to use whatever python version used to
  build python binding.
* move WITH_PYTHON3 option to $top_srcdir/CMakeLists.txt, as ceph-mgr
  and python binding will share this option.
* hardware ${PYTHON_VERSION} to 3
* hardware ${Python${PYTHON_VERSION}_VERSION_MAJOR} to 3
* only build boost library with python3
* s/Python_EXECUTABLE/Python3_EXECUTABLE/
* update the build scripts and packagings accordingly
* rename all cython${PYTHON_VERSION}_* targets to cython_*
* update distutils_install_module() so it does not take python_version
  parameter anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-12-19 10:58:58 +08:00
Kefu Chai
5e9a1d95c9 run-make-check.sh: extract run-make.sh
so we can reuse run-make.sh for building the artifact used by other
tests than "make check", for instance, dashboard's E2E test and
crimson's performance test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-05 21:07:37 +08:00
Kefu Chai
3234fb3982 cmake,run-make-check.sh,deb,rpm: disable SPDK by default
but we still enable it in `run-make-check.sh`

* cmake: disable SPDK by default
* run-make-check.sh: enable WITH_SPDK so at least we can ensure it
  builds
* deb,rpm: add uuid-dev / libuuid-devel as a "make check" dependency

Fixes: https://tracker.ceph.com/issues/41330
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-20 15:38:33 +08:00
Kefu Chai
11f9aae4ca run-make-check.sh: run sudo with absolute path
there is chance that `/sbin` is not listed in `secure_path`, see
sudoers(5) for more details

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-20 10:51:36 +08:00
Varsha Rao
e90d58208d
cephfs-shell: Add tox for testing with flake8
Fixes: https://tracker.ceph.com/issues/39947
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-07-30 10:34:24 -07:00
Kefu Chai
694495f621 install-deps.sh: selectively enable deps on deb distros
on debian derivatives, only enable required dependencies on demand.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-17 02:27:07 +08:00
Nathan Cutler
a691122a4e build/ops: do_cmake.sh: automate py3 build options for certain distros
Before this patch, run-make-check.sh was setting the cmake options for
py3 build based only on the absence of a python2 binary in the system.
This was not sufficiently deterministic, and it didn't help folks who
run do_cmake.sh directly.

With this patch, we explicitly force the py3 build in do_cmake.sh, for
those OS types/versions we know need it.

Fixes: https://tracker.ceph.com/issues/37330
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-21 16:51:16 +01:00
Kefu Chai
79b55473ef run-make-check,do_cmake: set BOOST_J if !WITH_SYSTEM_BOOS
it's also more readable if we can set the number of make jobs
in a single place.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-11 22:22:47 +08:00
Kefu Chai
f0d7616b37 install-deps,run-make-check: use ceph-libboost repo
ceph-libboost1.67 repo is setup to satisfy the libboost dependencies
used when building ceph. now it offers arm64 and amd64 builds for xenial.

* install-deps.sh: add ceph-libboost1.67 repo hosted on chacra if the
  building host does not have any of the listed ceph-libboost1.67
  packages installed
* run-make-check.sh: add "-DWITH_SYSTEM_BOOST=ON
  -DBOOST_ROOT=/opt/ceph" if include/boost/config.hpp is found
  under /opt/ceph

See-also: http://tracker.ceph.com/issues/25186
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-11 22:22:47 +08:00
Kefu Chai
b2ef5ddb9a run-make-check: should use sudo for running sysctl
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-23 19:47:11 +08:00
Kefu Chai
b928c7777c run-make-check: increase fs.aio-max-nr to 1048576
the solution was suggested by Yingxin Cheng. otherwise we could have
EAGAIN returned by io_setup(2).

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-22 19:52:46 +08:00
Erwan Velu
d1a807a277 run-make-check: Showing configuration before the build
The actual code is resetting the statistics before doing the actual compilation and prints them after the build.

That is nice to understand how much the cache was used but doesn't help understanding how much it _could_ have been used.

This patch is adding a reporting (-s) when cleaning the statistics so we can estimate :
- the actual number of files in cache
- the actual size of the cache

With this two missing information, its now possible estimate if there is some miss-usage of the cache.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-08-17 10:16:58 +02:00
Kefu Chai
8bec1035ba run-make-check: drop CMAKE_PYTHON_OPTS variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-02 17:23:11 +08:00
Kefu Chai
cee0122780 cmake,run-make-check: always enable WITH_GTEST_PARALLEL
now that https://github.com/google/gtest-parallel/pull/63 has been
merged, we can now use gtest-parallel with py2 and also py3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-02 17:13:14 +08:00
Kefu Chai
45bca777aa run-make-check: set WITH_SEASTAR with a non-empty string
otherwise "test -n $WITH_SEASTAR" fails, and the dependencies won't be
installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-18 10:25:21 +08:00
Kefu Chai
f1f1526166 run-make-check.sh: enable WITH_SEASTAR
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-12 17:51:22 +08:00
Nathan Cutler
693beaa8f3 run-make-check.sh: fill in missing DRY_RUN and add doc hint
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:05 +02:00
Nathan Cutler
458a0a6e62 run-make-check.sh: reset trap after sourcing install-deps.sh
Since we source install-deps.sh, it replaces our trap with its own,
so we have to reinstate our trap after it finishes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
b79b463f90 cleanup: run-make-check.sh: no space between -D and name of cmake option
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
3d6909b0f3 run-make-check.sh: gtest-parallel only with Python 2
Fixes: http://tracker.ceph.com/issues/24817
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
f9c0691f84 run-make-check.sh: inform user of make -j setting
This can help the user figure out why the build is so slow.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
43fe00e90c doc: cleanup: wordsmith a comment in run-make-check.sh
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
2315928acd run-make-check.sh: ccache goodness for everyone
Since run-make-check.sh already ensures that ccache is installed,
it makes sense to let everyone benefit from the ccache
tweaks introduced by 4cb5a59053

Note 1: The previous solution using "date" would cause build tools to reset
their timestamps after 24 hours, on subsequent runs of run-make-check.sh.
In order to maximize ccache effectiveness, this commit sets SOURCE_DATE_EPOCH
to a fixed value: the number of seconds elapsed since the Unix epoch as at
January 1, 2000 (chosen to commemorate Y2K armageddon).

Note 2: this commit introduces "set -e". This was actually in effect
before, via "source install-deps.sh". Better to make it explicit.

Fixes: http://tracker.ceph.com/issues/24777
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:01 +02:00
Nathan Cutler
c8691cd9a3 run-make-check.sh: drop workaround for undeclared jq build dependency
jq build dependency is properly declared now.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:25:40 +02:00
Erwan Velu
4cb5a59053 run-make-check.sh: Adding ccache tuning for the CI
When run-make-check is run by the CI, some tuning can be performed to
speedup the build.

This commit :
- Detect jenkins by searching JENKINS_HOME env variable
- Defines the SOURCE_DATE_EPOCH to enforce a stable date across builds
- Ask cmake not to use the git versioning which adds useless entropy for a temporary build (ENABLE_GIT_VERSION=OFF)
- Define the ccache slopiness to increase efficiency
- Increase the ccache size to save multiple builds to maximise cache hit between PRs
- Print ccache statistics to evaluate ccache efficiency

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-07-04 11:55:43 +02:00
Kefu Chai
eb615410d3 run-make-check: enable WITH_GTEST_PARALLEL
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-06-15 15:55:27 +02:00
Nathan Cutler
a7908a9147 build/ops: run-make-check.sh: handle Python 2 absence
Building Ceph without Python 2 requires passing certain options to cmake.

Fixes: http://tracker.ceph.com/issues/23035
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-04-23 10:50:01 +02:00
Chang Liu
1d3a67cf16 test: delete asok directories correctly
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2018-03-23 20:26:03 +08:00
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Igor Fedotov
939805165e test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-03-05 20:35:08 +03:00
Nathan Cutler
5200405bf0 build/ops: run-make-check.sh: fix SUSE support
Get /usr/bin/which from the right RPM.

Fixes: http://tracker.ceph.com/issues/22875
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-02-27 13:58:42 +01:00
Kefu Chai
3f8647a7e8 install-deps.sh: use DTS on centos if GCC is too old
please note, run-make-check.sh sources install-deps.sh here to import
the $PATH and other environmental variables, which could be changed by
the the DTS "enable" script.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-12-08 19:36:58 +08:00
yang.wang
4e16380ab7 run-make-check.sh: run ulimit without sudo
ulimit is a shell builtin like cd, not a separate program.
sudo looks for a binary to run, but there is no ulimit binary,
then you will get an error message like:
          "sudo: ulimit: command not found"
ulimit can only be run with root if you want raise the ulimit -n

Change-Id: I59ab09240cb15cabe5b0a7342c3f562cd50c75e9
Signed-off-by: yang.wang <yang.wang@easystack.cn>
2017-09-15 10:11:03 +08:00
Kefu Chai
30b5b4627c Merge pull request #16494 from asomers/bin_bash
misc: Fix bash path in shebangs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-27 10:14:14 +08:00
Dan Mick
9f3b1cf5ea run-make-check.sh: set ulimit -n (because no one else does)
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2017-08-02 21:12:13 -07:00
Alan Somers
3aae5ca6fd scripts: fix bash path in shebangs
/bin/bash is a Linuxism.  Other operating systems install bash to
different paths.  Use /usr/bin/env in shebangs to find bash.

Signed-off-by: Alan Somers <asomers@gmail.com>
2017-07-27 13:24:26 -06:00
Kefu Chai
10b88b5d82 test: create asok files in a temp directory under $TMPDIR
to shorten the pathname of unix domain socket created for admin socket,
so it does not exceed the limit of 107 on GNU/Linux:

* ceph-helper.sh: the temp directory is named ${TMPDIR:-/tmp}/ceph-asok.$$
* vstart.sh: the temp directory is named `mktemp -u -d "${TMPDIR:-/tmp}/ceph-asok.XXXXXX"`

Fixes: http://tracker.ceph.com/issues/16895
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-07-22 01:05:29 +08:00
Nathan Cutler
ec01d242cb build/ops: make sure which is installed in run-make-check.sh
Fixes: http://tracker.ceph.com/issues/20127
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2017-06-14 12:15:47 +02:00
Sage Weil
15e80fa2d8 Merge pull request #9853 from dx9/wip-alpine-dev-support
alpine: add alpine linux dev support
2017-04-21 18:54:25 -05:00
Nathan Cutler
71c6c8ebac build/ops: add sanity checks to run-make-check.sh
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-12-27 12:59:29 +01:00
John Coyle
3d0c778b54 run-make-check.sh: Init defaults after deps are installed.
get_processors() depends on coreutils nproc. Alpine does not install coreutils by default.

Signed-off-by: John Coyle <dx9err@gmail.com>
2016-12-20 20:14:16 -05:00
Nathan Cutler
dcfff6127c tests: check hostname --fqdn sanity before running make check
Fixes: http://tracker.ceph.com/issues/18134
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2016-12-03 23:37:08 +01:00
Sage Weil
b424a35f3a run-make-check.sh: use cmake exclusively
..and drop the separate cmake script

Signed-off-by: Sage Weil <sage@redhat.com>
2016-10-03 09:46:50 -04:00
Kefu Chai
c79ca73d17 Merge pull request #11105 from badone/wip-dnf-enable-build-scripts
build scripts: Enable dnf for Fedora >= 22

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-09-19 10:15:32 +08:00
Brad Hubbard
07a5a98982 build scripts: Enable dnf for Fedora >= 22
The use of yum triggers a warning and reference to yum2dnf on Fedora >= 22.
Use either yum or dnf based on the Fedora version.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-09-16 18:59:59 +10:00
Sage Weil
d61cfb5c71 Merge pull request #11092 from badone/wip-dry-run-run-make-check
run-make-check.sh: Make DRY_RUN actually do a dry run
2016-09-15 09:07:59 -05:00
Brad Hubbard
2d5326e16e run-make-check.sh: Make DRY_RUN actually do a dry run
The "cd build" command will cause an error during a dry run if the directory
does not exist.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-09-15 20:08:52 +10:00
Brad Hubbard
abe914467a run-*make-check.sh: Make DRY_RUN actually mean a dry run
DRY_RUN does not currently do an actual dry run as it actually runs an install
command.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2016-09-14 17:15:44 +10:00
John Coyle
a40815a4d5 run-make-check.sh: pass args to do_cmake.sh
Signed-off-by: John Coyle <dx9err@gmail.com>
2016-08-11 20:11:40 -04:00
Kefu Chai
1727fdd593 Revert "run-make-check.sh: run tests in two steps"
This reverts commit cd0dfc305d.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 16:30:24 +08:00