Commit Graph

42 Commits

Author SHA1 Message Date
Evgeny Vereshchagin 33621cb7c8 libsepol/cil: move the fuzz target and build script to the selinux repository
It should make it easier to reproduce bugs found by OSS-Fuzz locally
without docker. The fuzz target can be built and run with the corpus
OSS-Fuzz has accumulated so far by running the following commands:
```
./scripts/oss-fuzz.sh
wget https://storage.googleapis.com/selinux-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/selinux_secilc-fuzzer/public.zip
unzip -d CORPUS public.zip
./out/secilc-fuzzer CORPUS/
```

It was tested in https://github.com/google/oss-fuzz/pull/6026
by pointing OSS-Fuzz to the branch containing the patch and
running all the tests with all the sanitizers and fuzzing engines
there: https://github.com/google/oss-fuzz/actions/runs/1024673143

[v2]
[1] oss-fuzz: make shellcheck happy

[2] oss-fuzz: build libsepol only

The fuzz target covers libsepol so it's unnecessary to build everything
else. Apart from that, the "LDFLAGS" kludge was removed since libsepol
is compatible with the sanitizers flags passed via CFLAGS only. It
should be brought back one way or another eventually though to fix
build failures like
```
clang -L/home/vagrant/selinux/selinux/DESTDIR/usr/lib -L/home/vagrant/selinux/selinux/DESTDIR/usr/lib -L../src  sefcontext_compile.o ../src/regex.o  -lselinux  -lpcre  ../src/libselinux.a -lsepol -o sefcontext_compile
/usr/bin/ld: sefcontext_compile.o: in function `usage':
/home/vagrant/selinux/selinux/libselinux/utils/sefcontext_compile.c:271: undefined reference to `__asan_report_load8'
/usr/bin/ld: /home/vagrant/selinux/selinux/libselinux/utils/sefcontext_compile.c:292: undefined reference to `__asan_handle_no_return'
/usr/bin/ld: sefcontext_compile.o: in function `asan.module_ctor':
```

[3] oss-fuzz: make it possible to run the script more than once
by removing various build artifacts

[4] oss-fuzz: make it possible to run the script from any directory

[5] oss-fuzz: be a little bit more specific about what the script does

[6] oss-fuzz: stop overwriting all the Makefiles

Signed-off-by: Evgeny Vereshchagin <evvers@ya.ru>
Acked-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2021-08-17 10:33:47 -04:00
Christian Göttsche 54667cdb37 scripts/run-scan-build: update
- use multiple jobs
- define _FORTIFY_SOURCE=2 to enable checks on standard string handling
  functions due to macro/intrinsic overloads or function attributes
- allow to override clang and scan-build binaries, i.e. for using
  versioned ones
- set PYTHON_SETUP_ARGS accordingly on Debian
- enable common warning -Wextra
- print build result

Signed-off-by: Christian Göttsche <cgzones@googlemail.com>
2021-07-19 10:42:45 -04:00
Ondrej Mosnacek eb6e97ab7d scripts/ci: use F34 image instead of F33
Now that F34 has been released, it's time to update the CI Vagrantfile
to use the new Fedora version. This also fixes the failure in the
recently added vsock_socket test that depends on a bugfix, which made it
to the F34 image's kernel, but is not in the F33 image's.

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
Acked-by: Petr Lautrbach <plautrba@redhat.com>
2021-05-18 10:05:03 +02:00
Nicolas Iooss 108c8edd95 scripts/release: make the script more robust, and release a source repository snapshot
Following Petr Lautrbach's suggestion, release a snapshot of the source
repository next to the individual archives which constitute a release.

While at it, make scripts/release more robust:

- Fix many warnings reported by shellcheck, by quoting strings.
- Use bash arrays for DIRS and DIRS_NEED_PREFIX
- Merge DIRS and DIRS_NEED_PREFIX into a single array, in order to
  produce SHA256 digests that are directly in alphabetical order, for
  https://github.com/SELinuxProject/selinux/wiki/Releases
- Use "set -e" in order to fail as soon as a command fails
- Change to the top-level directory at the start of the script, in order
  to be able to run it from anywhere.
- Use `cat $DIR/VERSION` and `git -C $DIR` instead of `cd $i ; cat VERSION`
  in order to prevent unexpected issues from directory change.

Finally, if version tags already exists, re-use them. This enables using
this script to re-generate the release archive (and check that they
really match the git repository). Currently, running scripts/release
will produce the same archives as the ones published in the 3.2-rc1
release (with the same SHA256 digests as the ones on the release page,
https://github.com/SELinuxProject/selinux/wiki/Releases). This helps to
ensure that the behaviour of the script is still fine.

Suggested-by: Petr Lautrbach <plautrba@redhat.com>
Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2021-02-03 11:26:07 +01:00
Nicolas Iooss ce64c473e3 scripts/ci: add configuration for a Vagrant virtual machine
Using Vagrant makes reproducing and debugging CI issues easier: after
"vagrant up", a test virtual machine is up and running, and ready to run
"fedora-test-runner.sh". In order to make using this VM even easier, a
helper script, "run-selinux-test.sh" is created inside and instructions
on how to use it are documented at the beginning of Vagrantfile.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2020-12-08 09:12:44 +01:00
Hu Keping f5f63035fa Simplify the tarball generating scripts
Since the format of git archive is inferred from the output file,
it's safe to remove the pipe for gzip.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2020-12-02 08:48:48 +01:00
Hu Keping f63ac245f7 Use X.Y instead of date for release tag
As per discussed at github issue[1] and mailing list[2], we consider
a version format of X.Y starting at 3.2.

[1]: https://github.com/SELinuxProject/selinux/issues/270
[2]: https://lore.kernel.org/selinux/87d004wan2.fsf@redhat.com/T/#u

Signed-off-by: Hu Keping <hukeping@huawei.com>
2020-12-02 08:48:48 +01:00
Ondrej Mosnacek fe985a8c84 travis: run only selinux-testsuite
Now that the standard testing is run on GitHub Actions, we can remove it
from .travis.yml, leaving only the selinux-testsuite sanity check, which
can't be migrated. Hopefully this will save some precious minutesfrom
the limited "plan" Travis gave us...

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2020-11-27 15:37:31 +01:00
Ondrej Mosnacek e081620812
ci: add new dependencies needed by selinux-testsuite
The testsuite now runs filesystem tests also on other filesystems than
just the default ext4. [1] That means a few more userspace utilities are
needed to format these filesystems. [2]

[1] 071ec9c5e5
[2] d4e507f78a

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2020-11-12 21:24:00 +01:00
Ondrej Mosnacek 3de445af0b
ci: bump Fedora image version to 33
The testsuite will soon be switching to testing multiple filesystems,
which exposes a bug in F32 image's kernel. Since Fedora 33 has been
released recently and the testsuite runs just fine on it, just bump the
image version to avoid the bug.

This commit also fixes the script to read out the Fedora image version
from environment variables instead of using hard-coded values.

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2020-11-11 20:40:15 +01:00
William Roberts 071264c6c4 ci: fix stall on git log -1
git log -1 may use a pager to output long messages, and when the pager
is invoked, leads to stalls on the ci system waiting for user input.
Use --oneline to print the short part of the commit message and
the digest. This information is for debug/informational purposes only,
so truncating the output is sufficient.

Reported-by: Stephen Smalley <sds@tycho.nsa.gov>
Signed-off-by: William Roberts <william.c.roberts@intel.com>
2020-08-11 17:01:04 -05:00
William Roberts 91cc86d44e scripts/ci: license as MIT
License the ci scripts with a permissive, OSI approved license, such as
MIT.

Signed-off-by: William Roberts <william.c.roberts@intel.com>
[omosnace: removed the dummy copyright header from LICENSE]
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2020-08-07 10:20:10 +02:00
William Roberts 1af345d222 ci: dont use hardcoded project name
Not everyone's github project is "selinux" so use the projects
name, as derived from TRAVIS_BUILD_DIR. TRAVIS_BUILD_DIR is
the absolute path to the project checkout on disk, so the
basename should be sufficient. The script that runs in the KVM
environment also needs to be updated where it can find the
selinux project code, so we pass it in through an env variable
in the ssh command.

Tested on Travis CI here:
  - https://travis-ci.org/github/williamcroberts/selinux/jobs/697307824

Signed-off-by: William Roberts <william.c.roberts@intel.com>
Acked-by: Petr Lautrbach <plautrba@redhat.com>
2020-06-18 19:22:37 +02:00
William Roberts 562d6d1527 ci: run SELinux kernel test suite
The current Travis CI runs the userspace tooling and libraries against
policy files, but cannot test against an SELinux enabled kernel. Thus,
some tests are not being done in the CI. Travis, unfortunately only
provides Ubuntu images, so in order to run against a modern distro with
SELinux in enforcing mode, we need to launch a KVM with something like
Fedora.

This patch enables this support by launching a Fedora32 Cloud Image with
the SELinux userspace library passed on from the Travis clone, it then
builds and replaces the current SELinux bits on the Fedora32 image and
runs the SELinux testsuite.

The cloud image run can be controlled with the TRAVIS env variable:
TRAVIS_CLOUD_IMAGE_VERSION. That variable takes the major and minor
version numbers in a colon delimited string, eg: "32:1.6".

Signed-off-by: William Roberts <william.c.roberts@intel.com>
Acked-by: Stephen Smalley <stephen.smalley.work@gmail.com>
2020-06-18 19:22:12 +02:00
Petr Lautrbach d7b0207c5a run-flake8: Filter out ./.git/ directory
When a branch has '.py' suffix git creates a file with the same suffix and this
file is found by the `find . -name '*.py'` command. Such files from './git' need
to be filtered out.

Fixes:

    $ PATH="$VIRTUAL_ENV/bin:$PATH" ./scripts/run-flake8
    Analyzing 189 Python scripts
    ./.git/logs/refs/heads/semanage-test.py:1:42: E999 SyntaxError: invalid syntax
    ./.git/refs/heads/semanage-test.py:1:4: E999 SyntaxError: invalid syntax
    The command "PATH="$VIRTUAL_ENV/bin:$PATH" ./scripts/run-flake8" exited with 1.

Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
Acked-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2020-05-13 11:11:27 +02:00
Nicolas Iooss 1baa8ffadf scripts/env_use_destdir: propagate PREFIX, LIBDIR, BINDIR, etc.
On systems using non-default `PREFIX`, `LIBDIR`, `SHLIBDIR`, `BINDIR`
or `SBINDIR`, running
`DESTDIR=/path/to/destdir ./scripts/env_use_destdir make test`
does not perform the intended behavior, because the testing programs and
libraries are installed into locations that are not added to
`LD_LIBRARY_PATH` nor `PATH`.

More precisely, with `LIBDIR=/usr/lib64 SHLIBDIR=/lib64`, `env_use_destdir`
does not work. Fix this by adding the installation directories relative
to `DESTDIR` in `LD_LIBRARY_PATH` and `PATH`.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
Acked-by: Petr Lautrbach <plautrba@redhat.com>
2020-05-05 15:19:05 +02:00
Nicolas Iooss e505694406 scripts/env_use_destdir: fix Fedora support
libselinux and libsemanage use:

    PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig
    import *; print(get_python_lib(plat_specific=1,
    prefix='$(PREFIX)'))")

while python/semanage and python/sepolgen/src/sepolgen use:

    PYTHONLIBDIR ?= $(shell $(PYTHON) -c "from distutils.sysconfig
    import *; print(get_python_lib(prefix='$(PREFIX)'))")

This is right: libselinux and libsemanage's Python bindings use native
code (thus "plat_specific=1") while the others only install Python
files.

Nevertheless `scripts/env_use_destdir` only runs the second command
when computing `$PYTHONPATH`. When using this script to run `make test`
in a minimal Fedora 31 environment, this leads to an error such as:

    make[2]: Entering directory '/code/python/sepolicy'
    Traceback (most recent call last):
      File "test_sepolicy.py", line 117, in <module>
        import selinux
    ModuleNotFoundError: No module named 'selinux'

Fix this by also adding `get_python_lib(plat_specific=1)` to the
computed `$PYTHONPATH`.

While at it, preserve `$PYTHONPATH` instead of resetting it. This makes
it easier to work with Python virtual environments.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2020-05-05 15:18:46 +02:00
Nicolas Iooss 120681c1a3 libsepol, libsemanage: add a macro to silence static analyzer warnings in tests
Several static analyzers (clang's one, Facebook Infer, etc.) warn about
NULL pointer dereferences after a call to CU_ASSERT_PTR_NOT_NULL_FATAL()
in the test code written using CUnit framework. This is because this
CUnit macro is too complex for them to understand that the pointer
cannot be NULL: it is translated to a call to CU_assertImplementation()
with an argument as TRUE in order to mean that the call is fatal if the
asserted condition failed (cf.
http://cunit.sourceforge.net/doxdocs/group__Framework.html).

A possible solution could consist in replacing the
CU_ASSERT_..._FATAL() calls by assert() ones, as most static analyzers
know about assert(). Nevertheless this seems to go against CUnit's API.

An alternative solution consists in overriding CU_ASSERT_..._FATAL()
macros in order to expand to assert() after a call to the matching
CU_ASSERT_...() non-fatal macro. This appears to work fine and to remove
many false-positive warnings from various static analyzers.

As this substitution should only occur when using static analyzer, put
it under #ifdef __CHECKER__, which is the macro used by sparse when
analyzing the Linux kernel.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2019-09-30 08:43:41 -04:00
Petr Lautrbach 1952be65dc Switch to python3 by default
- Python 2.7 is planned to be the last of the 2.x releases
- It's generally advised to use Python 3
- Majority of python/ scripts are already switched python3
- Users with python 2 only can still use:

$ make PYTHON=/usr/bin/python ....

Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
2019-02-20 16:43:27 +01:00
Petr Lautrbach f0f68ab2ff scripts/release: Update links to use release assets instead of wiki links
- new release files are created in release/$RELEASE_TAG
- download links refers to new release assets

Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
2019-02-05 09:56:34 +01:00
Nicolas Iooss 9e0ed5ce17 scripts: introduce env_use_destdir.sh helper
Set-up environment variables in order to use DESTDIR in Python, Ruby,
etc. This makes testing Python scripts easier.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2019-01-21 12:13:41 +01:00
Nicolas Iooss bb518a01e9 scripts/run-flake8: run on Python scripts not ending with .py
When running flake8 on a directory, it does not analyze files without an
extension, like semanage_migrate_store, mlscolor-test, etc. Use grep to
find files with a Python shebang and build a list which is then given to
flake8.

This commit is possible now that some clean-up patches have been
applied, such as commit 69c56bd2f6 ("python/chcat: improve the code
readability") and b7227aaec1 ("mcstrans: fix Python linter warnings on
test scripts") and 3cb974d2d2 ("semanage_migrate_store: fix many
Python linter warnings").

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2019-01-08 13:21:09 +01:00
Nicolas Iooss 691231e612 python/sepolgen: upgrade ply to release 3.11
PLY (Python Lex-Yacc) 3.11 has been released in February 2018:
- http://www.dabeaz.com/ply/index.html
- https://github.com/dabeaz/ply/releases/tag/3.11

Copy lex.py and yacc.py from this new release.

This fixes the following warning from "make test":

    python run-tests.py
    ../src/./sepolgen/lex.py:634: DeprecationWarning: Using or importing
    the ABCs from 'collections' instead of from 'collections.abc' is
    deprecated, and in 3.8 it will stop working
     if isinstance(t, collections.Callable):

(Python 3.3 moved collections.Callable to collections.abc.Callable)

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2019-01-04 13:52:09 +01:00
Nicolas Iooss 6f01778406
python: remove semicolon from end of lines
Python does not need to end a statement with a semicolon. Doing this
gets reported by linters such as flake8 ("E703 statement ends with a
semicolon").

Remove such semicolons in the code and enable this warning in
scripts/run-flake8.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2018-08-19 17:55:19 +02:00
Nicolas Iooss 41764b73a7
Travis-CI: run flake8 on Python code
flake8 is a Python linter which is able to detect issues in Python code
(syntax errors, undefined variables, etc.). It has been used to find
bugs in the project. In order to prevent the introduction of new bugs
which can be detected by it, add a script which runs it and use it in
Travis-CI.

flake8 can be used to detect code which is not written according to PEP8
style guide (which forbids whitespaces in some places, enforces the use
of space-indenting, specifies how many blank lines are used between
functions, etc.). As SELinux code does not follow this style guide,
scripts/run-flake8 disables many warnings related to this when running
the linter.

In order to silence flake8 warnings, the Python code can also be
modified. However fixing every "do not use bare 'except'" in the project
needs to be done carefully and takes much time.
This is why the warnings which are disabled have been ordered in three
lists:
* The warnings which can be activated in a not-so-distant future after
  the code has been modified.
* The warnings related to PEP8 which cannot be activated without a major
  cleaning work of the codebase (for example to modify white spaces)
* The warnings which are introduced by code generated by SWIG 3.0.12,
  which would require patches in SWIG in order to be activated (there
  is right now only one such warning).

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2018-08-18 12:00:24 +02:00
Nicolas Iooss 416900cb11 scripts: add a helper script to run clang's static analyzer
Using clang's static analyzer is as simple as running "scan-build make",
but in order to obtain clean and reproducible results, the build
environment has to be cleaned beforehand ("make clean distclean").

Moreover the project requires running "make install" before "make test"
in order to install the dependencies needed for the tests, and running
these tests with the newly-built libraries requires a specific
LD_LIBRARY_PATH. This new script takes care of setting up everything
which is needed.

Signed-off-by: Nicolas Iooss <nicolas.iooss@m4x.org>
2018-06-15 09:03:17 -04:00
Stephen Smalley f0e61d336f Fix release script
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2016-11-16 11:19:51 -05:00
Stephen Smalley f0cc95435f Fix release script for packages that need prefixes.
We need prefixes for dbus, gui, python at least, and
we'll include sandbox too.

Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2016-11-16 11:19:51 -05:00
Stephen Smalley 1c8505dac2 Update release script for the new structure.
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2016-11-16 11:19:51 -05:00
Stephen Smalley 1a27b5add9 scripts/release: cleanups
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2016-10-14 11:47:29 -04:00
Stephen Smalley b5f2652a11 Fix release script
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2016-10-14 11:40:16 -04:00
Stephen Smalley b8390a6c75 Updated release script 2016-10-06 12:45:18 -04:00
Stephen Smalley 1b7f3248ec Add secilc to release script.
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2015-03-31 12:41:28 -04:00
Steve Lawrence 5b9a172bf9 scripts: update release script to not output file name twice
Signed-off-by: Steve Lawrence <slawrence@tresys.com>
2014-12-03 10:06:26 -05:00
Steve Lawrence 51516db96b Update release script for github 2014-08-27 11:47:04 -04:00
Stephen Smalley f64918ff71 Add make-update script and fix release script.
make-update script is for generating an update for an existing release,
e.g. for a bug fix update.

release script is for generating a new release.

Per-package tagging may fail on the release script if the version
has not changed; this is harmless.

Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2013-10-31 14:34:02 -04:00
Stephen Smalley c525be35fb Update to latest Lindent script from kernel tree.
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2013-10-30 14:34:25 -04:00
Stephen Smalley 6abcb4f293 Drop obsolete script. 2013-10-30 14:33:51 -04:00
Stephen Smalley 75740f4f34 Add tagging and wiki section generation to the release script.
Tag the release before generating it, and automatically generate
a template for use in updating the wiki page.  Also, make the
output directory unique to each release, delete any prior contents,
and tell the user where to find it.

Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
2013-10-30 14:28:30 -04:00
Eric Paris 3a4fc087ee scripts: release: do not complain if release dir exists
I just don't like the error message when building tar files.

Signed-off-by: Eric Paris <eparis@redhat.com>
2013-02-05 20:19:03 -05:00
Joshua Brindle ff317eb616 release script 2009-03-12 01:23:32 -04:00
Joshua Brindle 13cd4c8960 initial import from svn trunk revision 2950 2008-08-19 15:30:36 -04:00