Several class annotations were subscripted but pylint had a bug that did
not handle this. This is now fixed in pylint 2.8+.
Signed-off-by: Chris PeBenito <pebenito@ieee.org>
Includes some minor code changes to fix errors from static type checking.
Disable unsubscriptable-object pylint check on Wrapper subclass
declarations, as this hits the bug described in PyCQA/pylint#2822.
Signed-off-by: Chris PeBenito <pebenito@ieee.org>
Switch dict.items() use to dict.values() since the keys weren't being used.
Also remove an unused parameter in _av_generate_diffs().
Signed-off-by: Chris PeBenito <pebenito@ieee.org>
The primary motivation for the change is to correctly handle redundant
rules. Recent changes in the SELinux toolchain added support for an
optimization that removes redundant rules from a policy. These are
conditional rules that are either already specified in unconditional
policy or rules using types that are also specified more generally
through an attribute. Since attributes are always expanded in sediff,
the second type of redundant rules are already effectively removed. But
redundant conditional rules show up as differences when a binary version
of a policy that has been optimized is compared to one that has not been.
A secondary motivation for the change is to reduce memory usage and diff
times. A modern Fedora policy cannot be diffed with a system with less than
32Gb of memory and it takes over four hours to complete.
With this change AV rules are processed by creating a data structure which
consists of nested dictionaries that store BOTH the left and the right
policies. All of the keys are interned strings to save space.
The basic structure is
rule_db[cond_exp][block_bool][src][tgt][tclass]=sides
where:
cond_exp is a boolean expression
block_bool is either true or false
src is the source type
tgt is the target type
tclass is the target class
sides is a namedtuple with "left" and "right" attributes
Each side is either None or another namedtuple with "perms" and
"orig_rule" attributes
perms is the set of permissions for this rule
orig_rule is the original unexpanded rule
These changes improve diff times and memory usage.
Without the change
Time Memory Usage
Older Fedora Policy 3 min 17 sec 4.5Gb
Recent Refpolicy 4 min 19 sec 6.0Gb
Recent Fedora Policy 4 hrs 9 min 31.9Gb
With the change
Time Memory Usage
Older Fedora Policy 28 sec 1.7Gb
Recent Refpolicy 34 sec 1.9Gb
Recent Fedora Policy 4 min 45 sec 7.7Gb
Also added a unit test to check that redundant rules are removed.
Signed-off-by: James Carter <jwcart2@tycho.nsa.gov>
Record the rule list and expanded rule statistics for determining if
memory size issues are due to rule size vs. something else.
Signed-off-by: Chris PeBenito <pebenito@ieee.org>
Change to error logging so the processing continues, as the error is not
fatal. Also specify in the message that it is a bug, so users know it
isn't their fault.
