For log_normal and DIFF_FATAL messages, prefix them with the object name
to give more context, which is useful for patches which change multiple
objects. Also, no need to add the function and line number to
DIFF_FATAL messages, as the error strings already give enough
information.
Example messages:
meminfo.o: changed function: meminfo_proc_show
cmdline.o: no changed functions were found
This reverts commit ab29b1ff59.
Reverting this commit because it causes kpatch-build to ignore any
errors reported by create-diff-object, treating all errors as meaning
that no changes occurred, which is a dangerous assumption to make.
Consider following patch: https://lkml.org/lkml/diff/2014/1/7/637/1
Kpatch-build will generate two objects for it. however mlock.o has no
changed function and will cause kpatch-build die.
Signed-off-by: Madper Xie <cxie@redhat.com>
Create the applied-patch file only after the patch has been verified.
Otherwise if you accidentally supply a patch which had already been
applied to the source, the cleanup trap won't reverse apply it when
exiting the script.
If the patch had already been mistakenly applied to the source tree,
don't ask the user if it should be reverse applied. Instead, just exit
with an error.
Cleanup the kpatch-build argument parsing a little bit:
- gracefully handle no args
- allow white space in filenames
- use 'eval set -- $options' to allow use of $1 and $2 variables
When debugging kpatch-build failures it can be
beneficial to have the scratch files in /tmp that
kpatch-build was operating on. These are
removed by default, as they can quickly fill /tmp.
However, for debugging reasons, the option should
exist to keep them around.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
When CONFIG_MODVERSIONS is enabled, loading of the patch module fails
with "no symbol version for kpatch_register". When building the patch
module, we need to point it to the core module's Module.symvers file.
This also works when CONFIG_MODVERSIONS is disabled, since
Module.symvers is created regardless.
There are many cases where a section may have
changed due to soure-level change but the inclusion
logic has not selected it for output. Some of these
cases are real no-go situations like changing data
structures. Some are just situations that
create-diff-object isn't smart enough to figure out
(yet).
Either way, it should be considered fatal when a
changed section hasn't been selected for output.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
In some environments, some files in scripts/mod (devicetable-offsets.s
and file2alias.o) are always getting rebuilt, and thus get incorrectly
added to the changed_objs file, resulting in the following error:
strip:/root/.kpatch/3.10.0-115.el7.x86_64/obj2/scripts/mod/devicetable-offsets.s: File format not recognized
The indexes are in order when being read from the
table. Just index directly into the table; a benefit
of using an array for this structure instead of a linked
list.
Removes another hot path during the rela table initialization.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
remove "ughs" by changing macro to start at symbol
index 1. new for_each_symbol_zero will start at zero
for rare cases that need it.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Upon realizing that there is no point in correlating rela entries,
I also realized that tracking the status of rela entries is also
not needed.
Additionally, the rela section correlation path (really misnamed
as it is the rela section _comparison path) is VERY hot. Particularly
on files like fs/ext4/ext4.o (which create-diff-object currently can't
successfully parse entirely):
Samples: 40K of event 'cycles', Event count (approx.): 36516578362
49.49% create-diff-obj create-diff-object [.] rela_equal
31.85% create-diff-obj create-diff-object [.] kpatch_correlate_relas
16.22% create-diff-obj create-diff-object [.] find_symbol_by_index
The refactor does a few things:
- replaces nested for loops with single for loop when comparing rela entries
- removes status field for rela entires
- compares rela and nonrela sections in the same path
- removes unnecessary setting of status fields as the inclusion tree
will include them even if the section status isn't set to CHANGED. This is
even better as unchanged sections won't appear as CHANGED just because
their partner .text or .rela section is CHANGED.
This drastically reduced runtime for larger objects and cooled the rela
comparison path:
87.64% create-diff-obj create-diff-object [.] find_symbol_by_index
6.98% create-diff-obj libc-2.18.so [.] __GI___strcmp_ssse3
1.33% create-diff-obj create-diff-object [.] find_section_by_index
1.16% create-diff-obj create-diff-object [.] kpatch_correlate_symbols
0.61% create-diff-obj create-diff-object [.] kpatch_create_rela_table
0.52% create-diff-obj create-diff-object [.] kpatch_correlate_sections
Signed-off-by: Seth Jennings <sjenning@redhat.com>
I've created a test designed to exercise the ability
of create-diff-object to parse, compare, and return
"no changed functions" for the entire kernel source
tree one file at a time by passing the same file as
both the original and patched file.
Through this process, I realized that excluding every
special case from being bundled it not feasible. There
are many sections in the kernel that don't honor
-ffunction|data-sections, not just __ksymtab_strings and
.init.text.
Plus, excluding situations is not the best way. We are
really only looking for sections that _were_ the result
of -f[function|data]-sections for bundling.
To that end, this commit looks to bundle only symbol/section
pairs that should be bundled ensuringthe .text/.data suffix
and the FUNC/OBJECT symbol name match.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
A local changed function will not appear with a "changed function"
notice if it is a dependency of another changed function and that
function occurs before it in the symbol table.
Rearrange some logic to print the notice regardless of whether or not
the function symbols has already been selected for inclusion.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
When linking function bundles (.text/.rela/section sym/func sym)
ignore __init functions as they do not honor -ffunction-sections
and violate the one-to-one func/section assumption of the
function bundling.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
In some circumstances, the kernel is built outside from the source tree,
thus we should specify the .config file of build path.
This patch changes the basic option parsing by using getopt, and add more
information in usage().
Signed-off-by: Jincheng Miao <jincheng.miao@gmail.com>
Create a new function kpatch_copy_symbols() that copies symbols from
one kelf to another if the "select" function to return true for the
symbol.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
ld drops FUNC syms that appear after the last SECTION
sym in the symbol table.
Make sure we order the FUNC syms before the SECTION syms.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Like in add-patches-section, we should continue to
search the vmlinux symbol table to see if there
is a duplicate global symbol. This normally would
not be possible, however, because create-diff-object
modifies unchanged local functions to be global
as a hack so they can be linked, there is a chance
that these symbols could collide with an existing
global symbol. We should detect this case and error
out.
Hopefully we can figure out a way to avoid this
situation altogether. But for now, this is a
protection against improper linking.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
My apologies for the size of this commit. I combined these two features
(updating API and using a hash table) into a single commit because their
implementations are tightly coupled and I didn't want to have to add
support for the old kpatch_funcs array with the new API just for the
sake of splitting up the commit :-)
- Update the core module API to get a more clear separation between core
module and patch module. This is cleaner and will help our case for
getting the core module merged upstream into the kernel.
- Convert the old kpatch_funcs array into a hash table. This is so much
nicer performance-wise and everything-else-wise than that ugly old
array.
- Do the incremental patching in stop machine. This ensures that the
funcs hash is up to date and we don't miss anything.
- Disable preemption in the ftrace handler when accessing the func hash.
That way we don't get conflicts with the stop_machine handler updating
the hash.
Currently, add-patches-section just blindly looks in vmlinux
for a function symbol matching the name of the patched function
in the input object file. However, for local symbols, they may
appear multiple times in the vmlinux symbol table since the symbol
name may be reused locally in different files.
This commit add support for "file hinting". It tracks what
file the symbol is in and searches for local symbols within
that file in vmlinux first. If it doesn't find one, it then
searches globally like it always has.
Fixes issue #53
Signed-off-by: Seth Jennings <sjenning@redhat.com>
The current solution doesn't work because "$?" will always be 0, even if
there were no "CC" lines in the build log. Instead, just make sure the
changed_objs file isn't empty.
Add a patch testing step before compiling the kernel, so that users
don't have to wait for the kernel to compile before seeing if the patch
applies cleanly.
Also allow the printing of the patch command's stdout/stderr to make it
clear what files are being patched and whether there's any fuzz.
If the patch file fails to apply, it "corrupts" the cache by leaving the
old applied-patch file around. Fix that by always cleaning up after
ourselves.
Allow the user to supply a custom kernel source directory. This copies
the directory to ~/.kpatch/src instead of using it in place. Otherwise
the "make mrproper" (which is needed for compiling objects in a separate
tree) would modify the original source tree and remove its .config file.
There's currently no caching support for this option. If needed, we
could implement that pretty easily by calculating an md5sum of the
original source directory.
kpatch-build is outgrowing the kpatch script and probably is a better
fit as its own utility instead of being wrapped by kpatch. Install
kpatch-build into /usr/local/bin, remove the kpatch wrapper around it,
and update the README accordingly.
For a local object or function symbol, we expect that
the section offset, sym.st_value, be 0 because we used
-ffunction-sections and -fdata-section during compile.
If value != 0, it undermines assumptions we make and
should return an error. Exceptions should be handled
on a case by case basis, like __ksymtab_strings.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
In preparation for adding an automated test framework,
add an ability to create-object-diff that will create
a human readable list of included sections and symbols
with type and bind information so that the test framework
can compare against a known-good reference list with the
expected set of sections and symbols.
The file is created when the -i/--inventory option is
used. The inventory filename is the user supplied output
file name suffixed by .inventory
Signed-off-by: Seth Jennings <sjenning@redhat.com>
I'm tired of setting CFLAGS and people shouldn't have to
recompile to get debug output. This lays the foundations
proper option handling and logging levels.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
- Fixup debug messages
- Remove dead code
- No more DEPENDENCY state
- Reachability test is now the "Inclusion tree" for determining
which syms/sections will be included in the output
- 'reachable' field is now and 'include' and is the sole
consideration in including sections/symbols (no more complex
conditional checks)
- Order LOCAL before GLOBAL in the symbol table. Apparently, after
a FILE sym, all LOCAL symbols should precede GLOBAL syms or readelf
shows <corrupt>
- Handle __ksymtab_strings section and __ksymtab_* syms
Signed-off-by: Seth Jennings <sjenning@redhat.com>
The kernel Makefile look for localversion in the source tree,
not the object tree. The absense from the source tree results
in a patch module that will not load because the kernel versions
don't match.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Setting KCFLAGS="-ffunction-sections -fdata-sections" causes make to
invalidate all the kernel objects, resulting in all the objects getting
rebuilt on the next pass, thus no build caching.
To fix that, build the objects in a separate directory (obj) for normal
builds, and another separate directory (obj2) for the builds with added
cflags.
The compression of the cache during initial build time and
the removal and (re)decompression of the cache for subsequent
builds takes a large amount of time and causes significant I/O.
This commit removes the compressed cache and, instead, keeps
the cache uncompressed and maintained in a known state. If
the "applied-patch" file does not exist, then the cache is
in the unpatched state. If the file does exist, the cache is
in a patched state and can be returned to an unpatched state
with "patch -R -p1 < applied-patch".
The if cache is detected and is in the patched state, the patch
is removed and vmlinux is rebuilt to obtain the base vmlinux.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Right now kpatch-build fails when more than one object
file has changed because the quotes around the for loop
arrays cause the for loop to execute only once for all
elements in a single string.
Remove the quotes around the for loop arrays so that the
for loop is execute for each element.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
When calling kpatch-build from the git directory, the patch kmod build
fails because it can't find kpatch.h because the symlink is broken.
Copy the kpatch.h file (instead of the symlink) to TEMPDIR.
bash gets confused when a command and its args are quoted:
/usr/local/libexec/kpatch/kpatch-build: line 131: strip -d --keep-file-symbols: command not found
Build and install the kpatch core module with make and make install,
rather than building it every time with kpatch build.
The only downside to this approach is that the user has to make and make
install kpatch every time they get a new kernel. But this is only
temporary, until the kpatch module is delivered in an RPM.
The source tree directory structure should be maintained
in the TEMPDIR while building the hotpatch module so that
changed objects in different directories with the same
name don't collide during the build process.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
We need to keep the file symbols around for add-patches-section
and link-vmlinux-syms to resolve ambigous symbols in vmlinux.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
Use relative paths so that kpatch and kpatch-build will find the tools
and data they need, regardless of the installation prefix. Also add
support for running them straight from the git repo.
Be more consistent with the patch metaphor (as opposed to modules):
- change load/unload to apply/remove
- strip "kpatch-" prefix from the module name, so that patch name "foo"
corresponds to module "kpatch-foo.ko"
Before adding any more features to the scripts, standardize the bash
style. I prefer something close to the google shell style guidelines:
http://google-styleguide.googlecode.com/svn/trunk/shell.xml
- change [[ to [ (more robust)
- put variable references in quotes (more robust)
- put "then" on same line as "if" (more C-like, readable)
- print error messages on stdout
- avoid using external utilities like ls
Also added a few error handling improvements, including using a die
function where appropriate.
- setup the makefiles to support "make" and "make install", which builds
the kpatch-build tools and installs everything in /usr/local.
- update kpatch-build to support new paths
- add "kpatch build" wrapper around kpatch-build
NOBITS section may have a non-zero size, however, the have no data and
the data descriptor will have d_buf set to NULL.
This commit fixes as segfault that occurs from trying to compare the
data of such sections.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
There are cases in which the compiler will create symbols with NOTYPE
that map to a non-zero offset inside an .rodata section. In that case, there
may not be a one-to-one relationship between that symbol and section as
the section may contains the data for multiple NOTYPE symbols.
This commit checks for this case and does not assign the symbol pointer of the
section that contains its data to avoid multiple symbols referring to the same
section from overwriting one another. It also adds a check ensuring that all
symbols whose type is !NOTYPE start at offset 0 within the section. This
should be guarenteed by the -ffunction-sections and -fdata-sections options
compiler options.
Signed-off-by: Seth Jennings <sjenning@redhat.com>
