Kernel v6.2+ commit bea75b33895f ("x86/Kconfig: Introduce function
padding") added 16 bytes of NOP padding in front of each function.
For objects built with --function-sections, this means that function
symbols no longer sit at the beginning of their respective ELF sections,
but 16 bytes offset.
In the same release, kernel v6.2+ commit 9f2899fe36a6 ("objtool: Add
option to generate prefix symbols") adds ELF function symbols with
prefix "__pfx_" to indicate the start of a function, inclusive of
NOP-padding.
For example:
$ objdump -Dr -j.text.cmdline_proc_show fs/proc/cmdline.o
...
Disassembly of section .text.cmdline_proc_show:
0000000000000000 <__pfx_cmdline_proc_show>:
0: 90 nop
1: 90 nop
2: 90 nop
3: 90 nop
4: 90 nop
5: 90 nop
6: 90 nop
7: 90 nop
8: 90 nop
9: 90 nop
a: 90 nop
b: 90 nop
c: 90 nop
d: 90 nop
e: 90 nop
f: 90 nop
0000000000000010 <cmdline_proc_show>:
10: e8 00 00 00 00 callq 15 <cmdline_proc_show+0x5>
11: R_X86_64_PLT32 __fentry__-0x4
15: 55 push %rbp
16: 48 8b 35 00 00 00 00 mov 0x0(%rip),%rsi # 1d <cmdline_proc_show+0xd>
19: R_X86_64_PC32 saved_command_line-0x4
1d: 48 89 fd mov %rdi,%rbp
20: e8 00 00 00 00 callq 25 <cmdline_proc_show+0x15>
21: R_X86_64_PLT32 seq_puts-0x4
25: 48 89 ef mov %rbp,%rdi
28: be 0a 00 00 00 mov $0xa,%esi
2d: e8 00 00 00 00 callq 32 <cmdline_proc_show+0x22>
2e: R_X86_64_PLT32 seq_putc-0x4
32: 31 c0 xor %eax,%eax
34: 5d pop %rbp
35: e9 00 00 00 00 jmpq 3a <cmdline_proc_show+0x2a>
36: R_X86_64_PLT32 __x86_return_thunk-0x4
Kpatch-build needs to gracefully handle NOP-padding when it is present.
At the same time, it should include "__pfx_<function>" symbols when
their respective functions change, but not treat prefix such functions
as first-class functions.
This also adds support for CONFIG_CFI_CLANG, which also creates prefixed
symbols with the name "__cfi_<function>".
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
The term "dynrela" was invented before klp relocations ever existed.
They're basically the same thing: special livepatch-specific relocations
which get applied when the patched object gets loaded or patched.
They're necessary due to a) the need to access unexported symbols; and
b) late module patching.
The different names are confusing. Consolidate them by replacing
"dynrela" with "klp_reloc" (code) or "klp relocation" (English).
Note there's still some antiquated code in the kpatch core module and in
the pre-4.7 klp patch template which still use the "dynrela" naming.
That code is mostly dead anyway so I've just left it alone.
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Several functions have a boolean semantic, but don't actually return
bool, which is confusing. Fix that.
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Technically we don't support s390 yet, but it's coming soon and there's
no harm in merging this one early. In fact this came in handy for
testing my endian fixes with #1203.
Note it doesn't actually do anything since 'kelf->arch' can't actually
get set to 'S390' yet. But it should work nicely with #1203 as it
evolves.
This is based on the patch from C. Erastus Toe in #1243, though there
may still be a few outstanding issues to look at in that PR, based on
some of the code review comments.
Originally-by: C. Erastus Toe <ctoe@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
Symbol indexes over 64k don't fit into st_shndx and are stored in
extended symbol table. Make sure we properly handle these cases. It is
not normally useful during kpatch builds but will come up if we start
diffing linked objects.
Signed-off-by: Artem Savkov <asavkov@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
libelf can read and write various architecture ELF files that may
differ from the host system. Instead of using preprocessor directives
to build architecture-specific code as per the current host, detect the
intended target architecture from the input ELF files.
Based-on: https://github.com/dynup/kpatch/pull/1179
Signed-off-by: Bill Wendling <morbo@google.com>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com> [small tweaks]
Building data-read-mostly.patch on rhel-9.0-beta for ppc64le leads to a
segmentation fault:
Program received signal SIGSEGV, Segmentation fault.
kpatch_check_relocations (kelf=0x10040490) at create-diff-object.c:2571
2571 sdata = rela->sym->sec->data;
(gdb) bt
(gdb) p rela->sym->sec->data
Cannot access memory at address 0x160000007e
Valgrind narrows the problem down to invalid reads through rela->sym in
kpatch-check-relocations().
The culprits are kpatch_create_intermediate_sections(), which marks
symbols referenced by rela sections that are now dynrelas to be
stripped, and kpatch_strip_unneeded_syms(), which removes and frees
them.
The problem with the symbol stripping is that multiple relas may
reference the same ELF symbol. If any remaining relocation references a
shared symbol, we must keep it.
Replace the symbol->strip boolean with an enumeration:
SYMBOL_DEFAULT - initial value, symbol usage unknown
SYMBOL_USED - symbol is definitely used by a rela
SYMBOL_STRIP - symbol was only referenced by dynrela(s)
Allow transitions from SYMBOL_DEFAULT to SYMBOL_* and SYMBOL_STRIP to
SYMBOL_USED, but _not_ SYMBOL_USED to SYMBOL_*.
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
So far create-diff-object worked only with objectfiles built from a
single source file. To support object-files built from multiple sources
such as linked vmlinux.o, we call locals_match() for each of STT_FILE
symbols and store the pointer to the beginning of appropriate symbol
block in lookup table in each symbol.
Signed-off-by: Artem Savkov <asavkov@redhat.com>
Currently all the callers of kpatch_write_output_elf() are creating
.o object files or .ko kernel modules. Neither of these filetypes are
executable on their own, so enhance kpatch_write_output_elf() to accept
file creation mode and update its callers to pass 0664 to match
the expected permissions.
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
A symbol associated to a function can be split into multiple
sub-functions. Currently, kpatch only supports one child per function.
Extend this to support an arbitrary number of sub-function per function.
Signed-off-by: Julien Thierry <jthierry@redhat.com>
On x86_64, GCC generates the following instruction to compute
'empty_zero_page - __START_KERNEL_map' (__phys_addr_nodebug(), used in
the implementation of ZERO_PAGE()):
48 ba 00 00 00 00 00 00 00 00 movabs $0x0,%rdx
R_X86_64_64 empty_zero_page+0x80000000
__START_KERNEL_map is 0xffffffff80000000.
However, the relocation addend becomes wrong in the patch module:
48 ba 00 00 00 00 00 00 00 00 movabs $0x0,%rdx
R_X86_64_64 empty_zero_page-0x80000000
Note the sign of the addend.
As a result, ZERO_PAGE(0) returns a wrong value in any function touched
by the patch, which may lead to memory corruption and difficult-to-debug
kernel crashes.
The cause is that 'struct rela' uses 'int' for the addend, which is not
enough to store such values. r_addend from Elf64_Rela is int64_t
(Elf64_Sxword) for that.
Let us use 'long' instead of 'int' for the addend in 'struct rela'.
v2:
* Moved 'addend' field after 'offset' in struct rela to facilitate
structure packing (suggested by Kamalesh Babulal).
Fixes https://github.com/dynup/kpatch/issues/1064.
Signed-off-by: Evgenii Shatokhin <eshatokhin@virtuozzo.com>
Add a function that would detect parent/child symbol relations. So far
it only supports .cold.* symbols as children.
Signed-off-by: Artem Savkov <asavkov@redhat.com>
From Oracle's Linker and Libraries Guide [1]:
"The symbols in a symbol table are written in the following order ...
The global symbols immediately follow the local symbols in the symbol
table. The first global symbol is identified by the symbol table sh_info
value. Local and global symbols are always kept separate in this manner,
and cannot be mixed together."
[1] https://docs.oracle.com/cd/E19120-01/open.solaris/819-0690/chapter6-79797/index.htmlFixes#854.
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
- convert section/symbol indexes and rela->offset to unsigned int as I
couldn't find any way for them to become negative.
- cast a number of rela->addend comparisons to int (assuming an 64bit
system this should be enough)
- a number of simple for-loop counter conversions to the type it
compares against
Signed-off-by: Artem Savkov <asavkov@redhat.com>
It was observed by Evgenii Shatokhin in PR#755, that when the RCU
callback was called on the patched function, from unloaded livepatch
module triggered a kernel crash.
This patch implements the approach on PowerPC outlined in PR#755.
With -mcmodel=large, like any other data, function pointers are also
loaded relative to the current TOC base and are populated as
relocation entries in .toc section. Every function passing a function
pointer as the argument need to load the function address through
.toc section + offset. Convert such .toc + offset relocation into
a dynamic rela, which resolves to original function address, during
module load.
Also move the comment related to nested function check, into
may_need_dynrela().
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Evgenii Shatokhin <eshatokhin@virtuozzo.com>
Cc: Joe Lawrence <jdl1291@gmail.com>
Signed-off-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
With gcc-6 the function prologue is changeg by
moving the toc base resolution func - 0x8 bytes:
.globl my_func
.type my_func, @function
.quad .TOC.-my_func
my_func:
.reloc ., R_PPC64_ENTRY ; optional
ld r2,-8(r12)
add r2,r2,r12
.localentry my_func, .-my_func
Add support for function prologue, along with gcc-5.
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
symbol->has_fentry_call is x86 specfic. Rename it to more
generic name, representing the general idea of calling
profiling function at function entry.
This patch converts all instance of symbol->has_fentry_call
to symbol->has_func_profiling and also renames functions:
kpatch_check_fentry_calls() -> kpatch_check_func_profiling_calls()
kpatch_find_fentry_calls() -> kpatch_find_func_profiling_calls()
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
Move functions kpatch_reindex_elements() and kpatch_rebuild_rela_section_data()
from create-diff-object.c to kpatch-elf.c. These functions will be used
to rebuild kpatch elf data in create-klp-module and create-kpatch-module,
i.e. during the second "phase" of kpatch-build.
Introduce a common kpatch elf api by moving all functions and struct
declarations related to manipulating kpatch_elf objects from
create-diff-object to kpatch-elf.{h,c}. Move logging macros to a separate
file log.h, and have kpatch-elf.h include it. These changes will generalize
the kpatch-elf and logging api and make it available to other kpatch-build
tools.
Josh Poimboeuf
Joe Lawrence
Sumanth Korikkar
Artem Savkov
Bill Wendling
Julien Thierry
Evgenii Shatokhin
Kamalesh Babulal
Jessica Yu