Put funcs, num_funcs, and mod in their own struct called kpatch_module.
This allows us to keep patch module specific variables in one place (and
we'll have more of these variables soon).
Support live patching on NMI handlers. This adds checks for
possible inconsistency of live patching on NMI handlers.
The inconsistency problem means that any concurrent execution
of old function and new function, which can lead unexpected results.
Current kpatch checks possible inconsistency problem with
stop_machine, which can cover only threads and normal interrupts.
However, beacuse NMI can not stop with it, stop_machine is not
enough for live patching on NMI handlers or sub-functions which are
invoked in the NMI context.
To check for possible inconsistency of live patching on those
functions, add an atomic flag to count patching target functions
invoked in NMI context while updating kpatch hash table. If the
flag is set by the target functions in NMI, we can not ensure
there is no concurrent execution on it.
This fixes the issue #65.
Changes from v5:
- Fix to add a NULL check in kpatch_get_committed_func().
Changes from v4:
- Change kpatch_operation to atomic_t.
- Use smp_rmb/wmb barriers between kpatch_operation and kpatch_status.
- Check in_nmi() first and if true, access kpatch_operation.
Changes from v3:
- Fix kpatch_apply/remove_patch to return 0 if succeeded.
Changes from v2:
- Clean up kpatch_get_committed_func as same style of kpatch_get_func.
- Rename opr to op in kpatch_ftrace_handler.
- Consolidate in_nmi() and kpatch_operation check into one condition.
- Fix UNPATCH/PATCH mistype in kpatch_register.
Changes from v1:
- Rename inconsistent_flag to kpatch_status.
- Introduce new enums and helper functions for kpatch_status.
- Use hash_del_rcu instead of hlist_del_rcu.
- Rename get_committed_func to kpatch_get_committed_func.
- Use ACCESS_ONCE for kpatch_operation to prevent compiler optimization.
- Fix to remove (!func || func->updating) condition from NMI check.
- Add more precise comments.
- Fix setting order of kpatch_status and kpatch_operation.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Handle registering error to unroll the ftrace filter.
This also introduces get_kpatch_func() and
kpatch_remove_funcs_from_filter() for holding up
redundant loops.
Changes from v2:
- Rebased on the latest kpatch.
Changes from v1:
- Rename get_kpatch_func to kpatch_get_func.
- Fix function definition style issue.
- Do not jump to a label in "if" block.
- Rollback the ftrace user counter if we hit an error.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
For now, taint with TAINT_USER when loading a patch module so that the
user can always detect when a kpatch module has been previously loaded.
Eventually we will want a dedicated TAINT_KPATCH flag in the kernel.
When multiple patch modules patch the same function, there's no need to
patch all the intermediate functions. Just hook them all into the
original function and use the ftrace handler to find the newest one.
Also use a mutex in the register/unregister functions to protect changes
to kpatch_num_registered, kpatch_func_hash and calls to the ftrace
functions by other register/unregister invocations.
Use a mutex in the register/unregister functions to protect changes to
kpatch_num_registered, kpatch_func_hash and calls to the ftrace
functions by other register/unregister invocations.
My apologies for the size of this commit. I combined these two features
(updating API and using a hash table) into a single commit because their
implementations are tightly coupled and I didn't want to have to add
support for the old kpatch_funcs array with the new API just for the
sake of splitting up the commit :-)
- Update the core module API to get a more clear separation between core
module and patch module. This is cleaner and will help our case for
getting the core module merged upstream into the kernel.
- Convert the old kpatch_funcs array into a hash table. This is so much
nicer performance-wise and everything-else-wise than that ugly old
array.
- Do the incremental patching in stop machine. This ensures that the
funcs hash is up to date and we don't miss anything.
- Disable preemption in the ftrace handler when accessing the func hash.
That way we don't get conflicts with the stop_machine handler updating
the hash.
Print the loading/unloading messages after they have successfully
completed. Using the KERN_NOTICE log level which corresponds to a
"normal but significant condition."
Preemption shouldn't cause a problem with determining activeness safety.
Even if a thread is preempted, it'll be on the backtrace.
We may need to disable preemption when reading the kpatch_funcs array,
but I'm removing that comment for now because the kpatch_funcs array
will soon be replaced by a much better data structure, and we'll deal
with proper synchronization then.
Long ago, the kpatch_trampoline required being written in assembler, but
that's no longer needed now that it integrates nicely with ftrace.
Move it to a C function and rename it kpatch_ftrace_handler.
Build and install the kpatch core module with make and make install,
rather than building it every time with kpatch build.
The only downside to this approach is that the user has to make and make
install kpatch every time they get a new kernel. But this is only
temporary, until the kpatch module is delivered in an RPM.
- setup the makefiles to support "make" and "make install", which builds
the kpatch-build tools and installs everything in /usr/local.
- update kpatch-build to support new paths
- add "kpatch build" wrapper around kpatch-build
