RepoMirrors/libabigail
1
0
Fork 0
mirror of git://sourceware.org/git/libabigail.git synced 2025-01-11 11:59:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,363 Commits 74 Branches 25 Tags 673 MiB
16f5e95fde
Commit Graph

5 Commits

Author SHA1 Message Date
Guillermo E. Martinez
e64d32bee3 ctf-reader: Add support to read CTF information from the Linux Kernel 
This patch is meant to extract ABI information from the CTF data
stored in the Linux kernel build directory.  It depends on the
vmlinux.ctfa archive file.

In order to generate the CTF information, the Linux Kernel build
system must support the 'make ctf' command, which causes the compiler
to be run with -gctf, thus emitting the CTF information for the
Kernel.

The target 'ctf' in the Linux Makefile generates a 'vmlinux.ctfa' file
that will be used by the ctf reader in libabigail. The 'vmlinux.ctfa'
archive has multiple 'ctf dictionaries' called "CTF archive members".

There is one CTF archive member for built-in kernel modules (in
`vmlinux') and one for each out-of-tree kernel module organized in a
parent-child hierarchy.

There is also a CTF archive member called `shared_ctf' which is a
parent dictionary containing shared symbols and CTF types used by more
than one kernel object.  These common types are stored in 'types_map'
in the ctf reader, ignoring the ctf dictionary name.  The CTF API has
the machinery for looking for a shared type in the parent dictionary
referred to in a given child dictionary. This CTF layout can be dumped
by using the objdump tool.

Due to the fact that the _same_ ctf archive is used to build the
vmlinux corpus the corpora of the kernel module (which, by the way,
all belong to the same corpus group), the high number of open/close on
the CTF archive is very time consuming during the ctf extraction.

So, the performance is improved up to 300% (from ~2m:50s to ~50s) by
keeping the ctf archive open for a given group, and thus, by using the
same ctf_archive_t pointer while building all the various corpora.

We just invoke `reset_read_context' for each new corpus.  Note that
the `read_context::ctfa` data member should be updated if the
corpus::origin data member is set to `LINUX_KERNEL_BINARY_ORIGIN' and
the file to be process is not 'vmlinux'.

Note that `ctf_close' must be called after processing all group's
members so it is executed from the destructor of `reader_context'.

The basic algorithm used to generate the Linux corpus is the
following:

   1. Looking for: vmlinux, *.ko objects, and vmlinux.ctfa files. The
   first files are used to extract the ELF symbols, and the last one
   contains the CTF type information for non-static variables and
   functions symbols.

   2. `process_ctf_archive' iterates on public symbols for vmlinux and
   its modules, using the name of the symbol, ctf reader search for CTF
   information in its dictionary, if the information was found it
   builds a `var_decl' or `function_decl' depending of `ctf_type_kind'
   result.

This algorithm is also applied to ELF files (exec, dyn, rel), so
instead of iterating on all ctf_types it just loops on the public
symbols.

	* abg-elf-reader-common.h: Include ctf-api.h file.
	(read_and_add_corpus_to_group_from_elf, set_read_context_corpus_group)
	(reset_read_context, dic_type_key): Declare new member functions.
	* include/abg-ir.cc (types_defined_same_linux_kernel_corpus_public): Use
	bitwise to know the corpus `origin'.
	* src/abg-ctf-reader.cc: Include map, algorithms header files.
	(read_context::type_map): Change from unordered_map to std::map storing
	ctf dictionary name as part of the key.
	(read_context::is_elf_exec): Add new member variable.
	(read_context::{cur_corpus_, cur_corpus_group_}): Likewise.
	(read_context::unknown_types_set): Likewise.
	(read_context::{current_corpus_group, main_corpus_from_current_group,
	has_corpus_group, current_corpus_is_main_corpus_from_current_group,
	should_reuse_type_from_corpus_group}): Add new member functions.
	(read_context::{add_unknown_type, lookup_unknown_type, initialize}):
	Likewise.
	(read_context::{add_type, lookup_type}): Add new `ctf_dict_t' type
	argument.
	(ctf_reader::{process_ctf_typedef, process_ctf_base_type,
	process_ctf_function_type, process_ctf_forward_type,
	process_ctf_struct_type, process_ctf_union_type, process_ctf_array_type,
	process_ctf_qualified_type, process_ctf_enum_type}): Add code to `reuse'
	types already registered in main corpus `should_reuse_type_from_corpus_group'.
	Use new `lookup_type' and `add_type' operations on `read_context::types_map'.
	Replace function calls to the new ctf interface. Add verifier to not build
	types duplicated by recursive calling chain.
	(ctf_reader::process_ctf_type): Add code to return immediately if the
	ctf type is unknown. Add unknown types to `unknown_types_set'.
	(ctf_reader::process_ctf_archive): Change comment.
	Add code to iterate over global symbols, searching by symbol name in the
	ctf dictionary using `ctf_lookup_{variable,by_symbol_name}' depending of
	the ELF file type and corpus type, creating a `{var,fuc}_decl' using the
	return type of `ctf_type_kind'.  Also close the ctf dict and call
	`canonicalize_all_types'.
	(slurp_elf_info): Set `is_elf_exec' depending of ELF type.  Also return
	success if corpus origin is Linux and symbol table was read.
	(ctf_reader::read_corpus): Add current corpus.  Set corpus origin to
	`LINUX_KERNEL_BINARY_ORIGIN' if `is_linux_kernel' returns true.  Verify
	the ctf reader status, now the ctf archive is 'opened' using
	`ctf_arc{open,bufopen}' depending if the corpus origin has
	`corpus::LINUX_KERNEL_BINARY_ORIGIN' bit set. Use
	`sort_{function,variables}' calls after extract ctf information.
	`ctf_close' is called from `read_context' destructor.
	(read:context::{set_read_context_corpus_group, reset_read_context,
	read_and_add_corpus_to_group_from_elf, dic_type_key): Add new member
	function implementation.
	* include/abg-tools-utils.h (build_corpus_group_from_kernel_dist_under):
	Add `origin' parameter with default `corpus::DWARF_ORIGIN'.
	* src/abg-tools-utils.cc: Use `abg-ctf-reader.h' file.
	(maybe_load_vmlinux_dwarf_corpus): Add new function.
	(maybe_load_vmlinux_ctf_corpus): Likewise.
	(build_corpus_group_from_kernel_dist_under): Update comments.
	Add new `origin' argument. Use `maybe_load_vmlinux_dwarf_corpus'
	or `maybe_load_vmlinux_ctf_corpus' according to `origin' value.
	* src/abg-corpus.h (corpus::origin): Update `origin' type
	values in enum.
	* src/abg-corpus-priv.h (corpus::priv): Replace `origin' type
	from `corpus::origin' to `uint32_t'.
	* src/abg-corpus.cc (corpus::{get,set}_origin): Replace data
	type from `corpus::origin' to `uint32_t'.
	* tools/abidw.cc (main): Use of --ctf argument to set format debug.
	* tests/test-read-ctf.cc: Add new tests to harness.
	* tests/data/test-read-ctf/test-PR27700.abi: New test expected
	  result.
	* tests/data/test-read-ctf/test-anonymous-fields.o.abi: Likewise.
	* tests/data/test-read-ctf/test-enum-many-ctf.o.hash.abi: Likewise.
	* tests/data/test-read-ctf/test-enum-many.o.hash.abi: Likewise.
	* tests/data/test-read-ctf/test-enum-symbol-ctf.o.hash.abi: Likewise.
	* tests/data/test-read-common/test-PR26568-2.o: Adjust.
	* tests/data/test-read-ctf/test-PR26568-1.o.abi: Likewise.
	* tests/data/test-read-ctf/test-PR26568-2.o.abi: Likewise.
	* tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi: Likewise.
	* tests/data/test-read-ctf/test-ambiguous-struct-B.c: Likewise.
	* tests/data/test-read-ctf/test-ambiguous-struct-B.o: Likewise.
	* tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi: Likewise.
	* tests/data/test-read-ctf/test-array-of-pointers.abi: Likewise.
	* tests/data/test-read-ctf/test-callback.abi: Likewise.
	* tests/data/test-read-ctf/test-callback2.abi: Likewise.
	* tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi:
	Likewise.
	* tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi:
	Likewise.
	* tests/data/test-read-ctf/test-dynamic-array.o.abi: Likewise.
	* tests/data/test-read-ctf/test-enum-ctf.o.abi: Likewise.
	* tests/data/test-read-ctf/test-enum-symbol.o.hash.abi: Likewise.
	* tests/data/test-read-ctf/test-enum.o.abi: Likewise.
	* tests/data/test-read-ctf/test-forward-type-decl.abi: Likewise.
	* tests/data/test-read-ctf/test-functions-declaration.abi: Likewise.
	* tests/data/test-read-ctf/test-list-struct.abi: Likewise.
	* tests/data/test-read-ctf/test0: Likewise.
	* tests/data/test-read-ctf/test0.abi: Likewise.
	* tests/data/test-read-ctf/test0.c: Likewise.
	* tests/data/test-read-ctf/test0.hash.abi: Likewise.
	* tests/data/test-read-ctf/test1.so.abi: Likewise.
	* tests/data/test-read-ctf/test1.so.hash.abi: Likewise.
	* tests/data/test-read-ctf/test2.so.abi: Likewise.
	* tests/data/test-read-ctf/test2.so.hash.abi: Likewise.
	* tests/data/test-read-ctf/test3.so.abi: Likewise.
	* tests/data/test-read-ctf/test3.so.hash.abi: Likewise.
	* tests/data/test-read-ctf/test4.so.abi: Likewise.
	* tests/data/test-read-ctf/test4.so.hash.abi: Likewise.
	* tests/data/test-read-ctf/test5.o.abi: Likewise.
	* tests/data/test-read-ctf/test7.o.abi: Likewise.
	* tests/data/test-read-ctf/test8.o.abi: Likewise.
	* tests/data/test-read-ctf/test9.o.abi: Likewise.

Signed-off-by: Guillermo E. Martinez <guillermo.e.martinez@oracle.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
 2022-05-13 09:11:37 +02:00
Guillermo E. Martinez via Libabigail
7f60fbc96b ctf-reader: Make create_read_context return a smart pointer. 
* include/abg-ctf-reader.h (read_context_sptr): New typedef.
	(create_read_context): Use read_context_sptr as return value.
	(read_corpus): New overload that takes a read_context_sptr.
	* src/abg-ctf-reader.cc (create_read_context): Use
	read_context_sptr as return value.
	(read_corpus): New overload that takes a read_context_sptr.
	* tools/abidiff.cc (main): Use read_context_sptr.
	* tools/abidw.cc (load_corpus_and_write_abixml): Adjust call to
	create_read_context.
	* tools/abilint.cc: Likewise.

Signed-off-by: Guillermo E. Martinez <guillermo.e.martinez@oracle.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
 2021-11-23 15:25:05 +01:00
Guillermo E. Martinez via Libabigail
ab77eb0d50 ctf-reader: Use argument by reference reading the context 
* include/abg-ctf-reader.h (create_read_context): Pass the string
	by reference.
	* src/abg-ctf-reader.cc (ctf_reader::read_context): Likewise.
	(create_read_context): Likewise.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>
 2021-11-22 11:34:56 +01:00
Jose E. Marchesi via Libabigail
8324a223df ctf: ctf_reader::read_corpus now sets a status 
This patch makes ctf_reader::read_corpus to get a reference to a
`status' variable as an argument, and set it to reflect the result of
the read operation.  The utilities calling to ctf_reader::read_corpus
are updated accordingly.

	* include/abg-ctf-reader.h: Include abg-elf-reader-common.h.
	read_corpus now gets an extra argument `status'.
	* src/abg-ctf-reader.cc (read_corpus): Likewise, and set `status'
	accordingly when the debug info is not found.
	* tools/abilint.cc (main): Pass a status argument to
	ctf_reader::read_corpus.
	* tools/abidiff.cc (main): Likewise.

Signed-off-by: Jose E. Marchesi <jose.marchesi@oracle.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
 2021-11-17 17:55:53 +01:00
Jose E. Marchesi via Libabigail
8c22f5fdc9 Add support for the CTF debug format to libabigail. 
CTF (C Type Format) is a lightweight debugging format that provides
information about C types and the association between functions and
data symbols and types.  It is designed to be very compact and
simple.  More can be learned about it at https://ctfstd.org.

This patch introduces support in libabigail to extract ABI information
from CTF stored in ELF files.

A few notes on this implementation:

- The implementation is complete in terms of CTF support.  Every CTF
  feature is processed and handled to generate libabigail IR.  This
  includes basic types, typedefs, pointer, array and struct types.
  The CTF record of data objects (variables) and functions are also
  used in order to generate the corresponding libabigail IR artifacts.

- The decoding of CTF data is done using the libctf library which is
  part of binutils.  In order to link with it, binutils shall be built
  with --enable-shared for libctf.so to become available.

- This initial implementation is aimed to simplicity.  We have not
  tried to resolve any and every corner case that may require special
  handling.  We have observed that the DWARF front-end (which is
  naturally way more complex as the scope is way bigger) is plagued
  with hacks to handle such situations.  However, for the CTF support
  we prefer to proceed in a simpler and more modest way: we will
  handle these problems if/when we find them.  The fact that CTF only
  supports C (currently) certainly helps there.

- Likewise, in this basic support we are not handling symbol
  suppressions or other goodies that libabigail provides.  We are new
  to libabigail and ABI analysis, and at this point we simply don't
  have a clear picture about what is most useful/relevant to support
  or not.  With the maintainer's blesssing, we will tackle that
  functionaly after this basic support is applied upstream.

- The implementation in abg-ctf-reader.{cc,h} is pretty much
  self-contained.  As a result there is some duplication in terms of
  ELF handling with the DWARF reader, but since that logic is very
  simple and can be easily implemented, we don't consider this to be a
  big deal (for now.)  Hopefully the maintainers agree.

- The libabigail tools assume that ELF means to always use DWARF to
  generate the ABI IR.  We added a new command-line option --ctf to
  the tools in order to make them to use the CTF debug info instead.
  We are definitely not sure whether this is the best user interface.
  In fact I would be suprised if it was ;)

- We added support for --ctf to both abilint and abidiff.   We are not
  sure whether it would make sense to add support for CTF to the other
  tools.  Feedback welcome.

- We are pondering about what to do in terms of testing.  We have
  cursory tested this implementation using abilint and abidiff.  We
  know we are generating IR corpus that seem to be ok.  It would be
  good however to be able to run the libabigail testsuites using CTF.
  However the testsuites may need some non-trivial changes in order to
  make this possible.  Let's talk about that :)

	* configure.ac: Check for libctf.
	* src/abg-ctf-reader.cc: New file.
	* include/abg-ctf-reader.h: Likewise.
	* src/Makefile.am (libabigail_la_SOURCES): Add abg-ctf-reader.cc
	conditionally.
	* include/Makefile.am (pkginclude_HEADERS): Add abg-ctf-reader.h
	conditionally.
	* tools/abilint.cc (struct options): New option `use_ctf'.
	(display_usage): Documentation for --ctf.
	(parse_command_line): Handle --ctf.
	(main): Honour --ctf.
	* tools/abidiff.cc (struct options): New option `use_ctf'.
	(display_usage): Documentation for --ctf.
	(parse_command_line): Handle --ctf.
	(main): Honour --ctf.
	* doc/manuals/abidiff.rst: Document --ctf.
	* doc/manuals/abilint.rst: Likewise.

Signed-off-by: Jose E. Marchesi <jose.marchesi@oracle.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
 2021-11-02 12:43:14 +01:00