RepoMirrors
/
musl
mirror of git://git.musl-libc.org/musl
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
musl/ldso/dynlink.c

2307 lines
62 KiB
C
Raw Normal View History

dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
#define _GNU_SOURCE
overhaul i386 syscall mechanism not to depend on external asm source this is the first part of a series of patches intended to make __syscall fully self-contained in the object file produced using syscall.h, which will make it possible for crt1 code to perform syscalls. the (confusingly named) i386 __vsyscall mechanism, which this commit removes, was introduced before the presence of a valid thread pointer was mandatory; back then the thread pointer was setup lazily only if threads were used. the intent was to be able to perform syscalls using the kernel's fast entry point in the VDSO, which can use the sysenter (Intel) or syscall (AMD) instruction instead of int $128, but without inlining an access to the __syscall global at the point of each syscall, which would incur a significant size cost from PIC setup everywhere. the mechanism also shuffled registers/calling convention around to avoid spills of call-saved registers, and to avoid allocating ebx or ebp via asm constraints, since there are plenty of broken-but-supported compiler versions which are incapable of allocating ebx with -fPIC or ebp with -fno-omit-frame-pointer. the new mechanism preserves the properties of avoiding spills and avoiding allocation of ebx/ebp in constraints, but does it inline, using some fairly simple register shuffling, and uses a field of the thread structure rather than global data for the vdso-provided syscall code address. for now, the external __syscall function is refactored not to use the old __vsyscall so it can be kept, but the intent is to remove it too.
2019-04-10 21:10:36 +00:00
#define SYSCALL_NO_TLS 1
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
#include <stdio.h>
#include <stdlib.h>
reduce code duplication in dynamic linker error paths eventually this should help making dlerror thread-safe too.
2014-06-18 07:05:42 +00:00
#include <stdarg.h>
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
#include <stddef.h>
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
#include <string.h>
#include <unistd.h>
#include <stdint.h>
#include <elf.h>
#include <sys/mman.h>
#include <limits.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <errno.h>
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
#include <link.h>
error handling in dynamic linking some of the code is not yet used, and is in preparation for dlopen which needs to be able to handle failure loading libraries without terminating the program.
2011-06-26 21:39:17 +00:00
#include <setjmp.h>
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
#include <pthread.h>
LD_PRELOAD support
2011-08-16 04:24:36 +00:00
#include <ctype.h>
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
#include <dlfcn.h>
install dynamic tls synchronously at dlopen, streamline access previously, dynamic loading of new libraries with thread-local storage allocated the storage needed for all existing threads at load-time, precluding late failure that can't be handled, but left installation in existing threads to take place lazily on first access. this imposed an additional memory access and branch on every dynamic tls access, and imposed a requirement, which was not actually met, that the dynamic tlsdesc asm functions preserve all call-clobbered registers before calling C code to to install new dynamic tls on first access. the x86[_64] versions of this code wrongly omitted saving and restoring of fpu/vector registers, assuming the compiler would not generate anything using them in the called C code. the arm and aarch64 versions saved known existing registers, but failed to be future-proof against expansion of the register file. now that we track live threads in a list, it's possible to install the new dynamic tls for each thread at dlopen time. for the most part, synchronization is not needed, because if a thread has not synchronized with completion of the dlopen, there is no way it can meaningfully request access to a slot past the end of the old dtv, which remains valid for accessing slots which already existed. however, it is necessary to ensure that, if a thread sees its new dtv pointer, it sees correct pointers in each of the slots that existed prior to the dlopen. my understanding is that, on most real-world coherency architectures including all the ones we presently support, a built-in consume order guarantees this; however, don't rely on that. instead, the SYS_membarrier syscall is used to ensure that all threads see the stores to the slots of their new dtv prior to the installation of the new dtv. if it is not supported, the same is implemented in userspace via signals, using the same mechanism as __synccall. the __tls_get_addr function, variants, and dynamic tlsdesc asm functions are all updated to remove the fallback paths for claiming new dynamic tls, and are now all branch-free.
2019-02-18 04:22:27 +00:00
#include <semaphore.h>
add membarrier syscall wrapper, refactor dynamic tls install to use it the motivation for this change is twofold. first, it gets the fallback logic out of the dynamic linker, improving code readability and organization. second, it provides application code that wants to use the membarrier syscall, which depends on preregistration of intent before the process becomes multithreaded unless unbounded latency is acceptable, with a symbol that, when linked, ensures that this registration happens.
2019-02-22 07:56:10 +00:00
#include <sys/membarrier.h>
TLS (GNU/C11 thread-local storage) support for static-linked programs the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
2012-10-04 20:35:46 +00:00
#include "pthread_impl.h"
#include "libc.h"
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
#include "dynlink.h"
move declarations for malloc internals to malloc_impl.h
2018-09-06 20:32:49 +00:00
#include "malloc_impl.h"
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
static void error(const char *, ...);
make dlerror conform to posix the error status is required to be sticky after failure of dlopen or dlsym until cleared by dlerror. applications and especially libraries should never rely on this since it is not thread-safe and subject to race conditions, but glib does anyway.
2012-03-23 04:28:20 +00:00
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
#define MAXP2(a,b) (-(-(a)&-(b)))
#define ALIGN(x,y) ((x)+(y)-1 & -(y))
fix crash in new dynamic tls installation when last dep lacks tls code introduced in commit 9d44b6460ab603487dab4d916342d9ba4467e6b9 wrongly assumed the dso list tail was the right place to find new dtv storage. however, this is only true if the last-loaded dependency has tls. the correct place to get it is the dso corresponding to the tls module list tail. introduce a container_of macro to get it, and use it. ultimately, dynamic tls allocation should be refactored so that this is not an issue. there is no reason to be allocating new dtv space at each load_library; instead it could happen after all new libraries have been loaded but before they are committed. such changes may be made later, but this commit fixes the present regression.
2019-02-25 07:09:36 +00:00
#define container_of(p,t,m) ((t*)((char *)(p)-offsetof(t,m)))
avoid malloc of ctor queue for programs with no external deps together with the previous two commits, this completes restoration of the property that dynamic-linked apps with no external deps and no tls have no failure paths before entry.
2019-03-03 18:24:23 +00:00
#define countof(a) ((sizeof (a))/(sizeof (a)[0]))
fix crash in new dynamic tls installation when last dep lacks tls code introduced in commit 9d44b6460ab603487dab4d916342d9ba4467e6b9 wrongly assumed the dso list tail was the right place to find new dtv storage. however, this is only true if the last-loaded dependency has tls. the correct place to get it is the dso corresponding to the tls module list tail. introduce a container_of macro to get it, and use it. ultimately, dynamic tls allocation should be refactored so that this is not an issue. there is no reason to be allocating new dtv space at each load_library; instead it could happen after all new libraries have been loaded but before they are committed. such changes may be made later, but this commit fixes the present regression.
2019-02-25 07:09:36 +00:00
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
struct debug {
int ver;
void *head;
void (*bp)(void);
int state;
void *base;
};
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
struct td_index {
size_t args[2];
struct td_index *next;
};
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
struct dso {
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
#if DL_FDPIC
struct fdpic_loadmap *loadmap;
#else
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
unsigned char *base;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
#endif
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
char *name;
size_t *dynv;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
struct dso *next, *prev;
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
Phdr *phdr;
int phnum;
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
size_t phentsize;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
Sym *syms;
generalize ELF hash table types not to assume 32-bit entries alpha and s390x gratuitously use 64-bit entries (wasting 2x space and cache utilization) despite the values always being 32-bit. based on patch by Bobby Bingham, with changes suggested by Alexander Monakov to use the public Elf_Symndx type from link.h (and make it properly variable by arch) rather than adding new internal infrastructure for handling the type.
2016-11-11 17:30:24 +00:00
Elf_Symndx *hashtab;
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
uint32_t *ghashtab;
work around libraries with versioned symbols in dynamic linker this commit does not add versioning support; it merely fixes incorrect lookups of symbols in libraries that contain versioned symbols. previously, the version information was completely ignored, and empirically this seems to have resulted in the oldest version being chosen, but I am uncertain if that behavior was even reliable. the new behavior being introduced is to completely ignore symbols which are marked "hidden" (this seems to be the confusing nomenclature for non-current-version) when versioning is present. this should solve all problems related to libraries with symbol versioning as long as all binaries involved are up-to-date (compatible with the latest-version symbols), and it's the needed behavior for dlsym under all circumstances.
2013-08-08 20:10:35 +00:00
int16_t *versym;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
char *strings;
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
struct dso *syms_next, *lazy_next;
size_t *lazy, lazy_cnt;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
unsigned char *map;
size_t map_len;
dev_t dev;
ino_t ino;
reduce some wasted space in dso structure
2012-02-08 01:29:29 +00:00
char relocated;
char constructed;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
char kernel_mapped;
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
char mark;
char bfs_built;
fix unsafety of new ldso dep tracking in presence of malloc replacement commit 403555690775f7c8806372644f543518e6664e3b introduced runtime realloc of an array that may have been allocated before symbols were resolved outside of libc, which is invalid if the allocator has been replaced. track this condition and manually copy if needed.
2019-03-01 19:37:52 +00:00
char runtime_loaded;
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
struct dso **deps, *needed_by;
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
size_t ndeps_direct;
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
size_t next_dep;
int ctor_visitor;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
char *rpath_orig, *rpath;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
struct tls_module tls;
size_t tls_id;
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
size_t relro_start, relro_end;
combine arch ABI's DTP_OFFSET into DTV pointers as explained in commit 6ba5517a460c6c438f64d69464fdfc3269a4c91a, some archs use an offset (typicaly -0x8000) with their DTPOFF relocations, which __tls_get_addr needs to invert. on affected archs, which lack direct support for large immediates, this can cost multiple extra instructions in the hot path. instead, incorporate the DTP_OFFSET into the DTV entries. this means they are no longer valid pointers, so store them as an array of uintptr_t rather than void *; this also makes it easier to access slot 0 as a valid slot count. commit e75b16cf93ebbc1ce758d3ea6b2923e8b2457c68 left behind cruft in two places, __reset_tls and __tls_get_new, from back when it was possible to have uninitialized gap slots indicated by a null pointer in the DTV. since the concept of null pointer is no longer meaningful with an offset applied, remove this cruft. presently there are no archs with both TLSDESC and nonzero DTP_OFFSET, but the dynamic TLSDESC relocation code is also updated to apply an inverted offset to its offset field, so that the offset DTV would not impose a runtime cost in TLSDESC resolver functions.
2018-10-12 04:30:34 +00:00
uintptr_t *new_dtv;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
unsigned char *new_tls;
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
struct td_index *td_index;
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
struct dso *fini_next;
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
char *shortname;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
#if DL_FDPIC
unsigned char *base;
#else
struct fdpic_loadmap *loadmap;
#endif
struct funcdesc {
void *addr;
size_t *got;
} *funcdescs;
size_t *got;
error handling in dynamic linking some of the code is not yet used, and is in preparation for dlopen which needs to be able to handle failure loading libraries without terminating the program.
2011-06-26 21:39:17 +00:00
char buf[];
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
};
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
struct symdef {
Sym *sym;
struct dso *dso;
};
fix over-alignment of TLS, insufficient builtin TLS on 64-bit archs a conservative estimate of 4*sizeof(size_t) was used as the minimum alignment for thread-local storage, despite the only requirements being alignment suitable for struct pthread and void* (which struct pthread already contains). additional alignment required by the application or libraries is encoded in their headers and is already applied. over-alignment prevented the builtin_tls array from ever being used in dynamic-linked programs on 64-bit archs, thereby requiring allocation at startup even in programs with no TLS of their own.
2015-03-06 18:27:08 +00:00
static struct builtin_tls {
char c;
struct pthread pt;
void *space[16];
} builtin_tls[1];
#define MIN_TLS_ALIGN offsetof(struct builtin_tls, pt)
reprocess all libc/ldso symbolic relocations in dynamic linking stage 3 commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-26 03:33:59 +00:00
#define ADDEND_LIMIT 4096
static size_t *saved_addends, *apply_addends_to;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
static struct dso ldso;
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
static struct dso *head, *tail, *fini_head, *syms_tail, *lazy_head;
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
static char *env_path, *sys_path;
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
static unsigned long long gencnt;
error handling in dynamic linking some of the code is not yet used, and is in preparation for dlopen which needs to be able to handle failure loading libraries without terminating the program.
2011-06-26 21:39:17 +00:00
static int runtime;
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
static int ldd_mode;
make dynamic linker report all failures before exiting before, only the first library that failed to load or symbol that failed to resolve was reported, and then the dynamic linker immediately exited. when attempting to fix a library compatibility issue, this is about the worst possible behavior. now we print all errors as they occur and exit at the very end if errors were encountered.
2012-08-18 20:00:23 +00:00
static int ldso_fail;
add support for RTLD_NOLOAD to dlopen based on patch by Pierre Carrier <pierre@gcarrier.fr> that just added the flag constant, but with minimal additional code so that it actually works as documented. this is a nonstandard option but some major software (reportedly, Firefox) uses it and it was easy to add anyway.
2013-01-24 03:07:45 +00:00
static int noload;
synchronize shared library dtor exec against concurrent loads/ctors previously, going way back, there was simply no synchronization here. a call to exit concurrent with ctor execution from dlopen could cause a dtor to execute concurrently with its corresponding ctor, or could cause dtors for newly-constructed libraries to be skipped. introduce a shutting_down state that blocks further ctor execution, producing the quiescence the dtor execution loop needs to ensure any kind of consistency, and that blocks further calls to dlopen so that a call into dlopen from a dtor cannot deadlock. better approaches to some of this may be possible, but the changes here at least make things safe.
2019-03-02 03:47:29 +00:00
static int shutting_down;
move the dynamic linker's jmp_buf from static to automatic storage this more than compensates for the size increase of jmp_buf, and greatly reduces bss/data size on archs with huge jmp_buf.
2013-07-24 06:38:05 +00:00
static jmp_buf *rtld_fail;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
static pthread_rwlock_t lock;
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
static struct debug debug;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
static struct tls_module *tls_tail;
fix over-alignment of TLS, insufficient builtin TLS on 64-bit archs a conservative estimate of 4*sizeof(size_t) was used as the minimum alignment for thread-local storage, despite the only requirements being alignment suitable for struct pthread and void* (which struct pthread already contains). additional alignment required by the application or libraries is encoded in their headers and is already applied. over-alignment prevented the builtin_tls array from ever being used in dynamic-linked programs on 64-bit archs, thereby requiring allocation at startup even in programs with no TLS of their own.
2015-03-06 18:27:08 +00:00
static size_t tls_cnt, tls_offset, tls_align = MIN_TLS_ALIGN;
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
static size_t static_tls_cnt;
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
static pthread_mutex_t init_fini_lock;
static pthread_cond_t ctor_cond;
avoid malloc of deps array for programs with no external deps traditionally, we've provided a guarantee that dynamic-linked applications with no external dependencies (nothing but libc) and no thread-local storage have no failure paths before the entry point. normally, thanks to reclaim_gaps, such a malloc will not require a syscall anyway, but if segment alignment is unlucky, it might. use a builtin array for this common special case.
2019-03-03 17:12:59 +00:00
static struct dso *builtin_deps[2];
avoid malloc of deps arrays for ldso and vdso neither has or can have any dependencies, but since commit 403555690775f7c8806372644f543518e6664e3b, gratuitous zero-length deps arrays were being allocated for them. use a dummy array instead.
2019-03-03 17:42:34 +00:00
static struct dso *const no_deps[1];
avoid malloc of ctor queue for programs with no external deps together with the previous two commits, this completes restoration of the property that dynamic-linked apps with no external deps and no tls have no failure paths before entry.
2019-03-03 18:24:23 +00:00
static struct dso *builtin_ctor_queue[4];
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
static struct dso **main_ctor_queue;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
static struct fdpic_loadmap *app_loadmap;
static struct fdpic_dummy_loadmap app_dummy_loadmap;
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
struct debug *_dl_debug_addr = &debug;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
define and use internal macros for hidden visibility, weak refs this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-09-05 16:43:34 +00:00
extern hidden int __malloc_replaced;
reintroduce hardening against partially-replaced allocator commit 618b18c78e33acfe54a4434e91aa57b8e171df89 removed the previous detection and hardening since it was incorrect. commit 72141795d4edd17f88da192447395a48444afa10 already handled all that remained for hardening the static-linked case. in the dynamic-linked case, have the dynamic linker check whether malloc was replaced and make that information available. with these changes, the properties documented in commit c9f415d7ea2dace5bf77f6518b6afc36bb7a5732 are restored: if calloc is not provided, it will behave as malloc+memset, and any of the memalign-family functions not provided will fail with ENOMEM.
2018-04-20 02:19:29 +00:00
define and use internal macros for hidden visibility, weak refs this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-09-05 16:43:34 +00:00
hidden void (*const __init_array_start)(void)=0, (*const __fini_array_start)(void)=0;
remove undef weak refs to init/fini array symbols in libc.so commit ad1cd43a86645ba2d4f7c8747240452a349d6bc1 eliminated preprocessor-level omission of references to the init/fini array symbols from object files going into libc.so. the references are weak, and the intent was that the linker would resolve them to zero in libc.so, but instead it leaves undefined references that could be satisfied at runtime. normally these references would be harmless, since the code using them does not even get executed, but some older binutils versions produce a linking error: when linking a program against libc.so, ld first tries to use the hidden init/fini array symbols produced by the linker script to satisfy the references in libc.so, then produces an error because the definitions are hidden. ideally ld would have already provided definitions of these symbols when linking libc.so, but the linker script for -shared omits them. to avoid this situation, the dynamic linker now provides its own dummy definitions of the init/fini array symbols for libc.so. since they are hidden, everything binds at ld time and no references remain in the dynamic symbol table. with modern binutils and --gc-sections, both the dummy empty array objects and the code referencing them get dropped at link time, anyway. the _init and _fini symbols are also switched back to using weak definitions rather than weak references since the latter behave somewhat problematically in general, and the weak definition approach was known to work well.
2015-11-20 01:28:08 +00:00
define and use internal macros for hidden visibility, weak refs this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-09-05 16:43:34 +00:00
extern hidden void (*const __init_array_end)(void), (*const __fini_array_end)(void);
remove undef weak refs to init/fini array symbols in libc.so commit ad1cd43a86645ba2d4f7c8747240452a349d6bc1 eliminated preprocessor-level omission of references to the init/fini array symbols from object files going into libc.so. the references are weak, and the intent was that the linker would resolve them to zero in libc.so, but instead it leaves undefined references that could be satisfied at runtime. normally these references would be harmless, since the code using them does not even get executed, but some older binutils versions produce a linking error: when linking a program against libc.so, ld first tries to use the hidden init/fini array symbols produced by the linker script to satisfy the references in libc.so, then produces an error because the definitions are hidden. ideally ld would have already provided definitions of these symbols when linking libc.so, but the linker script for -shared omits them. to avoid this situation, the dynamic linker now provides its own dummy definitions of the init/fini array symbols for libc.so. since they are hidden, everything binds at ld time and no references remain in the dynamic symbol table. with modern binutils and --gc-sections, both the dummy empty array objects and the code referencing them get dropped at link time, anyway. the _init and _fini symbols are also switched back to using weak definitions rather than weak references since the latter behave somewhat problematically in general, and the weak definition approach was known to work well.
2015-11-20 01:28:08 +00:00
weak_alias(__init_array_start, __init_array_end);
weak_alias(__fini_array_start, __fini_array_end);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
static int dl_strcmp(const char *l, const char *r)
{
for (; *l==*r && *l; l++, r++);
return *(unsigned char *)l - *(unsigned char *)r;
}
#define strcmp(l,r) dl_strcmp(l,r)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
/* Compute load address for a virtual address in a given dso. */
fix breakage in non-fdpic dynamic linker init/fini processing a mistaken #ifdef instead of #if caused conversion of code addresses to function descriptors to be performed even on non-fdpic.
2015-09-22 20:20:39 +00:00
#if DL_FDPIC
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
static void *laddr(const struct dso *p, size_t v)
{
size_t j=0;
if (!p->loadmap) return p->base + v;
for (j=0; v-p->loadmap->segs[j].p_vaddr >= p->loadmap->segs[j].p_memsz; j++);
return (void *)(v - p->loadmap->segs[j].p_vaddr + p->loadmap->segs[j].addr);
}
enable reclaim_gaps for fdpic the existing laddr function for fdpic cannot translate ELF virtual addresses outside of the LOAD segments to runtime addresses because the fdpic loadmap only covers the logically-mapped part. however the whole point of reclaim_gaps is to recover the slack space up to the page boundaries, so it needs to work with such addresses. add a new laddr_pg function that accepts any address in the page range for the LOAD segment by expanding the loadmap records out to page boundaries. only use the new version for reclaim_gaps, so as not to impact performance of other address lookups. also, only use laddr_pg for the start address of a gap; the end address lies one byte beyond the end, potentially in a different page where it would get mapped differently. instead of mapping end, apply the length (end-start) to the mapped value of start.
2018-04-17 19:55:18 +00:00
static void *laddr_pg(const struct dso *p, size_t v)
{
size_t j=0;
size_t pgsz = PAGE_SIZE;
if (!p->loadmap) return p->base + v;
for (j=0; ; j++) {
size_t a = p->loadmap->segs[j].p_vaddr;
size_t b = a + p->loadmap->segs[j].p_memsz;
a &= -pgsz;
b += pgsz-1;
b &= -pgsz;
if (v-a<b-a) break;
}
return (void *)(v - p->loadmap->segs[j].p_vaddr + p->loadmap->segs[j].addr);
}
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
#define fpaddr(p, v) ((void (*)())&(struct funcdesc){ \
laddr(p, v), (p)->got })
#else
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
#define laddr(p, v) (void *)((p)->base + (v))
enable reclaim_gaps for fdpic the existing laddr function for fdpic cannot translate ELF virtual addresses outside of the LOAD segments to runtime addresses because the fdpic loadmap only covers the logically-mapped part. however the whole point of reclaim_gaps is to recover the slack space up to the page boundaries, so it needs to work with such addresses. add a new laddr_pg function that accepts any address in the page range for the LOAD segment by expanding the loadmap records out to page boundaries. only use the new version for reclaim_gaps, so as not to impact performance of other address lookups. also, only use laddr_pg for the start address of a gap; the end address lies one byte beyond the end, potentially in a different page where it would get mapped differently. instead of mapping end, apply the length (end-start) to the mapped value of start.
2018-04-17 19:55:18 +00:00
#define laddr_pg(p, v) laddr(p, v)
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
#define fpaddr(p, v) ((void (*)())laddr(p, v))
#endif
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
static void decode_vec(size_t *v, size_t *a, size_t cnt)
{
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
size_t i;
for (i=0; i<cnt; i++) a[i] = 0;
for (; v[0]; v+=2) if (v[0]-1<cnt-1) {
a[0] |= 1UL<<v[0];
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
a[v[0]] = v[1];
}
}
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
static int search_vec(size_t *v, size_t *r, size_t key)
{
for (; v[0]!=key; v+=2)
if (!v[0]) return 0;
*r = v[1];
return 1;
}
static uint32_t sysv_hash(const char *s0)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
fix char signedness bug in dynlinker hash function this only affects non-ascii symbol names, which are probably not in use anyway..
2012-01-17 05:34:58 +00:00
const unsigned char *s = (void *)s0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
uint_fast32_t h = 0;
while (*s) {
h = 16*h + *s++;
h ^= h>>24 & 0xf0;
}
return h & 0xfffffff;
}
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
static uint32_t gnu_hash(const char *s0)
{
const unsigned char *s = (void *)s0;
uint_fast32_t h = 5381;
for (; *s; s++)
dynlink.c: use a faster expression in gnu_hash With -Os, GCC uses a multiply rather than a shift and addition for 'h*33'. Use a more efficient expression explicitely.
2015-06-27 23:48:30 +00:00
h += h*32 + *s;
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
return h;
}
static Sym *sysv_lookup(const char *s, uint32_t h, struct dso *dso)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
size_t i;
dynamic linker internals cleanup
2012-08-05 06:38:35 +00:00
Sym *syms = dso->syms;
generalize ELF hash table types not to assume 32-bit entries alpha and s390x gratuitously use 64-bit entries (wasting 2x space and cache utilization) despite the values always being 32-bit. based on patch by Bobby Bingham, with changes suggested by Alexander Monakov to use the public Elf_Symndx type from link.h (and make it properly variable by arch) rather than adding new internal infrastructure for handling the type.
2016-11-11 17:30:24 +00:00
Elf_Symndx *hashtab = dso->hashtab;
dynamic linker internals cleanup
2012-08-05 06:38:35 +00:00
char *strings = dso->strings;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
for (i=hashtab[2+h%hashtab[0]]; i; i=hashtab[2+hashtab[0]+i]) {
work around libraries with versioned symbols in dynamic linker this commit does not add versioning support; it merely fixes incorrect lookups of symbols in libraries that contain versioned symbols. previously, the version information was completely ignored, and empirically this seems to have resulted in the oldest version being chosen, but I am uncertain if that behavior was even reliable. the new behavior being introduced is to completely ignore symbols which are marked "hidden" (this seems to be the confusing nomenclature for non-current-version) when versioning is present. this should solve all problems related to libraries with symbol versioning as long as all binaries involved are up-to-date (compatible with the latest-version symbols), and it's the needed behavior for dlsym under all circumstances.
2013-08-08 20:10:35 +00:00
if ((!dso->versym || dso->versym[i] >= 0)
&& (!strcmp(s, strings+syms[i].st_name)))
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
return syms+i;
}
return 0;
}
dynlink.c: pass gnu-hash table pointer to gnu_lookup The callers need to check the value of the pointer anyway, so make them pass the pointer to gnu_lookup instead of reloading it there. Reorder gnu_lookup arguments so that always-used ones are listed first. GCC can choose a calling convention with arguments in registers (e.g. up to 3 arguments in eax, ecx, edx on x86), but cannot reorder the arguments for static functions.
2015-06-27 23:48:33 +00:00
static Sym *gnu_lookup(uint32_t h1, uint32_t *hashtab, struct dso *dso, const char *s)
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
{
uint32_t nbuckets = hashtab[0];
uint32_t *buckets = hashtab + 4 + hashtab[2]*(sizeof(size_t)/4);
work around libraries with versioned symbols in dynamic linker this commit does not add versioning support; it merely fixes incorrect lookups of symbols in libraries that contain versioned symbols. previously, the version information was completely ignored, and empirically this seems to have resulted in the oldest version being chosen, but I am uncertain if that behavior was even reliable. the new behavior being introduced is to completely ignore symbols which are marked "hidden" (this seems to be the confusing nomenclature for non-current-version) when versioning is present. this should solve all problems related to libraries with symbol versioning as long as all binaries involved are up-to-date (compatible with the latest-version symbols), and it's the needed behavior for dlsym under all circumstances.
2013-08-08 20:10:35 +00:00
uint32_t i = buckets[h1 % nbuckets];
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
work around libraries with versioned symbols in dynamic linker this commit does not add versioning support; it merely fixes incorrect lookups of symbols in libraries that contain versioned symbols. previously, the version information was completely ignored, and empirically this seems to have resulted in the oldest version being chosen, but I am uncertain if that behavior was even reliable. the new behavior being introduced is to completely ignore symbols which are marked "hidden" (this seems to be the confusing nomenclature for non-current-version) when versioning is present. this should solve all problems related to libraries with symbol versioning as long as all binaries involved are up-to-date (compatible with the latest-version symbols), and it's the needed behavior for dlsym under all circumstances.
2013-08-08 20:10:35 +00:00
if (!i) return 0;
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
dynlink.c: slim down gnu_lookup Do not reference dso->syms and dso->strings until point of use. Check 'h1 == (h2|1)', the simplest condition, before the others.
2015-06-27 23:48:32 +00:00
uint32_t *hashval = buckets + nbuckets + (i - hashtab[1]);
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
work around libraries with versioned symbols in dynamic linker this commit does not add versioning support; it merely fixes incorrect lookups of symbols in libraries that contain versioned symbols. previously, the version information was completely ignored, and empirically this seems to have resulted in the oldest version being chosen, but I am uncertain if that behavior was even reliable. the new behavior being introduced is to completely ignore symbols which are marked "hidden" (this seems to be the confusing nomenclature for non-current-version) when versioning is present. this should solve all problems related to libraries with symbol versioning as long as all binaries involved are up-to-date (compatible with the latest-version symbols), and it's the needed behavior for dlsym under all circumstances.
2013-08-08 20:10:35 +00:00
for (h1 |= 1; ; i++) {
dynlink.c: slim down gnu_lookup Do not reference dso->syms and dso->strings until point of use. Check 'h1 == (h2|1)', the simplest condition, before the others.
2015-06-27 23:48:32 +00:00
uint32_t h2 = *hashval++;
if ((h1 == (h2|1)) && (!dso->versym || dso->versym[i] >= 0)
&& !strcmp(s, dso->strings + dso->syms[i].st_name))
return dso->syms+i;
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
if (h2 & 1) break;
}
return 0;
}
dynlink.c: pass gnu-hash table pointer to gnu_lookup The callers need to check the value of the pointer anyway, so make them pass the pointer to gnu_lookup instead of reloading it there. Reorder gnu_lookup arguments so that always-used ones are listed first. GCC can choose a calling convention with arguments in registers (e.g. up to 3 arguments in eax, ecx, edx on x86), but cannot reorder the arguments for static functions.
2015-06-27 23:48:33 +00:00
static Sym *gnu_lookup_filtered(uint32_t h1, uint32_t *hashtab, struct dso *dso, const char *s, uint32_t fofs, size_t fmask)
dynlink.c: use bloom filter in gnu hash lookup Introduce gnu_lookup_filtered and use it to speed up symbol lookups in find_sym (do_dlsym is left as is, based on an expectation that frequently dlsym queries will use a dlopen handle rather than RTLD_NEXT or RTLD_DEFAULT, and will not need to look at more than one DSO).
2015-06-27 23:48:31 +00:00
{
const size_t *bloomwords = (const void *)(hashtab+4);
size_t f = bloomwords[fofs & (hashtab[2]-1)];
if (!(f & fmask)) return 0;
f >>= (h1 >> hashtab[3]) % (8 * sizeof f);
if (!(f & 1)) return 0;
dynlink.c: pass gnu-hash table pointer to gnu_lookup The callers need to check the value of the pointer anyway, so make them pass the pointer to gnu_lookup instead of reloading it there. Reorder gnu_lookup arguments so that always-used ones are listed first. GCC can choose a calling convention with arguments in registers (e.g. up to 3 arguments in eax, ecx, edx on x86), but cannot reorder the arguments for static functions.
2015-06-27 23:48:33 +00:00
return gnu_lookup(h1, hashtab, dso, s);
dynlink.c: use bloom filter in gnu hash lookup Introduce gnu_lookup_filtered and use it to speed up symbol lookups in find_sym (do_dlsym is left as is, based on an expectation that frequently dlsym queries will use a dlopen handle rather than RTLD_NEXT or RTLD_DEFAULT, and will not need to look at more than one DSO).
2015-06-27 23:48:31 +00:00
}
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
#define OK_TYPES (1<<STT_NOTYPE | 1<<STT_OBJECT | 1<<STT_FUNC | 1<<STT_COMMON | 1<<STT_TLS)
support STB_GNU_UNIQUE symbol bindings in dynamic linker these are needed for some C++ library binaries including most builds of libstdc++. I'm not entirely clear on the rationale. this patch does not implement any special semantics for them, but as far as I can tell, no special treatment is needed in correctly-linked programs; this binding seems to exist only for catching incorrectly-linked programs.
2013-07-24 15:53:23 +00:00
#define OK_BINDS (1<<STB_GLOBAL | 1<<STB_WEAK | 1<<STB_GNU_UNIQUE)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
fix regression in mips dynamic linker this issue caused the address of functions in shared libraries to resolve to their PLT thunks in the main program rather than their correct addresses. it was observed causing crashes, though the mechanism of the crash was not thoroughly investigated. since the issue is very subtle, it calls for some explanation: on all well-behaved archs, GOT entries that belong to the PLT use a special relocation type, typically called JMP_SLOT, so that the dynamic linker can avoid having the jump destinations for the PLT resolve to PLT thunks themselves (they also provide a definition for the symbol, which must be used whenever the address of the function is taken so that all DSOs see the same address). however, the traditional mips PIC ABI lacked such a JMP_SLOT relocation type, presumably because, due to the way PIC works, the address of the PLT thunk was never needed and could always be ignored. prior to commit adf94c19666e687a728bbf398f9a88ea4ea19996, the mips version of reloc.h contained a hack that caused all symbol lookups to be treated like JMP_SLOT, inhibiting undefined symbols from ever being used to resolve symbolic relocations. this hack goes all the way back to commit babf820180368f00742ec65b2050a82380d7c542, when the mips dynamic linker was first made usable. during the recent refactoring to eliminate arch-specific relocation processing (commit adf94c19666e687a728bbf398f9a88ea4ea19996), this hack was overlooked and no equivalent functionality was provided in the new code. fixing the problem is not as simple as adding back an equivalent hack, since there is now also a "non-PIC ABI" that can be used for the main executable, which actually does use a PLT. the closest thing to official documentation I could find for this ABI is nonpic.txt, attached to Message-ID: 20080701202236.GA1534@caradoc.them.org, which can be found in the gcc mailing list archives and elsewhere. per this document, undefined symbols corresponding to PLT thunks have the STO_MIPS_PLT bit set in the symbol's st_other field. thus, I have added an arch-specific rule for mips, applied at the find_sym level rather than the relocation level, to reject undefined symbols with the STO_MIPS_PLT bit clear. the previous hack of treating all mips relocations as JMP_SLOT-like, rather than rejecting the unwanted symbols in find_sym, probably also caused dlsym to wrongly return PLT thunks in place of the correct address of a function under at least some conditions. this should now be fixed, at least for global-scope symbol lookups.
2014-06-30 05:18:14 +00:00
#ifndef ARCH_SYM_REJECT_UND
#define ARCH_SYM_REJECT_UND(s) 0
#endif
make relocation time symbol lookup and dlsym consistent Using common code path for all symbol lookups fixes three dlsym issues: - st_shndx of STT_TLS symbols were not checked and thus an undefined tls symbol reference could be incorrectly treated as a definition (the sysv hash lookup returns undefined symbols, gnu does not, so should be rare in practice). - symbol binding was not checked so a hidden symbol may be returned (in principle STB_LOCAL symbols may appear in the dynamic symbol table for hidden symbols, but linkers most likely don't produce it). - mips specific behaviour was not applied (ARCH_SYM_REJECT_UND) so undefined symbols may be returned on mips. always_inline is used to avoid relocation performance regression, the code generation for find_sym should not be affected.
2019-08-10 23:14:40 +00:00
#if defined(__GNUC__)
__attribute__((always_inline))
#endif
static inline struct symdef find_sym2(struct dso *dso, const char *s, int need_def, int use_deps)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
precalculate gnu hash rather than doing it lazily in find_sym inner loop this change was suggested based on testing done by Timo Teräs almost two years ago; the branch (and probably call prep overhead) in the inner loop was found to contribute noticably to total symbol lookup time. this change will make lookup slightly slower if libraries were built with only the traditional "sysv" ELF hash table, but based on how much slower lookup tends to be without the gnu hash table, it seems reasonable to assume that (1) users building without gnu hash don't care about dynamic linking performance, and (2) the extra time spent computing the gnu hash is likely to be dominated by the slowness of the sysv hash table lookup anyway.
2017-03-15 20:50:19 +00:00
uint32_t h = 0, gh = gnu_hash(s), gho = gh / (8*sizeof(size_t)), *ght;
size_t ghm = 1ul << gh % (8*sizeof(size_t));
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
struct symdef def = {0};
make relocation time symbol lookup and dlsym consistent Using common code path for all symbol lookups fixes three dlsym issues: - st_shndx of STT_TLS symbols were not checked and thus an undefined tls symbol reference could be incorrectly treated as a definition (the sysv hash lookup returns undefined symbols, gnu does not, so should be rare in practice). - symbol binding was not checked so a hidden symbol may be returned (in principle STB_LOCAL symbols may appear in the dynamic symbol table for hidden symbols, but linkers most likely don't produce it). - mips specific behaviour was not applied (ARCH_SYM_REJECT_UND) so undefined symbols may be returned on mips. always_inline is used to avoid relocation performance regression, the code generation for find_sym should not be affected.
2019-08-10 23:14:40 +00:00
struct dso **deps = use_deps ? dso->deps : 0;
for (; dso; dso=use_deps ? *deps++ : dso->syms_next) {
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
Sym *sym;
dynlink.c: pass gnu-hash table pointer to gnu_lookup The callers need to check the value of the pointer anyway, so make them pass the pointer to gnu_lookup instead of reloading it there. Reorder gnu_lookup arguments so that always-used ones are listed first. GCC can choose a calling convention with arguments in registers (e.g. up to 3 arguments in eax, ecx, edx on x86), but cannot reorder the arguments for static functions.
2015-06-27 23:48:33 +00:00
if ((ght = dso->ghashtab)) {
sym = gnu_lookup_filtered(gh, ght, dso, s, gho, ghm);
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
} else {
if (!h) h = sysv_hash(s);
sym = sysv_lookup(s, h, dso);
}
fix symbol acceptance/rejection rules for TLS symbol value of 0 is not "undefined" for TLS; it's the address of the first symbol in the TLS segment. however, non-definition TLS references also have values of 0, so check the section. hopefully the new logic is more clear, too.
2012-10-06 05:36:11 +00:00
if (!sym) continue;
if (!sym->st_shndx)
fix regression in mips dynamic linker this issue caused the address of functions in shared libraries to resolve to their PLT thunks in the main program rather than their correct addresses. it was observed causing crashes, though the mechanism of the crash was not thoroughly investigated. since the issue is very subtle, it calls for some explanation: on all well-behaved archs, GOT entries that belong to the PLT use a special relocation type, typically called JMP_SLOT, so that the dynamic linker can avoid having the jump destinations for the PLT resolve to PLT thunks themselves (they also provide a definition for the symbol, which must be used whenever the address of the function is taken so that all DSOs see the same address). however, the traditional mips PIC ABI lacked such a JMP_SLOT relocation type, presumably because, due to the way PIC works, the address of the PLT thunk was never needed and could always be ignored. prior to commit adf94c19666e687a728bbf398f9a88ea4ea19996, the mips version of reloc.h contained a hack that caused all symbol lookups to be treated like JMP_SLOT, inhibiting undefined symbols from ever being used to resolve symbolic relocations. this hack goes all the way back to commit babf820180368f00742ec65b2050a82380d7c542, when the mips dynamic linker was first made usable. during the recent refactoring to eliminate arch-specific relocation processing (commit adf94c19666e687a728bbf398f9a88ea4ea19996), this hack was overlooked and no equivalent functionality was provided in the new code. fixing the problem is not as simple as adding back an equivalent hack, since there is now also a "non-PIC ABI" that can be used for the main executable, which actually does use a PLT. the closest thing to official documentation I could find for this ABI is nonpic.txt, attached to Message-ID: 20080701202236.GA1534@caradoc.them.org, which can be found in the gcc mailing list archives and elsewhere. per this document, undefined symbols corresponding to PLT thunks have the STO_MIPS_PLT bit set in the symbol's st_other field. thus, I have added an arch-specific rule for mips, applied at the find_sym level rather than the relocation level, to reject undefined symbols with the STO_MIPS_PLT bit clear. the previous hack of treating all mips relocations as JMP_SLOT-like, rather than rejecting the unwanted symbols in find_sym, probably also caused dlsym to wrongly return PLT thunks in place of the correct address of a function under at least some conditions. this should now be fixed, at least for global-scope symbol lookups.
2014-06-30 05:18:14 +00:00
if (need_def || (sym->st_info&0xf) == STT_TLS
|| ARCH_SYM_REJECT_UND(sym))
fix symbol acceptance/rejection rules for TLS symbol value of 0 is not "undefined" for TLS; it's the address of the first symbol in the TLS segment. however, non-definition TLS references also have values of 0, so check the section. hopefully the new logic is more clear, too.
2012-10-06 05:36:11 +00:00
continue;
if (!sym->st_value)
if ((sym->st_info&0xf) != STT_TLS)
continue;
if (!(1<<(sym->st_info&0xf) & OK_TYPES)) continue;
if (!(1<<(sym->st_info>>4) & OK_BINDS)) continue;
def.sym = sym;
def.dso = dso;
treat STB_WEAK and STB_GNU_UNIQUE like STB_GLOBAL in find_sym A weak symbol definition is not special during dynamic linking, so don't let a strong definition in a later module override it. (glibc dynamic linker allows overriding weak definitions if LD_DYNAMIC_WEAK is set, musl does not.) STB_GNU_UNIQUE means that the symbol is global, even if it is in a module that's loaded with RTLD_LOCAL, and all references resolve to the same definition. This semantics is only relevant for c++ plugin systems and even there it's often not what the user wants (so it can be turned off in g++ by -fno-gnu-unique when the c++ shared lib is compiled). In musl just treat it like STB_GLOBAL.
2016-12-03 20:52:43 +00:00
break;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
fix resolution of weak symbols (hopefully right now) and vdso
2011-07-24 06:19:47 +00:00
return def;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
make relocation time symbol lookup and dlsym consistent Using common code path for all symbol lookups fixes three dlsym issues: - st_shndx of STT_TLS symbols were not checked and thus an undefined tls symbol reference could be incorrectly treated as a definition (the sysv hash lookup returns undefined symbols, gnu does not, so should be rare in practice). - symbol binding was not checked so a hidden symbol may be returned (in principle STB_LOCAL symbols may appear in the dynamic symbol table for hidden symbols, but linkers most likely don't produce it). - mips specific behaviour was not applied (ARCH_SYM_REJECT_UND) so undefined symbols may be returned on mips. always_inline is used to avoid relocation performance regression, the code generation for find_sym should not be affected.
2019-08-10 23:14:40 +00:00
static struct symdef find_sym(struct dso *dso, const char *s, int need_def)
{
return find_sym2(dso, s, need_def, 0);
}
more cleanup of dynamic linker internals
2012-08-05 06:49:02 +00:00
static void do_relocs(struct dso *dso, size_t *rel, size_t rel_size, size_t stride)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
more cleanup of dynamic linker internals
2012-08-05 06:49:02 +00:00
unsigned char *base = dso->base;
Sym *syms = dso->syms;
char *strings = dso->strings;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
Sym *sym;
const char *name;
void *ctx;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
int type;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
int sym_index;
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
struct symdef def;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
size_t *reloc_addr;
size_t sym_val;
size_t tls_val;
size_t addend;
reprocess all libc/ldso symbolic relocations in dynamic linking stage 3 commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-26 03:33:59 +00:00
int skip_relative = 0, reuse_addends = 0, save_slot = 0;
if (dso == &ldso) {
/* Only ldso's REL table needs addend saving/reuse. */
if (rel == apply_addends_to)
reuse_addends = 1;
skip_relative = 1;
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
for (; rel_size; rel+=stride, rel_size-=stride*sizeof(size_t)) {
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (skip_relative && IS_RELATIVE(rel[1], dso->syms)) continue;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
type = R_TYPE(rel[1]);
fix dynamic linker regression processing R_*_NONE type relocations commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 inadvertently removed the early check for "none" type relocations, causing the address dso->base+0 to be dereferenced to obtain an addend. shared libraries, (including libc.so) and PIE executables were unaffected, since their base addresses are the actual address of their mappings and are readable. non-PIE main executables, however, have a base address of 0 because their load addresses are absolute and not offset at load time. in practice none-type relocations do not arise with toolchains that are in use except on mips, and on mips it's moderately rare for a non-PIE executable to have a relocation table, since the mips-specific got processing serves in its place for most purposes.
2015-06-04 15:45:17 +00:00
if (type == REL_NONE) continue;
further refactoring of dynamic linker load address computations these are in do_relocs. the first one was omitted in commit 301335a80b85f12c018e4acf1a2c28615e119f8d because it slightly changes code (using dso->base rather than cached local var base) and would have prevented easy verification. the other was an oversight.
2015-09-17 17:50:43 +00:00
reloc_addr = laddr(dso, rel[0]);
reorder addend handling before symbol lookup in relocation code these two tasks are independent now, but in order to support lazy relocations, the failure path for symbol lookup may want the addend to be available.
2017-03-13 04:30:26 +00:00
if (stride > 2) {
addend = rel[2];
} else if (type==REL_GOT || type==REL_PLT|| type==REL_COPY) {
addend = 0;
} else if (reuse_addends) {
/* Save original addend in stage 2 where the dso
* chain consists of just ldso; otherwise read back
* saved addend since the inline one was clobbered. */
if (head==&ldso)
saved_addends[save_slot] = *reloc_addr;
addend = saved_addends[save_slot++];
} else {
addend = *reloc_addr;
}
sym_index = R_SYM(rel[1]);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
if (sym_index) {
sym = syms + sym_index;
name = strings + sym->st_name;
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
ctx = type==REL_COPY ? head->syms_next : head;
ldso: correct condition for local symbol handling in do_relocs commit 7a9669e977e5f750cf72ccbd2614f8b72ce02c4c added use of the symbol reference as the definition, in place of performing a lookup, for STT_SECTION symbol references that were first found used in FDPIC. such references may happen in certain other cases, such as local-dynamic TLS and with relocation types that require a symbol but that are being used for non-symbolic purposes, like the powerpc unaligned address relocations. in all such cases I'm aware of, the symbol referenced is a section symbol (STT_SECTION); however, the important semantic property is not its being a section, but rather its binding local (STB_LOCAL). check the latter instead of the former for greater generality and semantic correctness.
2019-08-12 22:19:38 +00:00
def = (sym->st_info>>4) == STB_LOCAL
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
? (struct symdef){ .dso = dso, .sym = sym }
: find_sym(ctx, name, type==REL_PLT);
fix crash in dynamic linker when certain copy relocations are unsatisfied STB_WEAK is only a weak reference for undefined symbols (those with a section of SHN_UNDEF). otherwise, it's a weak definition. normally this distinction would not matter, since a relocation referencing a symbol that also provides a definition (not SHN_UNDEF) will always succeed in finding the referenced symbol itself. however, in the case of copy relocations, the referenced symbol itself is ignored in order to search for another symbol to copy from, and thus it's possible that no definition is found. in this case, if the symbol being resolved happened to be a weak definition, it was misinterpreted as a weak reference, suppressing the error path and causing a crash when the copy relocation was performed with a null source pointer passed to memcpy. there are almost certainly still situations in which invalid combinations of symbol and relocation types can cause the dynamic linker to crash (this is pretty much inevitable), but the intent is that crashes not be possible for symbol/relocation tables produced by a valid linker.
2014-01-21 05:36:35 +00:00
if (!def.sym && (sym->st_shndx != SHN_UNDEF
|| sym->st_info>>4 != STB_WEAK)) {
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
if (dso->lazy && (type==REL_PLT || type==REL_GOT)) {
dso->lazy[3*dso->lazy_cnt+0] = rel[0];
dso->lazy[3*dso->lazy_cnt+1] = rel[1];
dso->lazy[3*dso->lazy_cnt+2] = addend;
dso->lazy_cnt++;
continue;
}
fix regression in dynamic linker error reporting due to a mistake when refactoring the error printing for the dynamic linker (commit 7c73cacd09a51a87484db5689864743e4984a84d), all messages were suppressed and replaced by blank lines.
2014-06-30 01:52:54 +00:00
error("Error relocating %s: %s: symbol not found",
more cleanup of dynamic linker internals
2012-08-05 06:49:02 +00:00
dso->name, name);
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
if (runtime) longjmp(*rtld_fail, 1);
make dynamic linker report all failures before exiting before, only the first library that failed to load or symbol that failed to resolve was reported, and then the dynamic linker immediately exited. when attempting to fix a library compatibility issue, this is about the worst possible behavior. now we print all errors as they occur and exit at the very end if errors were encountered.
2012-08-18 20:00:23 +00:00
continue;
error handling in dynamic linking some of the code is not yet used, and is in preparation for dlopen which needs to be able to handle failure loading libraries without terminating the program.
2011-06-26 21:39:17 +00:00
}
more changes that were lost when committing mips dynamic linker
2012-08-05 18:03:17 +00:00
} else {
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
sym = 0;
def.sym = 0;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
def.dso = dso;
}
further refactoring of dynamic linker load address computations these are in do_relocs. the first one was omitted in commit 301335a80b85f12c018e4acf1a2c28615e119f8d because it slightly changes code (using dso->base rather than cached local var base) and would have prevented easy verification. the other was an oversight.
2015-09-17 17:50:43 +00:00
sym_val = def.sym ? (size_t)laddr(def.dso, def.sym->st_value) : 0;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
tls_val = def.sym ? def.sym->st_value : 0;
block dlopen of libraries with initial-exec refs to dynamic TLS previously, this operation succeeded, and the relocation results worked for access from new threads created after dlopen, but produced invalid accesses (and possibly clobbered other memory) from threads that already existed. the way the check is written, it still permits dlopen of libraries containing initial-exec references to static TLS (TLS in the main program or in a dynamic library loaded at startup).
2018-07-16 16:32:57 +00:00
if ((type == REL_TPOFF || type == REL_TPOFF_NEG)
ldso: remove redundant runtime checks in static TLS logic as a result of commit ffab43602b5900c86b7040abdda8ccf6cdec95f5, static_tls_cnt is now valid during relocations at program startup, so it's no longer necessary to condition the check against static_tls_cnt on this being a runtime (dlopen) relocation.
2019-08-11 15:57:38 +00:00
&& def.dso->tls_id > static_tls_cnt) {
block dlopen of libraries with initial-exec refs to dynamic TLS previously, this operation succeeded, and the relocation results worked for access from new threads created after dlopen, but produced invalid accesses (and possibly clobbered other memory) from threads that already existed. the way the check is written, it still permits dlopen of libraries containing initial-exec references to static TLS (TLS in the main program or in a dynamic library loaded at startup).
2018-07-16 16:32:57 +00:00
error("Error relocating %s: %s: initial-exec TLS "
"resolves to dynamic definition in %s",
dso->name, name, def.dso->name);
longjmp(*rtld_fail, 1);
}
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
switch(type) {
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
case REL_NONE:
break;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
case REL_OFFSET:
addend -= (size_t)reloc_addr;
case REL_SYMBOLIC:
case REL_GOT:
case REL_PLT:
*reloc_addr = sym_val + addend;
break;
add support for powerpc/powerpc64 unaligned relocations R_PPC_UADDR32 (R_PPC64_UADDR64) has the same meaning as R_PPC_ADDR32 (R_PPC64_ADDR64), except that its address need not be aligned. For powerpc64, BFD ld(1) will automatically convert between ADDR<->UADDR relocations when the address is/isn't at its native alignment. This will happen if, for example, there is a pointer in a packed struct. gold and lld do not currently generate R_PPC64_UADDR64, but pass through misaligned R_PPC64_ADDR64 relocations from object files, possibly relaxing them to misaligned R_PPC64_RELATIVE. In both cases (relaxed or not) this violates the PSABI, which defines the relevant field type as "a 64-bit field occupying 8 bytes, the alignment of which is 8 bytes unless otherwise specified." All three linkers violate the PSABI on 32-bit powerpc, where the only difference is that the field is 32 bits wide, aligned to 4 bytes. Currently musl fails to load executables linked by BFD ld containing R_PPC64_UADDR64, with the error "unsupported relocation type 43". This change provides compatibility with BFD ld on powerpc64, and any static linker on either architecture that starts following the PSABI more closely.
2019-06-30 12:39:20 +00:00
case REL_USYMBOLIC:
memcpy(reloc_addr, &(size_t){sym_val + addend}, sizeof(size_t));
break;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
case REL_RELATIVE:
*reloc_addr = (size_t)base + addend;
break;
case REL_SYM_OR_REL:
if (sym) *reloc_addr = sym_val + addend;
else *reloc_addr = (size_t)base + addend;
break;
case REL_COPY:
memcpy(reloc_addr, (void *)sym_val, sym->st_size);
break;
case REL_OFFSET32:
*(uint32_t *)reloc_addr = sym_val + addend
- (size_t)reloc_addr;
break;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
case REL_FUNCDESC:
*reloc_addr = def.sym ? (size_t)(def.dso->funcdescs
+ (def.sym - def.dso->syms)) : 0;
break;
case REL_FUNCDESC_VAL:
if ((sym->st_info&0xf) == STT_SECTION) *reloc_addr += sym_val;
else *reloc_addr = sym_val;
reloc_addr[1] = def.sym ? (size_t)def.dso->got : 0;
break;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
case REL_DTPMOD:
*reloc_addr = def.dso->tls_id;
break;
case REL_DTPOFF:
fix local-dynamic model TLS on mips and powerpc the TLS ABI spec for mips, powerpc, and some other (presently unsupported) RISC archs has the return value of __tls_get_addr offset by +0x8000 and the result of DTPOFF relocations offset by -0x8000. I had previously assumed this part of the ABI was actually just an implementation detail, since the adjustments cancel out. however, when the local dynamic model is used for accessing TLS that's known to be in the same DSO, either of the following may happen: 1. the -0x8000 offset may already be applied to the argument structure passed to __tls_get_addr at ld time, without any opportunity for runtime relocations. 2. __tls_get_addr may be used with a zero offset argument to obtain a base address for the module's TLS, to which the caller then applies immediate offsets for individual objects accessed using the local dynamic model. since the immediate offsets have the -0x8000 adjustment applied to them, the base address they use needs to include the +0x8000 offset. it would be possible, but more complex, to store the pointers in the dtv[] array with the +0x8000 offset pre-applied, to avoid the runtime cost of adding 0x8000 on each call to __tls_get_addr. this change could be made later if measurements show that it would help.
2015-06-25 22:22:00 +00:00
*reloc_addr = tls_val + addend - DTP_OFFSET;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
break;
#ifdef TLS_ABOVE_TP
case REL_TPOFF:
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
*reloc_addr = tls_val + def.dso->tls.offset + TPOFF_K + addend;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
break;
#else
case REL_TPOFF:
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
*reloc_addr = tls_val - def.dso->tls.offset + addend;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
break;
case REL_TPOFF_NEG:
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
*reloc_addr = def.dso->tls.offset - tls_val + addend;
refactor to remove arch-specific relocation code from dynamic linker this was one of the main instances of ugly code duplication: all archs use basically the same types of relocations, but roughly equivalent logic was duplicated for each arch to account for the different naming and numbering of relocation types and variation in whether REL or RELA records are used. as an added bonus, both REL and RELA are now supported on all archs, regardless of which is used by the standard toolchain.
2014-06-18 06:44:02 +00:00
break;
#endif
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
case REL_TLSDESC:
if (stride<3) addend = reloc_addr[1];
ldso: remove redundant runtime checks in static TLS logic as a result of commit ffab43602b5900c86b7040abdda8ccf6cdec95f5, static_tls_cnt is now valid during relocations at program startup, so it's no longer necessary to condition the check against static_tls_cnt on this being a runtime (dlopen) relocation.
2019-08-11 15:57:38 +00:00
if (def.dso->tls_id > static_tls_cnt) {
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
struct td_index *new = malloc(sizeof *new);
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
if (!new) {
error(
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
"Error relocating %s: cannot allocate TLSDESC for %s",
dso->name, sym ? name : "(local)" );
remove always-true conditional in dynamic linker TLSDESC processing the allocating path which can fail is for dynamic TLS, which can only occur at runtime, and the check for runtime was already made in the outer conditional.
2015-04-21 17:22:48 +00:00
longjmp(*rtld_fail, 1);
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
}
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
new->next = dso->td_index;
dso->td_index = new;
new->args[0] = def.dso->tls_id;
combine arch ABI's DTP_OFFSET into DTV pointers as explained in commit 6ba5517a460c6c438f64d69464fdfc3269a4c91a, some archs use an offset (typicaly -0x8000) with their DTPOFF relocations, which __tls_get_addr needs to invert. on affected archs, which lack direct support for large immediates, this can cost multiple extra instructions in the hot path. instead, incorporate the DTP_OFFSET into the DTV entries. this means they are no longer valid pointers, so store them as an array of uintptr_t rather than void *; this also makes it easier to access slot 0 as a valid slot count. commit e75b16cf93ebbc1ce758d3ea6b2923e8b2457c68 left behind cruft in two places, __reset_tls and __tls_get_new, from back when it was possible to have uninitialized gap slots indicated by a null pointer in the DTV. since the concept of null pointer is no longer meaningful with an offset applied, remove this cruft. presently there are no archs with both TLSDESC and nonzero DTP_OFFSET, but the dynamic TLSDESC relocation code is also updated to apply an inverted offset to its offset field, so that the offset DTV would not impose a runtime cost in TLSDESC resolver functions.
2018-10-12 04:30:34 +00:00
new->args[1] = tls_val + addend - DTP_OFFSET;
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
reloc_addr[0] = (size_t)__tlsdesc_dynamic;
reloc_addr[1] = (size_t)new;
} else {
reloc_addr[0] = (size_t)__tlsdesc_static;
#ifdef TLS_ABOVE_TP
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
reloc_addr[1] = tls_val + def.dso->tls.offset
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
+ TPOFF_K + addend;
#else
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
reloc_addr[1] = tls_val - def.dso->tls.offset
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
+ addend;
#endif
}
add TLSDESC support for 32-bit arm unlike other asm where the baseline ISA is used, these functions are hot paths and use ISA-level specializations. call-clobbered vfp registers are saved before calling __tls_get_new, since there is no guarantee it won't use them. while setjmp/longjmp have to use hwcap to decide whether to the fpu is in use, since application code could be using vfp registers even if libc was compiled as pure softfloat, __tls_get_new is part of libc and can be assumed not to have access to vfp registers if tlsdesc.S does not. thus it suffices just to check the predefined preprocessor macros. the check for __ARM_PCS_VFP is redundant; !__SOFTFP__ must always be true if the target ISA level includes fpu instructions/registers.
2018-10-01 22:37:02 +00:00
#ifdef TLSDESC_BACKWARDS
/* Some archs (32-bit ARM at least) invert the order of
* the descriptor members. Fix them up here. */
size_t tmp = reloc_addr[0];
reloc_addr[0] = reloc_addr[1];
reloc_addr[1] = tmp;
#endif
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
break;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
default:
error("Error relocating %s: unsupported relocation type %d",
dso->name, type);
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
if (runtime) longjmp(*rtld_fail, 1);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
continue;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
}
}
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
static void redo_lazy_relocs()
{
struct dso *p = lazy_head, *next;
lazy_head = 0;
for (; p; p=next) {
next = p->lazy_next;
size_t size = p->lazy_cnt*3*sizeof(size_t);
p->lazy_cnt = 0;
do_relocs(p, p->lazy, size, 3);
if (p->lazy_cnt) {
p->lazy_next = lazy_head;
lazy_head = p;
} else {
free(p->lazy);
p->lazy = 0;
p->lazy_next = 0;
}
}
}
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
/* A huge hack: to make up for the wastefulness of shared libraries
* needing at least a page of dirty memory even if they have no global
* data, we reclaim the gaps at the beginning and end of writable maps
ldso, malloc: implement reclaim_gaps via __malloc_donate Split 'free' into unmap_chunk and bin_chunk, use the latter to introduce __malloc_donate and use it in reclaim_gaps instead of calling 'free'.
2018-04-16 17:54:36 +00:00
* and "donate" them to the heap. */
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
static void reclaim(struct dso *dso, size_t start, size_t end)
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
{
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
if (start >= dso->relro_start && start < dso->relro_end) start = dso->relro_end;
if (end >= dso->relro_start && end < dso->relro_end) end = dso->relro_start;
ldso, malloc: implement reclaim_gaps via __malloc_donate Split 'free' into unmap_chunk and bin_chunk, use the latter to introduce __malloc_donate and use it in reclaim_gaps instead of calling 'free'.
2018-04-16 17:54:36 +00:00
if (start >= end) return;
enable reclaim_gaps for fdpic the existing laddr function for fdpic cannot translate ELF virtual addresses outside of the LOAD segments to runtime addresses because the fdpic loadmap only covers the logically-mapped part. however the whole point of reclaim_gaps is to recover the slack space up to the page boundaries, so it needs to work with such addresses. add a new laddr_pg function that accepts any address in the page range for the LOAD segment by expanding the loadmap records out to page boundaries. only use the new version for reclaim_gaps, so as not to impact performance of other address lookups. also, only use laddr_pg for the start address of a gap; the end address lies one byte beyond the end, potentially in a different page where it would get mapped differently. instead of mapping end, apply the length (end-start) to the mapped value of start.
2018-04-17 19:55:18 +00:00
char *base = laddr_pg(dso, start);
__malloc_donate(base, base+(end-start));
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
}
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
static void reclaim_gaps(struct dso *dso)
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
{
cosmetic improvements in dynamic linker cleanup consistent use of braces in if/else structure, line length.
2014-03-25 20:21:50 +00:00
Phdr *ph = dso->phdr;
size_t phcnt = dso->phnum;
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
cosmetic improvements in dynamic linker cleanup consistent use of braces in if/else structure, line length.
2014-03-25 20:21:50 +00:00
for (; phcnt--; ph=(void *)((char *)ph+dso->phentsize)) {
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
if (ph->p_type!=PT_LOAD) continue;
if ((ph->p_flags&(PF_R|PF_W))!=(PF_R|PF_W)) continue;
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
reclaim(dso, ph->p_vaddr & -PAGE_SIZE, ph->p_vaddr);
reclaim(dso, ph->p_vaddr+ph->p_memsz,
reclaim the memory wasted by dynamic linking for use by malloc
2011-06-28 23:40:14 +00:00
ph->p_vaddr+ph->p_memsz+PAGE_SIZE-1 & -PAGE_SIZE);
}
}
make dynamic linker work around MAP_FAILED mmap failure on nommu kernels previously, loading of additional libraries beyond libc/ldso did not work on nommu kernels, nor did loading programs via invocation of the dynamic linker as a command.
2015-06-23 04:03:42 +00:00
static void *mmap_fixed(void *p, size_t n, int prot, int flags, int fd, off_t off)
{
fix dynamic loader library mapping for nommu systems on linux/nommu, non-writable private mappings of files may actually use memory shared with other processes or the fs cache. the old nommu loader code (used when mmap with MAP_FIXED fails) simply wrote over top of the original file mapping, possibly clobbering this shared memory. no such breakage was observed in practice, but it should have been possible. the new code starts by mapping anonymous writable memory on archs that might support nommu, then maps load segments over top of it, falling back to read if MAP_FIXED fails. we use an anonymous map rather than a writable file map to avoid reading more data from disk than needed. since pages cannot be loaded lazily on fault, in case of large data/bss, mapping the full file may read a lot of data that will subsequently be thrown away when processing additional LOAD segments. as a result, we cannot skip the first LOAD segment when operating in this mode. these changes affect only non-FDPIC nommu support.
2015-11-11 22:34:17 +00:00
static int no_map_fixed;
char *q;
if (!no_map_fixed) {
q = mmap(p, n, prot, flags|MAP_FIXED, fd, off);
if (!DL_NOMMU_SUPPORT || q != MAP_FAILED || errno != EINVAL)
return q;
no_map_fixed = 1;
}
make dynamic linker work around MAP_FAILED mmap failure on nommu kernels previously, loading of additional libraries beyond libc/ldso did not work on nommu kernels, nor did loading programs via invocation of the dynamic linker as a command.
2015-06-23 04:03:42 +00:00
/* Fallbacks for MAP_FIXED failure on NOMMU kernels. */
if (flags & MAP_ANONYMOUS) {
memset(p, 0, n);
return p;
}
ssize_t r;
if (lseek(fd, off, SEEK_SET) < 0) return MAP_FAILED;
for (q=p; n; q+=r, off+=r, n-=r) {
r = read(fd, q, n);
if (r < 0 && errno != EINTR) return MAP_FAILED;
if (!r) {
memset(q, 0, n);
break;
}
}
return p;
}
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
static void unmap_library(struct dso *dso)
{
if (dso->loadmap) {
size_t i;
for (i=0; i<dso->loadmap->nsegs; i++) {
if (!dso->loadmap->segs[i].p_memsz)
continue;
munmap((void *)dso->loadmap->segs[i].addr,
dso->loadmap->segs[i].p_memsz);
}
free(dso->loadmap);
} else if (dso->map && dso->map_len) {
munmap(dso->map, dso->map_len);
}
}
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
static void *map_library(int fd, struct dso *dso)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
fix possible (questionable) strict aliasing violations and ptr arith
2011-06-25 16:26:08 +00:00
Ehdr buf[(896+sizeof(Ehdr))/sizeof(Ehdr)];
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
void *allocated_buf=0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
size_t phsize;
size_t addr_min=SIZE_MAX, addr_max=0, map_len;
size_t this_min, this_max;
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
size_t nsegs = 0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
off_t off_start;
Ehdr *eh;
fix invalid library phdr pointers passed to callback from dl_iterate_phdr map_library was saving pointers to an automatic-storage buffer rather than pointers into the mapping. this should be a fairly simple fix, but the patch here is slightly complicated by two issues: 1. supporting gratuitously obfuscated ELF files where the program headers are not right at the beginning of the file. 2. cleaning up the map_library function so that data isn't clobbered by the time we need it.
2013-07-10 18:38:20 +00:00
Phdr *ph, *ph0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
unsigned prot;
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
unsigned char *map=MAP_FAILED, *base;
fix uninitialized dyn variable in map_library this can only happen for invalid library files, but they were not detected reliably because the variable was uninitialized.
2013-08-02 13:25:12 +00:00
size_t dyn=0;
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
size_t tls_image=0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
size_t i;
ssize_t l = read(fd, buf, sizeof buf);
fix possible (questionable) strict aliasing violations and ptr arith
2011-06-25 16:26:08 +00:00
eh = buf;
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
if (l<0) return 0;
if (l<sizeof *eh || (eh->e_type != ET_DYN && eh->e_type != ET_EXEC))
goto noexec;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
phsize = eh->e_phentsize * eh->e_phnum;
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
if (phsize > sizeof buf - sizeof *eh) {
allocated_buf = malloc(phsize);
if (!allocated_buf) return 0;
l = pread(fd, allocated_buf, phsize, eh->e_phoff);
if (l < 0) goto error;
if (l != phsize) goto noexec;
ph = ph0 = allocated_buf;
} else if (eh->e_phoff + phsize > l) {
fix possible (questionable) strict aliasing violations and ptr arith
2011-06-25 16:26:08 +00:00
l = pread(fd, buf+1, phsize, eh->e_phoff);
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
if (l < 0) goto error;
if (l != phsize) goto noexec;
fix invalid library phdr pointers passed to callback from dl_iterate_phdr map_library was saving pointers to an automatic-storage buffer rather than pointers into the mapping. this should be a fairly simple fix, but the patch here is slightly complicated by two issues: 1. supporting gratuitously obfuscated ELF files where the program headers are not right at the beginning of the file. 2. cleaning up the map_library function so that data isn't clobbered by the time we need it.
2013-07-10 18:38:20 +00:00
ph = ph0 = (void *)(buf + 1);
} else {
ph = ph0 = (void *)((char *)buf + eh->e_phoff);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
cosmetic improvements in dynamic linker cleanup consistent use of braces in if/else structure, line length.
2014-03-25 20:21:50 +00:00
if (ph->p_type == PT_DYNAMIC) {
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
dyn = ph->p_vaddr;
cosmetic improvements in dynamic linker cleanup consistent use of braces in if/else structure, line length.
2014-03-25 20:21:50 +00:00
} else if (ph->p_type == PT_TLS) {
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
tls_image = ph->p_vaddr;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
dso->tls.align = ph->p_align;
dso->tls.len = ph->p_filesz;
dso->tls.size = ph->p_memsz;
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
} else if (ph->p_type == PT_GNU_RELRO) {
dso->relro_start = ph->p_vaddr & -PAGE_SIZE;
dso->relro_end = (ph->p_vaddr + ph->p_memsz) & -PAGE_SIZE;
support setting of default thread stack size via PT_GNU_STACK header this facilitates building software that assumes a large default stack size without any patching to call pthread_setattr_default_np or pthread_attr_setstacksize at each thread creation site, using just LDFLAGS. normally the PT_GNU_STACK header is used only to reflect whether executable stack is desired, but with GNU ld at least, passing -Wl,-z,stack-size=N will set a size on the program header. with this patch, that size will be incorporated into the default stack size (subject to increase-only rule and DEFAULT_STACK_MAX limit). both static and dynamic linking honor the program header. for dynamic linking, all libraries loaded at program start, including preloaded ones, are considered. dlopened libraries are not considered, for several reasons. extra logic would be needed to defer processing until the load of the new library is commited, synchronization woud be needed since other threads may be running concurrently, and the effectiveness woud be limited since the larger size would not apply to threads that already existed at the time of dlopen. programs that will dlopen code expecting a large stack need to declare the requirement themselves, or pthread_setattr_default_np can be used.
2018-09-19 03:54:18 +00:00
} else if (ph->p_type == PT_GNU_STACK) {
if (!runtime && ph->p_memsz > __default_stacksize) {
__default_stacksize =
ph->p_memsz < DEFAULT_STACK_MAX ?
ph->p_memsz : DEFAULT_STACK_MAX;
}
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
if (ph->p_type != PT_LOAD) continue;
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
nsegs++;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
if (ph->p_vaddr < addr_min) {
addr_min = ph->p_vaddr;
off_start = ph->p_offset;
prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
((ph->p_flags&PF_X) ? PROT_EXEC : 0));
}
if (ph->p_vaddr+ph->p_memsz > addr_max) {
addr_max = ph->p_vaddr+ph->p_memsz;
}
}
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
if (!dyn) goto noexec;
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
if (DL_FDPIC && !(eh->e_flags & FDPIC_CONSTDISP_FLAG)) {
dso->loadmap = calloc(1, sizeof *dso->loadmap
+ nsegs * sizeof *dso->loadmap->segs);
if (!dso->loadmap) goto error;
dso->loadmap->nsegs = nsegs;
for (ph=ph0, i=0; i<nsegs; ph=(void *)((char *)ph+eh->e_phentsize)) {
if (ph->p_type != PT_LOAD) continue;
prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
((ph->p_flags&PF_X) ? PROT_EXEC : 0));
map = mmap(0, ph->p_memsz + (ph->p_vaddr & PAGE_SIZE-1),
use private maps even for read-only segments of FDPIC libraries the nommu kernel shares memory when it can anyway for private read-only maps, but semantically the map should be private. this can make a difference when debugging breakpoints are to be used, in which case the kernel may need to ensure that the mapping is not shared. the new behavior matches how the kernel FDPIC loader maps the main program and/or program interpreter (dynamic linker) binary.
2015-11-16 02:28:41 +00:00
prot, MAP_PRIVATE,
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
fd, ph->p_offset & -PAGE_SIZE);
if (map == MAP_FAILED) {
unmap_library(dso);
goto error;
}
dso->loadmap->segs[i].addr = (size_t)map +
(ph->p_vaddr & PAGE_SIZE-1);
dso->loadmap->segs[i].p_vaddr = ph->p_vaddr;
dso->loadmap->segs[i].p_memsz = ph->p_memsz;
i++;
fix missing bss handling in FDPIC ELF loader when a library being loaded has bss (i.e. data segment with p_memsz>p_filesz), this region needs to be zeroed with a combination of memset and/or mmap. the regular ELF loader always did this but the FDPIC code path omitted it, leading to objects in bss having uninitialized/junk contents.
2015-10-29 01:45:31 +00:00
if (prot & PROT_WRITE) {
size_t brk = (ph->p_vaddr & PAGE_SIZE-1)
+ ph->p_filesz;
size_t pgbrk = brk + PAGE_SIZE-1 & -PAGE_SIZE;
size_t pgend = brk + ph->p_memsz - ph->p_filesz
+ PAGE_SIZE-1 & -PAGE_SIZE;
if (pgend > pgbrk && mmap_fixed(map+pgbrk,
pgend-pgbrk, prot,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,
-1, off_start) == MAP_FAILED)
goto error;
memset(map + brk, 0, pgbrk-brk);
}
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
}
map = (void *)dso->loadmap->segs[0].addr;
map_len = 0;
goto done_mapping;
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
addr_max += PAGE_SIZE-1;
addr_max &= -PAGE_SIZE;
addr_min &= -PAGE_SIZE;
off_start &= -PAGE_SIZE;
map_len = addr_max - addr_min + off_start;
/* The first time, we map too much, possibly even more than
* the length of the file. This is okay because we will not
* use the invalid part; we just need to reserve the right
* amount of virtual address space to map over later. */
fix dynamic loader library mapping for nommu systems on linux/nommu, non-writable private mappings of files may actually use memory shared with other processes or the fs cache. the old nommu loader code (used when mmap with MAP_FIXED fails) simply wrote over top of the original file mapping, possibly clobbering this shared memory. no such breakage was observed in practice, but it should have been possible. the new code starts by mapping anonymous writable memory on archs that might support nommu, then maps load segments over top of it, falling back to read if MAP_FIXED fails. we use an anonymous map rather than a writable file map to avoid reading more data from disk than needed. since pages cannot be loaded lazily on fault, in case of large data/bss, mapping the full file may read a lot of data that will subsequently be thrown away when processing additional LOAD segments. as a result, we cannot skip the first LOAD segment when operating in this mode. these changes affect only non-FDPIC nommu support.
2015-11-11 22:34:17 +00:00
map = DL_NOMMU_SUPPORT
? mmap((void *)addr_min, map_len, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
: mmap((void *)addr_min, map_len, prot,
MAP_PRIVATE, fd, off_start);
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
if (map==MAP_FAILED) goto error;
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
dso->map = map;
dso->map_len = map_len;
add some sanity checks in dynamic loader code reject elf files which are not ET_EXEC/ET_DYN type as bad exec format, and reject ET_EXEC files when they cannot be loaded at the correct address, since they are not relocatable at runtime. the main practical benefit of this is to make dlopen of the main program fail rather than producing an unsafe-to-use handle.
2013-07-31 18:42:08 +00:00
/* If the loaded file is not relocatable and the requested address is
* not available, then the load operation must fail. */
if (eh->e_type != ET_DYN && addr_min && map!=(void *)addr_min) {
errno = EBUSY;
goto error;
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
base = map - addr_min;
fix invalid library phdr pointers passed to callback from dl_iterate_phdr map_library was saving pointers to an automatic-storage buffer rather than pointers into the mapping. this should be a fairly simple fix, but the patch here is slightly complicated by two issues: 1. supporting gratuitously obfuscated ELF files where the program headers are not right at the beginning of the file. 2. cleaning up the map_library function so that data isn't clobbered by the time we need it.
2013-07-10 18:38:20 +00:00
dso->phdr = 0;
dso->phnum = 0;
for (ph=ph0, i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
if (ph->p_type != PT_LOAD) continue;
fix invalid library phdr pointers passed to callback from dl_iterate_phdr map_library was saving pointers to an automatic-storage buffer rather than pointers into the mapping. this should be a fairly simple fix, but the patch here is slightly complicated by two issues: 1. supporting gratuitously obfuscated ELF files where the program headers are not right at the beginning of the file. 2. cleaning up the map_library function so that data isn't clobbered by the time we need it.
2013-07-10 18:38:20 +00:00
/* Check if the programs headers are in this load segment, and
* if so, record the address for use by dl_iterate_phdr. */
if (!dso->phdr && eh->e_phoff >= ph->p_offset
&& eh->e_phoff+phsize <= ph->p_offset+ph->p_filesz) {
dso->phdr = (void *)(base + ph->p_vaddr
+ (eh->e_phoff-ph->p_offset));
dso->phnum = eh->e_phnum;
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
dso->phentsize = eh->e_phentsize;
fix invalid library phdr pointers passed to callback from dl_iterate_phdr map_library was saving pointers to an automatic-storage buffer rather than pointers into the mapping. this should be a fairly simple fix, but the patch here is slightly complicated by two issues: 1. supporting gratuitously obfuscated ELF files where the program headers are not right at the beginning of the file. 2. cleaning up the map_library function so that data isn't clobbered by the time we need it.
2013-07-10 18:38:20 +00:00
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
this_min = ph->p_vaddr & -PAGE_SIZE;
this_max = ph->p_vaddr+ph->p_memsz+PAGE_SIZE-1 & -PAGE_SIZE;
off_start = ph->p_offset & -PAGE_SIZE;
prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
((ph->p_flags&PF_X) ? PROT_EXEC : 0));
fix dynamic linker mapping/clearing bss in first/only LOAD segment writable load segments can have size-in-memory larger than their size in the ELF file, representing bss or equivalent. the initial partial page has to be zero-filled, and additional anonymous pages have to be mapped such that accesses don't failt with SIGBUS. map_library skips redundant MAP_FIXED mapping of the initial (lowest-address) segment when processing LOAD segments since it was already mapped when reserving the virtual address range, but in doing so, inadvertently also skipped the code to fill/map bss. typical executable and library files have two or more LOAD segments, and the first one is text/rodata (non-writable) and thus has no bss, but it is syntactically valid for an ELF program/library to put its writable segment first, or to have only one segment (everything writable). the binutils bfd-based linker has been observed to create such programs in the presence of unusual sections or linker scripts. fix by moving only the mmap_fixed operation under the conditional rather than skipping the remainder of the loop body. add a check to avoid bss processing in the case where the segment is not writable; this should not happen, but if it does, the change would be a crashing regression without this check.
2018-06-26 16:15:13 +00:00
/* Reuse the existing mapping for the lowest-address LOAD */
if ((ph->p_vaddr & -PAGE_SIZE) != addr_min || DL_NOMMU_SUPPORT)
if (mmap_fixed(base+this_min, this_max-this_min, prot, MAP_PRIVATE|MAP_FIXED, fd, off_start) == MAP_FAILED)
goto error;
if (ph->p_memsz > ph->p_filesz && (ph->p_flags&PF_W)) {
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
size_t brk = (size_t)base+ph->p_vaddr+ph->p_filesz;
size_t pgbrk = brk+PAGE_SIZE-1 & -PAGE_SIZE;
memset((void *)brk, 0, pgbrk-brk & PAGE_SIZE-1);
make dynamic linker work around MAP_FAILED mmap failure on nommu kernels previously, loading of additional libraries beyond libc/ldso did not work on nommu kernels, nor did loading programs via invocation of the dynamic linker as a command.
2015-06-23 04:03:42 +00:00
if (pgbrk-(size_t)base < this_max && mmap_fixed((void *)pgbrk, (size_t)base+this_max-pgbrk, prot, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) == MAP_FAILED)
treat failure of mprotect in map_library as a fatal load failure the error will propagate up and be printed to the user at program start time; at runtime, dlopen will just fail and leave a message for dlerror. previously, if mprotect failed, subsequent attempts to perform relocations would crash the program. this was resulting in an increasing number of false bug reports on grsec systems where rwx permission is not possible in cases where users were wrongly attempting to use non-PIC code in shared libraries. supporting that usage is in theory possible, but the x86_64 toolchain does not even support textrels, and the cost of keeping around the necessary information to handle textrels without rwx permissions is disproportionate to the benefit (which is essentially just supporting broken library setups on grsec machines). also, i unified the error-out code in map_library now that there are 3 places from which munmap might have to be called.
2012-06-06 15:21:28 +00:00
goto error;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
}
textrel support, cheap and ugly
2011-06-29 04:29:08 +00:00
for (i=0; ((size_t *)(base+dyn))[i]; i+=2)
if (((size_t *)(base+dyn))[i]==DT_TEXTREL) {
ignore ENOSYS error from mprotect in pthread_create and dynamic linker this error simply indicated a system without memory protection (NOMMU) and should not cause failure in the caller.
2015-06-17 17:21:46 +00:00
if (mprotect(map, map_len, PROT_READ|PROT_WRITE|PROT_EXEC)
&& errno != ENOSYS)
treat failure of mprotect in map_library as a fatal load failure the error will propagate up and be printed to the user at program start time; at runtime, dlopen will just fail and leave a message for dlerror. previously, if mprotect failed, subsequent attempts to perform relocations would crash the program. this was resulting in an increasing number of false bug reports on grsec systems where rwx permission is not possible in cases where users were wrongly attempting to use non-PIC code in shared libraries. supporting that usage is in theory possible, but the x86_64 toolchain does not even support textrels, and the cost of keeping around the necessary information to handle textrels without rwx permissions is disproportionate to the benefit (which is essentially just supporting broken library setups on grsec machines). also, i unified the error-out code in map_library now that there are 3 places from which munmap might have to be called.
2012-06-06 15:21:28 +00:00
goto error;
textrel support, cheap and ugly
2011-06-29 04:29:08 +00:00
break;
}
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
done_mapping:
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
dso->base = base;
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
dso->dynv = laddr(dso, dyn);
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (dso->tls.size) dso->tls.image = laddr(dso, tls_image);
if map_library has allocated a buffer for phdrs, free it on success too this fixes an oversight in the previous commit.
2013-08-02 13:59:02 +00:00
free(allocated_buf);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
return map;
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
noexec:
errno = ENOEXEC;
treat failure of mprotect in map_library as a fatal load failure the error will propagate up and be printed to the user at program start time; at runtime, dlopen will just fail and leave a message for dlerror. previously, if mprotect failed, subsequent attempts to perform relocations would crash the program. this was resulting in an increasing number of false bug reports on grsec systems where rwx permission is not possible in cases where users were wrongly attempting to use non-PIC code in shared libraries. supporting that usage is in theory possible, but the x86_64 toolchain does not even support textrels, and the cost of keeping around the necessary information to handle textrels without rwx permissions is disproportionate to the benefit (which is essentially just supporting broken library setups on grsec machines). also, i unified the error-out code in map_library now that there are 3 places from which munmap might have to be called.
2012-06-06 15:21:28 +00:00
error:
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
if (map!=MAP_FAILED) unmap_library(dso);
improve error handling in map_library and support long phdrs previously, errno could be meaningless when the caller wrote it to the dlerror string or stderr. try to make it meaningful. also, fix incorrect check for over-long program headers and instead actually support them by allocating memory if needed.
2013-08-02 13:56:49 +00:00
free(allocated_buf);
treat failure of mprotect in map_library as a fatal load failure the error will propagate up and be printed to the user at program start time; at runtime, dlopen will just fail and leave a message for dlerror. previously, if mprotect failed, subsequent attempts to perform relocations would crash the program. this was resulting in an increasing number of false bug reports on grsec systems where rwx permission is not possible in cases where users were wrongly attempting to use non-PIC code in shared libraries. supporting that usage is in theory possible, but the x86_64 toolchain does not even support textrels, and the cost of keeping around the necessary information to handle textrels without rwx permissions is disproportionate to the benefit (which is essentially just supporting broken library setups on grsec machines). also, i unified the error-out code in map_library now that there are 3 places from which munmap might have to be called.
2012-06-06 15:21:28 +00:00
return 0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
make dynamic linker accept : or \n as path separator this allows /etc/ld-musl-$(ARCH).path to contain one path per line, which is much more convenient for users than the :-delimited format, which was a source of repeated and unnecessary confusion. for simplicity, \n is also accepted in environment variables, though it should probably not be used there. at the same time, issues with overly long paths invoking UB or getting truncated have been fixed. such issues should not have arisen with the environment (which is size-limited) but could have been generated by a path file larger than 2**31 bytes in length.
2013-04-20 15:51:58 +00:00
static int path_open(const char *name, const char *s, char *buf, size_t buf_size)
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
{
make dynamic linker accept : or \n as path separator this allows /etc/ld-musl-$(ARCH).path to contain one path per line, which is much more convenient for users than the :-delimited format, which was a source of repeated and unnecessary confusion. for simplicity, \n is also accepted in environment variables, though it should probably not be used there. at the same time, issues with overly long paths invoking UB or getting truncated have been fixed. such issues should not have arisen with the environment (which is size-limited) but could have been generated by a path file larger than 2**31 bytes in length.
2013-04-20 15:51:58 +00:00
size_t l;
int fd;
handle library paths better (ignore empty path components, etc.)
2011-06-25 21:49:16 +00:00
for (;;) {
make dynamic linker accept : or \n as path separator this allows /etc/ld-musl-$(ARCH).path to contain one path per line, which is much more convenient for users than the :-delimited format, which was a source of repeated and unnecessary confusion. for simplicity, \n is also accepted in environment variables, though it should probably not be used there. at the same time, issues with overly long paths invoking UB or getting truncated have been fixed. such issues should not have arisen with the environment (which is size-limited) but could have been generated by a path file larger than 2**31 bytes in length.
2013-04-20 15:51:58 +00:00
s += strspn(s, ":\n");
l = strcspn(s, ":\n");
if (l-1 >= INT_MAX) return -1;
harden dynamic linker library path search transient errors during the path search should not allow the search to continue and possibly open the wrong file. this patch eliminates most conditions where that could happen, but there is still a possibility that $ORIGIN-based rpath processing will have an allocation failure, causing the search to skip such a path. fixing this is left as a separate task. a small bug where overly-long path components caused an infinite loop rather than being skipped/ignored is also fixed.
2015-04-02 00:27:29 +00:00
if (snprintf(buf, buf_size, "%.*s/%s", (int)l, s, name) < buf_size) {
if ((fd = open(buf, O_RDONLY|O_CLOEXEC))>=0) return fd;
switch (errno) {
case ENOENT:
case ENOTDIR:
case EACCES:
case ENAMETOOLONG:
break;
default:
/* Any negative value but -1 will inhibit
* futher path search. */
return -2;
}
}
handle library paths better (ignore empty path components, etc.)
2011-06-25 21:49:16 +00:00
s += l;
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
}
}
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
static int fixup_rpath(struct dso *p, char *buf, size_t buf_size)
{
size_t n, l;
const char *s, *t, *origin;
char *d;
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
if (p->rpath || !p->rpath_orig) return 0;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
if (!strchr(p->rpath_orig, '$')) {
p->rpath = p->rpath_orig;
return 0;
}
n = 0;
s = p->rpath_orig;
fix bugs in $ORIGIN handling 1. an occurrence of ${ORIGIN} before $ORIGIN would be ignored due to the strstr logic. (note that rpath contains multiple :-delimited paths to be searched.) 2. data read by readlink was not null-terminated.
2013-08-23 19:51:59 +00:00
while ((t=strchr(s, '$'))) {
if (strncmp(t, "$ORIGIN", 7) && strncmp(t, "${ORIGIN}", 9))
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
return 0;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
s = t+1;
n++;
}
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
if (n > SSIZE_MAX/PATH_MAX) return 0;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
if (p->kernel_mapped) {
/* $ORIGIN searches cannot be performed for the main program
* when it is suid/sgid/AT_SECURE. This is because the
* pathname is under the control of the caller of execve.
* For libraries, however, $ORIGIN can be processed safely
* since the library's pathname came from a trusted source
* (either system paths or a call to dlopen). */
if (libc.secure)
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
return 0;
fix bugs in $ORIGIN handling 1. an occurrence of ${ORIGIN} before $ORIGIN would be ignored due to the strstr logic. (note that rpath contains multiple :-delimited paths to be searched.) 2. data read by readlink was not null-terminated.
2013-08-23 19:51:59 +00:00
l = readlink("/proc/self/exe", buf, buf_size);
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
if (l == -1) switch (errno) {
case ENOENT:
case ENOTDIR:
case EACCES:
break;
default:
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
return -1;
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
}
if (l >= buf_size)
return 0;
fix bugs in $ORIGIN handling 1. an occurrence of ${ORIGIN} before $ORIGIN would be ignored due to the strstr logic. (note that rpath contains multiple :-delimited paths to be searched.) 2. data read by readlink was not null-terminated.
2013-08-23 19:51:59 +00:00
buf[l] = 0;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
origin = buf;
} else {
origin = p->name;
}
t = strrchr(origin, '/');
honor rpath $ORIGIN for ldd/ldso command with program in working dir the rpath fixup code assumed any module's name field would contain at least one slash, an invariant which is usually met but not in the case of a main executable loaded from the current working directory by running ldd or ldso as a command. it would be possible to make this invariant always hold, but it has a higher runtime allocation cost and does not seem useful elsewhere, so just patch things up in fixup_rpath instead.
2018-02-07 19:27:08 +00:00
if (t) {
l = t-origin;
} else {
/* Normally p->name will always be an absolute or relative
* pathname containing at least one '/' character, but in the
* case where ldso was invoked as a command to execute a
* program in the working directory, app.name may not. Fix. */
origin = ".";
l = 1;
}
disallow non-absolute rpath $ORIGIN for suid/sgid/AT_SECURE processes in theory non-absolute origins can only arise when either the main program is invoked by running ldso as a command (inherently non-suid) or when dlopen was called with a relative pathname containing at least one slash. such usage would be inherently insecure in an suid program anyway, so the old behavior here does not seem to have been insecure. harden against it anyway.
2018-02-07 19:31:42 +00:00
/* Disallow non-absolute origins for suid/sgid/AT_SECURE. */
if (libc.secure && *origin != '/')
return 0;
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
p->rpath = malloc(strlen(p->rpath_orig) + n*l + 1);
if (!p->rpath) return -1;
d = p->rpath;
s = p->rpath_orig;
fix bugs in $ORIGIN handling 1. an occurrence of ${ORIGIN} before $ORIGIN would be ignored due to the strstr logic. (note that rpath contains multiple :-delimited paths to be searched.) 2. data read by readlink was not null-terminated.
2013-08-23 19:51:59 +00:00
while ((t=strchr(s, '$'))) {
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
memcpy(d, s, t-s);
d += t-s;
memcpy(d, origin, l);
d += l;
fix bugs in $ORIGIN handling 1. an occurrence of ${ORIGIN} before $ORIGIN would be ignored due to the strstr logic. (note that rpath contains multiple :-delimited paths to be searched.) 2. data read by readlink was not null-terminated.
2013-08-23 19:51:59 +00:00
/* It was determined previously that the '$' is followed
* either by "ORIGIN" or "{ORIGIN}". */
add rpath $ORIGIN processing to dynamic linker
2013-08-23 17:56:30 +00:00
s = t + 7 + 2*(t[1]=='{');
}
strcpy(d, s);
return 0;
}
cleanup dynamic linker, removing some code duplication
2012-01-23 05:57:38 +00:00
static void decode_dyn(struct dso *p)
{
remove initializers for decoded aux/dyn arrays in dynamic linker the zero initialization is redundant since decode_vec does its own clearing, and it increases the risk that buggy compilers will generate calls to memset. as long as symbols are bound at ld time, such a call will not break anything, but it may be desirable to turn off ld-time binding in the future.
2015-04-14 02:38:18 +00:00
size_t dyn[DYN_CNT];
cleanup dynamic linker, removing some code duplication
2012-01-23 05:57:38 +00:00
decode_vec(p->dynv, dyn, DYN_CNT);
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
p->syms = laddr(p, dyn[DT_SYMTAB]);
p->strings = laddr(p, dyn[DT_STRTAB]);
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
if (dyn[0]&(1<<DT_HASH))
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
p->hashtab = laddr(p, dyn[DT_HASH]);
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
if (dyn[0]&(1<<DT_RPATH))
remove some useless casts in dynamic linker
2015-09-17 19:21:55 +00:00
p->rpath_orig = p->strings + dyn[DT_RPATH];
adapt dynamic linker for new binutils versions that omit DT_RPATH the new DT_RUNPATH semantics for search order are always used, and since binutils had always set both DT_RPATH and DT_RUNPATH when the latter was used, processing only DT_RPATH worked fine. however, recent binutils has stopped generating DT_RPATH when DT_RUNPATH is used, which broke support for this feature completely.
2014-11-23 21:17:57 +00:00
if (dyn[0]&(1<<DT_RUNPATH))
remove some useless casts in dynamic linker
2015-09-17 19:21:55 +00:00
p->rpath_orig = p->strings + dyn[DT_RUNPATH];
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (dyn[0]&(1<<DT_PLTGOT))
p->got = laddr(p, dyn[DT_PLTGOT]);
add gnu hash support in the dynamic linker based on the patches contributed by boris brezillon.
2012-08-25 21:13:28 +00:00
if (search_vec(p->dynv, dyn, DT_GNU_HASH))
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
p->ghashtab = laddr(p, *dyn);
work around libraries with versioned symbols in dynamic linker this commit does not add versioning support; it merely fixes incorrect lookups of symbols in libraries that contain versioned symbols. previously, the version information was completely ignored, and empirically this seems to have resulted in the oldest version being chosen, but I am uncertain if that behavior was even reliable. the new behavior being introduced is to completely ignore symbols which are marked "hidden" (this seems to be the confusing nomenclature for non-current-version) when versioning is present. this should solve all problems related to libraries with symbol versioning as long as all binaries involved are up-to-date (compatible with the latest-version symbols), and it's the needed behavior for dlsym under all circumstances.
2013-08-08 20:10:35 +00:00
if (search_vec(p->dynv, dyn, DT_VERSYM))
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
p->versym = laddr(p, *dyn);
cleanup dynamic linker, removing some code duplication
2012-01-23 05:57:38 +00:00
}
factor symbol counting out of dladdr as its own function the fdpic code will need to count symbols, and it may be useful elsewhere in the future too. counting is trivial as long as sysv hash is present, but for gnu-hash-only libraries it's complex. the behavior of the count is changed slightly: we now include symbols that are not accessible by the gnu hash table in the count. this may make dladdr slightly slower. if this is a problem, dladdr can subtract out the part that should not be accessible. unlike in the old code, subtracting this out is easy even in the fast path where sysv hash is available too.
2015-09-21 21:47:50 +00:00
static size_t count_syms(struct dso *p)
{
if (p->hashtab) return p->hashtab[1];
size_t nsym, i;
uint32_t *buckets = p->ghashtab + 4 + (p->ghashtab[2]*sizeof(size_t)/4);
uint32_t *hashval;
for (i = nsym = 0; i < p->ghashtab[0]; i++) {
if (buckets[i] > nsym)
nsym = buckets[i];
}
if (nsym) {
hashval = buckets + p->ghashtab[0] + (nsym - p->ghashtab[1]);
do nsym++;
while (!(*hashval++ & 1));
}
return nsym;
}
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
static void *dl_mmap(size_t n)
{
void *p;
int prot = PROT_READ|PROT_WRITE, flags = MAP_ANONYMOUS|MAP_PRIVATE;
#ifdef SYS_mmap2
p = (void *)__syscall(SYS_mmap2, 0, n, prot, flags, -1, 0);
#else
p = (void *)__syscall(SYS_mmap, 0, n, prot, flags, -1, 0);
#endif
fix the use of syscall result in dl_mmap
2019-02-09 14:56:17 +00:00
return (unsigned long)p > -4096UL ? 0 : p;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
}
static void makefuncdescs(struct dso *p)
{
static int self_done;
size_t nsym = count_syms(p);
size_t i, size = nsym * sizeof(*p->funcdescs);
if (!self_done) {
p->funcdescs = dl_mmap(size);
self_done = 1;
} else {
p->funcdescs = malloc(size);
}
if (!p->funcdescs) {
if (!runtime) a_crash();
error("Error allocating function descriptors for %s", p->name);
longjmp(*rtld_fail, 1);
}
for (i=0; i<nsym; i++) {
if ((p->syms[i].st_info&0xf)==STT_FUNC && p->syms[i].st_shndx) {
p->funcdescs[i].addr = laddr(p, p->syms[i].st_value);
p->funcdescs[i].got = p->got;
} else {
p->funcdescs[i].addr = 0;
p->funcdescs[i].got = 0;
}
}
}
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
static struct dso *load_library(const char *name, struct dso *needed_by)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
char buf[2*NAME_MAX+2];
fix lots of breakage on dlopen, mostly with explicit pathnames most importantly, the name for such libs was being set from an uninitialized buffer. also, shortname always had an initial '/' character, making it useless for looking up already-loaded libraries by name, and thus causing repeated searches through the library path. major changes now: - shortname is the base name for library lookups with no explicit pathname. it's initially clear for libraries loaded with an explicit pathname (and for the main program), but will be set if the same library (detected via inodes match) is later found by a search. - exact name match is never used to identify libraries loaded with an explicit pathname. in this case, there's no explicit search, so we can just stat the file and check for inode match.
2012-07-11 05:41:20 +00:00
const char *pathname;
fix breakage in ldd (failure to print library load address)
2012-12-16 04:34:08 +00:00
unsigned char *map;
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
struct dso *p, temp_dso = {0};
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
int fd;
struct stat st;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
size_t alloc_size;
int n_th = 0;
prevent passing PT_INTERP name to dlopen from double-loading libc the dev/inode for the main app and the dynamic linker ("interpreter") are not available, so the subsequent checks don't work. in general we don't want to make exact string matches to existing libraries prevent loading new ones, since this breaks loading upgraded modules in module-loading systems. so instead, special-case it. the motivation for this fix is that calling dlopen on the names returned by dl_iterate_phdr or walking the link map (obtained by dlinfo) seem to be the only methods available to an application to actually get a list of open dso handles.
2013-07-31 18:59:36 +00:00
int is_self = 0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
explicitly reject empty names in dynamic linker load_library function previously passing an empty string for name resulted in failure, as expected, but only after spurious syscalls, and it produced confusing errno values (and thus dlerror strings). in addition to dlopen calls, this issue affected use of LD_PRELOAD with trailing whitespace or colon characters.
2014-07-11 04:29:44 +00:00
if (!*name) {
errno = EINVAL;
return 0;
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
/* Catch and block attempts to reload the implementation itself */
if (name[0]=='l' && name[1]=='i' && name[2]=='b') {
fix accidental global static pointer in ldso this was harmless as load_library is not called concurrently, but it used one word of bss.
2016-11-01 01:44:56 +00:00
static const char reserved[] =
fix ldso reserved library name handling If a DT_NEEDED entry was the prefix of a reserved library name (up to the first dot) then it was incorrectly treated as a libc reserved name. e.g. libp.so dependency was not loaded as it matched libpthread reserved name.
2016-11-01 01:49:09 +00:00
"c.pthread.rt.m.dl.util.xnet.";
const char *rp, *next;
for (rp=reserved; *rp; rp=next) {
next = strchr(rp, '.') + 1;
if (strncmp(name+3, rp, next-rp) == 0)
break;
}
if (*rp) {
if (ldd_mode) {
/* Track which names have been resolved
* and only report each one once. */
static unsigned reported;
unsigned mask = 1U<<(rp-reserved);
if (!(reported & mask)) {
reported |= mask;
dprintf(1, "\t%s => %s (%p)\n",
name, ldso.name,
ldso.base);
make ldd report the libc/dynamic linker itself
2013-07-26 18:41:12 +00:00
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
fix ldso reserved library name handling If a DT_NEEDED entry was the prefix of a reserved library name (up to the first dot) then it was incorrectly treated as a libc reserved name. e.g. libp.so dependency was not loaded as it matched libpthread reserved name.
2016-11-01 01:49:09 +00:00
is_self = 1;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
if (!strcmp(name, ldso.name)) is_self = 1;
prevent passing PT_INTERP name to dlopen from double-loading libc the dev/inode for the main app and the dynamic linker ("interpreter") are not available, so the subsequent checks don't work. in general we don't want to make exact string matches to existing libraries prevent loading new ones, since this breaks loading upgraded modules in module-loading systems. so instead, special-case it. the motivation for this fix is that calling dlopen on the names returned by dl_iterate_phdr or walking the link map (obtained by dlinfo) seem to be the only methods available to an application to actually get a list of open dso handles.
2013-07-31 18:59:36 +00:00
if (is_self) {
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
if (!ldso.prev) {
tail->next = &ldso;
ldso.prev = tail;
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
tail = &ldso;
prevent passing PT_INTERP name to dlopen from double-loading libc the dev/inode for the main app and the dynamic linker ("interpreter") are not available, so the subsequent checks don't work. in general we don't want to make exact string matches to existing libraries prevent loading new ones, since this breaks loading upgraded modules in module-loading systems. so instead, special-case it. the motivation for this fix is that calling dlopen on the names returned by dl_iterate_phdr or walking the link map (obtained by dlinfo) seem to be the only methods available to an application to actually get a list of open dso handles.
2013-07-31 18:59:36 +00:00
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
return &ldso;
prevent passing PT_INTERP name to dlopen from double-loading libc the dev/inode for the main app and the dynamic linker ("interpreter") are not available, so the subsequent checks don't work. in general we don't want to make exact string matches to existing libraries prevent loading new ones, since this breaks loading upgraded modules in module-loading systems. so instead, special-case it. the motivation for this fix is that calling dlopen on the names returned by dl_iterate_phdr or walking the link map (obtained by dlinfo) seem to be the only methods available to an application to actually get a list of open dso handles.
2013-07-31 18:59:36 +00:00
}
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
if (strchr(name, '/')) {
fix lots of breakage on dlopen, mostly with explicit pathnames most importantly, the name for such libs was being set from an uninitialized buffer. also, shortname always had an initial '/' character, making it useless for looking up already-loaded libraries by name, and thus causing repeated searches through the library path. major changes now: - shortname is the base name for library lookups with no explicit pathname. it's initially clear for libraries loaded with an explicit pathname (and for the main program), but will be set if the same library (detected via inodes match) is later found by a search. - exact name match is never used to identify libraries loaded with an explicit pathname. in this case, there's no explicit search, so we can just stat the file and check for inode match.
2012-07-11 05:41:20 +00:00
pathname = name;
fix some more O_CLOEXEC/SOCK_CLOEXEC issues
2012-09-29 21:59:50 +00:00
fd = open(name, O_RDONLY|O_CLOEXEC);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
} else {
fix lots of breakage on dlopen, mostly with explicit pathnames most importantly, the name for such libs was being set from an uninitialized buffer. also, shortname always had an initial '/' character, making it useless for looking up already-loaded libraries by name, and thus causing repeated searches through the library path. major changes now: - shortname is the base name for library lookups with no explicit pathname. it's initially clear for libraries loaded with an explicit pathname (and for the main program), but will be set if the same library (detected via inodes match) is later found by a search. - exact name match is never used to identify libraries loaded with an explicit pathname. in this case, there's no explicit search, so we can just stat the file and check for inode match.
2012-07-11 05:41:20 +00:00
/* Search for the name to see if it's already loaded */
for (p=head->next; p; p=p->next) {
if (p->shortname && !strcmp(p->shortname, name)) {
return p;
}
}
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
if (strlen(name) > NAME_MAX) return 0;
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
fd = -1;
move RPATH search after LD_LIBRARY_PATH search this is the modern way, and the only way that makes any sense. glibc has this complicated mechanism with RPATH and RUNPATH that controls whether RPATH is processed before or after LD_LIBRARY_PATH, presumably to support legacy binaries, but there is no compelling reason to support this, and better behavior is obtained by just fixing the search order.
2013-08-02 14:02:29 +00:00
if (env_path) fd = path_open(name, env_path, buf, sizeof buf);
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
for (p=needed_by; fd == -1 && p; p=p->needed_by) {
if (fixup_rpath(p, buf, sizeof buf) < 0)
fd = -2; /* Inhibit further search. */
if (p->rpath)
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
fd = path_open(name, p->rpath, buf, sizeof buf);
halt dynamic linker library search on errors resolving $ORIGIN in rpath this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
2015-04-03 20:35:43 +00:00
}
harden dynamic linker library path search transient errors during the path search should not allow the search to continue and possibly open the wrong file. this patch eliminates most conditions where that could happen, but there is still a possibility that $ORIGIN-based rpath processing will have an allocation failure, causing the search to skip such a path. fixing this is left as a separate task. a small bug where overly-long path components caused an infinite loop rather than being skipped/ignored is also fixed.
2015-04-02 00:27:29 +00:00
if (fd == -1) {
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
if (!sys_path) {
make the dynamic linker find its path file relative to its own location prior to this change, using a non-default syslibdir was impractical on systems where the ordinary library paths contain musl-incompatible library files. the file containing search paths was always taken from /etc, which would either correspond to a system-wide musl installation, or fail to exist at all, resulting in searching of the default library path. the new search strategy is safe even for suid programs because the pathname used comes from the PT_INTERP header of the program being run, rather than any external input. as part of this change, I have also begun differentiating the names of arch variants that differ by endianness or floating point calling convention. the corresponding changes in the build system and and gcc wrapper script (to use an alternate dynamic linker name) for these configurations have not yet been made.
2013-07-18 23:29:44 +00:00
char *prefix = 0;
size_t prefix_len;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
if (ldso.name[0]=='/') {
make the dynamic linker find its path file relative to its own location prior to this change, using a non-default syslibdir was impractical on systems where the ordinary library paths contain musl-incompatible library files. the file containing search paths was always taken from /etc, which would either correspond to a system-wide musl installation, or fail to exist at all, resulting in searching of the default library path. the new search strategy is safe even for suid programs because the pathname used comes from the PT_INTERP header of the program being run, rather than any external input. as part of this change, I have also begun differentiating the names of arch variants that differ by endianness or floating point calling convention. the corresponding changes in the build system and and gcc wrapper script (to use an alternate dynamic linker name) for these configurations have not yet been made.
2013-07-18 23:29:44 +00:00
char *s, *t, *z;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
for (s=t=z=ldso.name; *s; s++)
make the dynamic linker find its path file relative to its own location prior to this change, using a non-default syslibdir was impractical on systems where the ordinary library paths contain musl-incompatible library files. the file containing search paths was always taken from /etc, which would either correspond to a system-wide musl installation, or fail to exist at all, resulting in searching of the default library path. the new search strategy is safe even for suid programs because the pathname used comes from the PT_INTERP header of the program being run, rather than any external input. as part of this change, I have also begun differentiating the names of arch variants that differ by endianness or floating point calling convention. the corresponding changes in the build system and and gcc wrapper script (to use an alternate dynamic linker name) for these configurations have not yet been made.
2013-07-18 23:29:44 +00:00
if (*s=='/') z=t, t=s;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
prefix_len = z-ldso.name;
make the dynamic linker find its path file relative to its own location prior to this change, using a non-default syslibdir was impractical on systems where the ordinary library paths contain musl-incompatible library files. the file containing search paths was always taken from /etc, which would either correspond to a system-wide musl installation, or fail to exist at all, resulting in searching of the default library path. the new search strategy is safe even for suid programs because the pathname used comes from the PT_INTERP header of the program being run, rather than any external input. as part of this change, I have also begun differentiating the names of arch variants that differ by endianness or floating point calling convention. the corresponding changes in the build system and and gcc wrapper script (to use an alternate dynamic linker name) for these configurations have not yet been made.
2013-07-18 23:29:44 +00:00
if (prefix_len < PATH_MAX)
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
prefix = ldso.name;
make the dynamic linker find its path file relative to its own location prior to this change, using a non-default syslibdir was impractical on systems where the ordinary library paths contain musl-incompatible library files. the file containing search paths was always taken from /etc, which would either correspond to a system-wide musl installation, or fail to exist at all, resulting in searching of the default library path. the new search strategy is safe even for suid programs because the pathname used comes from the PT_INTERP header of the program being run, rather than any external input. as part of this change, I have also begun differentiating the names of arch variants that differ by endianness or floating point calling convention. the corresponding changes in the build system and and gcc wrapper script (to use an alternate dynamic linker name) for these configurations have not yet been made.
2013-07-18 23:29:44 +00:00
}
if (!prefix) {
prefix = "";
prefix_len = 0;
}
char etc_ldso_path[prefix_len + 1
+ sizeof "/etc/ld-musl-" LDSO_ARCH ".path"];
snprintf(etc_ldso_path, sizeof etc_ldso_path,
"%.*s/etc/ld-musl-" LDSO_ARCH ".path",
(int)prefix_len, prefix);
FILE *f = fopen(etc_ldso_path, "rbe");
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
if (f) {
fix dynamic linker handling of empty path file or error reading path file previously, the path string was being used despite being invalid. with this change, empty path file or error reading the path file is treated as an empty path. this is preferable to falling back to a default path, so that attacks to prevent reading of the path file could not result in loading incorrect and possibly dangerous (outdated or mismatching ABI) libraries from. the code to strip the final newline has also been removed; now that newline is accepted as a delimiter, it's harmless to leave it in place.
2013-06-26 14:17:29 +00:00
if (getdelim(&sys_path, (size_t[1]){0}, 0, f) <= 0) {
remove useless conditional before free from dynamic linker path code
2013-06-26 14:51:36 +00:00
free(sys_path);
fix dynamic linker handling of empty path file or error reading path file previously, the path string was being used despite being invalid. with this change, empty path file or error reading the path file is treated as an empty path. this is preferable to falling back to a default path, so that attacks to prevent reading of the path file could not result in loading incorrect and possibly dangerous (outdated or mismatching ABI) libraries from. the code to strip the final newline has also been removed; now that newline is accepted as a delimiter, it's harmless to leave it in place.
2013-06-26 14:17:29 +00:00
sys_path = "";
support ldso path files without final newline
2012-11-09 18:49:40 +00:00
}
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
fclose(f);
do not use default when dynamic linker fails to open existing path file if fopen fails for a reason other than ENOENT, we must assume the intent is that the path file be used. failure may be due to misconfiguration or intentional resource-exhaustion attack (against suid programs), in which case falling back to loading libraries from an unintended path could be dangerous.
2013-09-09 17:39:08 +00:00
} else if (errno != ENOENT) {
sys_path = "";
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
}
}
change ldso path file logic to replace rather than add to search path this change was originally intended just to avoid repeated attempts to open a nonexistant /etc/ls-musl-$(ARCH).path file, but I realized it also prevents the default paths from being searched when such a path file exists. despite the potential to break existing usage, I believe the new behavior is the right behavior, and it's better to fix it sooner rather than later. with the old behavior, it was impossible to inhibit search of default paths which might contain musl-incompatible libs (or even libs from a different cpu arch, on multi-arch machines).
2012-11-09 03:41:16 +00:00
if (!sys_path) sys_path = "/lib:/usr/local/lib:/usr/lib";
fd = path_open(name, sys_path, buf, sizeof buf);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
fix lots of breakage on dlopen, mostly with explicit pathnames most importantly, the name for such libs was being set from an uninitialized buffer. also, shortname always had an initial '/' character, making it useless for looking up already-loaded libraries by name, and thus causing repeated searches through the library path. major changes now: - shortname is the base name for library lookups with no explicit pathname. it's initially clear for libraries loaded with an explicit pathname (and for the main program), but will be set if the same library (detected via inodes match) is later found by a search. - exact name match is never used to identify libraries loaded with an explicit pathname. in this case, there's no explicit search, so we can just stat the file and check for inode match.
2012-07-11 05:41:20 +00:00
pathname = buf;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
if (fd < 0) return 0;
if (fstat(fd, &st) < 0) {
close(fd);
return 0;
}
for (p=head->next; p; p=p->next) {
if (p->dev == st.st_dev && p->ino == st.st_ino) {
fix lots of breakage on dlopen, mostly with explicit pathnames most importantly, the name for such libs was being set from an uninitialized buffer. also, shortname always had an initial '/' character, making it useless for looking up already-loaded libraries by name, and thus causing repeated searches through the library path. major changes now: - shortname is the base name for library lookups with no explicit pathname. it's initially clear for libraries loaded with an explicit pathname (and for the main program), but will be set if the same library (detected via inodes match) is later found by a search. - exact name match is never used to identify libraries loaded with an explicit pathname. in this case, there's no explicit search, so we can just stat the file and check for inode match.
2012-07-11 05:41:20 +00:00
/* If this library was previously loaded with a
* pathname but a search found the same inode,
* setup its shortname so it can be found by name. */
small dynamic linker module search fix libraries loaded more than once by pathname should not get shortnames that would cause them to later be used to satisfy non-pathname load requests.
2012-10-05 16:09:54 +00:00
if (!p->shortname && pathname != name)
p->shortname = strrchr(p->name, '/')+1;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
close(fd);
return p;
}
}
add support for RTLD_NOLOAD to dlopen based on patch by Pierre Carrier <pierre@gcarrier.fr> that just added the flag constant, but with minimal additional code so that it actually works as documented. this is a nonstandard option but some major software (reportedly, Firefox) uses it and it was easy to add anyway.
2013-01-24 03:07:45 +00:00
map = noload ? 0 : map_library(fd, &temp_dso);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
close(fd);
if (!map) return 0;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
avoid loading of multiple libc versions via explicit pathname such loading is unsafe, and can happen when programs use their own logic to locate a .so file then pass the absolute pathname to dlopen, or if an absolute pathname ends up in DT_NEEDED headers. multiple loads with only the base name were already precluded, provided libc was named appropriately, by special-casing standard library names. one function symbol (in the reserved namespace, but public, since it's part of the crt1 entry point ABI) and one data symbol are checked. this way we avoid likely false positives, particularly from libraries interposing and wrapping functions. there is no hard requirement to avoid breaking such usage, since trying to run a hook before libc is even initialized is not a supported usage case, but it's friendlier not to break things.
2017-03-14 22:51:27 +00:00
/* Avoid the danger of getting two versions of libc mapped into the
* same process when an absolute pathname was used. The symbols
* checked are chosen to catch both musl and glibc, and to avoid
* false positives from interposition-hack libraries. */
decode_dyn(&temp_dso);
if (find_sym(&temp_dso, "__libc_start_main", 1).sym &&
find_sym(&temp_dso, "stdin", 1).sym) {
unmap_library(&temp_dso);
return load_library("libc.so", needed_by);
}
fix malloc state corruption when ldso rejects loading a second libc commit c49d3c8adadfa24235fcf4779bb722b1aa6f480b added logic to detect attempts to load libc.so via another name and instead redirect to the existing libc, rather than loading two and producing dangerously inconsistent state. however, the check for and unmapping of the duplicate libc happened after reclaim_gaps was already called, donating the slack space around the writable segment to malloc. subsequent unmapping of the library then invalidated malloc's free lists. fix the issue by moving the call to reclaim_gaps out of map_library into load_library, after the duplicate libc check but before the first call to calloc, so that the gaps can still be used to satisfy the allocation of struct dso. this change also eliminates the need for an ugly hack (temporarily setting runtime=1) to avoid reclaim_gaps when loading the main program via map_library, which happens when ldso is invoked as a command. only programs/libraries erroneously containing a DT_NEEDED reference to libc.so via an absolute pathname or symlink were affected by this issue.
2017-11-13 20:27:10 +00:00
/* Past this point, if we haven't reached runtime yet, ldso has
* committed either to use the mapped library or to abort execution.
* Unmapping is not possible, so we can safely reclaim gaps. */
if (!runtime) reclaim_gaps(&temp_dso);
avoid loading of multiple libc versions via explicit pathname such loading is unsafe, and can happen when programs use their own logic to locate a .so file then pass the absolute pathname to dlopen, or if an absolute pathname ends up in DT_NEEDED headers. multiple loads with only the base name were already precluded, provided libc was named appropriately, by special-casing standard library names. one function symbol (in the reserved namespace, but public, since it's part of the crt1 entry point ABI) and one data symbol are checked. this way we avoid likely false positives, particularly from libraries interposing and wrapping functions. there is no hard requirement to avoid breaking such usage, since trying to run a hook before libc is even initialized is not a supported usage case, but it's friendlier not to break things.
2017-03-14 22:51:27 +00:00
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
/* Allocate storage for the new DSO. When there is TLS, this
* storage must include a reservation for all pre-existing
* threads to obtain copies of both the new TLS, and an
* extended DTV capable of storing an additional slot for
* the newly-loaded DSO. */
alloc_size = sizeof *p + strlen(pathname) + 1;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (runtime && temp_dso.tls.image) {
size_t per_th = temp_dso.tls.size + temp_dso.tls.align
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
+ sizeof(void *) * (tls_cnt+3);
fix namespace clash (libc) in dynlink.c this makes it so the #undef libc and __libc name are no longer needed, which were problematic because the "accessor function" mode for accessing the libc struct could not be used, breaking build on any compiler without (working) visibility.
2012-10-14 03:25:20 +00:00
n_th = libc.threads_minus_1 + 1;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
if (n_th > SSIZE_MAX / per_th) alloc_size = SIZE_MAX;
else alloc_size += n_th * per_th;
}
p = calloc(1, alloc_size);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
if (!p) {
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
unmap_library(&temp_dso);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
return 0;
}
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
memcpy(p, &temp_dso, sizeof temp_dso);
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
p->dev = st.st_dev;
p->ino = st.st_ino;
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
p->needed_by = needed_by;
error handling in dynamic linking some of the code is not yet used, and is in preparation for dlopen which needs to be able to handle failure loading libraries without terminating the program.
2011-06-26 21:39:17 +00:00
p->name = p->buf;
fix unsafety of new ldso dep tracking in presence of malloc replacement commit 403555690775f7c8806372644f543518e6664e3b introduced runtime realloc of an array that may have been allocated before symbols were resolved outside of libc, which is invalid if the allocator has been replaced. track this condition and manually copy if needed.
2019-03-01 19:37:52 +00:00
p->runtime_loaded = runtime;
fix lots of breakage on dlopen, mostly with explicit pathnames most importantly, the name for such libs was being set from an uninitialized buffer. also, shortname always had an initial '/' character, making it useless for looking up already-loaded libraries by name, and thus causing repeated searches through the library path. major changes now: - shortname is the base name for library lookups with no explicit pathname. it's initially clear for libraries loaded with an explicit pathname (and for the main program), but will be set if the same library (detected via inodes match) is later found by a search. - exact name match is never used to identify libraries loaded with an explicit pathname. in this case, there's no explicit search, so we can just stat the file and check for inode match.
2012-07-11 05:41:20 +00:00
strcpy(p->name, pathname);
/* Add a shortname only if name arg was not an explicit pathname. */
if (pathname != name) p->shortname = strrchr(p->name, '/')+1;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (p->tls.image) {
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
p->tls_id = ++tls_cnt;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
tls_align = MAXP2(tls_align, p->tls.align);
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
#ifdef TLS_ABOVE_TP
fix tls offsets when p_vaddr%p_align != 0 on TLS_ABOVE_TP targets currently the bfd linker does not seem to create tls segments where p_vaddr%p_align != 0, but this is valid in ELF and then the runtime computed tls offset must satisfy offset%p_align == (base+p_vaddr)%p_align and in case of local exec tls (main executable) the smallest such offset must be used (otherwise it is incompatible with the offset computed by the static linker). the !TLS_ABOVE_TP case is handled correctly (the offset is negative then in the formula). the ldso code for TLS_ABOVE_TP is changed so the static tls offset of each module satisfies the formula.
2019-05-13 18:47:11 +00:00
p->tls.offset = tls_offset + ( (p->tls.align-1) &
(-tls_offset + (uintptr_t)p->tls.image) );
fix static tls offsets of shared libs on TLS_ABOVE_TP targets tls_offset should always point to the end of the allocated static tls area, but this was not handled correctly on "tls variant 1" targets in the dynamic linker: after application tls was allocated, tls_offset was aligned up, potentially wasting tls space. (alignment may be needed at the begining of the tls area, not at the end, but that will be fixed separately as it is unlikely to affect real binaries.) when static tls was allocated for a shared library, tls_offset was only updated with the size of the tls segment which does not include alignment gaps, which can easily happen if the tls size update for one library leaves tls_offset misaligned for the next one. this can cause oob access in __copy_tls or arbitrary breakage at tls access. (the issue was observed on aarch64 with rust binaries)
2019-05-16 17:15:33 +00:00
tls_offset = p->tls.offset + p->tls.size;
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
#else
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
tls_offset += p->tls.size + p->tls.align - 1;
tls_offset -= (tls_offset + (uintptr_t)p->tls.image)
& (p->tls.align-1);
p->tls.offset = tls_offset;
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
#endif
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
p->new_dtv = (void *)(-sizeof(size_t) &
(uintptr_t)(p->name+strlen(p->name)+sizeof(size_t)));
p->new_tls = (void *)(p->new_dtv + n_th*(tls_cnt+1));
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (tls_tail) tls_tail->next = &p->tls;
else libc.tls_head = &p->tls;
tls_tail = &p->tls;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
tail->next = p;
p->prev = tail;
tail = p;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (DL_FDPIC) makefuncdescs(p);
fix breakage in ldd (failure to print library load address)
2012-12-16 04:34:08 +00:00
if (ldd_mode) dprintf(1, "\t%s => %s (%p)\n", name, pathname, p->base);
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
return p;
}
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
static void load_direct_deps(struct dso *p)
{
record preloaded libraries as direct pseudo-dependencies of main app this makes calling dlsym on the main app more consistent with the global symbol table (load order), and is a prerequisite for dependency-order ctor execution to work correctly with LD_PRELOAD.
2019-03-01 20:09:16 +00:00
size_t i, cnt=0;
avoid malloc of deps array for programs with no external deps traditionally, we've provided a guarantee that dynamic-linked applications with no external dependencies (nothing but libc) and no thread-local storage have no failure paths before the entry point. normally, thanks to reclaim_gaps, such a malloc will not require a syscall anyway, but if segment alignment is unlucky, it might. use a builtin array for this common special case.
2019-03-03 17:12:59 +00:00
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (p->deps) return;
record preloaded libraries as direct pseudo-dependencies of main app this makes calling dlsym on the main app more consistent with the global symbol table (load order), and is a prerequisite for dependency-order ctor execution to work correctly with LD_PRELOAD.
2019-03-01 20:09:16 +00:00
/* For head, all preloads are direct pseudo-dependencies.
* Count and include them now to avoid realloc later. */
if (p==head) for (struct dso *q=p->next; q; q=q->next)
cnt++;
for (i=0; p->dynv[i]; i+=2)
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (p->dynv[i] == DT_NEEDED) cnt++;
avoid malloc of deps array for programs with no external deps traditionally, we've provided a guarantee that dynamic-linked applications with no external dependencies (nothing but libc) and no thread-local storage have no failure paths before the entry point. normally, thanks to reclaim_gaps, such a malloc will not require a syscall anyway, but if segment alignment is unlucky, it might. use a builtin array for this common special case.
2019-03-03 17:12:59 +00:00
/* Use builtin buffer for apps with no external deps, to
* preserve property of no runtime failure paths. */
p->deps = (p==head && cnt<2) ? builtin_deps :
calloc(cnt+1, sizeof *p->deps);
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (!p->deps) {
error("Error loading dependencies for %s", p->name);
if (runtime) longjmp(*rtld_fail, 1);
}
record preloaded libraries as direct pseudo-dependencies of main app this makes calling dlsym on the main app more consistent with the global symbol table (load order), and is a prerequisite for dependency-order ctor execution to work correctly with LD_PRELOAD.
2019-03-01 20:09:16 +00:00
cnt=0;
if (p==head) for (struct dso *q=p->next; q; q=q->next)
p->deps[cnt++] = q;
for (i=0; p->dynv[i]; i+=2) {
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (p->dynv[i] != DT_NEEDED) continue;
struct dso *dep = load_library(p->strings + p->dynv[i+1], p);
if (!dep) {
error("Error loading shared library %s: %m (needed by %s)",
p->strings + p->dynv[i+1], p->name);
if (runtime) longjmp(*rtld_fail, 1);
continue;
}
p->deps[cnt++] = dep;
}
record preloaded libraries as direct pseudo-dependencies of main app this makes calling dlsym on the main app more consistent with the global symbol table (load order), and is a prerequisite for dependency-order ctor execution to work correctly with LD_PRELOAD.
2019-03-01 20:09:16 +00:00
p->deps[cnt] = 0;
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
p->ndeps_direct = cnt;
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
static void load_deps(struct dso *p)
{
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (p->deps) return;
for (; p; p=p->next)
load_direct_deps(p);
}
static void extend_bfs_deps(struct dso *p)
{
size_t i, j, cnt, ndeps_all;
struct dso **tmp;
fix unsafety of new ldso dep tracking in presence of malloc replacement commit 403555690775f7c8806372644f543518e6664e3b introduced runtime realloc of an array that may have been allocated before symbols were resolved outside of libc, which is invalid if the allocator has been replaced. track this condition and manually copy if needed.
2019-03-01 19:37:52 +00:00
/* Can't use realloc if the original p->deps was allocated at
avoid malloc of deps array for programs with no external deps traditionally, we've provided a guarantee that dynamic-linked applications with no external dependencies (nothing but libc) and no thread-local storage have no failure paths before the entry point. normally, thanks to reclaim_gaps, such a malloc will not require a syscall anyway, but if segment alignment is unlucky, it might. use a builtin array for this common special case.
2019-03-03 17:12:59 +00:00
* program entry and malloc has been replaced, or if it's
* the builtin non-allocated trivial main program deps array. */
int no_realloc = (__malloc_replaced && !p->runtime_loaded)
|| p->deps == builtin_deps;
fix unsafety of new ldso dep tracking in presence of malloc replacement commit 403555690775f7c8806372644f543518e6664e3b introduced runtime realloc of an array that may have been allocated before symbols were resolved outside of libc, which is invalid if the allocator has been replaced. track this condition and manually copy if needed.
2019-03-01 19:37:52 +00:00
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (p->bfs_built) return;
ndeps_all = p->ndeps_direct;
/* Mark existing (direct) deps so they won't be duplicated. */
for (i=0; p->deps[i]; i++)
p->deps[i]->mark = 1;
/* For each dependency already in the list, copy its list of direct
* dependencies to the list, excluding any items already in the
* list. Note that the list this loop iterates over will grow during
* the loop, but since duplicates are excluded, growth is bounded. */
for (i=0; p->deps[i]; i++) {
struct dso *dep = p->deps[i];
for (j=cnt=0; j<dep->ndeps_direct; j++)
if (!dep->deps[j]->mark) cnt++;
fix unsafety of new ldso dep tracking in presence of malloc replacement commit 403555690775f7c8806372644f543518e6664e3b introduced runtime realloc of an array that may have been allocated before symbols were resolved outside of libc, which is invalid if the allocator has been replaced. track this condition and manually copy if needed.
2019-03-01 19:37:52 +00:00
tmp = no_realloc ?
malloc(sizeof(*tmp) * (ndeps_all+cnt+1)) :
realloc(p->deps, sizeof(*tmp) * (ndeps_all+cnt+1));
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (!tmp) {
error("Error recording dependencies for %s", p->name);
if (runtime) longjmp(*rtld_fail, 1);
continue;
}
fix unsafety of new ldso dep tracking in presence of malloc replacement commit 403555690775f7c8806372644f543518e6664e3b introduced runtime realloc of an array that may have been allocated before symbols were resolved outside of libc, which is invalid if the allocator has been replaced. track this condition and manually copy if needed.
2019-03-01 19:37:52 +00:00
if (no_realloc) {
memcpy(tmp, p->deps, sizeof(*tmp) * (ndeps_all+1));
no_realloc = 0;
}
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
p->deps = tmp;
for (j=0; j<dep->ndeps_direct; j++) {
if (dep->deps[j]->mark) continue;
dep->deps[j]->mark = 1;
p->deps[ndeps_all++] = dep->deps[j];
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
p->deps[ndeps_all] = 0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
p->bfs_built = 1;
for (p=head; p; p=p->next)
p->mark = 0;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
LD_PRELOAD support
2011-08-16 04:24:36 +00:00
static void load_preload(char *s)
{
int tmp;
char *z;
for (z=s; *z; s=z) {
make dynamic linker accept colon as a separator for LD_PRELOAD
2014-07-11 04:26:12 +00:00
for ( ; *s && (isspace(*s) || *s==':'); s++);
for (z=s; *z && !isspace(*z) && *z!=':'; z++);
LD_PRELOAD support
2011-08-16 04:24:36 +00:00
tmp = *z;
*z = 0;
add recursive rpath support to dynamic linker previously, rpath was only honored for direct dependencies. in other words, if A depends on B and B depends on C, only B's rpath (if any), not A's rpath, was being searched for C. this limitation made rpath-based deployment difficult in the presence of multiple levels of library dependency. at present, $ORIGIN processing in rpath is still unsupported.
2013-08-23 15:15:40 +00:00
load_library(s, 0);
LD_PRELOAD support
2011-08-16 04:24:36 +00:00
*z = tmp;
}
}
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
static void add_syms(struct dso *p)
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
{
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
if (!p->syms_next && syms_tail != p) {
syms_tail->syms_next = p;
syms_tail = p;
}
}
static void revert_syms(struct dso *old_tail)
{
struct dso *p, *next;
/* Chop off the tail of the list of dsos that participate in
* the global symbol table, reverting them to RTLD_LOCAL. */
for (p=old_tail; p; p=next) {
next = p->syms_next;
p->syms_next = 0;
}
syms_tail = old_tail;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
static void do_mips_relocs(struct dso *p, size_t *got)
{
size_t i, j, rel[2];
unsigned char *base = p->base;
i=0; search_vec(p->dynv, &i, DT_MIPS_LOCAL_GOTNO);
reprocess all libc/ldso symbolic relocations in dynamic linking stage 3 commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-26 03:33:59 +00:00
if (p==&ldso) {
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
got += i;
} else {
while (i--) *got++ += (size_t)base;
}
j=0; search_vec(p->dynv, &j, DT_MIPS_GOTSYM);
i=0; search_vec(p->dynv, &i, DT_MIPS_SYMTABNO);
Sym *sym = p->syms + j;
rel[0] = (unsigned char *)got - base;
for (i-=j; i; i--, sym++, rel[0]+=sizeof(size_t)) {
generalize mips-specific reloc code not to hard-code sym/type encoding this change is made in preparation for adding the mips64 port, which needs a 64-bit (and mips64-specific) form of the R_INFO macro, but it's a better abstraction anyway. based on part of the mips64 port patch by Mahesh Bodapati and Jaydeep Patil of Imagination Technologies.
2016-03-06 17:25:52 +00:00
rel[1] = R_INFO(sym-p->syms, R_MIPS_JUMP_SLOT);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
do_relocs(p, rel, sizeof rel, 2);
}
}
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
static void reloc_all(struct dso *p)
{
remove initializers for decoded aux/dyn arrays in dynamic linker the zero initialization is redundant since decode_vec does its own clearing, and it increases the risk that buggy compilers will generate calls to memset. as long as symbols are bound at ld time, such a call will not break anything, but it may be desirable to turn off ld-time binding in the future.
2015-04-14 02:38:18 +00:00
size_t dyn[DYN_CNT];
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
for (; p; p=p->next) {
if (p->relocated) continue;
decode_vec(p->dynv, dyn, DYN_CNT);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
if (NEED_MIPS_GOT_RELOCS)
refactor some more dynamic linker load address computations these were just missed in the previous commits.
2015-09-17 19:45:45 +00:00
do_mips_relocs(p, laddr(p, dyn[DT_PLTGOT]));
do_relocs(p, laddr(p, dyn[DT_JMPREL]), dyn[DT_PLTRELSZ],
more cleanup of dynamic linker internals
2012-08-05 06:49:02 +00:00
2+(dyn[DT_PLTREL]==DT_RELA));
refactor some more dynamic linker load address computations these were just missed in the previous commits.
2015-09-17 19:45:45 +00:00
do_relocs(p, laddr(p, dyn[DT_REL]), dyn[DT_RELSZ], 2);
do_relocs(p, laddr(p, dyn[DT_RELA]), dyn[DT_RELASZ], 3);
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
if (head != &ldso && p->relro_start != p->relro_end &&
refactor some more dynamic linker load address computations these were just missed in the previous commits.
2015-09-17 19:45:45 +00:00
mprotect(laddr(p, p->relro_start), p->relro_end-p->relro_start, PROT_READ)
ignore ENOSYS error from mprotect in pthread_create and dynamic linker this error simply indicated a system without memory protection (NOMMU) and should not cause failure in the caller.
2015-06-17 17:21:46 +00:00
&& errno != ENOSYS) {
fix regression in dynamic linker error reporting due to a mistake when refactoring the error printing for the dynamic linker (commit 7c73cacd09a51a87484db5689864743e4984a84d), all messages were suppressed and replaced by blank lines.
2014-06-30 01:52:54 +00:00
error("Error relocating %s: RELRO protection failed: %m",
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
p->name);
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
if (runtime) longjmp(*rtld_fail, 1);
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
}
keep track of which dsos have been relocated
2011-06-25 04:18:19 +00:00
p->relocated = 1;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
}
}
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
static void kernel_mapped_dso(struct dso *p)
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
{
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
size_t min_addr = -1, max_addr = 0, cnt;
Phdr *ph = p->phdr;
for (cnt = p->phnum; cnt--; ph = (void *)((char *)ph + p->phentsize)) {
if (ph->p_type == PT_DYNAMIC) {
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
p->dynv = laddr(p, ph->p_vaddr);
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
} else if (ph->p_type == PT_GNU_RELRO) {
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
p->relro_start = ph->p_vaddr & -PAGE_SIZE;
p->relro_end = (ph->p_vaddr + ph->p_memsz) & -PAGE_SIZE;
support setting of default thread stack size via PT_GNU_STACK header this facilitates building software that assumes a large default stack size without any patching to call pthread_setattr_default_np or pthread_attr_setstacksize at each thread creation site, using just LDFLAGS. normally the PT_GNU_STACK header is used only to reflect whether executable stack is desired, but with GNU ld at least, passing -Wl,-z,stack-size=N will set a size on the program header. with this patch, that size will be incorporated into the default stack size (subject to increase-only rule and DEFAULT_STACK_MAX limit). both static and dynamic linking honor the program header. for dynamic linking, all libraries loaded at program start, including preloaded ones, are considered. dlopened libraries are not considered, for several reasons. extra logic would be needed to defer processing until the load of the new library is commited, synchronization woud be needed since other threads may be running concurrently, and the effectiveness woud be limited since the larger size would not apply to threads that already existed at the time of dlopen. programs that will dlopen code expecting a large stack need to declare the requirement themselves, or pthread_setattr_default_np can be used.
2018-09-19 03:54:18 +00:00
} else if (ph->p_type == PT_GNU_STACK) {
if (!runtime && ph->p_memsz > __default_stacksize) {
__default_stacksize =
ph->p_memsz < DEFAULT_STACK_MAX ?
ph->p_memsz : DEFAULT_STACK_MAX;
}
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
}
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
if (ph->p_type != PT_LOAD) continue;
if (ph->p_vaddr < min_addr)
min_addr = ph->p_vaddr;
if (ph->p_vaddr+ph->p_memsz > max_addr)
max_addr = ph->p_vaddr+ph->p_memsz;
}
min_addr &= -PAGE_SIZE;
max_addr = (max_addr + PAGE_SIZE-1) & -PAGE_SIZE;
p->map = p->base + min_addr;
p->map_len = max_addr - min_addr;
clean up internal dynamic linker functions enumerating phdrs record phentsize in struct dso, so the phdrs can be easily enumerated via it. simplify all functions enumerating phdrs to require only struct dso. also merge find_map_range and find_dso to kernel_mapped_dso function that does both tasks during single phdr enumeration.
2014-03-25 18:59:50 +00:00
p->kernel_mapped = 1;
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
}
unify static and dynamic libc init/fini code paths use weak definitions that the dynamic linker can override instead of preprocessor conditionals on SHARED so that the same libc start and exit code can be used for both static and dynamic linking.
2015-11-12 03:08:23 +00:00
void __libc_exit_fini()
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
{
struct dso *p;
remove initializers for decoded aux/dyn arrays in dynamic linker the zero initialization is redundant since decode_vec does its own clearing, and it increases the risk that buggy compilers will generate calls to memset. as long as symbols are bound at ld time, such a call will not break anything, but it may be desirable to turn off ld-time binding in the future.
2015-04-14 02:38:18 +00:00
size_t dyn[DYN_CNT];
synchronize shared library dtor exec against concurrent loads/ctors previously, going way back, there was simply no synchronization here. a call to exit concurrent with ctor execution from dlopen could cause a dtor to execute concurrently with its corresponding ctor, or could cause dtors for newly-constructed libraries to be skipped. introduce a shutting_down state that blocks further ctor execution, producing the quiescence the dtor execution loop needs to ensure any kind of consistency, and that blocks further calls to dlopen so that a call into dlopen from a dtor cannot deadlock. better approaches to some of this may be possible, but the changes here at least make things safe.
2019-03-02 03:47:29 +00:00
int self = __pthread_self()->tid;
/* Take both locks before setting shutting_down, so that
* either lock is sufficient to read its value. The lock
* order matches that in dlopen to avoid deadlock. */
pthread_rwlock_wrlock(&lock);
pthread_mutex_lock(&init_fini_lock);
shutting_down = 1;
pthread_rwlock_unlock(&lock);
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
for (p=fini_head; p; p=p->fini_next) {
synchronize shared library dtor exec against concurrent loads/ctors previously, going way back, there was simply no synchronization here. a call to exit concurrent with ctor execution from dlopen could cause a dtor to execute concurrently with its corresponding ctor, or could cause dtors for newly-constructed libraries to be skipped. introduce a shutting_down state that blocks further ctor execution, producing the quiescence the dtor execution loop needs to ensure any kind of consistency, and that blocks further calls to dlopen so that a call into dlopen from a dtor cannot deadlock. better approaches to some of this may be possible, but the changes here at least make things safe.
2019-03-02 03:47:29 +00:00
while (p->ctor_visitor && p->ctor_visitor!=self)
pthread_cond_wait(&ctor_cond, &init_fini_lock);
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
if (!p->constructed) continue;
decode_vec(p->dynv, dyn, DYN_CNT);
add support for init_array/fini_array ctors/dtors to dynamic linker
2013-07-20 22:26:17 +00:00
if (dyn[0] & (1<<DT_FINI_ARRAY)) {
size_t n = dyn[DT_FINI_ARRAYSZ]/sizeof(size_t);
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
size_t *fn = (size_t *)laddr(p, dyn[DT_FINI_ARRAY])+n;
fix order of fini_array execution for shared libs
2013-07-21 06:35:46 +00:00
while (n--) ((void (*)(void))*--fn)();
add support for init_array/fini_array ctors/dtors to dynamic linker
2013-07-20 22:26:17 +00:00
}
disable legacy init/fini processing on ARM since the old, poorly-thought-out musl approach to init/fini arrays on ARM (when it was the only arch that needed them) was to put the code in crti/crtn and have the legacy _init/_fini code run the arrays, adding proper init/fini array support caused the arrays to get processed twice on ARM. I'm not sure skipping legacy init/fini processing is the best solution to the problem, but it works, and it shouldn't break anything since the legacy init/fini system was never used for ARM EABI.
2013-07-22 18:08:33 +00:00
#ifndef NO_LEGACY_INITFINI
don't call null pointer if DT_INIT/DT_FINI are null it's not clear to me why the linker even outputs these headers if they are null, but apparently it does so. with the default startfiles, they will never be null anyway, but this patch allows eliminating crti, crtn, crtbegin, and crtend (leaving only crt1) if the toolchain is using init_array/fini_array (or for a C-only, no-ctor environment).
2013-07-31 04:04:10 +00:00
if ((dyn[0] & (1<<DT_FINI)) && dyn[DT_FINI])
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
fpaddr(p, dyn[DT_FINI])();
disable legacy init/fini processing on ARM since the old, poorly-thought-out musl approach to init/fini arrays on ARM (when it was the only arch that needed them) was to put the code in crti/crtn and have the legacy _init/_fini code run the arrays, adding proper init/fini array support caused the arrays to get processed twice on ARM. I'm not sure skipping legacy init/fini processing is the best solution to the problem, but it works, and it shouldn't break anything since the legacy init/fini system was never used for ARM EABI.
2013-07-22 18:08:33 +00:00
#endif
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
}
}
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
static struct dso **queue_ctors(struct dso *dso)
add support for init/finit (constructors and destructors) this is mainly in hopes of supporting c++ (not yet possible for other reasons) but will also help applications/libraries which use (and more often, abuse) the gcc __attribute__((__constructor__)) feature in "C" code. x86_64 and arm versions of the new startup asm are untested and may have minor problems.
2012-02-06 19:39:09 +00:00
{
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
size_t cnt, qpos, spos, i;
struct dso *p, **queue, **stack;
if (ldd_mode) return 0;
/* Bound on queue size is the total number of indirect deps.
* If a bfs deps list was built, we can use it. Otherwise,
* bound by the total number of DSOs, which is always safe and
* is reasonable we use it (for main app at startup). */
if (dso->bfs_built) {
for (cnt=0; dso->deps[cnt]; cnt++)
dso->deps[cnt]->mark = 0;
cnt++; /* self, not included in deps */
} else {
for (cnt=0, p=head; p; cnt++, p=p->next)
p->mark = 0;
}
cnt++; /* termination slot */
avoid malloc of ctor queue for programs with no external deps together with the previous two commits, this completes restoration of the property that dynamic-linked apps with no external deps and no tls have no failure paths before entry.
2019-03-03 18:24:23 +00:00
if (dso==head && cnt <= countof(builtin_ctor_queue))
queue = builtin_ctor_queue;
else
queue = calloc(cnt, sizeof *queue);
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
if (!queue) {
error("Error allocating constructor queue: %m\n");
if (runtime) longjmp(*rtld_fail, 1);
return 0;
}
/* Opposite ends of the allocated buffer serve as an output queue
* and a working stack. Setup initial stack with just the argument
* dso and initial queue empty... */
avoid malloc of ctor queue for programs with no external deps together with the previous two commits, this completes restoration of the property that dynamic-linked apps with no external deps and no tls have no failure paths before entry.
2019-03-03 18:24:23 +00:00
stack = queue;
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
qpos = 0;
spos = cnt;
stack[--spos] = dso;
dso->next_dep = 0;
dso->mark = 1;
/* Then perform pseudo-DFS sort, but ignoring circular deps. */
while (spos<cnt) {
p = stack[spos++];
while (p->next_dep < p->ndeps_direct) {
if (p->deps[p->next_dep]->mark) {
p->next_dep++;
} else {
stack[--spos] = p;
p = p->deps[p->next_dep];
p->next_dep = 0;
p->mark = 1;
}
}
queue[qpos++] = p;
}
queue[qpos] = 0;
for (i=0; i<qpos; i++) queue[i]->mark = 0;
return queue;
}
static void do_init_fini(struct dso **queue)
{
struct dso *p;
size_t dyn[DYN_CNT], i;
int self = __pthread_self()->tid;
pthread_mutex_lock(&init_fini_lock);
for (i=0; (p=queue[i]); i++) {
synchronize shared library dtor exec against concurrent loads/ctors previously, going way back, there was simply no synchronization here. a call to exit concurrent with ctor execution from dlopen could cause a dtor to execute concurrently with its corresponding ctor, or could cause dtors for newly-constructed libraries to be skipped. introduce a shutting_down state that blocks further ctor execution, producing the quiescence the dtor execution loop needs to ensure any kind of consistency, and that blocks further calls to dlopen so that a call into dlopen from a dtor cannot deadlock. better approaches to some of this may be possible, but the changes here at least make things safe.
2019-03-02 03:47:29 +00:00
while ((p->ctor_visitor && p->ctor_visitor!=self) || shutting_down)
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
pthread_cond_wait(&ctor_cond, &init_fini_lock);
if (p->ctor_visitor || p->constructed)
continue;
p->ctor_visitor = self;
add support for init/finit (constructors and destructors) this is mainly in hopes of supporting c++ (not yet possible for other reasons) but will also help applications/libraries which use (and more often, abuse) the gcc __attribute__((__constructor__)) feature in "C" code. x86_64 and arm versions of the new startup asm are untested and may have minor problems.
2012-02-06 19:39:09 +00:00
decode_vec(p->dynv, dyn, DYN_CNT);
add support for init_array/fini_array ctors/dtors to dynamic linker
2013-07-20 22:26:17 +00:00
if (dyn[0] & ((1<<DT_FINI) | (1<<DT_FINI_ARRAY))) {
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
p->fini_next = fini_head;
fini_head = p;
}
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
pthread_mutex_unlock(&init_fini_lock);
disable legacy init/fini processing on ARM since the old, poorly-thought-out musl approach to init/fini arrays on ARM (when it was the only arch that needed them) was to put the code in crti/crtn and have the legacy _init/_fini code run the arrays, adding proper init/fini array support caused the arrays to get processed twice on ARM. I'm not sure skipping legacy init/fini processing is the best solution to the problem, but it works, and it shouldn't break anything since the legacy init/fini system was never used for ARM EABI.
2013-07-22 18:08:33 +00:00
#ifndef NO_LEGACY_INITFINI
don't call null pointer if DT_INIT/DT_FINI are null it's not clear to me why the linker even outputs these headers if they are null, but apparently it does so. with the default startfiles, they will never be null anyway, but this patch allows eliminating crti, crtn, crtbegin, and crtend (leaving only crt1) if the toolchain is using init_array/fini_array (or for a C-only, no-ctor environment).
2013-07-31 04:04:10 +00:00
if ((dyn[0] & (1<<DT_INIT)) && dyn[DT_INIT])
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
fpaddr(p, dyn[DT_INIT])();
disable legacy init/fini processing on ARM since the old, poorly-thought-out musl approach to init/fini arrays on ARM (when it was the only arch that needed them) was to put the code in crti/crtn and have the legacy _init/_fini code run the arrays, adding proper init/fini array support caused the arrays to get processed twice on ARM. I'm not sure skipping legacy init/fini processing is the best solution to the problem, but it works, and it shouldn't break anything since the legacy init/fini system was never used for ARM EABI.
2013-07-22 18:08:33 +00:00
#endif
add support for init_array/fini_array ctors/dtors to dynamic linker
2013-07-20 22:26:17 +00:00
if (dyn[0] & (1<<DT_INIT_ARRAY)) {
size_t n = dyn[DT_INIT_ARRAYSZ]/sizeof(size_t);
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
size_t *fn = laddr(p, dyn[DT_INIT_ARRAY]);
add support for init_array/fini_array ctors/dtors to dynamic linker
2013-07-20 22:26:17 +00:00
while (n--) ((void (*)(void))*fn++)();
}
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
pthread_mutex_lock(&init_fini_lock);
p->ctor_visitor = 0;
p->constructed = 1;
pthread_cond_broadcast(&ctor_cond);
add support for init/finit (constructors and destructors) this is mainly in hopes of supporting c++ (not yet possible for other reasons) but will also help applications/libraries which use (and more often, abuse) the gcc __attribute__((__constructor__)) feature in "C" code. x86_64 and arm versions of the new startup asm are untested and may have minor problems.
2012-02-06 19:39:09 +00:00
}
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
pthread_mutex_unlock(&init_fini_lock);
add support for init/finit (constructors and destructors) this is mainly in hopes of supporting c++ (not yet possible for other reasons) but will also help applications/libraries which use (and more often, abuse) the gcc __attribute__((__constructor__)) feature in "C" code. x86_64 and arm versions of the new startup asm are untested and may have minor problems.
2012-02-06 19:39:09 +00:00
}
move calls to application init functions after crt1 entry point this change is needed to be compatible with fdpic, where some of the main application's relocations may be performed as part of the crt1 entry point. if we call init functions before passing control, these relocations will not yet have been performed, and the init code will potentially make use of invalid pointers. conceptually, no code provided by the application or third-party libraries should run before the application entry point. the difference is not observable to programs using the crt1 we provide, but it could come into play if custom entry point code is used, so it's better to be doing this right anyway.
2015-09-22 20:24:28 +00:00
void __libc_start_init(void)
{
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
do_init_fini(main_ctor_queue);
avoid malloc of ctor queue for programs with no external deps together with the previous two commits, this completes restoration of the property that dynamic-linked apps with no external deps and no tls have no failure paths before entry.
2019-03-03 18:24:23 +00:00
if (!__malloc_replaced && main_ctor_queue != builtin_ctor_queue)
free(main_ctor_queue);
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
main_ctor_queue = 0;
move calls to application init functions after crt1 entry point this change is needed to be compatible with fdpic, where some of the main application's relocations may be performed as part of the crt1 entry point. if we call init functions before passing control, these relocations will not yet have been performed, and the init code will potentially make use of invalid pointers. conceptually, no code provided by the application or third-party libraries should run before the application entry point. the difference is not observable to programs using the crt1 we provide, but it could come into play if custom entry point code is used, so it's better to be doing this right anyway.
2015-09-22 20:24:28 +00:00
}
ensure debugger hook for dynamic linker does not point to a PLT slot this change is made in preparation to support linking without -Bsymbolic-functions.
2015-04-18 03:23:05 +00:00
static void dl_debug_state(void)
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
{
}
ensure debugger hook for dynamic linker does not point to a PLT slot this change is made in preparation to support linking without -Bsymbolic-functions.
2015-04-18 03:23:05 +00:00
weak_alias(dl_debug_state, _dl_debug_state);
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
void __init_tls(size_t *auxv)
add system for resetting TLS to initial values this is needed for reused threads in the SIGEV_THREAD timer notification system, and could be reused elsewhere in the future if needed, though it should be refactored for such use. for static linking, __init_tls.c is simply modified to export the TLS info in a structure with external linkage, rather than using statics. this perhaps makes the code more clear, since the statics were poorly named for statics. the new __reset_tls.c is only linked if it is used. for dynamic linking, the code is in dynlink.c. sharing code with __copy_tls is not practical since __reset_tls must also re-zero thread-local bss.
2013-08-03 20:27:30 +00:00
{
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
}
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
static void update_tls_size()
{
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
libc.tls_cnt = tls_cnt;
libc.tls_align = tls_align;
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
libc.tls_size = ALIGN(
(1+tls_cnt) * sizeof(void *) +
tls_offset +
sizeof(struct pthread) +
tls_align * 2,
tls_align);
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
}
install dynamic tls synchronously at dlopen, streamline access previously, dynamic loading of new libraries with thread-local storage allocated the storage needed for all existing threads at load-time, precluding late failure that can't be handled, but left installation in existing threads to take place lazily on first access. this imposed an additional memory access and branch on every dynamic tls access, and imposed a requirement, which was not actually met, that the dynamic tlsdesc asm functions preserve all call-clobbered registers before calling C code to to install new dynamic tls on first access. the x86[_64] versions of this code wrongly omitted saving and restoring of fpu/vector registers, assuming the compiler would not generate anything using them in the called C code. the arm and aarch64 versions saved known existing registers, but failed to be future-proof against expansion of the register file. now that we track live threads in a list, it's possible to install the new dynamic tls for each thread at dlopen time. for the most part, synchronization is not needed, because if a thread has not synchronized with completion of the dlopen, there is no way it can meaningfully request access to a slot past the end of the old dtv, which remains valid for accessing slots which already existed. however, it is necessary to ensure that, if a thread sees its new dtv pointer, it sees correct pointers in each of the slots that existed prior to the dlopen. my understanding is that, on most real-world coherency architectures including all the ones we presently support, a built-in consume order guarantees this; however, don't rely on that. instead, the SYS_membarrier syscall is used to ensure that all threads see the stores to the slots of their new dtv prior to the installation of the new dtv. if it is not supported, the same is implemented in userspace via signals, using the same mechanism as __synccall. the __tls_get_addr function, variants, and dynamic tlsdesc asm functions are all updated to remove the fallback paths for claiming new dynamic tls, and are now all branch-free.
2019-02-18 04:22:27 +00:00
static void install_new_tls(void)
{
sigset_t set;
pthread_t self = __pthread_self(), td;
fix crash in new dynamic tls installation when last dep lacks tls code introduced in commit 9d44b6460ab603487dab4d916342d9ba4467e6b9 wrongly assumed the dso list tail was the right place to find new dtv storage. however, this is only true if the last-loaded dependency has tls. the correct place to get it is the dso corresponding to the tls module list tail. introduce a container_of macro to get it, and use it. ultimately, dynamic tls allocation should be refactored so that this is not an issue. there is no reason to be allocating new dtv space at each load_library; instead it could happen after all new libraries have been loaded but before they are committed. such changes may be made later, but this commit fixes the present regression.
2019-02-25 07:09:36 +00:00
struct dso *dtv_provider = container_of(tls_tail, struct dso, tls);
uintptr_t (*newdtv)[tls_cnt+1] = (void *)dtv_provider->new_dtv;
install dynamic tls synchronously at dlopen, streamline access previously, dynamic loading of new libraries with thread-local storage allocated the storage needed for all existing threads at load-time, precluding late failure that can't be handled, but left installation in existing threads to take place lazily on first access. this imposed an additional memory access and branch on every dynamic tls access, and imposed a requirement, which was not actually met, that the dynamic tlsdesc asm functions preserve all call-clobbered registers before calling C code to to install new dynamic tls on first access. the x86[_64] versions of this code wrongly omitted saving and restoring of fpu/vector registers, assuming the compiler would not generate anything using them in the called C code. the arm and aarch64 versions saved known existing registers, but failed to be future-proof against expansion of the register file. now that we track live threads in a list, it's possible to install the new dynamic tls for each thread at dlopen time. for the most part, synchronization is not needed, because if a thread has not synchronized with completion of the dlopen, there is no way it can meaningfully request access to a slot past the end of the old dtv, which remains valid for accessing slots which already existed. however, it is necessary to ensure that, if a thread sees its new dtv pointer, it sees correct pointers in each of the slots that existed prior to the dlopen. my understanding is that, on most real-world coherency architectures including all the ones we presently support, a built-in consume order guarantees this; however, don't rely on that. instead, the SYS_membarrier syscall is used to ensure that all threads see the stores to the slots of their new dtv prior to the installation of the new dtv. if it is not supported, the same is implemented in userspace via signals, using the same mechanism as __synccall. the __tls_get_addr function, variants, and dynamic tlsdesc asm functions are all updated to remove the fallback paths for claiming new dynamic tls, and are now all branch-free.
2019-02-18 04:22:27 +00:00
struct dso *p;
size_t i, j;
size_t old_cnt = self->dtv[0];
__block_app_sigs(&set);
__tl_lock();
/* Copy existing dtv contents from all existing threads. */
for (i=0, td=self; !i || td!=self; i++, td=td->next) {
memcpy(newdtv+i, td->dtv,
(old_cnt+1)*sizeof(uintptr_t));
newdtv[i][0] = tls_cnt;
}
/* Install new dtls into the enlarged, uninstalled dtv copies. */
for (p=head; ; p=p->next) {
fix crash/misbehavior from oob read in new dynamic tls installation code introduced in commit 9d44b6460ab603487dab4d916342d9ba4467e6b9 wrongly attempted to read past the end of the currently-installed dtv to determine if a dso provides new, not-already-installed tls. this logic was probably leftover from an earlier draft of the code that wrongly installed the new dtv before populating it. it would work if we instead queried the new, not-yet-installed dtv, but instead, replace the incorrect check with a simple range check against old_cnt. this also catches modules that have no tls at all with a single condition.
2019-02-27 17:02:49 +00:00
if (p->tls_id <= old_cnt) continue;
install dynamic tls synchronously at dlopen, streamline access previously, dynamic loading of new libraries with thread-local storage allocated the storage needed for all existing threads at load-time, precluding late failure that can't be handled, but left installation in existing threads to take place lazily on first access. this imposed an additional memory access and branch on every dynamic tls access, and imposed a requirement, which was not actually met, that the dynamic tlsdesc asm functions preserve all call-clobbered registers before calling C code to to install new dynamic tls on first access. the x86[_64] versions of this code wrongly omitted saving and restoring of fpu/vector registers, assuming the compiler would not generate anything using them in the called C code. the arm and aarch64 versions saved known existing registers, but failed to be future-proof against expansion of the register file. now that we track live threads in a list, it's possible to install the new dynamic tls for each thread at dlopen time. for the most part, synchronization is not needed, because if a thread has not synchronized with completion of the dlopen, there is no way it can meaningfully request access to a slot past the end of the old dtv, which remains valid for accessing slots which already existed. however, it is necessary to ensure that, if a thread sees its new dtv pointer, it sees correct pointers in each of the slots that existed prior to the dlopen. my understanding is that, on most real-world coherency architectures including all the ones we presently support, a built-in consume order guarantees this; however, don't rely on that. instead, the SYS_membarrier syscall is used to ensure that all threads see the stores to the slots of their new dtv prior to the installation of the new dtv. if it is not supported, the same is implemented in userspace via signals, using the same mechanism as __synccall. the __tls_get_addr function, variants, and dynamic tlsdesc asm functions are all updated to remove the fallback paths for claiming new dynamic tls, and are now all branch-free.
2019-02-18 04:22:27 +00:00
unsigned char *mem = p->new_tls;
for (j=0; j<i; j++) {
unsigned char *new = mem;
new += ((uintptr_t)p->tls.image - (uintptr_t)mem)
& (p->tls.align-1);
memcpy(new, p->tls.image, p->tls.len);
newdtv[j][p->tls_id] =
(uintptr_t)new + DTP_OFFSET;
mem += p->tls.size + p->tls.align;
}
if (p->tls_id == tls_cnt) break;
}
/* Broadcast barrier to ensure contents of new dtv is visible
add membarrier syscall wrapper, refactor dynamic tls install to use it the motivation for this change is twofold. first, it gets the fallback logic out of the dynamic linker, improving code readability and organization. second, it provides application code that wants to use the membarrier syscall, which depends on preregistration of intent before the process becomes multithreaded unless unbounded latency is acceptable, with a symbol that, when linked, ensures that this registration happens.
2019-02-22 07:56:10 +00:00
* if the new dtv pointer is. The __membarrier function has a
* fallback emulation using signals for kernels that lack the
* feature at the syscall level. */
__membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED, 0);
install dynamic tls synchronously at dlopen, streamline access previously, dynamic loading of new libraries with thread-local storage allocated the storage needed for all existing threads at load-time, precluding late failure that can't be handled, but left installation in existing threads to take place lazily on first access. this imposed an additional memory access and branch on every dynamic tls access, and imposed a requirement, which was not actually met, that the dynamic tlsdesc asm functions preserve all call-clobbered registers before calling C code to to install new dynamic tls on first access. the x86[_64] versions of this code wrongly omitted saving and restoring of fpu/vector registers, assuming the compiler would not generate anything using them in the called C code. the arm and aarch64 versions saved known existing registers, but failed to be future-proof against expansion of the register file. now that we track live threads in a list, it's possible to install the new dynamic tls for each thread at dlopen time. for the most part, synchronization is not needed, because if a thread has not synchronized with completion of the dlopen, there is no way it can meaningfully request access to a slot past the end of the old dtv, which remains valid for accessing slots which already existed. however, it is necessary to ensure that, if a thread sees its new dtv pointer, it sees correct pointers in each of the slots that existed prior to the dlopen. my understanding is that, on most real-world coherency architectures including all the ones we presently support, a built-in consume order guarantees this; however, don't rely on that. instead, the SYS_membarrier syscall is used to ensure that all threads see the stores to the slots of their new dtv prior to the installation of the new dtv. if it is not supported, the same is implemented in userspace via signals, using the same mechanism as __synccall. the __tls_get_addr function, variants, and dynamic tlsdesc asm functions are all updated to remove the fallback paths for claiming new dynamic tls, and are now all branch-free.
2019-02-18 04:22:27 +00:00
/* Install new dtv for each thread. */
for (j=0, td=self; !j || td!=self; j++, td=td->next) {
td->dtv = td->dtv_copy = newdtv[j];
}
__tl_unlock();
__restore_sigs(&set);
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
/* Stage 1 of the dynamic linker is defined in dlstart.c. It calls the
* following stage 2 and stage 3 functions via primitive symbolic lookup
* since it does not have access to their addresses to begin with. */
/* Stage 2 of the dynamic linker is called after relative relocations
* have been processed. It can make function calls to static functions
* and access string literals and static data, but cannot use extern
* symbols. Its job is to perform symbolic relocations on the dynamic
* linker itself, but some of the relocations performed may need to be
* replaced later due to copy relocations in the main program. */
define and use internal macros for hidden visibility, weak refs this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-09-05 16:43:34 +00:00
hidden void __dls2(unsigned char *base, size_t *sp)
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
{
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (DL_FDPIC) {
void *p1 = (void *)sp[-2];
void *p2 = (void *)sp[-1];
if (!p1) {
size_t *auxv, aux[AUX_CNT];
fix misleading placement of statement on same line as for loop in ldso the placement triggered -Wmisleading-indentation warnings if enabled, and was gratuitously confusing to anyone reading the code.
2018-10-15 19:31:02 +00:00
for (auxv=sp+1+*sp+1; *auxv; auxv++);
auxv++;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
decode_vec(auxv, aux, AUX_CNT);
if (aux[AT_BASE]) ldso.base = (void *)aux[AT_BASE];
else ldso.base = (void *)(aux[AT_PHDR] & -4096);
}
app_loadmap = p2 ? p1 : 0;
ldso.loadmap = p2 ? p2 : p1;
ldso.base = laddr(&ldso, 0);
} else {
ldso.base = base;
}
Ehdr *ehdr = (void *)ldso.base;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
ldso.name = ldso.shortname = "libc.so";
ldso.phnum = ehdr->e_phnum;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
ldso.phdr = laddr(&ldso, ehdr->e_phoff);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
ldso.phentsize = ehdr->e_phentsize;
kernel_mapped_dso(&ldso);
decode_dyn(&ldso);
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (DL_FDPIC) makefuncdescs(&ldso);
reprocess all libc/ldso symbolic relocations in dynamic linking stage 3 commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-26 03:33:59 +00:00
/* Prepare storage for to save clobbered REL addends so they
* can be reused in stage 3. There should be very few. If
* something goes wrong and there are a huge number, abort
* instead of risking stack overflow. */
size_t dyn[DYN_CNT];
decode_vec(ldso.dynv, dyn, DYN_CNT);
refactor some more dynamic linker load address computations these were just missed in the previous commits.
2015-09-17 19:45:45 +00:00
size_t *rel = laddr(&ldso, dyn[DT_REL]);
reprocess all libc/ldso symbolic relocations in dynamic linking stage 3 commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-26 03:33:59 +00:00
size_t rel_size = dyn[DT_RELSZ];
size_t symbolic_rel_cnt = 0;
apply_addends_to = rel;
for (; rel_size; rel+=2, rel_size-=2*sizeof(size_t))
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (!IS_RELATIVE(rel[1], ldso.syms)) symbolic_rel_cnt++;
reprocess all libc/ldso symbolic relocations in dynamic linking stage 3 commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
2015-05-26 03:33:59 +00:00
if (symbolic_rel_cnt >= ADDEND_LIMIT) a_crash();
size_t addends[symbolic_rel_cnt+1];
saved_addends = addends;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
head = &ldso;
reloc_all(&ldso);
ldso.relocated = 0;
move call to dynamic linker stage-3 into stage-2 function this move eliminates a duplicate "by-hand" symbol lookup loop from the stage-1 code and replaces it with a call to find_sym, which can be used once we're in stage 2. it reduces the size of the stage 1 code, which is helpful because stage 1 will become the crt start file for static-PIE executables, and it will allow stage 3 to access stage 2's automatic storage, which will be important in an upcoming commit.
2015-05-25 23:15:17 +00:00
add new stage 2b to dynamic linker bootstrap for thread pointer commit a603a75a72bb469c6be4963ed1b55fabe675fe15 removed attribute const from __errno_location and pthread_self, and the same reasoning forced arch definitions of __pthread_self to use volatile asm, significantly impacting code generation and imposing manual caching of pointers where the impact might be noticable. reorder the thread pointer setup and place it across a strong barrier (symbolic function lookup) so that there is no assumed ordering between the initialization and the accesses to the thread pointer in stage 3.
2018-10-16 17:36:51 +00:00
/* Call dynamic linker stage-2b, __dls2b, looking it up
move call to dynamic linker stage-3 into stage-2 function this move eliminates a duplicate "by-hand" symbol lookup loop from the stage-1 code and replaces it with a call to find_sym, which can be used once we're in stage 2. it reduces the size of the stage 1 code, which is helpful because stage 1 will become the crt start file for static-PIE executables, and it will allow stage 3 to access stage 2's automatic storage, which will be important in an upcoming commit.
2015-05-25 23:15:17 +00:00
* symbolically as a barrier against moving the address
* load across the above relocation processing. */
add new stage 2b to dynamic linker bootstrap for thread pointer commit a603a75a72bb469c6be4963ed1b55fabe675fe15 removed attribute const from __errno_location and pthread_self, and the same reasoning forced arch definitions of __pthread_self to use volatile asm, significantly impacting code generation and imposing manual caching of pointers where the impact might be noticable. reorder the thread pointer setup and place it across a strong barrier (symbolic function lookup) so that there is no assumed ordering between the initialization and the accesses to the thread pointer in stage 3.
2018-10-16 17:36:51 +00:00
struct symdef dls2b_def = find_sym(&ldso, "__dls2b", 0);
if (DL_FDPIC) ((stage3_func)&ldso.funcdescs[dls2b_def.sym-ldso.syms])(sp);
else ((stage3_func)laddr(&ldso, dls2b_def.sym->st_value))(sp);
}
/* Stage 2b sets up a valid thread pointer, which requires relocations
* completed in stage 2, and on which stage 3 is permitted to depend.
* This is done as a separate stage, with symbolic lookup as a barrier,
* so that loads of the thread pointer and &errno can be pure/const and
* thereby hoistable. */
remove unnecessary and problematic _Noreturn from crt/ldso startup after commit a48ccc159a5fa061a18419296100ee48a1cd6cc9 removed the use of _Noreturn on the stage3_func type (which only worked due to it being defined to the "GNU C" attribute in C99 mode), GCC could no longer assume that the ends of __dls2 and __dls2b are unreachable, and produced a warning that a function marked _Noreturn returns. also, since commit 4390383b32250a941ec616e8bff6f568a801b1c0, the _Noreturn declaration for __libc_start_main in crt1/rcrt1 has been not only inconsistent with the definition, but wrong. formally, __libc_start_main does return, via a (hopefully) tail call to a helper function after the barrier. incorrect usage of _Noreturn in the declaration was probably formal UB. the _Noreturn specifiers were not useful in any of these places, so remove them all. now, the only remaining usage of _Noreturn is in public interfaces where _Noreturn is part of their contract.
2019-06-25 22:50:05 +00:00
void __dls2b(size_t *sp)
add new stage 2b to dynamic linker bootstrap for thread pointer commit a603a75a72bb469c6be4963ed1b55fabe675fe15 removed attribute const from __errno_location and pthread_self, and the same reasoning forced arch definitions of __pthread_self to use volatile asm, significantly impacting code generation and imposing manual caching of pointers where the impact might be noticable. reorder the thread pointer setup and place it across a strong barrier (symbolic function lookup) so that there is no assumed ordering between the initialization and the accesses to the thread pointer in stage 3.
2018-10-16 17:36:51 +00:00
{
/* Setup early thread pointer in builtin_tls for ldso/libc itself to
* use during dynamic linking. If possible it will also serve as the
* thread pointer at runtime. */
libc.tls_size = sizeof builtin_tls;
libc.tls_align = tls_align;
if (__init_tp(__copy_tls((void *)builtin_tls)) < 0) {
a_crash();
}
move call to dynamic linker stage-3 into stage-2 function this move eliminates a duplicate "by-hand" symbol lookup loop from the stage-1 code and replaces it with a call to find_sym, which can be used once we're in stage 2. it reduces the size of the stage 1 code, which is helpful because stage 1 will become the crt start file for static-PIE executables, and it will allow stage 3 to access stage 2's automatic storage, which will be important in an upcoming commit.
2015-05-25 23:15:17 +00:00
struct symdef dls3_def = find_sym(&ldso, "__dls3", 0);
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (DL_FDPIC) ((stage3_func)&ldso.funcdescs[dls3_def.sym-ldso.syms])(sp);
else ((stage3_func)laddr(&ldso, dls3_def.sym->st_value))(sp);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
}
/* Stage 3 of the dynamic linker is called with the dynamic linker/libc
* fully functional. Its job is to load (if not already loaded) and
* process dependencies and relocations for the main application and
* transfer control to its entry point. */
remove unnecessary and problematic _Noreturn from crt/ldso startup after commit a48ccc159a5fa061a18419296100ee48a1cd6cc9 removed the use of _Noreturn on the stage3_func type (which only worked due to it being defined to the "GNU C" attribute in C99 mode), GCC could no longer assume that the ends of __dls2 and __dls2b are unreachable, and produced a warning that a function marked _Noreturn returns. also, since commit 4390383b32250a941ec616e8bff6f568a801b1c0, the _Noreturn declaration for __libc_start_main in crt1/rcrt1 has been not only inconsistent with the definition, but wrong. formally, __libc_start_main does return, via a (hopefully) tail call to a helper function after the barrier. incorrect usage of _Noreturn in the declaration was probably formal UB. the _Noreturn specifiers were not useful in any of these places, so remove them all. now, the only remaining usage of _Noreturn is in public interfaces where _Noreturn is part of their contract.
2019-06-25 22:50:05 +00:00
void __dls3(size_t *sp)
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
{
static struct dso app, vdso;
remove initializers for decoded aux/dyn arrays in dynamic linker the zero initialization is redundant since decode_vec does its own clearing, and it increases the risk that buggy compilers will generate calls to memset. as long as symbols are bound at ld time, such a call will not break anything, but it may be desirable to turn off ld-time binding in the future.
2015-04-14 02:38:18 +00:00
size_t aux[AUX_CNT], *auxv;
experimental dynamic linker! some notes: - library search path is hard coded - x86_64 code is untested and may not work - dlopen/dlsym is not yet implemented - relocations in read-only memory won't work
2011-06-18 23:48:42 +00:00
size_t i;
LD_PRELOAD support
2011-08-16 04:24:36 +00:00
char *env_preload=0;
allow specifying argv[0] when invoking a program via ldso command previously, the pathname used to load the program was always used as argv[0]. the default remains the same, but a new --argv0 option can be used to provide a different value.
2017-07-04 20:58:28 +00:00
char *replace_argv0=0;
clean up search_vec usage for vdso
2012-08-25 21:31:59 +00:00
size_t vdso_base;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
int argc = *sp;
char **argv = (void *)(sp+1);
char **argv_orig = argv;
add support for init/fini array in main program, and greatly simplify modern (4.7.x and later) gcc uses init/fini arrays, rather than the legacy _init/_fini function pasting and crtbegin/crtend ctors/dtors system, on most or all archs. some archs had already switched a long time ago. without following this change, global ctors/dtors will cease to work under musl when building with new gcc versions. the most surprising part of this patch is that it actually reduces the size of the init code, for both static and shared libc. this is achieved by (1) unifying the handling main program and shared libraries in the dynamic linker, and (2) eliminating the glibc-inspired rube goldberg machine for passing around init and fini function pointers. to clarify, some background: the function signature for __libc_start_main was based on glibc, as part of the original goal of being able to run some glibc-linked binaries. it worked by having the crt1 code, which is linked into every application, static or dynamic, obtain and pass pointers to the init and fini functions, which __libc_start_main is then responsible for using and recording for later use, as necessary. however, in neither the static-linked nor dynamic-linked case do we actually need crt1.o's help. with dynamic linking, all the pointers are available in the _DYNAMIC block. with static linking, it's safe to simply access the _init/_fini and __init_array_start, etc. symbols directly. obviously changing the __libc_start_main function signature in an incompatible way would break both old musl-linked programs and glibc-linked programs, so let's not do that. instead, the function can just ignore the information it doesn't need. new archs need not even provide the useless args in their versions of crt1.o. existing archs should continue to provide it as long as there is an interest in having newly-linked applications be able to run on old versions of musl; at some point in the future, this support can be removed.
2013-07-21 07:00:54 +00:00
char **envp = argv+argc+1;
move thread pointer setup to beginning of dynamic linker stage 3 this allows the dynamic linker itself to run with a valid thread pointer, which is a prerequisite for stack protector on archs where the ssp canary is stored in TLS. it will also allow us to remove some remaining runtime checks for whether the thread pointer is valid. as long as the application and its libraries do not require additional size or alignment, this early thread pointer will be kept and reused at runtime. otherwise, a new static TLS block is allocated after library loading has finished and the thread pointer is switched over.
2015-04-13 22:40:52 +00:00
fix regression in pre-v7 arm on kernels with kuser helper removed the arm atomics/TLS runtime selection code is called from __set_thread_area and depends on having libc.auxv and __hwcap available. commit 71f099cb7db821c51d8f39dfac622c61e54d794c moved the first call to __set_thread_area to the top of dynamic linking stage 3, before this data is made available, causing the runtime detection code to always see __hwcap as zero and thereby select the atomics/TLS implementations based on kuser helper. upcoming work on superh will use similar runtime detection. ideally this early-init code should be cleanly refactored and shared between the dynamic linker and static-linked startup.
2015-06-07 20:55:23 +00:00
/* Find aux vector just past environ[] and use it to initialize
* global data that may be needed before we can make syscalls. */
__environ = envp;
for (i=argc+1; argv[i]; i++);
libc.auxv = auxv = (void *)(argv+i+1);
decode_vec(auxv, aux, AUX_CNT);
__hwcap = aux[AT_HWCAP];
overhaul i386 syscall mechanism not to depend on external asm source this is the first part of a series of patches intended to make __syscall fully self-contained in the object file produced using syscall.h, which will make it possible for crt1 code to perform syscalls. the (confusingly named) i386 __vsyscall mechanism, which this commit removes, was introduced before the presence of a valid thread pointer was mandatory; back then the thread pointer was setup lazily only if threads were used. the intent was to be able to perform syscalls using the kernel's fast entry point in the VDSO, which can use the sysenter (Intel) or syscall (AMD) instruction instead of int $128, but without inlining an access to the __syscall global at the point of each syscall, which would incur a significant size cost from PIC setup everywhere. the mechanism also shuffled registers/calling convention around to avoid spills of call-saved registers, and to avoid allocating ebx or ebp via asm constraints, since there are plenty of broken-but-supported compiler versions which are incapable of allocating ebx with -fPIC or ebp with -fno-omit-frame-pointer. the new mechanism preserves the properties of avoiding spills and avoiding allocation of ebx/ebp in constraints, but does it inline, using some fairly simple register shuffling, and uses a field of the thread structure rather than global data for the vdso-provided syscall code address. for now, the external __syscall function is refactored not to use the old __vsyscall so it can be kept, but the intent is to remove it too.
2019-04-10 21:10:36 +00:00
search_vec(auxv, &__sysinfo, AT_SYSINFO);
__pthread_self()->sysinfo = __sysinfo;
fix regression in pre-v7 arm on kernels with kuser helper removed the arm atomics/TLS runtime selection code is called from __set_thread_area and depends on having libc.auxv and __hwcap available. commit 71f099cb7db821c51d8f39dfac622c61e54d794c moved the first call to __set_thread_area to the top of dynamic linking stage 3, before this data is made available, causing the runtime detection code to always see __hwcap as zero and thereby select the atomics/TLS implementations based on kuser helper. upcoming work on superh will use similar runtime detection. ideally this early-init code should be cleanly refactored and shared between the dynamic linker and static-linked startup.
2015-06-07 20:55:23 +00:00
libc.page_size = aux[AT_PAGESZ];
libc.secure = ((aux[0]&0x7800)!=0x7800 || aux[AT_UID]!=aux[AT_EUID]
|| aux[AT_GID]!=aux[AT_EGID] || aux[AT_SECURE]);
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
/* Only trust user/env if kernel says we're not suid/sgid */
fix regression in pre-v7 arm on kernels with kuser helper removed the arm atomics/TLS runtime selection code is called from __set_thread_area and depends on having libc.auxv and __hwcap available. commit 71f099cb7db821c51d8f39dfac622c61e54d794c moved the first call to __set_thread_area to the top of dynamic linking stage 3, before this data is made available, causing the runtime detection code to always see __hwcap as zero and thereby select the atomics/TLS implementations based on kuser helper. upcoming work on superh will use similar runtime detection. ideally this early-init code should be cleanly refactored and shared between the dynamic linker and static-linked startup.
2015-06-07 20:55:23 +00:00
if (!libc.secure) {
env_path = getenv("LD_LIBRARY_PATH");
env_preload = getenv("LD_PRELOAD");
proper path searching for dynamic linker first, use $LD_LIBRARY_PATH unless suid. if that fails, read path from /etc/ld-musl-$ARCH.path and fallback to a builtin default.
2011-06-25 05:56:34 +00:00
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
/* If the main program was already loaded by the kernel,
* AT_PHDR will point to some location other than the dynamic
* linker's program headers. */
if (aux[AT_PHDR] != (size_t)ldso.phdr) {
make dynamic linker tell the debugger its own pathname use the main program's PT_INTERP header if possible, since this is sure to be a correct (and hopefully absolute) pathname.
2012-07-13 05:31:02 +00:00
size_t interp_off = 0;
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
size_t tls_image = 0;
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
/* Find load address of the main program, via AT_PHDR vs PT_PHDR. */
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
Phdr *phdr = app.phdr = (void *)aux[AT_PHDR];
app.phnum = aux[AT_PHNUM];
app.phentsize = aux[AT_PHENT];
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
for (i=aux[AT_PHNUM]; i; i--, phdr=(void *)((char *)phdr + aux[AT_PHENT])) {
if (phdr->p_type == PT_PHDR)
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
app.base = (void *)(aux[AT_PHDR] - phdr->p_vaddr);
make dynamic linker tell the debugger its own pathname use the main program's PT_INTERP header if possible, since this is sure to be a correct (and hopefully absolute) pathname.
2012-07-13 05:31:02 +00:00
else if (phdr->p_type == PT_INTERP)
interp_off = (size_t)phdr->p_vaddr;
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
else if (phdr->p_type == PT_TLS) {
tls_image = phdr->p_vaddr;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
app.tls.len = phdr->p_filesz;
app.tls.size = phdr->p_memsz;
app.tls.align = phdr->p_align;
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
}
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
}
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (DL_FDPIC) app.loadmap = app_loadmap;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (app.tls.size) app.tls.image = laddr(&app, tls_image);
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
if (interp_off) ldso.name = laddr(&app, interp_off);
use AT_EXECFN, if available, for dynamic linker to identify main program fallback to argv[0] as before. unlike argv[0], AT_EXECFN was a valid (but possibly relative) pathname for the new program image at the time the execve syscall was made. as a special case, ignore AT_EXECFN if it begins with "/proc/", in order not to give bogus (and possibly harmful) results when fexecve was used.
2013-08-23 18:14:47 +00:00
if ((aux[0] & (1UL<<AT_EXECFN))
&& strncmp((char *)aux[AT_EXECFN], "/proc/", 6))
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
app.name = (char *)aux[AT_EXECFN];
use AT_EXECFN, if available, for dynamic linker to identify main program fallback to argv[0] as before. unlike argv[0], AT_EXECFN was a valid (but possibly relative) pathname for the new program image at the time the execve syscall was made. as a special case, ignore AT_EXECFN if it begins with "/proc/", in order not to give bogus (and possibly harmful) results when fexecve was used.
2013-08-23 18:14:47 +00:00
else
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
app.name = argv[0];
kernel_mapped_dso(&app);
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
} else {
int fd;
char *ldname = argv[0];
fix more unused variable warnings some of these were coming from stdio functions locking files without unlocking them. I believe it's useful for this to throw a warning, so I added a new macro that's self-documenting that the file will never be unlocked to avoid the warning in the few places where it's wrong.
2012-11-02 03:46:39 +00:00
size_t l = strlen(ldname);
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
if (l >= 3 && !strcmp(ldname+l-3, "ldd")) ldd_mode = 1;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
argv++;
add options when explicitly invoking dynamic loader so far the options are --library-path and --preload which override the corresponding environment variables, and --list which forces the behavior of ldd even if the invocation name is not ldd. both the two-arg form and the one-arg form using an equals sign are supported. based loosely on a patch proposed by Rune.
2014-04-16 16:45:36 +00:00
while (argv[0] && argv[0][0]=='-' && argv[0][1]=='-') {
char *opt = argv[0]+2;
*argv++ = (void *)-1;
if (!*opt) {
break;
} else if (!memcmp(opt, "list", 5)) {
ldd_mode = 1;
} else if (!memcmp(opt, "library-path", 12)) {
if (opt[12]=='=') env_path = opt+13;
else if (opt[12]) *argv = 0;
else if (*argv) env_path = *argv++;
} else if (!memcmp(opt, "preload", 7)) {
if (opt[7]=='=') env_preload = opt+8;
else if (opt[7]) *argv = 0;
else if (*argv) env_preload = *argv++;
allow specifying argv[0] when invoking a program via ldso command previously, the pathname used to load the program was always used as argv[0]. the default remains the same, but a new --argv0 option can be used to provide a different value.
2017-07-04 20:58:28 +00:00
} else if (!memcmp(opt, "argv0", 5)) {
if (opt[5]=='=') replace_argv0 = opt+6;
else if (opt[5]) *argv = 0;
else if (*argv) replace_argv0 = *argv++;
add options when explicitly invoking dynamic loader so far the options are --library-path and --preload which override the corresponding environment variables, and --list which forces the behavior of ldd even if the invocation name is not ldd. both the two-arg form and the one-arg form using an equals sign are supported. based loosely on a patch proposed by Rune.
2014-04-16 16:45:36 +00:00
} else {
argv[0] = 0;
}
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
argv[-1] = (void *)(argc - (argv-argv_orig));
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
if (!argv[0]) {
add arch/abi info to dynamic linker's id/version output
2016-01-22 04:04:16 +00:00
dprintf(2, "musl libc (" LDSO_ARCH ")\n"
add infrastructure to record and report the version of libc.so this is still experimental and subject to change. for git checkouts, an attempt is made to record the exact revision to aid in bug reports and debugging. no version information is recorded in the static libc.a or binaries it's linked into.
2013-12-01 22:27:25 +00:00
"Version %s\n"
"Dynamic Program Loader\n"
add options when explicitly invoking dynamic loader so far the options are --library-path and --preload which override the corresponding environment variables, and --list which forces the behavior of ldd even if the invocation name is not ldd. both the two-arg form and the one-arg form using an equals sign are supported. based loosely on a patch proposed by Rune.
2014-04-16 16:45:36 +00:00
"Usage: %s [options] [--] pathname%s\n",
improve machinery for ldso to report libc version eliminate gratuitous glue function for reporting the version, which was probably leftover from the old dynamic linker design which lacked a clear barrier for when/how it could access global data. put the declaration for the data object that replaces it in libc.h where it can be type checked.
2018-09-06 20:17:56 +00:00
__libc_version, ldname,
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
ldd_mode ? "" : " [args]");
_exit(1);
}
fd = open(argv[0], O_RDONLY);
if (fd < 0) {
dprintf(2, "%s: cannot load %s: %s\n", ldname, argv[0], strerror(errno));
_exit(1);
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
Ehdr *ehdr = (void *)map_library(fd, &app);
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
if (!ehdr) {
dprintf(2, "%s: %s: Not a valid dynamic program\n", ldname, argv[0]);
_exit(1);
}
close(fd);
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
ldso.name = ldname;
app.name = argv[0];
begin refactoring load address computations in dynamic linker for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
2015-09-17 17:18:09 +00:00
aux[AT_ENTRY] = (size_t)laddr(&app, ehdr->e_entry);
make ldd report the libc/dynamic linker itself
2013-07-26 18:41:12 +00:00
/* Find the name that would have been used for the dynamic
* linker had ldd not taken its place. */
if (ldd_mode) {
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
for (i=0; i<app.phnum; i++) {
if (app.phdr[i].p_type == PT_INTERP)
fix resolving interp string address on fdpic ldd command
2015-09-22 19:21:57 +00:00
ldso.name = laddr(&app, app.phdr[i].p_vaddr);
make ldd report the libc/dynamic linker itself
2013-07-26 18:41:12 +00:00
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
dprintf(1, "\t%s (%p)\n", ldso.name, ldso.base);
make ldd report the libc/dynamic linker itself
2013-07-26 18:41:12 +00:00
}
dynamic linker support for PIE binaries (position-independent main program) even with this change, PIE will not work yet due to deficiencies in the crt1.o startup code.
2012-01-23 07:02:59 +00:00
}
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (app.tls.size) {
fix regression in dynamic-linked tls when both main app & libs have tls commit d56460c939c94a6c547abe8238f442b8de10bfbd introduced this bug by setting up the tls module chain incorrectly when the main app has tls. the singly-linked list head pointer was setup correctly, but the tail pointer was not, so the first attempt to append to the list (for a shared library with tls) would treat the list as empty and effectively removed the main app from the list. this left all tls module id numbers off-by-one. this bug did not appear in any released versions.
2016-01-30 19:34:45 +00:00
libc.tls_head = tls_tail = &app.tls;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
app.tls_id = tls_cnt = 1;
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
#ifdef TLS_ABOVE_TP
fix TLS layout of TLS variant I when there is a gap above TP In TLS variant I the TLS is above TP (or above a fixed offset from TP) but on some targets there is a reserved gap above TP before TLS starts. This matters for the local-exec tls access model when the offsets of TLS variables from the TP are hard coded by the linker into the executable, so the libc must compute these offsets the same way as the linker. The tls offset of the main module has to be alignup(GAP_ABOVE_TP, main_tls_align). If there is no TLS in the main module then the gap can be ignored since musl does not use it and the tls access models of shared libraries are not affected. The previous setup only worked if (tls_align & -GAP_ABOVE_TP) == 0 (i.e. TLS did not require large alignment) because the gap was treated as a fixed offset from TP. Now the TP points at the end of the pthread struct (which is aligned) and there is a gap above it (which may also need alignment). The fix required changing TP_ADJ and __pthread_self on affected targets (aarch64, arm and sh) and in the tlsdesc asm the offset to access the dtv changed too.
2018-06-01 23:52:01 +00:00
app.tls.offset = GAP_ABOVE_TP;
fix tls offsets when p_vaddr%p_align != 0 on TLS_ABOVE_TP targets currently the bfd linker does not seem to create tls segments where p_vaddr%p_align != 0, but this is valid in ELF and then the runtime computed tls offset must satisfy offset%p_align == (base+p_vaddr)%p_align and in case of local exec tls (main executable) the smallest such offset must be used (otherwise it is incompatible with the offset computed by the static linker). the !TLS_ABOVE_TP case is handled correctly (the offset is negative then in the formula). the ldso code for TLS_ABOVE_TP is changed so the static tls offset of each module satisfies the formula.
2019-05-13 18:47:11 +00:00
app.tls.offset += (-GAP_ABOVE_TP + (uintptr_t)app.tls.image)
& (app.tls.align-1);
fix static tls offsets of shared libs on TLS_ABOVE_TP targets tls_offset should always point to the end of the allocated static tls area, but this was not handled correctly on "tls variant 1" targets in the dynamic linker: after application tls was allocated, tls_offset was aligned up, potentially wasting tls space. (alignment may be needed at the begining of the tls area, not at the end, but that will be fixed separately as it is unlikely to affect real binaries.) when static tls was allocated for a shared library, tls_offset was only updated with the size of the tls segment which does not include alignment gaps, which can easily happen if the tls size update for one library leaves tls_offset misaligned for the next one. this can cause oob access in __copy_tls or arbitrary breakage at tls access. (the issue was observed on aarch64 with rust binaries)
2019-05-16 17:15:33 +00:00
tls_offset = app.tls.offset + app.tls.size;
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
#else
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
tls_offset = app.tls.offset = app.tls.size
+ ( -((uintptr_t)app.tls.image + app.tls.size)
& (app.tls.align-1) );
add support for TLS variant I, presently needed for arm and mips despite documentation that makes it sound a lot different, the only ABI-constraint difference between TLS variants II and I seems to be that variant II stores the initial TLS segment immediately below the thread pointer (i.e. the thread pointer points to the end of it) and variant I stores the initial TLS segment above the thread pointer, requiring the thread descriptor to be stored below. the actual value stored in the thread pointer register also tends to have per-arch random offsets applied to it for silly micro-optimization purposes. with these changes applied, TLS should be basically working on all supported archs except microblaze. I'm still working on getting the necessary information and a working toolchain that can build TLS binaries for microblaze, but in theory, static-linked programs with TLS and dynamic-linked programs where only the main executable uses TLS should already work on microblaze. alignment constraints have not yet been heavily tested, so it's possible that this code does not always align TLS segments correctly on archs that need TLS variant I.
2012-10-15 22:51:53 +00:00
#endif
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
tls_align = MAXP2(tls_align, app.tls.align);
partial TLS support for dynamic-linked programs only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
2012-10-05 00:04:13 +00:00
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
decode_dyn(&app);
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
if (DL_FDPIC) {
makefuncdescs(&app);
if (!app.loadmap) {
app.loadmap = (void *)&app_dummy_loadmap;
app.loadmap->nsegs = 1;
fix dlsym RTLD_NEXT behavior for fdpic lookup the dso an address falls in based on the loadmap and not just a base/length. fix the main app's fake loadmap used when loaded by a non-fdpic-aware loader so that it does not cover the whole memory space. function descriptor addresses are also matched for future use by dladdr, but reverse lookups of function descriptors via dladdr have not been implemented yet. some revisions may be needed in the future once reclaim_gaps supports fdpic, so that function descriptors allocated in reclaimed heap space do not get detected as belonging to the module whose gaps they were allocated in.
2015-09-22 23:41:41 +00:00
app.loadmap->segs[0].addr = (size_t)app.map;
app.loadmap->segs[0].p_vaddr = (size_t)app.map
- (size_t)app.base;
app.loadmap->segs[0].p_memsz = app.map_len;
add general fdpic support in dynamic linker and arch support for sh at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
2015-09-22 03:54:42 +00:00
}
argv[-3] = (void *)app.loadmap;
}
cleanup dynamic linker, removing some code duplication
2012-01-23 05:57:38 +00:00
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
/* Initial dso chain consists only of the app. */
head = tail = syms_tail = &app;
/* Donate unused parts of app and library mapping to malloc */
reclaim_gaps(&app);
reclaim_gaps(&ldso);
/* Load preload/needed libraries, add symbols to global namespace. */
avoid malloc of deps arrays for ldso and vdso neither has or can have any dependencies, but since commit 403555690775f7c8806372644f543518e6664e3b, gratuitous zero-length deps arrays were being allocated for them. use a dummy array instead.
2019-03-03 17:42:34 +00:00
ldso.deps = (struct dso **)no_deps;
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
if (env_preload) load_preload(env_preload);
load_deps(&app);
for (struct dso *p=head; p; p=p->next)
add_syms(p);
/* Attach to vdso, if provided by the kernel, last so that it does
* not become part of the global namespace. */
treat null vdso base same as missing On s390x, the kernel provides AT_SYSINFO_EHDR, but sets it to zero, if the program being run does not have a program interpreter. This causes problems when running the dynamic linker directly.
2016-07-26 03:52:58 +00:00
if (search_vec(auxv, &vdso_base, AT_SYSINFO_EHDR) && vdso_base) {
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
Ehdr *ehdr = (void *)vdso_base;
Phdr *phdr = vdso.phdr = (void *)(vdso_base + ehdr->e_phoff);
vdso.phnum = ehdr->e_phnum;
vdso.phentsize = ehdr->e_phentsize;
load vdso, if present, into the dso list
2011-07-24 04:54:55 +00:00
for (i=ehdr->e_phnum; i; i--, phdr=(void *)((char *)phdr + ehdr->e_phentsize)) {
if (phdr->p_type == PT_DYNAMIC)
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
vdso.dynv = (void *)(vdso_base + phdr->p_offset);
load vdso, if present, into the dso list
2011-07-24 04:54:55 +00:00
if (phdr->p_type == PT_LOAD)
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
vdso.base = (void *)(vdso_base - phdr->p_vaddr + phdr->p_offset);
load vdso, if present, into the dso list
2011-07-24 04:54:55 +00:00
}
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
vdso.name = "";
vdso.shortname = "linux-gate.so.1";
vdso.relocated = 1;
avoid malloc of deps arrays for ldso and vdso neither has or can have any dependencies, but since commit 403555690775f7c8806372644f543518e6664e3b, gratuitous zero-length deps arrays were being allocated for them. use a dummy array instead.
2019-03-03 17:42:34 +00:00
vdso.deps = (struct dso **)no_deps;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
decode_dyn(&vdso);
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
vdso.prev = tail;
tail->next = &vdso;
tail = &vdso;
load vdso, if present, into the dso list
2011-07-24 04:54:55 +00:00
}
ldso: fix GDB dynamic linker info on MIPS GDB is looking for a pointer to the ldso debug info in the data of the ..rld_map section. Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-01-30 20:14:05 +00:00
for (i=0; app.dynv[i]; i+=2) {
if (!DT_DEBUG_INDIRECT && app.dynv[i]==DT_DEBUG)
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
app.dynv[i+1] = (size_t)&debug;
ldso: fix GDB dynamic linker info on MIPS GDB is looking for a pointer to the ldso debug info in the data of the ..rld_map section. Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-01-30 20:14:05 +00:00
if (DT_DEBUG_INDIRECT && app.dynv[i]==DT_DEBUG_INDIRECT) {
size_t *ptr = (size_t *) app.dynv[i+1];
*ptr = (size_t)&debug;
}
}
implement PT_GNU_RELRO support
2014-03-25 12:13:27 +00:00
fix malloc misuse for startup ctor queue, breakage on fdpic archs in the case where malloc is being replaced, it's not valid to call malloc between final relocations and main app's crt1 entry point; on fdpic archs the main app's entry point will not yet have performed the self-fixups necessary to call its code. to fix, reorder queue_ctors before final relocations. an alternative solution would be doing the allocation from __libc_start_init, after the entry point but before any ctors run. this is less desirable, since it would leave a call to malloc that might be provided by the application happening at startup when doing so can be easily avoided.
2019-03-03 14:57:19 +00:00
/* This must be done before final relocations, since it calls
* malloc, which may be provided by the application. Calling any
* application code prior to the jump to its entry point is not
* valid in our model and does not work with FDPIC, where there
* are additional relocation-like fixups that only the entry point
* code can see to perform. */
main_ctor_queue = queue_ctors(&app);
ldso: fix calloc misuse allocating initial tls this is analogous to commit 2f1f51ae7b2d78247568e7fdb8462f3c19e469a4, and should have been caught at the same time since it was right next to the code moved in that commit. between final stage 3 reloc_all and the jump to the main program's entry point, it is not valid to call any functions which may be interposed by the application; doing so results in execution of application code before ctors have run, and on fdpic archs, before the main program's fdpic self-fixups have taken place, which will produce runaway wrong execution.
2019-08-11 15:48:06 +00:00
/* Initial TLS must also be allocated before final relocations
* might result in calloc being a call to application code. */
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
update_tls_size();
fix regression whereby main thread didn't get TLS relocations commit ffab43602b5900c86b7040abdda8ccf6cdec95f5 broke this by moving relocations after not only the allocation of storage for the main thread's static TLS, but after the copying of the TLS image. thus, relocation results were not reflected in the main thread's copy. this could be fixed by calling __reset_tls after relocations, but instead split the allocation and installation before/after relocations so that there's not a redundant copy. due to commit 71af5309874269bcc9e4b84ea716fab33d888c1d, updating of static_tls_cnt needs to be kept with allocation of static TLS, before relocations, rather than after installation.
2019-08-14 01:53:30 +00:00
void *initial_tls = builtin_tls;
move thread pointer setup to beginning of dynamic linker stage 3 this allows the dynamic linker itself to run with a valid thread pointer, which is a prerequisite for stack protector on archs where the ssp canary is stored in TLS. it will also allow us to remove some remaining runtime checks for whether the thread pointer is valid. as long as the application and its libraries do not require additional size or alignment, this early thread pointer will be kept and reused at runtime. otherwise, a new static TLS block is allocated after library loading has finished and the thread pointer is switched over.
2015-04-13 22:40:52 +00:00
if (libc.tls_size > sizeof builtin_tls || tls_align > MIN_TLS_ALIGN) {
fix regression whereby main thread didn't get TLS relocations commit ffab43602b5900c86b7040abdda8ccf6cdec95f5 broke this by moving relocations after not only the allocation of storage for the main thread's static TLS, but after the copying of the TLS image. thus, relocation results were not reflected in the main thread's copy. this could be fixed by calling __reset_tls after relocations, but instead split the allocation and installation before/after relocations so that there's not a redundant copy. due to commit 71af5309874269bcc9e4b84ea716fab33d888c1d, updating of static_tls_cnt needs to be kept with allocation of static TLS, before relocations, rather than after installation.
2019-08-14 01:53:30 +00:00
initial_tls = calloc(libc.tls_size, 1);
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 20:57:11 +00:00
if (!initial_tls) {
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
dprintf(2, "%s: Error getting %zu bytes thread-local storage: %m\n",
fix namespace clash (libc) in dynlink.c this makes it so the #undef libc and __libc name are no longer needed, which were problematic because the "accessor function" mode for accessing the libc struct could not be used, breaking build on any compiler without (working) visibility.
2012-10-14 03:25:20 +00:00
argv[0], libc.tls_size);
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
_exit(127);
}
fix regression whereby main thread didn't get TLS relocations commit ffab43602b5900c86b7040abdda8ccf6cdec95f5 broke this by moving relocations after not only the allocation of storage for the main thread's static TLS, but after the copying of the TLS image. thus, relocation results were not reflected in the main thread's copy. this could be fixed by calling __reset_tls after relocations, but instead split the allocation and installation before/after relocations so that there's not a redundant copy. due to commit 71af5309874269bcc9e4b84ea716fab33d888c1d, updating of static_tls_cnt needs to be kept with allocation of static TLS, before relocations, rather than after installation.
2019-08-14 01:53:30 +00:00
}
static_tls_cnt = tls_cnt;
/* The main program must be relocated LAST since it may contain
* copy relocations which depend on libraries' relocations. */
reloc_all(app.next);
reloc_all(&app);
/* Actual copying to new TLS needs to happen after relocations,
* since the TLS images might have contained relocated addresses. */
if (initial_tls != builtin_tls) {
move thread pointer setup to beginning of dynamic linker stage 3 this allows the dynamic linker itself to run with a valid thread pointer, which is a prerequisite for stack protector on archs where the ssp canary is stored in TLS. it will also allow us to remove some remaining runtime checks for whether the thread pointer is valid. as long as the application and its libraries do not require additional size or alignment, this early thread pointer will be kept and reused at runtime. otherwise, a new static TLS block is allocated after library loading has finished and the thread pointer is switched over.
2015-04-13 22:40:52 +00:00
if (__init_tp(__copy_tls(initial_tls)) < 0) {
remove remnants of support for running in no-thread-pointer mode since 1.1.0, musl has nominally required a thread pointer to be setup. most of the remaining code that was checking for its availability was doing so for the sake of being usable by the dynamic linker. as of commit 71f099cb7db821c51d8f39dfac622c61e54d794c, this is no longer necessary; the thread pointer is now valid before any libc code (outside of dynamic linker bootstrap functions) runs. this commit essentially concludes "phase 3" of the "transition path for removing lazy init of thread pointer" project that began during the 1.1.0 release cycle.
2015-04-13 23:24:51 +00:00
a_crash();
move thread pointer setup to beginning of dynamic linker stage 3 this allows the dynamic linker itself to run with a valid thread pointer, which is a prerequisite for stack protector on archs where the ssp canary is stored in TLS. it will also allow us to remove some remaining runtime checks for whether the thread pointer is valid. as long as the application and its libraries do not require additional size or alignment, this early thread pointer will be kept and reused at runtime. otherwise, a new static TLS block is allocated after library loading has finished and the thread pointer is switched over.
2015-04-13 22:40:52 +00:00
}
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 20:57:11 +00:00
} else {
move thread pointer setup to beginning of dynamic linker stage 3 this allows the dynamic linker itself to run with a valid thread pointer, which is a prerequisite for stack protector on archs where the ssp canary is stored in TLS. it will also allow us to remove some remaining runtime checks for whether the thread pointer is valid. as long as the application and its libraries do not require additional size or alignment, this early thread pointer will be kept and reused at runtime. otherwise, a new static TLS block is allocated after library loading has finished and the thread pointer is switched over.
2015-04-13 22:40:52 +00:00
size_t tmp_tls_size = libc.tls_size;
pthread_t self = __pthread_self();
/* Temporarily set the tls size to the full size of
* builtin_tls so that __copy_tls will use the same layout
* as it did for before. Then check, just to be safe. */
libc.tls_size = sizeof builtin_tls;
if (__copy_tls((void*)builtin_tls) != self) a_crash();
libc.tls_size = tmp_tls_size;
dynamic-linked TLS support for everything but dlopen'd libs currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
2012-10-05 02:48:33 +00:00
}
ldso: fix calloc misuse allocating initial tls this is analogous to commit 2f1f51ae7b2d78247568e7fdb8462f3c19e469a4, and should have been caught at the same time since it was right next to the code moved in that commit. between final stage 3 reloc_all and the jump to the main program's entry point, it is not valid to call any functions which may be interposed by the application; doing so results in execution of application code before ctors have run, and on fdpic archs, before the main program's fdpic self-fixups have taken place, which will produce runaway wrong execution.
2019-08-11 15:48:06 +00:00
make dynamic linker report all failures before exiting before, only the first library that failed to load or symbol that failed to resolve was reported, and then the dynamic linker immediately exited. when attempting to fix a library compatibility issue, this is about the worst possible behavior. now we print all errors as they occur and exit at the very end if errors were encountered.
2012-08-18 20:00:23 +00:00
if (ldso_fail) _exit(127);
add ldd and main program loading support to dynamic linker
2012-05-27 20:01:44 +00:00
if (ldd_mode) _exit(0);
reintroduce hardening against partially-replaced allocator commit 618b18c78e33acfe54a4434e91aa57b8e171df89 removed the previous detection and hardening since it was incorrect. commit 72141795d4edd17f88da192447395a48444afa10 already handled all that remained for hardening the static-linked case. in the dynamic-linked case, have the dynamic linker check whether malloc was replaced and make that information available. with these changes, the properties documented in commit c9f415d7ea2dace5bf77f6518b6afc36bb7a5732 are restored: if calloc is not provided, it will behave as malloc+memset, and any of the memalign-family functions not provided will fail with ENOMEM.
2018-04-20 02:19:29 +00:00
/* Determine if malloc was interposed by a replacement implementation
* so that calloc and the memalign family can harden against the
* possibility of incomplete replacement. */
if (find_sym(head, "malloc", 1).dso != &ldso)
__malloc_replaced = 1;
cleanup dynamic linker, removing some code duplication
2012-01-23 05:57:38 +00:00
/* Switch to runtime mode: any further failures in the dynamic
* linker are a reportable failure rather than a fatal startup
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
* error. */
simplify dynamic linker startup instead of creating temp dso objects on the stack and moving them to the heap if dlopen/dlsym are used, use static objects to begin with, and just donate them to malloc if we no longer need them.
2011-07-24 04:26:12 +00:00
runtime = 1;
add support for init/finit (constructors and destructors) this is mainly in hopes of supporting c++ (not yet possible for other reasons) but will also help applications/libraries which use (and more often, abuse) the gcc __attribute__((__constructor__)) feature in "C" code. x86_64 and arm versions of the new startup asm are untested and may have minor problems.
2012-02-06 19:39:09 +00:00
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
debug.ver = 1;
ensure debugger hook for dynamic linker does not point to a PLT slot this change is made in preparation to support linking without -Bsymbolic-functions.
2015-04-18 03:23:05 +00:00
debug.bp = dl_debug_state;
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
debug.head = head;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
debug.base = ldso.base;
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
debug.state = 0;
_dl_debug_state();
allow specifying argv[0] when invoking a program via ldso command previously, the pathname used to load the program was always used as argv[0]. the default remains the same, but a new --argv0 option can be used to provide a different value.
2017-07-04 20:58:28 +00:00
if (replace_argv0) argv[0] = replace_argv0;
add support for init/fini array in main program, and greatly simplify modern (4.7.x and later) gcc uses init/fini arrays, rather than the legacy _init/_fini function pasting and crtbegin/crtend ctors/dtors system, on most or all archs. some archs had already switched a long time ago. without following this change, global ctors/dtors will cease to work under musl when building with new gcc versions. the most surprising part of this patch is that it actually reduces the size of the init code, for both static and shared libc. this is achieved by (1) unifying the handling main program and shared libraries in the dynamic linker, and (2) eliminating the glibc-inspired rube goldberg machine for passing around init and fini function pointers. to clarify, some background: the function signature for __libc_start_main was based on glibc, as part of the original goal of being able to run some glibc-linked binaries. it worked by having the crt1 code, which is linked into every application, static or dynamic, obtain and pass pointers to the init and fini functions, which __libc_start_main is then responsible for using and recording for later use, as necessary. however, in neither the static-linked nor dynamic-linked case do we actually need crt1.o's help. with dynamic linking, all the pointers are available in the _DYNAMIC block. with static linking, it's safe to simply access the _init/_fini and __init_array_start, etc. symbols directly. obviously changing the __libc_start_main function signature in an incompatible way would break both old musl-linked programs and glibc-linked programs, so let's not do that. instead, the function can just ignore the information it doesn't need. new archs need not even provide the useless args in their versions of crt1.o. existing archs should continue to provide it as long as there is an interest in having newly-linked applications be able to run on old versions of musl; at some point in the future, this support can be removed.
2013-07-21 07:00:54 +00:00
errno = 0;
dynamic linker bootstrap overhaul this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
2015-04-13 06:56:26 +00:00
CRTJMP((void *)aux[AT_ENTRY], argv-1);
for(;;);
fix ordering of shared library ctors with respect to libc init previously, shared library constructors were being called before important internal things like the environment (extern char **environ) and hwcap flags (needed for sjlj to work right with float on arm) were initialized in __libc_start_main. rather than trying to have to dynamic linker make sure this stuff all gets initialized right, I've opted to just defer calling shared library constructors until after the main program's entry point is reached. this also fixes the order of ctors to be the exact reverse of dtors, which is a desirable property and possibly even mandated by some languages. the main practical effect of this change is that shared libraries calling getenv from ctors will no longer fail.
2012-11-30 22:56:23 +00:00
}
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
static void prepare_lazy(struct dso *p)
{
size_t dyn[DYN_CNT], n, flags1=0;
decode_vec(p->dynv, dyn, DYN_CNT);
search_vec(p->dynv, &flags1, DT_FLAGS_1);
if (dyn[DT_BIND_NOW] || (dyn[DT_FLAGS] & DF_BIND_NOW) || (flags1 & DF_1_NOW))
return;
n = dyn[DT_RELSZ]/2 + dyn[DT_RELASZ]/3 + dyn[DT_PLTRELSZ]/2 + 1;
if (NEED_MIPS_GOT_RELOCS) {
size_t j=0; search_vec(p->dynv, &j, DT_MIPS_GOTSYM);
size_t i=0; search_vec(p->dynv, &i, DT_MIPS_SYMTABNO);
n += i-j;
}
p->lazy = calloc(n, 3*sizeof(size_t));
if (!p->lazy) {
error("Error preparing lazy relocation for %s: %m", p->name);
longjmp(*rtld_fail, 1);
}
p->lazy_next = lazy_head;
lazy_head = p;
}
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
void *dlopen(const char *file, int mode)
{
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
struct dso *volatile p, *orig_tail, *orig_syms_tail, *orig_lazy_head, *next;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
struct tls_module *orig_tls_tail;
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
size_t orig_tls_cnt, orig_tls_offset, orig_tls_align;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
size_t i;
protect against cancellation in dlopen i'm not sure that it's "correct" for dlopen to block cancellation when calling constructors for libraries it loads, but it sure seems like the right thing. in any case, dlopen itself needs cancellation blocked.
2012-02-08 01:31:27 +00:00
int cs;
move the dynamic linker's jmp_buf from static to automatic storage this more than compensates for the size increase of jmp_buf, and greatly reduces bss/data size on archs with huge jmp_buf.
2013-07-24 06:38:05 +00:00
jmp_buf jb;
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
struct dso **volatile ctor_queue = 0;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
if (!file) return head;
protect against cancellation in dlopen i'm not sure that it's "correct" for dlopen to block cancellation when calling constructors for libraries it loads, but it sure seems like the right thing. in any case, dlopen itself needs cancellation blocked.
2012-02-08 01:31:27 +00:00
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &cs);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
pthread_rwlock_wrlock(&lock);
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
__inhibit_ptc();
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
p = 0;
synchronize shared library dtor exec against concurrent loads/ctors previously, going way back, there was simply no synchronization here. a call to exit concurrent with ctor execution from dlopen could cause a dtor to execute concurrently with its corresponding ctor, or could cause dtors for newly-constructed libraries to be skipped. introduce a shutting_down state that blocks further ctor execution, producing the quiescence the dtor execution loop needs to ensure any kind of consistency, and that blocks further calls to dlopen so that a call into dlopen from a dtor cannot deadlock. better approaches to some of this may be possible, but the changes here at least make things safe.
2019-03-02 03:47:29 +00:00
if (shutting_down) {
error("Cannot dlopen while program is exiting.");
goto end;
}
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
orig_tls_tail = tls_tail;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
orig_tls_cnt = tls_cnt;
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
orig_tls_offset = tls_offset;
orig_tls_align = tls_align;
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
orig_lazy_head = lazy_head;
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
orig_syms_tail = syms_tail;
fix race condition in dlopen orig_tail was being saved before the lock was obtained, allowing dlopen failure to roll-back other dlopens that had succeeded.
2012-10-05 05:15:25 +00:00
orig_tail = tail;
add support for RTLD_NOLOAD to dlopen based on patch by Pierre Carrier <pierre@gcarrier.fr> that just added the flag constant, but with minimal additional code so that it actually works as documented. this is a nonstandard option but some major software (reportedly, Firefox) uses it and it was easy to add anyway.
2013-01-24 03:07:45 +00:00
noload = mode & RTLD_NOLOAD;
fix race condition in dlopen orig_tail was being saved before the lock was obtained, allowing dlopen failure to roll-back other dlopens that had succeeded.
2012-10-05 05:15:25 +00:00
move the dynamic linker's jmp_buf from static to automatic storage this more than compensates for the size increase of jmp_buf, and greatly reduces bss/data size on archs with huge jmp_buf.
2013-07-24 06:38:05 +00:00
rtld_fail = &jb;
if (setjmp(*rtld_fail)) {
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
/* Clean up anything new that was (partially) loaded */
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
revert_syms(orig_syms_tail);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
for (p=orig_tail->next; p; p=next) {
next = p->next;
add arch-generic support for tlsdesc relocations to dynamic linker this code is non-functional without further changes to link up the arch-specific reloc types for tlsdesc and add asm implementations of __tlsdesc_static and __tlsdesc_dynamic.
2014-06-19 06:01:06 +00:00
while (p->td_index) {
void *tmp = p->td_index->next;
free(p->td_index);
p->td_index = tmp;
}
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
free(p->funcdescs);
fix rpath string memory leak on failed dlopen when dlopen fails, all partially-loaded libraries need to be unmapped and freed. any of these libraries using an rpath with $ORIGIN expansion may have an allocated string for the expanded rpath; previously, this string was not freed when freeing the library data structures.
2015-04-04 04:15:19 +00:00
if (p->rpath != p->rpath_orig)
free(p->rpath);
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
free(p->deps);
add real fdpic loading of shared libraries previously, the normal ELF library loading code was used even for fdpic, so only the kernel-loaded dynamic linker and main app could benefit from separate placement of segments and shared text.
2015-09-22 19:12:48 +00:00
unmap_library(p);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
free(p);
}
fix invalid-/double-/use-after-free in new dlopen ctor execution this affected the error path where dlopen successfully found and loaded the requested dso and all its dependencies, but failed to resolve one or more relocations, causing the operation to fail after storage for the ctor queue was allocated. commit 188759bbee057aa94db2bbb7cf7f5855f3b9ab53 wrongly put the free for the ctor_queue array in the error path inside a loop over each loaded dso that needed to be backed-out, rather than just doing it once. in addition, the exit path also observed the ctor_queue pointer still being nonzero, and would attempt to call ctors on the backed-out dsos unless the double-free crashed the process first.
2019-03-10 17:16:59 +00:00
free(ctor_queue);
ctor_queue = 0;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
if (!orig_tls_tail) libc.tls_head = 0;
tls_tail = orig_tls_tail;
fix crash from corrupted tls module list after failed dlopen commit d56460c939c94a6c547abe8238f442b8de10bfbd introduced this regression as part of splitting the tls module list out of the dso list. the new code added to dlopen's failure path to undo the changes adding the partially-loaded libraries reset the tls_tail pointer correctly, but did not clear its link to the next list entry. thus, at least until the next successful dlopen, the list was not terminated but ended with an invalid next pointer, which __copy_tls attempted to follow when a new thread was created. patch by Mikael Vidstedt.
2017-01-05 03:54:06 +00:00
if (tls_tail) tls_tail->next = 0;
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
tls_cnt = orig_tls_cnt;
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
tls_offset = orig_tls_offset;
tls_align = orig_tls_align;
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
lazy_head = orig_lazy_head;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
tail = orig_tail;
tail->next = 0;
protect against cancellation in dlopen i'm not sure that it's "correct" for dlopen to block cancellation when calling constructors for libraries it loads, but it sure seems like the right thing. in any case, dlopen itself needs cancellation blocked.
2012-02-08 01:31:27 +00:00
p = 0;
make dlerror produce informative results note that dlerror is specified to be non-thread-safe, so no locking is performed on the error flag or message aside from the rwlock already held by dlopen or dlsym. if 2 invocations of dlsym are generating errors at the same time, they could clobber each other's results, but the resulting string, albeit corrupt, will still be null-terminated. any use of dlerror in such a situation could not be expected to give meaningful results anyway.
2012-04-23 16:03:31 +00:00
goto end;
make dlopen honor the rpath of the main program this seems to match what other systems do, and seems useful for programs that have their libraries and plugins stored relative to the executable.
2013-08-24 03:13:25 +00:00
} else p = load_library(file, head);
make dlerror conform to posix the error status is required to be sticky after failure of dlopen or dlsym until cleared by dlerror. applications and especially libraries should never rely on this since it is not thread-safe and subject to race conditions, but glib does anyway.
2012-03-23 04:28:20 +00:00
if (!p) {
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
error(noload ?
add support for RTLD_NOLOAD to dlopen based on patch by Pierre Carrier <pierre@gcarrier.fr> that just added the flag constant, but with minimal additional code so that it actually works as documented. this is a nonstandard option but some major software (reportedly, Firefox) uses it and it was easy to add anyway.
2013-01-24 03:07:45 +00:00
"Library %s is not already loaded" :
"Error loading shared library %s: %m",
file);
protect against cancellation in dlopen i'm not sure that it's "correct" for dlopen to block cancellation when calling constructors for libraries it loads, but it sure seems like the right thing. in any case, dlopen itself needs cancellation blocked.
2012-02-08 01:31:27 +00:00
goto end;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
}
/* First load handling */
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
load_deps(p);
extend_bfs_deps(p);
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
pthread_mutex_lock(&init_fini_lock);
if (!p->constructed) ctor_queue = queue_ctors(p);
pthread_mutex_unlock(&init_fini_lock);
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (!p->relocated && (mode & RTLD_LAZY)) {
prepare_lazy(p);
for (i=0; p->deps[i]; i++)
if (!p->deps[i]->relocated)
prepare_lazy(p->deps[i]);
fix regression in dlopen promotion from RTLD_LOCAL to RTLD_GLOBAL commit 4ff234f6cba96403b5de6d29d48a59fd73252040 inadvertently removed the logic to do this when changing the representation of global status.
2017-07-04 15:34:39 +00:00
}
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (!p->relocated || (mode & RTLD_GLOBAL)) {
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
/* Make new symbols global, at least temporarily, so we can do
* relocations. If not RTLD_GLOBAL, this is reverted below. */
add_syms(p);
ldso: avoid spurious & possible erroneous work for libs with no deps a null pointer for a library's deps list was ambiguous: it could indicate either no dependencies or that the dependency list had not yet been populated. inability to distinguish could lead to spurious work when dlopen is called multiple times on a library with no deps, and due to related bugs, could actually cause other libraries to falsely appear as dependencies, translating into false positives for dlsym. avoid the problem by always initializing the deps pointer, pointing to an empty list if there are no deps. rather than wasting memory and introducing another failure path by allocating an empty list per library, simply share a global dummy list. further fixes will be needed for related bugs, and much of this code may end up being replaced.
2017-07-04 14:58:13 +00:00
for (i=0; p->deps[i]; i++)
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
add_syms(p->deps[i]);
fix regression in dlopen promotion from RTLD_LOCAL to RTLD_GLOBAL commit 4ff234f6cba96403b5de6d29d48a59fd73252040 inadvertently removed the logic to do this when changing the representation of global status.
2017-07-04 15:34:39 +00:00
}
fix and overhaul dlsym depedency order, always record direct deps dlsym with an explicit handle is specified to use "dependency order", a breadth-first search rooted at the argument. this has always been implemented by iterating a flattened dependency list built at dlopen time. however, the logic for building this list was completely wrong except in trivial cases; it simply used the list of libraries loaded since a given library, and their direct dependencies, as that library's dependencies, which could result in misordering, wrongful omission of deep dependencies from the search, and wrongful inclusion of unrelated libraries in the search. further, libraries did not have any recorded list of resolved dependencies until they were explicitly dlopened, meaning that DT_NEEDED entries had to be resolved again whenever a library participated as a dependency of more than one dlopened library. with this overhaul, the resolved direct dependency list of each library is always recorded when it is first loaded, and can be extended to a full flattened breadth-first search list if dlopen is called on the library. the extension is performed using the direct dependency list as a queue and appending copies of the direct dependency list of each dependency in the queue, excluding duplicates, until the end of the queue is reached. the direct deps remain available for future use as the initial subarray of the full deps array. first-load logic in dlopen is updated to match these changes, and clarified.
2019-02-26 23:05:19 +00:00
if (!p->relocated) {
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
reloc_all(p);
}
rework ldso handling of global symbol table for consistency when loading libraries with dlopen, the caller can request that the library's symbols become part of the global symbol table, or that they only be used for resolving relocations in the loaded library and its dependencies. in the latter case, a subsequent dlopen of the same library can upgrade it to global status. previously, if a library was upgraded from local to global mode, its symbols entered the symbol lookup search order at the point where the library was originally loaded. this means that a new call to dlopen could change the value of a symbol that already had a visible definition, an inconsistency which applications could observe. POSIX is unclear whether this should happen or whether it's permitted to happen, but the resolution of Austin Group issue #982 made it formally unspecified. with this patch, a library whose mode is upgraded from local to global enters the symbol lookup order at the point where it was made global, so that symbol resolution before and after the upgrade are consistent. in order to implement this change, the per-dso global flag is replaced with a separate set of linked-list pointers for participation in the global symbol table. this permits the order of dso objects for symbol resolution to differ from the order used for iteration of all loaded libraries. it also improves performance of find_sym, by avoiding a branch per iteration and skipping, and especially in the case where many non-global libraries have been loaded, by allowing the loop to skip over them entirely. logic for temporarily adding non-global libraries to the symbol table for relocation purposes is also mildly simplified.
2017-03-13 01:03:05 +00:00
/* If RTLD_GLOBAL was not specified, undo any new additions
* to the global symbol table. This is a nop if the library was
* previously loaded and already global. */
if (!(mode & RTLD_GLOBAL))
revert_syms(orig_syms_tail);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
emulate lazy relocation as deferrable relocation traditional lazy relocation with call-time plt resolver is intentionally not implemented, as it is a huge bug surface and demands significant amounts of arch-specific code and requires ongoing maintenance to ensure compatibility with applications which make use of new additions to the arch's register file in passing function arguments. some applications, however, depend on the ability to dlopen modules which have unsatisfied symbol references at the time they are loaded, either avoiding use of the affected interfaces or manually loading another module to provide the missing definition via their own module dependency tracking outside the ELF data structures. while such usage is non-conforming, failure to support it has been a significant obstacle for users/distributions trying to support affected software, particularly the X.org server. instead of resolving lazy relocations at call time, this patch saves unresolved GOT/PLT relocations for deferral and retries them after each subsequent dlopen until they are resolved. since dlopen is the only time at which the effective global symbol table can change, this behavior is not observably different from traditional lazy binding, and the required code is minimal.
2017-03-13 12:52:41 +00:00
/* Processing of deferred lazy relocations must not happen until
* the new libraries are committed; otherwise we could end up with
* relocations resolved to symbol definitions that get removed. */
redo_lazy_relocs();
TLS fixes, mainly alignment handling compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
2012-10-06 05:22:51 +00:00
update_tls_size();
install dynamic tls synchronously at dlopen, streamline access previously, dynamic loading of new libraries with thread-local storage allocated the storage needed for all existing threads at load-time, precluding late failure that can't be handled, but left installation in existing threads to take place lazily on first access. this imposed an additional memory access and branch on every dynamic tls access, and imposed a requirement, which was not actually met, that the dynamic tlsdesc asm functions preserve all call-clobbered registers before calling C code to to install new dynamic tls on first access. the x86[_64] versions of this code wrongly omitted saving and restoring of fpu/vector registers, assuming the compiler would not generate anything using them in the called C code. the arm and aarch64 versions saved known existing registers, but failed to be future-proof against expansion of the register file. now that we track live threads in a list, it's possible to install the new dynamic tls for each thread at dlopen time. for the most part, synchronization is not needed, because if a thread has not synchronized with completion of the dlopen, there is no way it can meaningfully request access to a slot past the end of the old dtv, which remains valid for accessing slots which already existed. however, it is necessary to ensure that, if a thread sees its new dtv pointer, it sees correct pointers in each of the slots that existed prior to the dlopen. my understanding is that, on most real-world coherency architectures including all the ones we presently support, a built-in consume order guarantees this; however, don't rely on that. instead, the SYS_membarrier syscall is used to ensure that all threads see the stores to the slots of their new dtv prior to the installation of the new dtv. if it is not supported, the same is implemented in userspace via signals, using the same mechanism as __synccall. the __tls_get_addr function, variants, and dynamic tlsdesc asm functions are all updated to remove the fallback paths for claiming new dynamic tls, and are now all branch-free.
2019-02-18 04:22:27 +00:00
if (tls_cnt != orig_tls_cnt)
install_new_tls();
gdb shared library debugging support provide the minimal level of dynamic linker-to-debugger glue needed to let gdb find loaded libraries and load their symbols.
2012-04-25 04:05:42 +00:00
_dl_debug_state();
fix/improve shared library ctor/dtor handling, allow recursive dlopen some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
2012-10-05 17:09:09 +00:00
orig_tail = tail;
don't leave the lock held on dlopen failure..
2011-06-27 02:09:32 +00:00
end:
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 15:51:50 +00:00
__release_ptc();
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
if (p) gencnt++;
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
pthread_rwlock_unlock(&lock);
overhaul shared library ctor execution for dependency order, concurrency previously, shared library constructors at program start and dlopen time were executed in reverse load order. some libraries, however, rely on a depth-first dependency order, which most other dynamic linker implementations provide. this is a much more reasonable, less arbitrary order, and it turns out to have much better properties with regard to how slow-running ctors affect multi-threaded programs, and how recursive dlopen behaves. this commit builds on previous work tracking direct dependencies of each dso (commit 403555690775f7c8806372644f543518e6664e3b), and performs a topological sort on the dependency graph at load time while the main ldso lock is held and before success is committed, producing a queue of constructors needed by the newly-loaded dso (or main application). in the case of circular dependencies, the dependency chain is simply broken at points where it becomes circular. when the ctor queue is run, the init_fini_lock is held only for iteration purposes; it's released during execution of each ctor, so that arbitrarily-long-running application code no longer runs with a lock held in the caller. this prevents a dlopen with slow ctors in one thread from arbitrarily delaying other threads that call dlopen. fully-independent ctors can run concurrently; when multiple threads call dlopen with a shared dependency, one will end up executing the ctor while the other waits on a condvar for it to finish. another corner case improved by these changes is recursive dlopen (call from a ctor). previously, recursive calls to dlopen could cause a ctor for a library to be executed before the ctor for its dependency, even when there was no relation between the calling library and the library it was loading, simply due to the naive reverse-load-order traversal. now, we can guarantee that recursive dlopen in non-circular-dependency usage preserves the desired ctor execution order properties, and that even in circular usage, at worst the libraries whose ctors call dlopen will fail to have completed construction when ctors that depend on them run. init_fini_lock is changed to a normal, non-recursive mutex, since it is no longer held while calling back into application code.
2019-03-02 02:06:23 +00:00
if (ctor_queue) {
do_init_fini(ctor_queue);
free(ctor_queue);
}
protect against cancellation in dlopen i'm not sure that it's "correct" for dlopen to block cancellation when calling constructors for libraries it loads, but it sure seems like the right thing. in any case, dlopen itself needs cancellation blocked.
2012-02-08 01:31:27 +00:00
pthread_setcancelstate(cs, 0);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
return p;
}
define and use internal macros for hidden visibility, weak refs this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-09-05 16:43:34 +00:00
hidden int __dl_invalid_handle(void *h)
check for invalid handles in dlsym/dlclose this is wasteful and useless from a standpoint of sane programs, but it is required by the standard, and the current requirements were upheld with the closure of Austin Group issue #639: http://austingroupbugs.net/view.php?id=639
2013-01-10 19:05:40 +00:00
{
struct dso *p;
for (p=head; p; p=p->next) if (h==p) return 0;
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
error("Invalid library handle %p", (void *)h);
check for invalid handles in dlsym/dlclose this is wasteful and useless from a standpoint of sane programs, but it is required by the standard, and the current requirements were upheld with the closure of Austin Group issue #639: http://austingroupbugs.net/view.php?id=639
2013-01-10 19:05:40 +00:00
return 1;
}
fix dlsym RTLD_NEXT behavior for fdpic lookup the dso an address falls in based on the loadmap and not just a base/length. fix the main app's fake loadmap used when loaded by a non-fdpic-aware loader so that it does not cover the whole memory space. function descriptor addresses are also matched for future use by dladdr, but reverse lookups of function descriptors via dladdr have not been implemented yet. some revisions may be needed in the future once reclaim_gaps supports fdpic, so that function descriptors allocated in reclaimed heap space do not get detected as belonging to the module whose gaps they were allocated in.
2015-09-22 23:41:41 +00:00
static void *addr2dso(size_t a)
{
struct dso *p;
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
size_t i;
if (DL_FDPIC) for (p=head; p; p=p->next) {
i = count_syms(p);
if (a-(size_t)p->funcdescs < i*sizeof(*p->funcdescs))
return p;
}
fix dlsym RTLD_NEXT behavior for fdpic lookup the dso an address falls in based on the loadmap and not just a base/length. fix the main app's fake loadmap used when loaded by a non-fdpic-aware loader so that it does not cover the whole memory space. function descriptor addresses are also matched for future use by dladdr, but reverse lookups of function descriptors via dladdr have not been implemented yet. some revisions may be needed in the future once reclaim_gaps supports fdpic, so that function descriptors allocated in reclaimed heap space do not get detected as belonging to the module whose gaps they were allocated in.
2015-09-22 23:41:41 +00:00
for (p=head; p; p=p->next) {
if (DL_FDPIC && p->loadmap) {
for (i=0; i<p->loadmap->nsegs; i++) {
if (a-p->loadmap->segs[i].p_vaddr
< p->loadmap->segs[i].p_memsz)
return p;
}
} else {
avoid spurious dso matches by dladdr outside bounds of load segments since slack space at the beginning and/or end of writable load maps is donated to malloc, the application could obtain valid pointers in these ranges which dladdr would erroneously identify as part of the shared object whose mapping they came from. instead of checking the queried address against the mapping base and length, check it against the load segments from the program headers, and only match the dso if it lies within the bounds of one of them. as a shortcut, if the address does match the range of the mapping but not any of the load segments, we know it cannot match any other dso and can immediately return failure.
2018-06-28 16:20:58 +00:00
Phdr *ph = p->phdr;
size_t phcnt = p->phnum;
size_t entsz = p->phentsize;
size_t base = (size_t)p->base;
for (; phcnt--; ph=(void *)((char *)ph+entsz)) {
if (ph->p_type != PT_LOAD) continue;
if (a-base-ph->p_vaddr < ph->p_memsz)
return p;
}
fix dlsym RTLD_NEXT behavior for fdpic lookup the dso an address falls in based on the loadmap and not just a base/length. fix the main app's fake loadmap used when loaded by a non-fdpic-aware loader so that it does not cover the whole memory space. function descriptor addresses are also matched for future use by dladdr, but reverse lookups of function descriptors via dladdr have not been implemented yet. some revisions may be needed in the future once reclaim_gaps supports fdpic, so that function descriptors allocated in reclaimed heap space do not get detected as belonging to the module whose gaps they were allocated in.
2015-09-22 23:41:41 +00:00
if (a-(size_t)p->map < p->map_len)
avoid spurious dso matches by dladdr outside bounds of load segments since slack space at the beginning and/or end of writable load maps is donated to malloc, the application could obtain valid pointers in these ranges which dladdr would erroneously identify as part of the shared object whose mapping they came from. instead of checking the queried address against the mapping base and length, check it against the load segments from the program headers, and only match the dso if it lies within the bounds of one of them. as a shortcut, if the address does match the range of the mapping but not any of the load segments, we know it cannot match any other dso and can immediately return failure.
2018-06-28 16:20:58 +00:00
return 0;
fix dlsym RTLD_NEXT behavior for fdpic lookup the dso an address falls in based on the loadmap and not just a base/length. fix the main app's fake loadmap used when loaded by a non-fdpic-aware loader so that it does not cover the whole memory space. function descriptor addresses are also matched for future use by dladdr, but reverse lookups of function descriptors via dladdr have not been implemented yet. some revisions may be needed in the future once reclaim_gaps supports fdpic, so that function descriptors allocated in reclaimed heap space do not get detected as belonging to the module whose gaps they were allocated in.
2015-09-22 23:41:41 +00:00
}
}
return 0;
}
RTLD_NEXT support the asm wrapper is needed to get the return address without compiler-specific extensions.
2011-08-16 04:42:13 +00:00
static void *do_dlsym(struct dso *p, const char *s, void *ra)
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
{
make relocation time symbol lookup and dlsym consistent Using common code path for all symbol lookups fixes three dlsym issues: - st_shndx of STT_TLS symbols were not checked and thus an undefined tls symbol reference could be incorrectly treated as a definition (the sysv hash lookup returns undefined symbols, gnu does not, so should be rare in practice). - symbol binding was not checked so a hidden symbol may be returned (in principle STB_LOCAL symbols may appear in the dynamic symbol table for hidden symbols, but linkers most likely don't produce it). - mips specific behaviour was not applied (ARCH_SYM_REJECT_UND) so undefined symbols may be returned on mips. always_inline is used to avoid relocation performance regression, the code generation for find_sym should not be affected.
2019-08-10 23:14:40 +00:00
int use_deps = 0;
if (p == head || p == RTLD_DEFAULT) {
p = head;
} else if (p == RTLD_NEXT) {
p = addr2dso((size_t)ra);
if (!p) p=head;
p = p->next;
} else if (__dl_invalid_handle(p)) {
fix regression in dlsym: rejection of special RTLD_* handles as invalid
2013-01-24 01:21:36 +00:00
return 0;
make relocation time symbol lookup and dlsym consistent Using common code path for all symbol lookups fixes three dlsym issues: - st_shndx of STT_TLS symbols were not checked and thus an undefined tls symbol reference could be incorrectly treated as a definition (the sysv hash lookup returns undefined symbols, gnu does not, so should be rare in practice). - symbol binding was not checked so a hidden symbol may be returned (in principle STB_LOCAL symbols may appear in the dynamic symbol table for hidden symbols, but linkers most likely don't produce it). - mips specific behaviour was not applied (ARCH_SYM_REJECT_UND) so undefined symbols may be returned on mips. always_inline is used to avoid relocation performance regression, the code generation for find_sym should not be affected.
2019-08-10 23:14:40 +00:00
} else
use_deps = 1;
struct symdef def = find_sym2(p, s, 0, use_deps);
if (!def.sym) {
error("Symbol not found: %s", s);
return 0;
}
if ((def.sym->st_info&0xf) == STT_TLS)
return __tls_get_addr((tls_mod_off_t []){def.dso->tls_id, def.sym->st_value-DTP_OFFSET});
if (DL_FDPIC && (def.sym->st_info&0xf) == STT_FUNC)
return def.dso->funcdescs + (def.sym - def.dso->syms);
return laddr(def.dso, def.sym->st_value);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
}
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
int dladdr(const void *addr_arg, Dl_info *info)
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
{
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
size_t addr = (size_t)addr_arg;
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
struct dso *p;
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
Sym *sym, *bestsym;
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
uint32_t nsym;
char *strings;
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
size_t best = 0;
fix symtab-order-dependent spurious matches in dladdr commit 8b8fb7f03721c42445f982582f462144ab60a1a0 added logic to prevent matching a symbol with no recorded size (closest-match) when there is an intervening symbol whose size was recorded, but it only worked when the intervening symbol was encountered later in the search. instead of rejecting symbols where addr falls outside their recorded size during the closest-match search, accept them to find the true closest-match, then reject such a result only once the search has finished.
2018-06-28 15:51:43 +00:00
size_t besterr = -1;
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
pthread_rwlock_rdlock(&lock);
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
p = addr2dso(addr);
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
pthread_rwlock_unlock(&lock);
if (!p) return 0;
sym = p->syms;
strings = p->strings;
factor symbol counting out of dladdr as its own function the fdpic code will need to count symbols, and it may be useful elsewhere in the future too. counting is trivial as long as sysv hash is present, but for gnu-hash-only libraries it's complex. the behavior of the count is changed slightly: we now include symbols that are not accessible by the gnu hash table in the count. this may make dladdr slightly slower. if this is a problem, dladdr can subtract out the part that should not be accessible. unlike in the old code, subtracting this out is easy even in the fast path where sysv hash is available too.
2015-09-21 21:47:50 +00:00
nsym = count_syms(p);
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
if (DL_FDPIC) {
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
size_t idx = (addr-(size_t)p->funcdescs)
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
/ sizeof(*p->funcdescs);
if (idx < nsym && (sym[idx].st_info&0xf) == STT_FUNC) {
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
best = (size_t)(p->funcdescs + idx);
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
bestsym = sym + idx;
fix symtab-order-dependent spurious matches in dladdr commit 8b8fb7f03721c42445f982582f462144ab60a1a0 added logic to prevent matching a symbol with no recorded size (closest-match) when there is an intervening symbol whose size was recorded, but it only worked when the intervening symbol was encountered later in the search. instead of rejecting symbols where addr falls outside their recorded size during the closest-match search, accept them to find the true closest-match, then reject such a result only once the search has finished.
2018-06-28 15:51:43 +00:00
besterr = 0;
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
}
}
if (!best) for (; nsym; nsym--, sym++) {
fix bug in dladdr that prevented resolving addresses in the PLT
2013-01-16 16:47:35 +00:00
if (sym->st_value
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
&& (1<<(sym->st_info&0xf) & OK_TYPES)
&& (1<<(sym->st_info>>4) & OK_BINDS)) {
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
size_t symaddr = (size_t)laddr(p, sym->st_value);
make dladdr consistently produce the first symbol in presence of aliases the early-exit condition for the symbol match loop on exact matches caused dladdr to produce the first match for an exact match, but the last match for an inexact match. in the interest of consistency, require a strictly-closer match to replace an already-found one.
2018-06-28 16:07:51 +00:00
if (symaddr > addr || symaddr <= best)
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
continue;
best = symaddr;
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
bestsym = sym;
fix symtab-order-dependent spurious matches in dladdr commit 8b8fb7f03721c42445f982582f462144ab60a1a0 added logic to prevent matching a symbol with no recorded size (closest-match) when there is an intervening symbol whose size was recorded, but it only worked when the intervening symbol was encountered later in the search. instead of rejecting symbols where addr falls outside their recorded size during the closest-match search, accept them to find the true closest-match, then reject such a result only once the search has finished.
2018-06-28 15:51:43 +00:00
besterr = addr - symaddr;
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
if (addr == symaddr)
break;
}
}
fix inadvertent use of uninitialized variable in dladdr commit c8b49b2fbc7faa8bf065220f11963d76c8a2eb93 introduced code that checked bestsym to determine whether a matching symbol was found, but bestsym is uninitialized if not. instead use best, consistent with use in the rest of the function. simplified from bug report and patch by Cheng Liu.
2019-07-06 21:47:43 +00:00
if (best && besterr > bestsym->st_size-1) {
fix symtab-order-dependent spurious matches in dladdr commit 8b8fb7f03721c42445f982582f462144ab60a1a0 added logic to prevent matching a symbol with no recorded size (closest-match) when there is an intervening symbol whose size was recorded, but it only worked when the intervening symbol was encountered later in the search. instead of rejecting symbols where addr falls outside their recorded size during the closest-match search, accept them to find the true closest-match, then reject such a result only once the search has finished.
2018-06-28 15:51:43 +00:00
best = 0;
bestsym = 0;
}
correctly handle non-matching symbols in dladdr based on patch by Axel Siebenborn, with fixes discussed on the mailing list after submission and and rebased around the UB fix in commit e829695fcc880f8578c2b964ea2d090f0016c9d7. avoid spurious symbol matches by dladdr beyond symbol size. for symbols with a size recorded, only match if the queried address lies within the address range determined by the symbol address and size. for symbols with no size recorded, the old closest-match behavior is kept, as long as there is no intervening symbol with a recorded size. the case where no symbol is matched, but the address does lie within the memory range of a shared object, is specified as success. fix the return value and produce a valid (with null dli_sname and dli_saddr) Dl_info structure.
2018-06-27 19:32:09 +00:00
info->dli_fname = p->name;
info->dli_fbase = p->map;
if (!best) {
info->dli_sname = 0;
info->dli_saddr = 0;
return 1;
}
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
if (DL_FDPIC && (bestsym->st_info&0xf) == STT_FUNC)
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
best = (size_t)(p->funcdescs + (bestsym - p->syms));
fix dladdr treatment of function descriptors for fdpic when determining which module an address belongs to, all function descriptor ranges must be checked first, in case the allocated memory falls inside another module's memory range. dladdr itself must also check addresses against function descriptors before doing a best-match search against the symbol table. even when doing the latter (e.g. for code addresses obtained from mcontext_t), also check whether the best-match was a function, and if so, replace the result with a function descriptor address. which is the nominal "base address" of the function and which the caller needs if it intends to subsequently call the matching function.
2015-10-16 02:51:56 +00:00
info->dli_sname = strings + bestsym->st_name;
avoid using undefined pointer arithmetic in dladdr
2018-06-27 19:29:12 +00:00
info->dli_saddr = (void *)best;
dladdr support for dynamic linker (nonstandard extension) based on patches submitted by boris brezillon. this commit also fixes the issue whereby the main application and libc don't have the address ranges of their mappings stored, which was theoretically a problem for RTLD_NEXT support in dlsym; it didn't actually matter because libc never calls dlsym, and it seemed to be doing the right thing (by chance) for symbols in the main program as well.
2012-08-27 01:09:26 +00:00
return 1;
}
define and use internal macros for hidden visibility, weak refs this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
2018-09-05 16:43:34 +00:00
hidden void *__dlsym(void *restrict p, const char *restrict s, void *restrict ra)
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
{
void *res;
pthread_rwlock_rdlock(&lock);
RTLD_NEXT support the asm wrapper is needed to get the return address without compiler-specific extensions.
2011-08-16 04:42:13 +00:00
res = do_dlsym(p, s, ra);
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
pthread_rwlock_unlock(&lock);
return res;
}
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
add time64 redirect for, and redirecting implementation of, dlsym if symbols are being redirected to provide the new time64 ABI, dlsym must perform matching redirections; otherwise, it would poke a hole in the magic and return pointers to functions that are not safe to call from a caller using time64 types. rather than duplicating a table of redirections, use the time64 symbols present in libc's symbol table to derive the decision for whether a particular symbol needs to be redirected.
2019-08-09 19:26:23 +00:00
hidden void *__dlsym_redir_time64(void *restrict p, const char *restrict s, void *restrict ra)
{
#if _REDIR_TIME64
const char *suffix, *suffix2 = "";
char redir[36];
/* Map the symbol name to a time64 version of itself according to the
* pattern used for naming the redirected time64 symbols. */
size_t l = strnlen(s, sizeof redir);
if (l<4 || l==sizeof redir) goto no_redir;
if (s[l-2]=='_' && s[l-1]=='r') {
l -= 2;
suffix2 = s+l;
}
if (l<4) goto no_redir;
if (!strcmp(s+l-4, "time")) suffix = "64";
else suffix = "_time64";
/* Use the presence of the remapped symbol name in libc to determine
* whether it's one that requires time64 redirection; replace if so. */
snprintf(redir, sizeof redir, "__%.*s%s%s", (int)l, s, suffix, suffix2);
if (find_sym(&ldso, redir, 1).sym) s = redir;
no_redir:
#endif
return __dlsym(p, s, ra);
}
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
int dl_iterate_phdr(int(*callback)(struct dl_phdr_info *info, size_t size, void *data), void *data)
{
struct dso *current;
struct dl_phdr_info info;
int ret = 0;
for(current = head; current;) {
info.dlpi_addr = (uintptr_t)current->base;
info.dlpi_name = current->name;
info.dlpi_phdr = current->phdr;
info.dlpi_phnum = current->phnum;
info.dlpi_adds = gencnt;
info.dlpi_subs = 0;
info.dlpi_tls_modid = current->tls_id;
unify static and dynamic linked implementations of thread-local storage this both allows removal of some of the main remaining uses of the SHARED macro and clears one obstacle to static-linked dlopen support, which may be added at some point in the future. specialized single-TLS-module versions of __copy_tls and __reset_tls are removed and replaced with code adapted from their dynamic-linked versions, capable of operating on a whole chain of TLS modules, and use of the dynamic linker's DSO chain (which contains large struct dso objects) by these functions is replaced with a new chain of struct tls_module objects containing only the information needed for implementing TLS. this may also yield some performance benefit initializing TLS for a new thread when a large number of modules without TLS have been loaded, since since there is no need to walk structures for modules without TLS.
2015-11-12 20:50:26 +00:00
info.dlpi_tls_data = current->tls.image;
add dl_iterate_phdr interface patches by Alex Caudill (npx). the dynamic-linked version is almost identical to the final submitted patch; I just added a couple missing lines for saving the phdr address when the dynamic linker is invoked directly to run a program, and removed a couple to avoid introducing another unnecessary type. the static-linked version is based on npx's draft. it could use some improvements which are contingent on the startup code saving some additional information for later use.
2012-11-01 01:27:48 +00:00
ret = (callback)(&info, sizeof (info), data);
if (ret != 0) break;
pthread_rwlock_rdlock(&lock);
current = current->next;
pthread_rwlock_unlock(&lock);
}
return ret;
}
experimental dlopen/dlsym and dynlink changes needed to support them
2011-06-26 23:23:28 +00:00
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
static void error(const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
if (!runtime) {
vdprintf(2, fmt, ap);
dprintf(2, "\n");
ldso_fail = 1;
va_end(ap);
return;
}
factor dlerror and error-setting code out of dynlink.c the ultimate goal of this change is to get all code used in libc.a out of dynlink.c, so that the dynamic linker code can be moved to its own tree and object files in the src tree can all be shared between libc.a and libc.so.
2016-01-25 22:56:00 +00:00
__dl_vseterr(fmt, ap);
make dlerror state and message thread-local and dynamically-allocated this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
2015-04-18 22:00:22 +00:00
va_end(ap);
}