RepoMirrors
/
musl
mirror of git://git.musl-libc.org/musl
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
musl/arch/i386/atomic_arch.h

110 lines
2.4 KiB
C
Raw Normal View History

refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_ctz_64 a_ctz_64
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline int a_ctz_64(uint64_t x)
{
int r;
__asm__( "bsf %1,%0 ; jnz 1f ; bsf %2,%0 ; addl $32,%0\n1:"
fix missing earlyclobber flag in i386 a_ctz_64 asm this error was only found by reading the code, but it seems to have been causing gcc to produce wrong code in malloc: the same register was used for the output and the high word of the input. in principle this could have caused an infinite loop searching for an available bin, but in practice most x86 models seem to implement the "undefined" result of the bsf instruction as "unchanged".
2015-09-09 07:18:28 +00:00
: "=&r"(r) : "r"((unsigned)x), "r"((unsigned)(x>>32)) );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
return r;
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_ctz_l a_ctz_l
add word-sized ctz function to atomic.h strictly speaking this and a few other ops should be factored into asm.h or the file should just be renamed to asm.h, but whatever. clean it up someday.
2011-04-27 16:19:49 +00:00
static inline int a_ctz_l(unsigned long x)
{
long r;
__asm__( "bsf %1,%0" : "=r"(r) : "r"(x) );
return r;
}
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_and_64 a_and_64
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_and_64(volatile uint64_t *p, uint64_t v)
{
__asm__( "lock ; andl %1, (%0) ; lock ; andl %2, 4(%0)"
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
: : "r"((long *)p), "r"((unsigned)v), "r"((unsigned)(v>>32)) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_or_64 a_or_64
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_or_64(volatile uint64_t *p, uint64_t v)
{
__asm__( "lock ; orl %1, (%0) ; lock ; orl %2, 4(%0)"
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
: : "r"((long *)p), "r"((unsigned)v), "r"((unsigned)(v>>32)) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_or_l a_or_l
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_or_l(volatile void *p, long v)
{
__asm__( "lock ; orl %1, %0"
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
: "=m"(*(long *)p) : "r"(v) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_cas a_cas
optimize contended normal mutex case; add int compare-and-swap atomic
2011-03-17 16:14:40 +00:00
static inline int a_cas(volatile int *p, int t, int s)
{
__asm__( "lock ; cmpxchg %3, %1"
: "=a"(t), "=m"(*p) : "a"(t), "r"(s) : "memory" );
return t;
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_or a_or
fix inconsistency in a_and and a_or argument types on x86[_64] conceptually, and on other archs, these functions take a pointer to int, but in the i386, x86_64, and x32 versions of atomic.h, they took a pointer to void instead.
2015-05-20 04:17:35 +00:00
static inline void a_or(volatile int *p, int v)
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
{
__asm__( "lock ; orl %1, %0"
fix inconsistency in a_and and a_or argument types on x86[_64] conceptually, and on other archs, these functions take a pointer to int, but in the i386, x86_64, and x32 versions of atomic.h, they took a pointer to void instead.
2015-05-20 04:17:35 +00:00
: "=m"(*p) : "r"(v) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_and a_and
fix inconsistency in a_and and a_or argument types on x86[_64] conceptually, and on other archs, these functions take a pointer to int, but in the i386, x86_64, and x32 versions of atomic.h, they took a pointer to void instead.
2015-05-20 04:17:35 +00:00
static inline void a_and(volatile int *p, int v)
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
{
__asm__( "lock ; andl %1, %0"
fix inconsistency in a_and and a_or argument types on x86[_64] conceptually, and on other archs, these functions take a pointer to int, but in the i386, x86_64, and x32 versions of atomic.h, they took a pointer to void instead.
2015-05-20 04:17:35 +00:00
: "=m"(*p) : "r"(v) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_swap a_swap
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline int a_swap(volatile int *x, int v)
{
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
__asm__( "xchg %0, %1" : "=r"(v), "=m"(*x) : "0"(v) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
return v;
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_fetch_add a_fetch_add
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline int a_fetch_add(volatile int *x, int v)
{
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
__asm__( "lock ; xadd %0, %1" : "=r"(v), "=m"(*x) : "0"(v) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
return v;
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_inc a_inc
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_inc(volatile int *x)
{
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
__asm__( "lock ; incl %0" : "=m"(*x) : "m"(*x) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_dec a_dec
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_dec(volatile int *x)
{
ensure that the compiler doesn't try to reorder around atomic ops
2011-02-12 05:26:24 +00:00
__asm__( "lock ; decl %0" : "=m"(*x) : "m"(*x) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_store a_store
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_store(volatile int *p, int x)
{
fix missing synchronization in atomic store on i386 and x86_64 despite being strongly ordered, the x86 memory model does not preclude reordering of loads across earlier stores. while a plain store suffices as a release barrier, we actually need a full barrier, since users of a_store subsequently load a waiter count to determine whether to issue a futex wait, and using a stale count will result in soft (fail-to-wake) deadlocks. these deadlocks were observed in malloc and possible with stdio locks and other libc-internal locking. on i386, an atomic operation on the caller's stack is used as the barrier rather than performing the store itself using xchg; this avoids the need to read the cache line on which the store is being performed. mfence is used on x86_64 where it's always available, and could be used on i386 with the appropriate cpu model checks if it's shown to perform better.
2015-07-28 18:40:18 +00:00
__asm__( "movl %1, %0 ; lock ; orl $0,(%%esp)" : "=m"(*p) : "r"(x) : "memory" );
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_spin a_spin
initial check-in, version 0.5.0
2011-02-12 05:22:29 +00:00
static inline void a_spin()
{
__asm__ __volatile__( "pause" : : : "memory" );
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_barrier a_barrier
add explicit barrier operation to internal atomic.h API
2014-10-10 22:17:09 +00:00
static inline void a_barrier()
{
__asm__ __volatile__( "" : : : "memory" );
}
refactor internal atomic.h rather than having each arch provide its own atomic.h, there is a new shared atomic.h in src/internal which pulls arch-specific definitions from arc/$(ARCH)/atomic_arch.h. the latter can be extremely minimal, defining only a_cas or new ll/sc type primitives which the shared atomic.h will use to construct everything else. this commit avoids making heavy changes to the individual archs' atomic implementations. definitions which are identical or near-identical to what the new shared atomic.h would produce have been removed, but otherwise the changes made are just hooking up the arch-specific files to the new infrastructure. major changes to take advantage of the new system will come in subsequent commits.
2016-01-21 19:08:54 +00:00
#define a_crash a_crash
security hardening: ensure suid programs have valid stdin/out/err this behavior (opening fds 0-2 for a suid program) is explicitly allowed (but not required) by POSIX to protect badly-written suid programs from clobbering files they later open. this commit does add some cost in startup code, but the availability of auxv and the security flag will be useful elsewhere in the future. in particular auxv is needed for static-linked vdso support, which is still waiting to be committed (sorry nik!)
2011-08-23 13:37:39 +00:00
static inline void a_crash()
{
__asm__ __volatile__( "hlt" : : : "memory" );
}