BUG/MAJOR: hashes: fix the signedness of the hash inputs

Wietse Venema reported in the thread below that we have a signedness
issue with our hashes implementations: due to the use of const char*
for the input key that's often text, the crc32, sdbm, djb2, and wt6
algorithms return a platform-dependent value for binary input keys
containing bytes with bit 7 set. This means that an ARM or PPC
platform will hash binary inputs differently from an x86 typically.
Worse, some algorithms are well defined in the industry (like CRC32)
and do not provide the expected result on x86, possibly causing
interoperability issues (e.g. a user-agent would fail to compare the
CRC32 of a message body against the one computed by haproxy).

Fortunately, and contrary to the first impression, the CRC32c variant
used in the PROXY protocol processing is not affected. Thus the impact
remains very limited (the vast majority of input keys are text-based,
such as user-agent headers for exmaple).

This patch addresses the issue by fixing all hash functions' prototypes
(even those not affected, for API consistency). A reg test will follow
in another patch.

The vast majority of users do not use these hashes. And among those
using them, very few will pass them on binary inputs. However, for the
rare ones doing it, this fix MAY have an impact during the upgrade. For
example if the package is upgraded on one LB then on another one, and
the CRC32 of a binary input is used as a stick table key (why?) then
these CRCs will not match between both nodes. Similarly, if
"hash-type ... crc32" is used, LB inconsistency may appear during the
transition. For this reason it is preferable to apply the patch on all
nodes using such hashes at the same time. Systems upgraded via their
distros will likely observe the least impact since they're expected to
be upgraded within a short time frame.

And it is important for distros NOT to skip this fix, in order to avoid
distributing an incompatible implementation of a hash. This is the
reason why this patch is tagged as MAJOR, eventhough it's extremely
unlikely that anyone will ever notice a change at all.

This patch must be backported to all supported branches since the
hashes were introduced in 1.5-dev20 (commit 98634f0c). Some parts
may be dropped since implemented later.

Link to Wietse's report:
  https://marc.info/?l=postfix-users&m=157879464518535&w=2
This commit is contained in:
Willy Tarreau 2020-01-15 10:54:42 +01:00
parent bb9da0b8e2
commit 340b07e868
2 changed files with 16 additions and 11 deletions

View File

@ -24,10 +24,10 @@
#include <inttypes.h>
unsigned int hash_djb2(const char *key, int len);
unsigned int hash_wt6(const char *key, int len);
unsigned int hash_sdbm(const char *key, int len);
unsigned int hash_crc32(const char *key, int len);
uint32_t hash_crc32c(const char *key, int len);
unsigned int hash_djb2(const void *input, int len);
unsigned int hash_wt6(const void *input, int len);
unsigned int hash_sdbm(const void *input, int len);
unsigned int hash_crc32(const void *input, int len);
uint32_t hash_crc32c(const void *input, int len);
#endif /* _COMMON_HASH_H_ */

View File

@ -17,8 +17,9 @@
#include <common/hash.h>
unsigned int hash_wt6(const char *key, int len)
unsigned int hash_wt6(const void *input, int len)
{
const unsigned char *key = input;
unsigned h0 = 0xa53c965aUL;
unsigned h1 = 0x5ca6953aUL;
unsigned step0 = 6;
@ -27,7 +28,7 @@ unsigned int hash_wt6(const char *key, int len)
for (; len > 0; len--) {
unsigned int t;
t = ((unsigned int)*key);
t = *key;
key++;
h0 = ~(h0 ^ t);
@ -44,8 +45,9 @@ unsigned int hash_wt6(const char *key, int len)
return h0 ^ h1;
}
unsigned int hash_djb2(const char *key, int len)
unsigned int hash_djb2(const void *input, int len)
{
const unsigned char *key = input;
unsigned int hash = 5381;
/* the hash unrolled eight times */
@ -72,8 +74,9 @@ unsigned int hash_djb2(const char *key, int len)
return hash;
}
unsigned int hash_sdbm(const char *key, int len)
unsigned int hash_sdbm(const void *input, int len)
{
const unsigned char *key = input;
unsigned int hash = 0;
int c;
@ -92,8 +95,9 @@ unsigned int hash_sdbm(const char *key, int len)
* this hash already sustains gigabit speed which is far faster than what
* we'd ever need. Better preserve the CPU's cache instead.
*/
unsigned int hash_crc32(const char *key, int len)
unsigned int hash_crc32(const void *input, int len)
{
const unsigned char *key = input;
unsigned int hash;
int bit;
@ -174,8 +178,9 @@ static const uint32_t crctable[256] = {
0xBE2DA0A5L, 0x4C4623A6L, 0x5F16D052L, 0xAD7D5351L
};
uint32_t hash_crc32c(const char *buf, int len)
uint32_t hash_crc32c(const void *input, int len)
{
const unsigned char *buf = input;
uint32_t crc = 0xffffffff;
while (len-- > 0) {
crc = (crc >> 8) ^ crctable[(crc ^ (*buf++)) & 0xff];