Commit Graph

26 Commits

Author SHA1 Message Date
David Sterba 00610f5853 btrfs-progs: libbtrfs: remove unneeded BTRFS_FLAT_INCLUDES protections
Remove the switch for local and system-wide headers in headers that are
not part of libbtrfs anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba 732d73dc1f btrfs-progs: remove btrfs_crc32c alias
There's an ancient macro btrfs_crc32c which is just wrapping crc32c and
not doing anything else, so we can use the crc helper directly.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba c3ee6a8a09 btrfs-progs: unify GPL header comments
Add the GPL v2 header to files where it was missing and is not from an
external source, update to the most recent version with the address.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 13:58:44 +02:00
David Sterba 9527bc0649 btrfs-progs: crypto: add perf support to speed test
Use perf events to read the cycle count, this should work on all
architectures. Enabled by option --perf and the sysctl
kernel.perf_event_paranoid must be 0 or 1.

The results are roughly the same as for raw cycles on x86_64 but worse
because of the additional overhead (read, context switch):

Block size:     4096
Iterations:     100000
Implementation: builtin
Units:          CPU cycles

    NULL-NOP: cycles:     42719688, cycles/i      427
 NULL-MEMCPY: cycles:     72941208, cycles/i      729,    18670.314 MiB/s
      CRC32C: cycles:    183709926, cycles/i     1837,     7413.009 MiB/s
      XXHASH: cycles:    136727614, cycles/i     1367,     9960.264 MiB/s
      SHA256: cycles:  10711594532, cycles/i   107115,      127.137 MiB/s
      BLAKE2: cycles:   2256957529, cycles/i    22569,      603.398 MiB/s

Block size:     4096
Iterations:     100000
Implementation: builtin
Units:          perf event: CPU cycles

    NULL-NOP: perf_c:     29649530, perf_c/i      296
 NULL-MEMCPY: perf_c:     59954062, perf_c/i      599,    15137.464 MiB/s
      CRC32C: perf_c:    179009071, perf_c/i     1790,     6929.460 MiB/s
      XXHASH: perf_c:    136413509, perf_c/i     1364,     9982.950 MiB/s
      SHA256: perf_c:  10997356664, perf_c/i   109973,      127.046 MiB/s
      BLAKE2: perf_c:   2379077576, perf_c/i    23790,      588.780 MiB/s

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-01 22:19:38 +02:00
David Sterba 1d4bab875a btrfs-progs: drop "2b" from blake2 in speed test
Internally it's blake2b but for the user facing output or other command
line interfaces let's call it just BLAKE2.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-01 21:17:50 +02:00
David Sterba 5734073b15 btrfs-progs: crypto: fix printf warnings in hash-speedtest
With explicit width the default alignment is to the right, using space
is a gnu extension. Fix the following warnings:

  crypto/hash-speedtest.c: In function ‘main’:
  crypto/hash-speedtest.c:152:15: warning: ' ' flag used with ‘%s’ gnu_printf format [-Wformat=]
    152 |   printf("% 12s: ", c->name);
	|               ^
  crypto/hash-speedtest.c:172:21: warning: ' ' flag used with ‘%u’ gnu_printf format [-Wformat=]
    172 |   printf("%s: % 12llu, %s/i % 8llu",
	|                     ^
  crypto/hash-speedtest.c:172:34: warning: ' ' flag used with ‘%u’ gnu_printf format [-Wformat=]
    172 |   printf("%s: % 12llu, %s/i % 8llu",
	|                                  ^

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-27 11:00:17 +02:00
David Sterba 133dd6c6c3 btrfs-progs: crypto: print throughput in hash-speedtest
Calculate the estimated throughput as a number that's comparable across
machines.

  $ ./hash-speedtest --cycles
  Block size:     4096
  Iterations:     100000
  Implementation: builtin
  Units:          cycles

      NULL-NOP: cycles:     42928902, cycles/i      429
   NULL-MEMCPY: cycles:     73014868, cycles/i      730,    18651.186 MiB/s
	CRC32C: cycles:    182293290, cycles/i     1822,     7470.579 MiB/s
	XXHASH: cycles:    138085981, cycles/i     1380,     9862.272 MiB/s
	SHA256: cycles:  10576270837, cycles/i   105762,      128.764 MiB/s
       BLAKE2b: cycles:   2263761293, cycles/i    22637,      601.585 MiB/s

  $ ./hash-speedtest --time
  Block size:     4096
  Iterations:     100000
  Implementation: builtin
  Units:          nsecs

      NULL-NOP: nsecs:     12164607, nsecs/i      121
   NULL-MEMCPY: nsecs:     20423641, nsecs/i      204,    19095.518 MiB/s
	CRC32C: nsecs:     51972794, nsecs/i      519,     7503.926 MiB/s
	XXHASH: nsecs:     38935164, nsecs/i      389,    10016.651 MiB/s
	SHA256: nsecs:   3030944497, nsecs/i    30309,      128.673 MiB/s
       BLAKE2b: nsecs:    648489262, nsecs/i     6484,      601.398 MiB/s

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-26 23:09:52 +02:00
David Sterba 55bf9b749d btrfs-progs: crypto: add time-based measurement to hash-speedtest
People are interested in measuring the hash performance on non-x86_64
architectures. Add option to do time-based measurements (in nanoseconds)
in case there's no support for clock-based measurements.

  $ ./hash-speedtest --cycles
  Block size:     4096
  Iterations:     100000
  Implementation: builtin
  Units:          cycles

      NULL-NOP: cycles:     43035633, cycles/i      430
   NULL-MEMCPY: cycles:     72478624, cycles/i      724
	CRC32C: cycles:    181712982, cycles/i     1817
	XXHASH: cycles:    136251305, cycles/i     1362
	SHA256: cycles:  10758567410, cycles/i   107585
       BLAKE2b: cycles:   2249704806, cycles/i    22497

  $ ./hash-speedtest --time
  Block size:     4096
  Iterations:     100000
  Implementation: builtin
  Units:          nsecs

      NULL-NOP:  nsecs:     12459033, nsecs/i      124
   NULL-MEMCPY:  nsecs:     20687845, nsecs/i      206
	CRC32C:  nsecs:     52648264, nsecs/i      526
	XXHASH:  nsecs:     39591766, nsecs/i      395
	SHA256:  nsecs:   3079668837, nsecs/i    30796
       BLAKE2b:  nsecs:    644766582, nsecs/i     6447

Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-26 22:42:59 +02:00
David Sterba fb98c1e9fc btrfs-progs: crypto: remove unused sha256 definitions
Remove:

- sha1, sha384, sha512 related structures and helpers
- HKDF definitions
- USHA definitions

Keep HMAC.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
David Sterba 05a92e6424 btrfs-progs: crypto: add test vectors
Add test vectors, a subset without keys as found in linux kernel sources
in crypto/test-mgr.h for all supported hash algorithms.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
Adam Borowski c615287cc0 btrfs-progs: a bunch of typo fixes
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:10 +01:00
David Sterba 297c71ee3b btrfs-progs: build: add support for libkcapi as crypto backend
https://github.com/smuellerDD/libkcapi allows user-space to access the
Linux kernel crypto API.  Uses netlink interface and exports easy to use
APIs.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-06-09 22:19:07 +02:00
David Sterba 82464a03e7 btrfs-progs: add more hash implementation providers
For environments that require certified implementations of cryptographic
primitives allow to select a library providing them. The requirements
are SHA256 and BLAKE2 (with the 2b variant and 256 bit digest).

For now there are two: libgrcrypt and libsodium (openssl does not
provide the BLAKE2b-256). Accellerated versions are typically provided
and automatically selected.

Signed-off-by: David Sterba <dsterba@suse.com>
2020-05-04 20:48:40 +02:00
Adam Borowski 3d379b1341 btrfs-progs: lots of typo fixes (codespell)
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:38 +02:00
Nikolay Borisov 5b13164cf9 btrfs-progs: fix xxhash on big endian machines
xxhash's state and results are always in little, but in progs after the
hash was calculated it was copied to the final buffer via memcpy,
meaning it'd be parsed as a big endian number on big endian machines.
This is incompatible with the kernel implementation of xxhash which
results in erroneous "checksum didn't match" errors on mount.

Fix it by using put_unaligned_le64 which always ensures the resulting
checksum will be copied in little endian format as the kernel expects
it.

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=206835
Fixes: f070ece2e9 ("btrfs-progs: add xxhash64 to mkfs")
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-03-31 18:37:35 +02:00
David Sterba 34ef695a81 btrfs-progs: add BLAKE2 to hash-speedtest
Sample results, Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz

Block size: 4096
Iterations: 1000000

    NULL-NOP: cycles:    314296257, c/i      314
 NULL-MEMCPY: cycles:    582807266, c/i      582
      CRC32C: cycles:   1738544130, c/i     1738
      XXHASH: cycles:   1449519934, c/i     1449
      SHA256: cycles: 110648340548, c/i   110648
     BLAKE2b: cycles:  29743769472, c/i    29743

Note this is unoptimized reference implementation.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:06 +01:00
David Sterba f5e952b13d btrfs-progs: add SHA256 to hash-speedtest
Sample results, Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz

Block size: 4096
Iterations: 1000000

    NULL-NOP: cycles:    314296257, c/i      314
 NULL-MEMCPY: cycles:    582807266, c/i      582
      CRC32C: cycles:   1738544130, c/i     1738
      XXHASH: cycles:   1449519934, c/i     1449
      SHA256: cycles: 110648340548, c/i   110648

Note this is unoptimized reference implementation.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:06 +01:00
David Sterba 2047b6de3d btrfs-progs: add blake2b support
Add definition, crypto wrappers and support to mkfs for blake2 for
checksumming. There are 2 aliases either blake2 or blake2b.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:06 +01:00
David Sterba 3778ece7ff btrfs-progs: add blake2b reference implementation
Upstream commit 997fa5ba1e14b52c554fb03ce39e579e6f27b90c,
git repository: git://github.com/BLAKE2/BLAKE2

The reference implemetation added in this patch is unchanged and will be
modified only to compile in current code base and with minimal other
modifications in case of future sync with upstream code. IOW, the coding
style should stay as-is and does not conform to the other btrfs-progs
code. This is an exception for xxhash and sha256 code as well.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:05 +01:00
Johannes Thumshirn ae9f8bff30 btrfs-progs: add sha256 as supported checksumming algorithm
Add the definition to the checksum types and let mkfs accept it.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:40 +01:00
David Sterba 785073f658 btrfs-progs: crypto: add hash speedtest utility
A simple tool to microbenchmark performance of the hashes. Uses rdtsc
for timing, so works only on x86_64.

 $ make hash-speedtest
 $ ./hash-speedtest [iterations]

   Block size: 4096
   Iterations: 100000

       NULL-NOP: cycles:     56061823, c/i      560
    NULL-MEMCPY: cycles:     61296469, c/i      612
	 CRC32C: cycles:    179961796, c/i     1799
	 XXHASH: cycles:    138434590, c/i     1384

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:03 +01:00
David Sterba ebb79f1644 btrfs-progs: add crc32c to hash wrappers
Unify the interface for crc32c too.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:03 +01:00
David Sterba 3a73bc1b37 btrfs-progs: move sha256 from tests to crypto/
The SHA256 is going to be used in the future, so this makes it a second
user and we also have the appropriate directory now.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:02 +01:00
Johannes Thumshirn c04bcdcacc btrfs-progs: move crc32c implementation to crypto/
With the introduction of xxhash64 to btrfs-progs we created a crypto/
directory for all the hashes used in btrfs (although no
cryptographically secure hash is there yet).

Move the crc32c implementation from kernel-lib/ to crypto/ as well so we
have all hashes consolidated.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:02 +01:00
Johannes Thumshirn f070ece2e9 btrfs-progs: add xxhash64 to mkfs
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:20:00 +01:00
David Sterba 2a264c6b0a btrfs-progs: add xxhash sources v0.7.1
Copy of xxhash.[ch] from git://github.com/Cyan4973/xxHash, version
v0.7.1. The include xxh3.h has been commented out as we don't have it
here, otherwise the copy is unchnaged.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-25 14:35:39 +02:00