Commit Graph

243 Commits

Author SHA1 Message Date
Josh Soref
bc72922789 btrfs-progs: fix typos in Makefile
Besides the comments, there's a slight change as the file config.log
will be deleted by the 'clean-gen' rule.

Generated by https://github.com/jsoref/spelling

Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-11-26 17:48:04 +01:00
Nikolay Borisov
c6039704c5 btrfs-progs: Add delayed refs infrastructure
This commit pulls those portions of the kernel implementation of
delayed refs which are necessary to have them working in user-space.
I've done the following modifications:

1. Replaced all kmem_cache_alloc calls to kmalloc.

2. Removed all locking-related code, since we are single threaded in
   userspace.

3. Removed code which deals with data refs - delayed refs in user space
   are going to be used only for cowonly trees.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Omar Sandoval
a4770b0a01 btrfs-progs: build: add --disable-shared and --disable-static
The build system mentioned in the previous commit builds libraries in
both PIC and non-PIC mode. Shared libraries don't work in PIC mode, so
it expects a --disable-shared configure option, which most open source
libraries using autoconf have. Let's add it, too.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:41 +02:00
Omar Sandoval
996a07c602 btrfs-progs: build: add --disable-programs
We have a build system internally which only needs to build and install
the libraries out of a repository, not any binaries. There's no easy way
to do this in btrfs-progs currently. Add --disable-programs to
./configure to support this.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:38 +02:00
Omar Sandoval
0cbcc58b96 btrfs-progs: remove stale dir-test
It does not 't build anymore and we don't have any use for it.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:01:38 +02:00
Gu JinXiang
bb9b8bcfd9 btrfs-progs: Add DEBUG_CFLAGS_INTERNAL for libbtrfsutil
Add DEBUG_CFLAGS_INTERNAL to LIBBTRFSUTIL_CFLAGS for libbtrfsutil's
build.

Signed-off-by: Gu JinXiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-06-07 16:37:38 +02:00
Jeff Mahoney
eb7b42003a btrfs-progs: build: detect whether -std=gnu90 is supported
GCC releases prior to 4.5.0 don't support -std=gnu90 so btrfs-progs won't
build at all on older distros.  We can detect whether the compiler
supports -std=gnu90 and fall back to -std=gnu89 if it doesn't.

AX_CHECK_COMPILE_FLAG is the right way to do this, but it depends on
autoconf 2.64.  AX_GCC_VERSION has been deprecated, so we'll use that
only for earlier autoconf versions so we can drop it when we drop
support for older autoconf releases.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-06-07 16:37:35 +02:00
Baruch Siach
cc66055f97 btrfs-progs: build btrfs.static needs libbtrfsutil to build
Add libbtrfsutil objects to btrfs.static link command. This fixes static
build failure:

utils.static.o: In function `parse_qgroupid':
utils.c:(.text.parse_qgroupid+0xb0): undefined reference to `btrfs_util_is_subvolume'
props.static.o: In function `prop_read_only':
props.c:(.text.prop_read_only+0x70): undefined reference to `btrfs_util_set_subvolume_read_only'
...
Makefile:457: recipe for target 'btrfs.static' failed
make[1]: *** [btrfs.static] Error 1

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:12 +02:00
Nikolay Borisov
631bf9f2b9 btrfs-progs: Remove deprecated btrfs-calc-size tool
Its function has been superseded by btrfs inspect-internal tree-stats.
Just remove it.

Deprecated since 4.8.

Issue: #97
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:10 +02:00
Nikolay Borisov
5a87a95115 btrfs-progs: Remove deprecated btrfs-show-super
Its function has been superseded by btrfs inspect-internal show-super.
Furthermore the tools is currently not built by default. Just remove it.

Deprecated since 4.8.

Issue: #97
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:10 +02:00
Nikolay Borisov
df9158f20d btrfs-progs: Remove deprecated btrfs-zero-log standalone tool
Its function has been subsumed by "btrfs rescue zero-log". Remove its
source file and adjust make/tests soruces accordingly.

Deprecated since 4.0.

Issue: #97
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:10 +02:00
Nikolay Borisov
de34a33a82 btrfs-progs: Remove btrfs-debug-tree command
There is already a replacement in the face of btrfs inspect-internal
dump-tree. And this command is just a simple wrapper around it. Just
remove it and adjust the show-blocks script to call the main btrfs
binary to achieve the same effect.

Informally deprecated since 4.4.

Issue: #97
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:10 +02:00
Peter Kjellerstedt
6cedd68811 btrfs-progs: build: Do not use cp -a to install library links
Using cp -a to install files will preserve the ownership of the original
files (if possible), which is typically not wanted. E.g. if the files
were built by a normal user, but are being installed by root, then the
installed files would maintain the UIDs/GIDs of the user that built the
files rather than be owned by root.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-06 15:06:50 +02:00
Omar Sandoval
7cdd58b2e9 libbtrfsutil: always build libbtrfsutil.so.$MAJOR
Otherwise, make test-libbtrfsutil from a fresh checkout fails.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:54:51 +02:00
Omar Sandoval
660adc0e1c libbtrfsutil: use local mkfs.btrfs for tests if it exists
The system might not have mkfs installed at all.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:54:41 +02:00
Lu Fengqi
3fd6a955f9 btrfs-progs: build: modify cscope/ctags rules to include directories such as check
Modify cscope/ctags rule to include directories such as check/
libbtrfsutil/kernel-lib/kernel-shared.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:55 +02:00
David Sterba
e0d173c6c4 libbtrfsutil: add stub for reallocarray
This function is new in glibc 2.26 and breaks build in CI and possibly
other environments.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-06 11:28:37 +01:00
Omar Sandoval
92d4035074 libbtrfsutil: add btrfs_util_is_subvolume() and btrfs_util_subvolume_id()
These are the most trivial helpers in the library and will be used to
implement several of the more involved functions.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-06 11:28:36 +01:00
Omar Sandoval
9615c23d15 libbtrfsutil: add filesystem sync helpers
Namely, sync, start_sync, and wait_sync.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:17 +01:00
Omar Sandoval
bad4208da3 libbtrfsutil: add qgroup inheritance helpers
We want to hide struct btrfs_qgroup_inherit from the user because that
comes from the Btrfs UAPI headers. Instead, wrap it in a struct
btrfs_util_qgroup_inherit and provide helpers to manipulate it. This
will be used for subvolume and snapshot creation.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:17 +01:00
Omar Sandoval
23c01b3c1b libbtrfsutil: add Python bindings
The C libbtrfsutil library isn't very useful for scripting, so we also
want bindings for Python. Writing unit tests in Python is also much
easier than doing so in C. Only Python 3 is supported; if someone really
wants Python 2 support, they can write their own bindings. This commit
is just the scaffolding.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:17 +01:00
Omar Sandoval
502e2a3510 Add libbtrfsutil
Currently, users wishing to manage Btrfs filesystems programatically
have to shell out to btrfs-progs and parse the output. This isn't ideal.
The goal of libbtrfsutil is to provide a library version of as many of
the operations of btrfs-progs as possible and to migrate btrfs-progs to
use it.

Rather than simply refactoring the existing btrfs-progs code, the code
has to be written from scratch for a couple of reasons:

* A lot of the btrfs-progs code was not designed with a nice library API
  in mind in terms of reusability, naming, and error reporting.
* libbtrfsutil is licensed under the LGPL, whereas btrfs-progs is under
  the GPL, which makes it dubious to directly copy or move the code.

Eventually, most of the low-level btrfs-progs code should either live in
libbtrfsutil or the shared kernel/userspace filesystem code, and
btrfs-progs will just be the CLI wrapper.

This first commit just includes the build system changes, license,
README, and error reporting helper.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:16 +01:00
David Sterba
e6477462d3 btrfs-progs: reorder tests in make target
Run the shorter tests first, convert can take very long, we're more
interested in the misc test.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-13 16:22:55 +01:00
David Sterba
04cd2c0bca btrfs-progs: rework testsuite export
Move the testsuite to tests/ and make the tarball generation more
deterministic. As there could be many random temporary files left in the
test directories, we can't just copy them. Use 'git ls-tree' to
filter just what we want, this needs a slight extension of the file list
specification.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-13 15:41:32 +01:00
Gu Jinxiang
eb2fe7d1c4 btrfs-progs: Add make testsuite command for export tests
Export the testsuite files to a separate tar.  Since fsck tests depend
on btrfs-corrupt-block, and misc tests depends on both
btrfs-corrupt-block and fssum, so set it as prerequisites for package
commad.

Because, althougth fssum can be generated by source that are all in
tests directory, and has no rely on the btrfs's structure.  But
btrfs-corrupt-block deeply relys on btrfs's structure.  For consistency,
at the present stage, generete the two when create test tar.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ applied without changes, the generated tarball will be different from
  the one after the follow up commits ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-13 15:41:32 +01:00
Gu Jinxiang
d5238ad711 btrfs-progs: add prerequisite btrfs-convert for test-misc
Since tests/misc-tests/005-convert-progress-thread-crash/test.sh need
the btrfs-convert for prerequisite.  So add the dependency in Makefile.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-13 15:41:32 +01:00
Gu Jinxiang
64a938ff70 btrfs-progs: add prerequisite btrfs-image for test-fuzz
Since tests/fuzz-tests/002-simple-image/test.sh need the btrfs-image for
prerequisite.  So add the dependency in Makefile.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-13 15:41:32 +01:00
Gu Jinxiang
837c850415 btrfs-progs: add prerequisite mkfs.btrfs for test-cli
Since tests/cli-tests/002-balance-full-no-filters/test.sh need
the mkfs.btrfs for prerequisite.
So add the dependency in Makefile.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-13 15:41:32 +01:00
David Sterba
6d3fea8f46 btrfs-progs: check: rename files after moving code
Add prefixes to the files so it's clear they belong to the mode group.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:59 +01:00
Qu Wenruo
b8d706e58e btrfs-progs: check: Move lowmem check code to its own check/lowmem.[ch]
Since lowmem mode code is highly internally connected, it's pretty hard to
move them piece by piece.
In theory it's possible to move part of the functions and temporarily
export them, but it will just cause extra temporarily modifications.

So this patch moves the whole lowmem check part into its own
check/lowmem.[ch].

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:59 +01:00
Qu Wenruo
b695ef1ddc btrfs-progs: check: Move count_csum_range function to check/common.c
Despite of moving it to check/common.c, also:

1) Add extra comment of the function
2) Change @root parameter to @fs_info
   Since @root is never used, csum_root is picked from fs_info anyway.

Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:58 +01:00
Qu Wenruo
c772db30e9 btrfs-progs: Move cmds-check.c to check/main.c
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:57 +01:00
David Sterba
2e1932e6a3 btrfs-progs: build: simplify version tracking
There will be a plain file tracking the last released version.

The rest will be simplified to print it where needed. The version
augmented by the current git status was not working anyway since we've
switched to autoconf. The result of version.h with the potential git
status was generated at configure time, which does not mean it's
accurate regarding the git status.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:01 +01:00
Gu Jinxiang
60fa9e945f btrfs-progs: build: Remove unused variable TESTS
Variable is never used.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 19:25:05 +01:00
Hans van Kranenburg
9303f1dd1b btrfs-progs: Fix progs_extra build dependencies
The Makefile does not have a dependency path that builds dependencies
for tools listed in progs_extra.

E.g. doing make btrfs-show-super in a clean build environment results in:
    gcc: error: cmds-inspect-dump-super.o: No such file or directory
    Makefile:389: recipe for target 'btrfs-show-super' failed

Signed-off-by: Hans van Kranenburg <hans@knorrie.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:18:29 +01:00
Qu Wenruo
075580471e btrfs-progs: mkfs: move image creation of rootdir to its own files
In fact, --rootdir option is getting more and more independent from
normal mkfs code.

So move image creation function, make_image() and its related code to
mkfs/rootdir.[ch], and rename the function to btrfs_mkfs_fill_dir().

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:11:24 +01:00
David Sterba
ea085932dc btrfs-progs: image: move sanitization to new file
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
David Sterba
af524c9539 btrfs-progs: build: use variables for btrfs-image images
More separate sources will be created for btrfs-image.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Naohiro Aota
0792c17028 btrfs-progs: build: set include path relatively
Currently, gcc is passed the include directory with full path. As a result,
dependency files (*.o.d) also record the full path at the build time. Such
full path dependency is annoying for sharing the source between multiple
machines, containers, or anything the path differ.

And this is the same way what other program using autotools e.g. e2fsprogs
is doing:

$ grep top_builddir Makefile
top_builddir = .
CPPFLAGS = -I. -I$(top_builddir)/lib -I$(top_srcdir)/lib
BUILD_CFLAGS = -g -O2  -I. -I$(top_builddir)/lib -I$(top_srcdir)/lib -DHAVE_CONFIG_H
<snip>

Signed-off-by: Naohiro Aota <naota@elisp.net>
[ set TOPDIR=. instead of -I as discussed, does not harm linker ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:16:01 +02:00
David Sterba
aafb4593fe btrfs-progs: build: add absolute path for topdir
We'll need TOPDIR to be ./ but library-test is intentionally built
outside of the git repository so we need to make them separate.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:58 +02:00
David Sterba
15beadeaa2 btrfs-progs: build: add more sparse warning checks
- declarations after statements (style issue)
- bitfields without defined signedness

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:44 +02:00
David Sterba
5d57bdeb23 btrfs-progs: build: use -std=gnu89 for sparse
Sparse does not seem to support gnu90 that we'd like to keep for the
regular build. Use gnu89 for C=1 build.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:40 +02:00
David Sterba
4498f53707 btrfs-progs: build: add missing defines for the C=1 build
Sparse complains that BTRFSCONVERT_EXT2 and _RESIERFS are not defined.
The per-target options are missing from the default CFLAGS and only
added in the specific rule.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:37 +02:00
Naohiro Aota
3efd322db2 btrfs-progs: build: omit unnecessary -MD flag
According to gcc(1), "-MD is equivalent to -M -MF file, except that -E is not
implied." Since the rule in the Makefile is just generating dependency file
and not building object file, it is no use to have "-MD" here. Also, it's
overridden and conflicting with the following "-MM" flag. I guess we can drop
it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:45 +02:00
Naohiro Aota
fd8c78a47c btrfs-progs: build: generate all dependency files
We're missing several dependency files like:

$ diff -u <(find -name '*.o'|cut -d. -f2|sort) <(find -name '*.o.d'|cut -d. -f2|sort)
@@ -3,7 +3,6 @@
 /btrfs-corrupt-block
 /btrfs-debug-tree
 /btrfs-find-root
-/btrfs-list
 /btrfs-map-logical
 /btrfs-select-super
 /btrfstune
@@ -29,11 +28,6 @@
 /cmds-scrub
 /cmds-send
 /cmds-subvolume
-/convert/common
-/convert/main
-/convert/source-ext2
-/convert/source-fs
-/convert/source-reiserfs
 /ctree
 /dir-item
 /disk-io
<snip>

This is due to moving things out of objects and cmds_objects variables. Such
missing dependency files cause mis-building of some source files (try touch
utils.h; make mkfs/main.o).

This patch introduce a new variable "all_objects" to keep all the objects and
use the variable to generate proper dependency file building rules.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:42 +02:00
David Sterba
bcaadc2e49 btrfs-progs: build: whitespace adjustment of [LD] fssum
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:35 +02:00
Nick Terrell
717a8b1e5d btrfs-progs: Add zstd support
Adds zstd support to the btrfs program. An optional dependency on libzstd
>= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and
defaults to detecting if libzstd is present using `pkg-config'.

The patch is also available in my fork of btrfs-progs [1], which passes
Travis-CI with the new tests. The prebuilt binary is available there.

I haven't updated Android.mk.

[1] https://github.com/terrelln/btrfs-progs/tree/devel

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:13 +02:00
David Sterba
37c271b216 btrfs-progs: move transaction implementation out of header
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
a31458b966 btrfs-progs: build: fix asan and ubsan build
Probably with a new gcc (7.1.1) I started to see asan/ubsan link failures.
Fixed by explicitly linking the libraries.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
2e5617c9a6 btrfs-progs: build: fix building libbtrfs.so with tsan
Except libbtrfs.so object, all other tools compile fine. The error is:

ld: send-stream.o: relocation R_X86_64_PC32 against symbol
`stderr@@GLIBC_2.2.5' can not be used when making a shared object;
recompile with -fPIC

Compiling with -fPIC fixes the problem.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00