Commit Graph

4014 Commits

Author SHA1 Message Date
David Sterba
a7a1ea0f4f
Btrfs progs v4.13.3
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-16 18:47:48 +02:00
David Sterba
877494861d btrfs-progs: update CHANGES for v4.13.3
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-16 18:37:24 +02:00
Lu Fengqi
19b18536d1 btrfs-progs: subvol list: don't print deleted subvol as TOPLEVEL
We should use entry->root_id instead of top_id to determine whether it is
the toplevel subvolume. Introduced in 4.13.2.

Issue: #72
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:16:03 +02:00
Naohiro Aota
0792c17028 btrfs-progs: build: set include path relatively
Currently, gcc is passed the include directory with full path. As a result,
dependency files (*.o.d) also record the full path at the build time. Such
full path dependency is annoying for sharing the source between multiple
machines, containers, or anything the path differ.

And this is the same way what other program using autotools e.g. e2fsprogs
is doing:

$ grep top_builddir Makefile
top_builddir = .
CPPFLAGS = -I. -I$(top_builddir)/lib -I$(top_srcdir)/lib
BUILD_CFLAGS = -g -O2  -I. -I$(top_builddir)/lib -I$(top_srcdir)/lib -DHAVE_CONFIG_H
<snip>

Signed-off-by: Naohiro Aota <naota@elisp.net>
[ set TOPDIR=. instead of -I as discussed, does not harm linker ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:16:01 +02:00
David Sterba
aafb4593fe btrfs-progs: build: add absolute path for topdir
We'll need TOPDIR to be ./ but library-test is intentionally built
outside of the git repository so we need to make them separate.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:58 +02:00
David Sterba
5286ec58f5 btrfs-progs: tests: add more configure option coverage
* test convert spec string
* explicitly ask for zstd, as it is now autodetected

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:54 +02:00
Sergei Trofimovich
2e5d35409a btrfs-progs: build: allow '--with-convert=' to accept comma-separated values
Before the change configure refused to accept it's defaults explicitly:
    $ ./configure --enable-convert --with-convert=ext2,reiserfs
    ...
    configure: error: unknown tokens for --with-convert: ,

After the change both converters are enabled:
    $ ./configure --enable-convert --with-convert=ext2,reiserfs
    ...
            btrfs-convert:      yes (ext2,reiserfs)

Cc: linux-btrfs@vger.kernel.org
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:51 +02:00
David Sterba
5a69c58be6 btrfs-progs: build: move travis helper script to own directory
We're going to add move build check integration scripts and
configuration, so put everything under travis/ now.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:50 +02:00
David Sterba
a69c1917c2 btrfs-progs: tests: fsck/007 fix so check --force works
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:47 +02:00
David Sterba
0a694ff4b0 btrfs-progs: check: drop exclusive open mode for --force
The check opens the given device in exclusive by default. In the forced
mode we want to access a device in use, so we have to drop the
exclusivity bit.

This works for block devices but not for files, that could be mounted
via a loop device. In that respect test check/007 is broken and will be
fixed.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-13 20:15:43 +02:00
David Sterba
37f1faef15
Btrfs progs v4.13.2
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 14:37:57 +02:00
David Sterba
3e7bd0dcea btrfs-progs: update CHANGES for v4.13.2
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 14:36:50 +02:00
Su Yue
7dbd7f21d5 btrfs-progs: tests: arg override in command line
Lowmem mode only repairs few cases which has a beacon file
".lowmem_repairable" in the case' directory.

However, defining TEST_ENABLE_OVERRIDE=true in command line does work
in above strategy.
Because _skip_spec() in tests/common.local isn't interpreted by shell
in that case.

Solve it by making _skip_spec() always be defined in common.local.

Reported-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[ keep the _skip_spec check ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:46 +02:00
David Sterba
1d4224e4aa btrfs-progs: tests: don't list toplevel subvolme in 'subvol list'
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:42 +02:00
David Sterba
f4a82d9e67 btrfs-progs: subvol list: don't list FS_TREE as deleted
Adding support for 'btrfs subvol show' for the toplevel subvolume
accidentally started to list the toplevel subvolume among the deleted.
Since version 4.8.3.

Don't panic. The toplevel subvolume (id 5) cannot be deleted.

Fixes: d4aa2bc07e ("btrfs-progs: subvol show: print more details about toplevel subvolume")
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:37 +02:00
Misono, Tomohiro
fa5b3a7fe2 btrfs-progs: misc-test: use raid1 for data to enable mount with -o degraded
kernel 4.14 introduces new function for checking if all chunks is ok for
mount with -o degraded option.

  commit 21634a19f646 ("btrfs: Introduce a function to check if all
  chunks a OK for degraded rw mount")

As a result, raid0 profile cannot be mounted with -o degraded on 4.14.
This causes failure of the misc-test 011 "delete missing device".

Fix this by using raid1 profile for both data and metadata.
This also should work for kernel before 4.13.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:34 +02:00
Holger Hoffstätte
0fc8e78088 btrfs-progs: right-align number columns in btrfs-debugfs output
The values for block group offset, length etc. in btrfs-debugfs' output
are left-aligned, which creates unaligned output and makes the usage
percentage hard to read/process further. This patch adds right-aligning
format specifiers for the number values.
Ideally the format values wouldn't be hardcoded but instead derived from
the filesystem size, but this seems to work for now.

Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:31 +02:00
David Sterba
e4c6772f69 btrfs-progs: set fixed with for printing uuids of subvolumes
The tabular output format looks better if the empty uuids are aligned
with the other. In the list output (now default) it's not that nice but
the whole list format is not nice anyway.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:28 +02:00
David Sterba
7b533f161c btrfs-progs: subvol: group options in help
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:21 +02:00
David Sterba
f96546d01c btrfs-progs: docs: update btrfs-balance
Copy unexpected edits from wiki so they do not get lost at next
git->wiki sync.

Author: Bill S.
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:12 +02:00
Jeff Mahoney
aa0cc10a4e btrfs-progs: backref: use separate list for indirect refs
Rather than iterate over all outstanding backrefs to resolve indirect refs,
use a separate list that only contains indirect refs.

When we process missing keys, the ref moves to the indirect ref list.
Once the indirect ref is resolved, move the ref to the pending list.

Eventually these lists will be replaced by rbtrees.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ added assertion fix from Josef ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:10 +02:00
Jeff Mahoney
b77ec6c6d5 btrfs-progs: backref: use separate list for missing keys
Rather than iterate over all outstanding backrefs to resolve missing keys,
use a separate list that only contains refs that need missing keys resolved.

Once the missing key is resolved, move the ref to the pending list.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:09 +02:00
Jeff Mahoney
e520de0dbb btrfs-progs: backref: add list_first_pref helper
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:08 +02:00
Jeff Mahoney
3ae36b3b0f btrfs-progs: backref: push state tracking into a helper structure
Eventually, we'll have several lists and trees, as well as some statistics.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:07 +02:00
Jeff Mahoney
a5ce5d2198 btrfs-progs: extent-cache: actually cache extent buffers
We have the infrastructure to cache extent buffers but we don't actually
do the caching.  As soon as the last reference is dropped, the buffer
is dropped.  This patch keeps the extent buffers around until the max
cache size is reached (defaults to 25% of memory) and then it drops
the last 10% of the LRU to free up cache space for reallocation.  The
cache size is configurable (for use by e.g. lowmem) when the cache is
initialized.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ update codingstyle, switch total_memory to bytes ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:06 +02:00
Jeff Mahoney
ad39e6252b btrfs-progs: check: switch to iterating over the backref_tree
We now have two data structures that can be used to iterate the same data
set, and there may be quite a few of them in memory.  Eliminating the
list_head member will reduce memory consumption while iterating over
the extent backrefs.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:05 +02:00
Jeff Mahoney
756105181e btrfs-progs: check: supplement extent backref list with rbtree
For the pathlogical case, like xfstests generic/297 that creates a
large file consisting of one, repeating reflinked extent, fsck can
take hours.  The root cause is that calling find_data_backref while
iterating the extent records is an O(n^2) algorithm.  For my
example test run, n was 2*2^20 and fsck was at 8 hours and counting.

This patch supplements the list with an rbtree and drops the runtime
of that testcase to about 20 seconds.

A previous version of this patch introduced a regression that would
have corrupted file systems during repair.  It was traced to the
compare algorithm honoring ->bytes regardless of whether the
reference had been found and a failure to reinsert nodes after
the target reference was found.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:41:01 +02:00
David Sterba
530ca51307 btrfs-progs: mkfs: remove unused function
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:46 +02:00
David Sterba
d07e349800 btrfs-progs: mkfs: use existing helper for path concatenation
Mkfs uses make_path that is duplicate of path_cat* functions, so we can
switch to them and add the error handling.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:44 +02:00
Yingyi Luo
fe667f6e7d btrfs-progs: add a parameter to btrfs_mksubvol
A convert parameter is added as a flag to indicate if btrfs_mksubvol()
is used for btrfs-convert. The change cascades down to the callchain.

Signed-off-by: Yingyi Luo <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:37 +02:00
Yingyi Luo
8a820b5765 btrfs-progs: convert: move link_subvol out of main
link_subvol() is moved to inode.c and renamed as btrfs_mksubvol().
The change cascades down to the callchain.

Signed-off-by: Yingyi Luo <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:34 +02:00
Misono, Tomohiro
8d93d71f6a btrfs-progs: subvol: fix subvol del --commit-after
Fix 'subvolume delete --commit-after' to work properly:
- SYNC ioctl will be issued even when last delete fails
- SYNC ioctl will be issued on each file system only once in the end

To achieve this, get_fsid() and add_seen_fsid() are called after each
delete to keep only one fd for each fs.

In the end, seen_fsid_hash will be traversed and SYNC is issued on each
fs.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:31 +02:00
Misono, Tomohiro
26908f6146 btrfs-progs: change seen_fsid to hold fd and DIR
Change seen_fsid to hold fd and DIR in order to keep access to each fs.
This will be used for 'subvol delete --commit-after'.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:30 +02:00
Misono, Tomohiro
836aa5202a btrfs-progs: move seen_fsid to utils.c
Move is_seen_fsid()/add_seen_fsid()/free_seen_fsid() to common functions.
This will be used for 'subvol delete --commit-after'.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:28 +02:00
Misono, Tomohiro
448763c64a btrfs-progs: move get_fsid() to utils.c
Make get_fsid() to a common functions.
This will be used for 'subvol delete --commit-after'.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:27 +02:00
Misono, Tomohiro
75716f6a8f btrfs-progs: subvol: exchange subvol del --commit-after and --commit-each
Current code is reversed in --commit-after and --commit-each operation,
i.e. --commit-after means --commit-each actually. This patch fixes this
and also introduces enum type for more readable code.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:26 +02:00
David Sterba
2a30baea9e
Btrfs progs v4.13.1
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 18:25:18 +02:00
David Sterba
bef1270572 btrfs-progs: update CHANGES for v4.13.1
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:24:25 +02:00
David Sterba
5faec98439 btrfs-progs: tests: check there are no unprintable characters in btrfs-image -ss output
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:46 +02:00
David Sterba
15beadeaa2 btrfs-progs: build: add more sparse warning checks
- declarations after statements (style issue)
- bitfields without defined signedness

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:44 +02:00
David Sterba
5d57bdeb23 btrfs-progs: build: use -std=gnu89 for sparse
Sparse does not seem to support gnu90 that we'd like to keep for the
regular build. Use gnu89 for C=1 build.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:40 +02:00
David Sterba
4498f53707 btrfs-progs: build: add missing defines for the C=1 build
Sparse complains that BTRFSCONVERT_EXT2 and _RESIERFS are not defined.
The per-target options are missing from the default CFLAGS and only
added in the specific rule.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:37 +02:00
David Sterba
628307f5c4 btrfs-progs: updated and add missing function attributes to the definition
Reported by sparse. Make the prototype match the definition and put the
attributes before the function.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:32 +02:00
David Sterba
b4b069dcb2 btrfs-progs: free-space-cache: fix endianity when reading from disk_key
Sparse reports:

free-space-cache.c:921:13: warning: incorrect type in assignment (different base types)
free-space-cache.c:921:13:    expected unsigned long long [unsigned] [usertype] ino
free-space-cache.c:921:13:    got restricted __le64 [addressable] [usertype] objectid

we need to use the helper to read objectid, as read_eb_member just
copies the data without disk->cpu transformation. This could lead to
bogus results on bigendian machines.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:27 +02:00
David Sterba
510a282f7f btrfs-progs: print-tree: use proper helper for reading offset
Sparse warns

print-tree.c:1261:49: warning: incorrect type in argument 4 (different base types)
print-tree.c:1261:49:    expected unsigned long long [unsigned] [usertype] start
print-tree.c:1261:49:    got restricted __le64 [addressable] [usertype] offset

which means we have to use the helper instead of directly accessing the offset
member. This could print bogus number on bigendian arch.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:19 +02:00
David Sterba
8eccfa86c0 btrfs-progs: fix debugging macro checks
Sparse warns that the DEBUG_*_ERROR macros are not defined when checked.
Use the ifdef check rather then relying on the implicit behaviour.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:17 +02:00
Nicholas D Steeves
5b2fbc6f4e btrfs-progs: tests: Remove misleading BCP 78 boilerplate from SHA implementation
BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD.

This causes the following lintian error when building on Debian and
Debian derivatives:

    E: btrfs-progs source: license-problem-non-free-RFC-BCP78
       tests/sha224-256.c

Please consult the following email from debian-legal@lists.debian.org
for more information:

https://lists.debian.org/debian-legal/2017/08/msg00004.html
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
[ I've copied too much from the RFC that's not related to the code,
  covered by the explicit copyright notice in the file ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:15 +02:00
Nicholas D Steeves
ac71751828 btrfs-progs: tests: Add required IETF Trust copyright to SHA implementation
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:12 +02:00
David Sterba
bc2c60ad31 btrfs-progs: docs: start ioctl documentation manual page
Start documenting the ioctl interface to btrfs. The overall structure
should be settled, the formatting of the ioctl description may change in
the future, newly added ioctl descriptions should follow the examples of
BTRFS_IOC_SUBVOL_CREATE.

The document is not finished yet and will not be installed until most of
ioctls' details are filled in.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:09 +02:00
David Sterba
509af95c92 btrfs-progs: tests: make sure _is_file_or_command does not get confused
The test cli/007-check-force reports something like:

$ type -p '--string that starts with dashes'
bash: type: --: invalid option

Add the option/argument separator.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:05 +02:00