Commit Graph

135 Commits

Author SHA1 Message Date
David Sterba
116e27a87a btrfs-progs: path-utils: rename is_path_exist
Rename so it fits the path_ group.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-04 02:06:33 +02:00
David Sterba
c3b0e6970b btrfs-progs: path-utils: rename is_block_device
Add the path_ prefix and update all callers.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-04 02:06:33 +02:00
David Sterba
f83e81c61b btrfs-progs: utils: split path related utils to own file
Group helpers that return some status of a given path to own file so we
debloat utils.c a bit.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-04 02:06:31 +02:00
David Sterba
c8bea2b84b btrfs-progs: move rbtree-utils.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba
94fced6353 btrfs-progs: build: drop kernel-lib from -I and update paths
Include the files by full path to avoid any confusion in case of
potentially duplicate names.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba
c07960c8be btrfs-progs: move utils.[ch] to common/
Update include paths and remove some duplicates.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba
07ca08ffba btrfs-progs: move fsfeatures.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba
f93b471143 btrfs-progs: move help.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba
d1efe50d0a btrfs-progs: move messages.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba
f63f29e9e9 btrfs-progs: move internal.h to common/
Create directory for all sources that can be used by anything that's not
rellated to a relevant kernel part, all common functions, helpers,
utilities that do not fit any other specific category.

The traditional location would be probably lib/ with all things that are
statically linked to the main binaries, but we have libbtrfs and
libbtrfsutil so this would be confusing.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
Qu Wenruo
c31edf610c btrfs-progs: Fix false ENOSPC alert by tracking used space correctly
[BUG]
There is a bug report of unexpected ENOSPC from btrfs-convert, issue #123.

After some debugging, even when we have enough unallocated space, we
still hit ENOSPC at btrfs_reserve_extent().

[CAUSE]
Btrfs-progs relies on chunk preallocator to make enough space for
data/metadata.

However after the introduction of delayed-ref, it's no longer reliable
to rely on btrfs_space_info::bytes_used and
btrfs_space_info::bytes_pinned to calculate used metadata space.

For a running transaction with a lot of allocated tree blocks,
btrfs_space_info::bytes_used stays its original value, and will only be
updated when running delayed ref.

This makes btrfs-progs chunk preallocator completely useless. And for
btrfs-convert/mkfs.btrfs --rootdir, if we're going to have enough
metadata to fill a metadata block group in one transaction, we will hit
ENOSPC no matter whether we have enough unallocated space.

[FIX]
This patch will introduce btrfs_space_info::bytes_reserved to track how
many space we have reserved but not yet committed to extent tree.

To support this change, this commit also introduces the following
modification:

- More comment on btrfs_space_info::bytes_*
  To make code a little easier to read

- Export update_space_info() to preallocate empty data/metadata space
  info for mkfs.
  For mkfs, we only have a temporary fs image with SYSTEM chunk only.
  Export update_space_info() so that we can preallocate empty
  data/metadata space info before we start a transaction.

- Proper btrfs_space_info::bytes_reserved update
  The timing is the as kernel (except we don't need to update
  bytes_reserved for data extents)
  * Increase bytes_reserved when call alloc_reserved_tree_block()
  * Decrease bytes_reserved when running delayed refs
    With the help of head->must_insert_reserved to determine whether we
    need to decrease.

Issue: #123
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 13:31:14 +02:00
Qu Wenruo
d490933d14 btrfs-progs: Enable crc32c optimization probe for convert and mkfs
Although moderm hardware is fast enough and crc32c calculation is not a
hotspot, doing such optimization won't hurt anyway.

Issue: #175
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-05-27 16:39:51 +02:00
Qu Wenruo
8addcab0d0 btrfs-progs: Create uuid tree with proper contents
Commit 2a496a5b8b ("btrfs-progs: mkfs: precreate the uuid tree")
creates uuid tree at mkfs time.  However it doesn't populate uuid tree
correctly nor creates an empty root.

It uses create_tree(), which just
copies the content of fs root, containing a meaningless INODE_ITEM:

v4.15 mkfs (no uuid tree creation) + kernel mount:
  uuid tree key (UUID_TREE ROOT_ITEM 0)
  leaf 30572544 items 1 free space 16250 generation 7 owner UUID_TREE
  leaf 30572544 flags 0x1(WRITTEN) backref revision 1
  fs uuid 33ecddef-fc86-481a-93ce-846b01c11376
  chunk uuid 9e58f646-b0da-43ca-9c7d-8bbe3e120246
	item 0 key (0x92457c59d31491be UUID_KEY_SUBVOL 0xef908b5e79aa76a1) itemoff 16275 itemsize 8
		subvol_id 5

v4.19.1 mkfs (incorrect one), no kernel mount:
  uuid tree key (UUID_TREE ROOT_ITEM 0)
  leaf 30507008 items 2 free space 16061 generation 4 owner UUID_TREE
  leaf 30507008 flags 0x1(WRITTEN) backref revision 1
  fs uuid 162f5333-9b5d-4217-877c-ddaeaa79398e
  chunk uuid 7bc2c5c6-a6d2-4eec-a513-142b549c6541
	item 0 key (256 INODE_ITEM 0) itemoff 16123 itemsize 160
		generation 3 transid 0 size 0 nbytes 16384
		block group 0 mode 40755 links 1 uid 0 gid 0 rdev 0
		sequence 0 flags 0x0(none)
	item 1 key (256 INODE_REF 256) itemoff 16111 itemsize 12
		index 0 namelen 2 name: ..

This patchset will fix it by populuating uuid tree properly:
(NOTE: due to tree-checker, kernel doesn't accept empty uuid tree, so we
can only fix it by populating uuid tree correctly)

With this patchset, no kernel mount:
  uuid tree key (UUID_TREE ROOT_ITEM 0)
  leaf 30507008 items 1 free space 16250 generation 4 owner UUID_TREE
  leaf 30507008 flags 0x1(WRITTEN) backref revision 1
  fs uuid ae53079e-dbbc-409b-a565-5326c7b27731
  chunk uuid b5fb1bea-f20d-4af1-80f8-6ca3f0038d67
	item 0 key (0x334ba6b032d89c07 UUID_KEY_SUBVOL 0x86cde09cb78bcca0) itemoff 16275 itemsize 8
		subvol_id 5

For kernel, except tree-checker needs an non-empty uuid tree, both of
the above behavior won't cause problem, but it's always better to keep a
good standardized behavior.

Fixes: 2a496a5b8b ("btrfs-progs: mkfs: precreate the uuid tree")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo
e4ca845778 btrfs-progs: mkfs: Create data reloc tree from scratch
For data reloc tree creation, we copy its contents from the fs tree just
for its INODE_ITEM, INODE_REF and dirid.  This hides the detail and is
not obvious for why we're copying from fs root.

This patch will create data reloc tree from scratch:
- Create root, including root item and new tree root
- Change dirid to BTRFS_FIRST_FREE_OBJECTID
- Insert root INODE_ITEM and INODE_REF

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Josh Soref
b7439eeb3a btrfs-progs: mkfs: fix typo in "multipler" variables
Generated by https://github.com/jsoref/spelling

Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-11-26 18:24:49 +01:00
David Sterba
e578b59bf6 btrfs-progs: convert strerror to implicit %m
Similar to the changes where strerror(errno) was converted, continue
with the remaining cases where the argument was stored in another
variable.

The savings in object size are about 4500 bytes:

 $ size btrfs.old btrfs.new
   text    data     bss     dec     hex filename
 805055   24248   19748  849051   cf49b btrfs.old
 804527   24248   19748  848523   cf28b btrfs.new

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:14 +01:00
Rosen Penev
251d32ea5c btrfs-progs: Eliminate remaining uses of strerror(errno)
%m allows a smaller filesize. Useful on embedded systems.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-25 16:11:40 +02:00
Yevgeny Popovych
9ff39d48aa btrfs-progs: mkfs: Fix typos in strings and comments
Signed-off-by: Yevgeny Popovych <yevgenyp@pointgrab.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-06-07 16:37:40 +02:00
Matthias Benkard
db84e5908d btrfs-progs: mkfs: traverse_directory: Reset error code on continue
In case add_inode_items returned -EEXIST, traverse_directory would
handle the condition and still continue under certain circumstances, but
it would not reset the error code, leading to spurious failure later.
This patch fixes that.

Pull-request: #124
Author: Matthias Benkard <matthias.benkard@egym.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-06-07 16:37:40 +02:00
Qu Wenruo
c57ed6ca6b btrfs-progs: Rename OPEN_CTREE_FS_PARTIAL to OPEN_CTREE_TEMPORARY_SUPER
The old flag OPEN_CTREE_FS_PARTIAL is in fact quite easy to be confused
with OPEN_CTREE_PARTIAL, which allow btrfs-progs to open damaged
filesystem (like corrupted extent/csum tree).

However OPEN_CTREE_FS_PARTIAL, unlike its name, is just allowing
btrfs-progs to open fs with temporary superblocks (which only has 6
basic trees on SINGLE meta/sys chunks).

The usage of FS_PARTIAL is really confusing here.

So rename OPEN_CTREE_FS_PARTIAL to OPEN_CTREE_TEMPORARY_SUPER, and add
extra comment for its behavior.
Also rename BTRFS_MAGIC_PARTIAL to BTRFS_MAGIC_TEMPORARY to keep the
naming consistent.

And with above comment, the usage of FS_PARTIAL in dump-tree is
obviously incorrect, fix it.

Fixes: 8698a2b9ba ("btrfs-progs: Allow inspect dump-tree to show specified tree block even some tree roots are corrupted")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:12 +02:00
Su Yue
0d4adfd336 btrfs-progs: mkfs: return nozero value on thin provisioned device
With mkfs.btrfs on a thin provisioned device with very small backing
size and big virtual size, all code works well in mkfs.btrfs until
close_ctree() is called.
close_ctree() fails to sync device due to small backing size while
closing devices.  However, mkfs returns 0 in such situation which causes
failure of fstests generic/405.

So, let mkfs returns nonzero value if previous steps succeeded but
close_ctree() failed. Then fstests generic/405 passes now.

Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-11 15:37:36 +02:00
Misono Tomohiro
eb9dd12d0e btrfs-progs: mkfs rootdir: use lgetxattr() not to follow a symbolic link
mkfs-test 016 "rootdir-bad-symbolic-link" fails when selinux is enabled.
This is because add_xattr_item() uses getxattr() and tries to follow a
bad symbolic link for selinux item, which causes ENOENT error.

The line above already uses llistxattr() for getting list of xattr in
order not to follow a symbolic link, so just use lgetxattr() too.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-06 15:06:50 +02:00
David Sterba
2a496a5b8b btrfs-progs: mkfs: precreate the uuid tree
We can easily create the uuid tree that's usually created after first
mount. The kernel will still check the tree on first mount so we don't
try to fake the uuid tree generation so it appears consistent, even if
it's empty.

Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:55 +02:00
Qu Wenruo
4e6e5804f2 btrfs-progs: mkfs/rootdir: Don't follow symbolic link when calcuating size
[BUG]
If we have a symbolic link in rootdir pointing to non-existing location,
mkfs.btrfs --rootdir will just fail:
------
$ mkfs.btrfs  -f --rootdir /tmp/rootdir/ /dev/data/btrfs
btrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.

ERROR: ftw subdir walk of /tmp/rootdir/ failed: No such file or directory
------

[CAUSE]
Commit 599a0abed5 ("btrfs-progs: mkfs/rootdir: Use over-reserve method
to make size estimate easier") add extra ftw walk to estimate the
filesystem size.

Such default ftw walk will follow symbolic link and gives ENOENT error.

[FIX]
Use nftw() to specify FTW_PHYS so we won't follow symbolic link for size
calculation.

Issue: #109
Reported-by: Alexander Kanavin <alexander.kanavin@intel.com>
Fixes: 599a0abed5 ("btrfs-progs: mkfs/rootdir: Use over-reserve method to make size estimate easier")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:55 +02:00
Misono, Tomohiro
c074434fb5 btrfs-progs: remove BTRFS_CRC32_SIZE definition
The kernel code no longer has BTRFS_CRC32_SIZE and only uses
btrfs_csum_sizes[]. So, update the progs code as well.

Suggested-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:55 +02:00
Misono Tomohiro
0a0a03554a btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of, FS_TREE
Currently, the top-level subvolume lacks the UUID. As a result, both
non-snapshot subvolume and snapshot of top-level subvolume do not have
Parent UUID and cannot be distinguisued. Therefore "fi show" of
top-level lists all the subvolumes which lacks the UUID in
"Snapshot(s)" filed.  Also, it lacks the otime information.

Fix this by adding the UUID and otime at the mkfs time.  As a
consequence, snapshots of top-level subvolume now have a Parent UUID and
UUID tree will create an entry for top-level subvolume at mount time.
This should not cause the problem for current kernel, but user program
which relies on the empty Parent UUID may be affected by this change.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:54 +02:00
Qu Wenruo
51667947e4 btrfs-progs: mkfs/rootdir: Fix inline extent creation check
Just like convert, we need extra check against sector size for creating
inline extent.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-03-30 22:15:54 +02:00
Qu Wenruo
78c5a90ebf btrfs-progs: mkfs/rootdir: Fix memory leak in traverse_directory()
The bug is exposed by mkfs test case 009, with D=asan.

We are leaking memory of parent_dir_entry->path() which ,except the
rootdir, is allocated by strdup().

Before fixing it, unifiy the allocation of parent_dir_entry() to heap
allocation.

Then fix it by adding "free(parent_dir_entry->path);" in
traverse_directory() and error handler.

Issue: #92
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-14 16:31:19 +01:00
David Sterba
3aa1bbdd89 btrfs-progs: mkfs: fix build on musl
Another build failure on musl.

Issue: #90
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-03 01:15:42 +01:00
Gu Jinxiang
c958110785 btrfs-progs: Sync code with kernel for BTRFS_MAX_INLINE_DATA_SIZE
Do a cleanup. Also make it consistent with kernel.  Use fs_info instead
of root for BTRFS_MAX_INLINE_DATA_SIZE, since maybe in some situation we
do not know root, but just know fs_info.

Change macro to inline function to be consistent with kernel.  And
change the function body to match kernel.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:57 +01:00
Qu Wenruo
1945854e01 btrfs-progs: Remove unnecessary parameter for btrfs_add_block_group
@chunk_objectid of btrfs_make_block_group() function is always fixed to
BTRFS_FIRST_FREE_OBJECTID, so there is no need to pass it as parameter
explicitly.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:03 +01:00
Rosen Penev
e4df433b8a btrfs-progs: treewide: Replace strerror(errno) with %m.
As btrfs is specific to Linux, %m can be used instead of strerror(errno)
in format strings. This has some size reduction benefits for embedded
systems.

glibc, musl, and uclibc-ng all support %m as a modifier to printf.
A quick glance at the BIONIC libc source indicates that it has
support for %m as well. BSDs and Windows do not but I do believe
them to be beyond the scope of btrfs-progs.

Compiled sizes on Ubuntu 16.04:

Before:
3916512 btrfs
233688  libbtrfs.so.0.1
4899    bcp
2367672 btrfs-convert
2208488 btrfs-corrupt-block
13302   btrfs-debugfs
2152160 btrfs-debug-tree
2136024 btrfs-find-root
2287592 btrfs-image
2144600 btrfs-map-logical
2130760 btrfs-select-super
2152608 btrfstune
2131760 btrfs-zero-log
2277752 mkfs.btrfs
9166    show-blocks

After:
3908744 btrfs
233256  libbtrfs.so.0.1
4899    bcp
2366560 btrfs-convert
2207432 btrfs-corrupt-block
13302   btrfs-debugfs
2151104 btrfs-debug-tree
2134968 btrfs-find-root
2281864 btrfs-image
2143536 btrfs-map-logical
2129704 btrfs-select-super
2151552 btrfstune
2130696 btrfs-zero-log
2276272 mkfs.btrfs
9166    show-blocks

Total savings: 23928 (24 kilo)bytes

Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:03 +01:00
Qu Wenruo
997f9977c2 btrfs-progs: mkfs: Prevent temporary system chunk to use space in reserved 1M range
When creating btrfs, mkfs.btrfs will firstly create a temporary system
chunk as basis, and then created needed trees or new devices.

However the layout temporary system chunk is hard-coded and uses
reserved [0, 1M) range of devid 1.

Change the temporary chunk layout from old:

0	1M				4M	5M
|<----------- temp chunk -------------->|
  And it's 1:1 mapped, which means it's a SINGLE chunk,
  and stripe offset is also 0.

to new layout:

0	1M				4M	5M
	|<----------- temp chunk -------------->|
  And still keeps the 1:1 mapping.

However this also affects btrfs_min_dev_size() which still assume
temporary chunks starts at device offset 0.

The problem can only be exposed by "-m single" or "-M" where we reuse the
temporary chunk.

With other meta profiles, system and meta chunks are allocated by later
btrfs_alloc_chunk() call, and old SINGLE chunks are removed, so it will
be no such problem for other meta profiles.

Reported-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
[ folded fix for the minimal device size calculation ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:02 +01:00
David Sterba
fab56e8c84 btrfs-progs: docs: update manual for mkfs --shrink
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:15:24 +01:00
Qu Wenruo
1733989352 btrfs-progs: mkfs: Use the whole file or block device to mkfs for rootdir
For --rootdir, even for large existing file or block device, it will
always shrink the resulting filesystem.

The problem is, mkfs.btrfs will try to calculate the dir size, and use
it as @block_count to mkfs, which makes the filesystem shrunk.

Fix it by trying to get the original block device or file size as
@block_count, so mkfs.btrfs can use the full file/block device for
--rootdir option.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:15:20 +01:00
Qu Wenruo
0855a8cd43 btrfs-progs: mkfs: fix regression preventing --rootdir to create file
Commit 460e93f257 ("btrfs-progs: mkfs: check the status of file at mkfs")
will try to check the file state before creating fs on it.

The check is mostly fine for normal mkfs case, while for --rootdir
option, it's allowed to create a new file if the destination file
doesn't exist.

Fix it by allowing non-existent file if --rootdir is specified.

Fixes: 460e93f257 ("btrfs-progs: mkfs: check the status of file at mkfs")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:15:16 +01:00
Qu Wenruo
1c9c5f7fb3 btrfs-progs: mkfs: Separate shrink from rootdir
Make --shrink a separate option for --rootdir, and change the default to
off.

The shrinking behaviour is not a commonly used feature but can be useful
for creating minimal pre-filled images, in one step, without requiring
to mount.

Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update changelog and error messages ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:15:15 +01:00
Qu Wenruo
c28418daed btrfs-progs: mkfs/rootdir: Shrink fs for rootdir option
Use the new dev extent based shrink method for rootdir option. This
restores the original behaviour when --rootdir will create a minimal
filesystem size.

Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:15:13 +01:00
Qu Wenruo
599a0abed5 btrfs-progs: mkfs/rootdir: Use over-reserve method to make size estimate easier
Use an easier method to calculate the estimate device size for
mkfs.btrfs --rootdir.

The new method will over-estimate, but should ensure we won't encounter
ENOSPC.

It relies on the following data:
1) number of inodes -- for metadata chunk size
2) rounded up data size of each regular inode -- for data chunk size

Total meta chunk size = round_up(nr_inode * (PATH_MAX * 3 + sectorsize),
min_chunk_size) * profile_multiplier

PATH_MAX is the maximum size possible for INODE_REF/DIR_INDEX/DIR_ITEM.
Sectorsize is the maximum size possible for inline extent.
min_chunk_size is 8M for SINGLE, and 32M for DUP, get from
btrfs_alloc_chunk().
profile_multiplier is 1 for Single, 2 for DUP.

Total data chunk size is much easier.
Total data chunk size = round_up(total_data_usage, min_chunk_size) *
profile_multiplier

Total_data_usage is the sum of *rounded up* size of each regular inode
use.
min_chunk_size is 8M for SINGLE, 64M for DUP, get from btrfS_alloc_chunk().
Same profile_multiplier for meta.

This over-estimate calculate is, of course inacurrate, but since we will
later shrink the fs to its real usage, it doesn't matter much now.

Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update comments ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:15:11 +01:00
Qu Wenruo
c7bc72264a btrfs-progs: mkfs: Don't use custom chunk allocator for rootdir
Remove the custom chunk allocator for mkfs. It is buggy in connection to
the --rootdir option and puts file data to the reerved 1M area. The
feature of the custom allocator was to reserve only minimal amount of
blockgroup space. This will temporarily stop working and will need an
explicit request by option, added by following patches.

Use the generic chunk allocator.

Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:14:33 +01:00
Qu Wenruo
2fd0f3a980 btrfs-progs: mkfs: Cleanup temporary chunks before filling rootdir
Cleanup of temporary chunks should be done as soon as possible, and it
should be especially before doing large tree operations, like filling
the filesystem when using --rootdir.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:11:38 +01:00
Qu Wenruo
41bc987511 btrfs-progs: mkfs: Update allocation info before verbose output
Since new --rootdir can allocate chunk, it will modify the chunk
allocation result.

This patch will update allocation info before verbose output to reflect
such info.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:11:37 +01:00
Qu Wenruo
0a7a48e1e8 btrfs-progs: mkfs/rootdir: Introduce function to get end position of last device extent
Useful for later 'mkfs.btrfs --rootdir' shrink support.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:11:36 +01:00
Qu Wenruo
8719161b4c btrfs-progs: mkfs: move source dir size calculation to its own files
Also rename the function from size_sourcedir() to mkfs_size_dir().

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:11:33 +01:00
Qu Wenruo
075580471e btrfs-progs: mkfs: move image creation of rootdir to its own files
In fact, --rootdir option is getting more and more independent from
normal mkfs code.

So move image creation function, make_image() and its related code to
mkfs/rootdir.[ch], and rename the function to btrfs_mkfs_fill_dir().

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-08 18:11:24 +01:00
Qu Wenruo
6dd8669a1c btrfs-progs: mkfs: Only zero out the first 1M for rootdir
It's a waste of IO to fill the whole image before creating btrfs on it,
just wiping the first 1M, and then write 1 byte to the last position to
create a sparse file.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-03 17:29:19 +01:00
Qu Wenruo
31d228a2eb btrfs-progs: mkfs: Enhance minimal device size calculation to fix mkfs failure on small file
Since commit c11e36a29e ("Btrfs-progs: Do not force mixed block group
creation unless '-M' option is specified"), mkfs no longer use mixed
block group unless specified manually.

This breaks the minimal device size calculation, which only considered
mixed block group use case.

This patch enhances minimal device size calculation for mkfs, by using
different minimal stripe length (calculated from code) for different
profiles, and use them to calculate minimal device size.

Reported-by: Wesley Aptekar-Cassels <W.Aptekar@gmail.com>
Fixes: c11e36a29e ("Btrfs-progs: Do not force mixed block group creation unless '-M' option is specified")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
[ updated comments ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-03 17:10:11 +01:00
Misono, Tomohiro
e460ccd6d1 btrfs-progs: mkfs: check the status of file at mkfs
Currently, only the status of block devices is checked at mkfs,
but we should also check for regular files whether they are already
formatted or mounted to prevent overwrite accidentally.

Device status is checked by test_dev_for_mkfs().
The part which is not related to block device is split from this
and used for both block device and regular file.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-03 17:09:57 +01:00
Qu Wenruo
b84aa6033c btrfs-progs: mkfs: refactor test_minimum_size to use the calculated minimal size
test_minimum_size() function is only called to check if provided device
is large enough.

However the minimal device size only needs to be calculated once, and
can be reused everywhere.

Refactor that function to make later modification easier.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo
fec462240d btrfs-progs: mkfs: error out gracefully for --rootdir
--rootdir option will start a transaction to fill the fs, however if
something goes wrong, from ENOSPC to lack of permission, we won't commit
the transaction and cause BUG_ON triggered by uncommitted transaction:

------
extent buffer leak: start 29392896 len 16384
extent_io.c:579: free_extent_buffer: BUG_ON `eb->flags & EXTENT_DIRTY` triggered, value 1
------

The root fix is to introduce btrfs_abort_transaction() in btrfs-progs,
however in this particular case, we can workaround it by force
committing the transaction.

Since during mkfs, the magic of btrfs is set to an invalid one, without
setting fs_info->finalize_on_close() the fs is never able to be mounted.
So even we force to commit wrong transaction we won't screw up things
worse.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo
625223903e btrfs-progs: mkfs: fix overwritten return value for mkfs
For mkfs failure, especially --rootdir errors like EPERM/ENOSPC, the out
branch will overwrite the return value, causing wrong status code.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo
a00bfc200c btrfs-progs: mkfs: avoid positive return value from cleanup_temp_chunks
Since we're calling btrfs_search_slot() the return value can be
positive.  However we just pass that return value out, causing undefined
return value.

This can cause mkfs to return 1, which indicates something wrong.

Fix it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
David Sterba
530ca51307 btrfs-progs: mkfs: remove unused function
Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:46 +02:00
David Sterba
d07e349800 btrfs-progs: mkfs: use existing helper for path concatenation
Mkfs uses make_path that is duplicate of path_cat* functions, so we can
switch to them and add the error handling.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-10-06 13:23:44 +02:00
yingyil
f7267bc85c btrfs-progs: mkfs: refactor create_data_reloc_tree
Add an objectid parameter to make the function a general one for
inserting root items and rename it to create_tree. The change cascades
down to the callchain.

Signed-off-by: yingyil <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:39 +02:00
Qu Wenruo
081e4e9bb8 btrfs-progs: mkfs: Fix wrong file type for dir items and indexes when specifying root directory
[Bug]
If using mkfs.btrfs with "-r" parameter and specified directory has
fifo/socket/char/block special file, then created filesystem can't pass
fsck:

------
checking fs roots
	unresolved ref dir 241158 index 3 namelen 9 name S.dirmngr filetype 0 errors 80, filetype mismatch
ERROR: errors found in fs roots
------

[Reason]
Btrfs dir items/indexes records inode type, while "-r" only handles
directories, regular files and symlink, it makes such special files type
to be regular file and caused the problem.

[Fix]
Add missing types for add_directory_items(), so that result of
"mkfs.btrfs -r" can pass mkfs.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Gu Jinxiang
944e14da14 btrfs-progs: mkfs: Move the tree root creation to own function
make_btrfs is too long to understand, make creatation of root tree
in a function.

Some of the tree roots are now created in a loop, where the code is just
copypasted. We now make use of the reference_root_table to translate
block index to root objectid.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ updated changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba
448999d84d btrfs-progs: add crude error handling when transaction start fails
Currently transaction bugs out insided btrfs_start_transaction in case
of error, we want to lift the error handling to the callers. This patch
adds the BUG_ON anywhere it's been missing so far. This is not the best
way of course. Transforming BUG_ON to a proper error handling highly
depends on the caller and should be dealt with case by case.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Nikolay Borisov
cbaa70b265 btrfs-progs: Use named constants for common sizes
There multiple places where we use well-known sizes - 1,8,16,32 megabytes. We
also have them defined as constants in the sizes.h header. So let's use them.
No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Gu Jinxiang
c0d783347f btrfs-progs: mkfs: Replace number with enum
For code maintainability and scalability,
replace hardcoded constant with a meaningful enum.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ add MKFS_ prefix ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Gu Jinxiang
6450314980 btrfs-progs: mkfs: Make in-place exit to a common exit block
Replace in-place exit with a common exit block in the main function.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-08-24 19:06:51 +02:00
Gu Jinxiang
4b54ca8ba4 btrfs-progs: mkfs: delete un-used parameter fd
Parameter fd is not used in function make_image and traverse_directory
of mkfs.  Delete it.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-08-24 19:06:46 +02:00
Qu Wenruo
0544aafcbf btrfs-progs: Refactor chunk creation functions to use btrfs_fs_info
4 functions are involved in this refactor: btrfs_make_block_group()
btrfs_make_block_groups(), btrfs_alloc_chunk, btrfs_alloc_data_chunk().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:54:16 +02:00
Qu Wenruo
790b5950f3 btrfs-progs: Refactor write_and_map_eb to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:52:49 +02:00
Qu Wenruo
b0f14e9b3d btrfs-progs: Refactor sectorsize users in mkfs/main.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
2017-07-03 13:35:10 +02:00
Qu Wenruo
3c7a04e592 btrfs-progs: Remove deprecated leafsize usage
Leafsize is deprecated for a long time, and kernel has already updated
ctree.h to rename sb->leafsize to sb->__unused_leafsize.

This patch will remove normal users of leafsize:
1) Remove leafsize member from btrfs_root structure
   Now only root->nodesize and root->sectorisze.
   No longer root->leafsize.

2) Remove @leafsize parameter from btrfs_setup_root() function
   Since no root->leafsize, no need for @leafsize parameter.

The remaining user of leafsize will be:
1) btrfs inspect-internal dump-super
   Reformat the "leafsize" output to "leafsize (deprecated)" and
   use le32_to_cpu() to do the cast manually.

2) mkfs
   We still need to set sb->__unused_leafsize to nodesize.
   Do the manual cast too.

3) convert
   Same as mkfs, these two superblock setup should be merged later

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
2017-07-03 13:35:10 +02:00
Baruch Siach
153d4e6c01 btrfs-progs: add missing limits headers
Build under musl libc fails because of missing PATH_MAX and XATTR_NAME_MAX
macro declarations. Add the required headers.

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
2017-03-16 17:02:44 +01:00
David Sterba
b8888cb176 btrfs-progs: mkfs: remove unused argument from add_file_items
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:48 +01:00
David Sterba
d9d9314e12 btrfs-progs: mkfs: remove unused arguments from add_inode_items
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:48 +01:00
David Sterba
b72df6551b btrfs-progs: mkfs: remove unused argument from make_root_dir
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:48 +01:00
David Sterba
e1a67a59a9 btrfs-progs: remove unused argument from set_extent_dirty
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:47 +01:00
David Sterba
b53af22c24 btrfs-progs: remove unused argument from write_and_map_eb
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:47 +01:00
David Sterba
d1d99f50e3 btrfs-progs: mkfs: use const char for label
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba
6783002cd3 btrfs-progs: mkfs: make make_cfg::blocks an internal member
The caller of make_btrfs does not need to set it, it's for internal use
and can be read after mkfs ends.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba
a2ed85a11d btrfs-progs: mkfs: describe fields of btrfs_mkfs_config
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba
ed4c7428b7 btrfs-progs: move mkfs helper implementation out of utils
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba
11c83cefb8 btrfs-progs: move more mkfs declarations to the common header
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:46 +01:00
David Sterba
14f9565c11 btrfs-progs: move fs features declarations to own header from utils
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:45 +01:00
David Sterba
ef9ddf7a5c btrfs-progs: mkfs: move common api implementation to own file
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:45 +01:00
David Sterba
ad7c469ac3 btrfs-progs: move mkfs definitions to own header
Create a header for filesystem creation API, the config and the main
entry function.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:45 +01:00
David Sterba
1c880f34f1 btrfs-progs: move help defines to own header
Signed-off-by: David Sterba <dsterba@suse.com>
2017-03-08 13:00:45 +01:00
David Sterba
c855e60f7e btrfs-progs: mkfs/convert: separate the convert part from make_btrfs
The regulare mkfs_btrfs does not anything special about convert and just
passes the arguments. Make that two functions.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-01-27 12:20:43 +01:00
Nicholas D Steeves
b757cf4ba7 btrfs-progs: Fix spelling/typos in user-facing strings
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-01-25 09:47:40 +01:00
David Sterba
a0ac307b0c btrfs-progs: mkfs: annotation of a case
Annotate to silence static analysis warning.

Resolves-coverity-id: 1396656
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:35 +01:00
David Sterba
088ed0e557 btrfs-progs: move mkfs.btrfs sources to own directory
Signed-off-by: David Sterba <dsterba@suse.com>
2016-12-14 15:06:35 +01:00