David Woodhouse originally contributed this code, and Chris Mason
changed it around to reflect the current design goals for raid56.
The original code expected all metadata and data writes to be full
stripes. This meant metadata block size == stripe size, and had a few
other restrictions.
This version allows metadata blocks smaller than the stripe size. It
implements both raid5 and raid6, although it does not have code to
rebuild from parity if one of the drives is missing or incorrect.
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
This patch turns on the BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF superblock flag
when creating a new file system in mkfs, enabling extended inode refs.
Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Commit 605e806166 broke the
mkfs.btrfs -r option, because it calls make_btrfs
without ever setting dev_block_count, in the -r case,
so we tell it to make a filesystem of size 0.
Then we wander into ENOSPC land and segfault.
As a quick one-line-fix, just set the dev_block_count
to the size of the destination image file.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
This patch kills a check in mkfs's label stuff which doesn't allow labels that
have /'s in them. This causes problems for Anaconda which try to label volumes
with their mountpoints. Thanks,
Signed-off-by: Josef Bacik <jbacik@redhat.com>
Patch rebased because of changes in mkfs.c but otherwise the same
as created by Josef Bacik
SSD's do not gain anything by having metadata DUP turned on. The underlying
file system that is a part of all SSD's could easily map duplicate metadat
blocks into the same erase block which effectively eliminates the benefit of
duplicating the metadata on disk. So detect if we are formatting a single
SSD drive and if we are do not use DUP. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Gene Czarcinski <gene@czarc.net>
Rawhide is getting cranky with posix compliance, and a few
things have stopped building.
getpagesize() is now only available -with- __USE_XOPEN_EXTENDED
or __USE_BSD, and NOT __USE_XOPEN2K.
_GNU_SOURCE must define __USE_XOPEN2K because getpagesize()
has gone away for mkfs. I gave up and used sysconf.
Also, something used to pull in stat that no longer does, so
things like S_ISREG weren't getting defined.
The following fixes things for me.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
The kernel uses unsigned long long for u64, but PPC64 uses unsigned
long by default. This results in compilation warnings such as:
print-tree.c:333: warning: format '%llu' expects type 'long long
unsigned int', but argument 4 has type 'u64'
To fix this, the macro __KERNEL__ needs to be defined before including
the file <asm/types.h>. This can be done by defining the macro in
"kerncompat.h" and making it the first included file in the relevant
header files; this fixes the compiler warnings on PPC64.
Reviewed-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Wade Cline <clinew@linux.vnet.ibm.com>
Using mkfs.btrfs like:
mkfs.btrfs -l 131072 /dev/sda
will return no error, but after mount it, the dmesg will report:
BTRFS: couldn't mount because metadata blocksize (131072) was too large
The leafsize and nodesize are equal at present, so we just use one function
"check_leaf_or_node_size" to limit leaf and node size below BTRFS_MAX_METADATA_BLOCKSIZE.
Signed-off-by: Robin Dong <sanbai@taobao.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
My patch
04609add88
introduced a regression where if you mkfs'ed a group of disks with different
sizes it limited the disks to the size of the first one that is specified.
This was not the intent of my patch, I only want it to limit the size based
on the -b option, so I've reworked the code to pass in a max block count and
that fixes the issue. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
I had a test that creates a 7gig raid1 device but it was ending up wonky
because the second device that gets added is the full size of the disk
instead of the limited size. So enforce the limited size on all disks
passed in at mkfs time, otherwise our threshold calculations end up wonky
when doing chunk allocations. Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
On Wed 08-02-12 22:05:26, Phillip Susi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 02/08/2012 06:20 PM, Jan Kara wrote:
> > Thanks for your reply. I admit I was not sure what exactly size argument
> > should be. So after looking into the code for a while I figured it should
> > be a total size of the filesystem - or differently it should be size of
> > virtual block address space in the filesystem. Thus when filesystem has
> > more devices (or admin wants to add more devices later), it can be larger
> > than the first device. But I'm not really a btrfs developper so I might be
> > wrong and of course feel free to fix the issue as you deem fit.
>
> The size of the fs is the total size of the individual disks. When you
> limit the size, you limit the size of a disk, not the whole fs. IIRC,
> mkfs initializes the fs on the first disk, which is why it was using that
> size as the size of the whole fs, and then adds the other disks after (
> which then add their size to the total fs size ).
OK, I missed that btrfs_add_to_fsid() increases total size of the
filesystem. So now I agree with you. New patch is attached. Thanks for your
review.
> It might be nice if
> mkfs could take sizes for each disk, but it only seems to take one size
> for the initial disk.
Yes, but I don't see a realistic usecase so I don't think it's really
worth the work.
Honza
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
>From e5f46872232520310c56327593c02ef6a7f5ea33 Mon Sep 17 00:00:00 2001
From: Jan Kara <jack@suse.cz>
Date: Fri, 10 Feb 2012 11:44:44 +0100
Subject: [PATCH] mkfs: Handle creation of filesystem larger than the first device
mkfs does not properly check requested size of the filesystem. Thus if the
requested size is larger than the first device, it happily creates larger
filesystem than a device it resides on which results in 'attemp to access
beyond end of device' messages from the kernel. So verify specified filesystem
size against the size of the first device.
CC: David Sterba <dsterba@suse.cz>
Signed-off-by: Jan Kara <jack@suse.cz>
* mkfs.c (parse_size): ./mkfs.btrfs -A '' would read and possibly
write the byte before beginning of strdup'd heap buffer. All other
size-accepting options were similarly affected.
Reviewed-by: Josef Bacik <josef@redhat.com>
We don't allow different leaf and node blocksizes, so
this just makes the two options mean the same thing
Signed-off-by: Chris Mason <chris.mason@oracle.com>
Before commit a46e7ff2 was merged it was possible to create dup for
data+metadata chunks (mixed mode) by giving -m raid1 -d raid1 -M to
mkfs. a46e7ff2 purposefully disabled behind the scenes profile
upgrading/downgrading, so give users a chance to pick dup explicitly and
bail if dup for data is requested in normal mode.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Currently mkfs in response to
mkfs.btrfs -d raid10 dev1 dev2
instead of telling "you can't do that" creates a SINGLE on two devices,
and only rebalance can transform it to raid0. Generally, it never warns
users about decisions it makes and it's not at all obvious which profile
it picks when.
Fix this by checking the number of effective devices and reporting back
if the specified profile is impossible to create. Do not create FS in
case invalid profile was given.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* Initialize ret in btrfs_csum_file_block
* Do not abort when xattr is not supported in the source directory
* Remove size limitation of 256M
* Alloc data chunk in a smaller size (8M) to make btrfs image smaller
* Let user specify the btrfs image name
Depends on below patch from samsung guys:
http://marc.info/?l=linux-btrfs&m=127858068226025&w=2
Signed-off-by: Zhong, Xin <xin.zhong@intel.com>
Hello,
While going through the mkfs.c, I noticed there is an issue for label
length checking, mkfs.btrfs will crashed if the label length exceeding
255 bytes, it's easy to triggered that out as below:
jeff@pibroch:~/opensource/btrfs-progs$ sudo ./mkfs.btrfs -L `perl -e
'print "A"x256'` /usr/src/linux-3.0/img0
WARNING! - Btrfs v0.19-35-g1b444cd IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using
*** buffer overflow detected ***: ./mkfs.btrfs terminated
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(__fortify_fail+0x50)[0xb7774df0]
/lib/i386-linux-gnu/libc.so.6(+0xe4cca)[0xb7773cca]
/lib/i386-linux-gnu/libc.so.6(__strcpy_chk+0x3f)[0xb777305f]
./mkfs.btrfs[0x805acc4]
./mkfs.btrfs[0x805def6]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0xb76a5e37]
./mkfs.btrfs[0x8048ef1]
======= Memory map: ========
......
a tiny patch could fix it.
Signed-off-by: Jie Liu <jeff.liu@oracle.com>
gcc 4.6 complains about several possible use-before-initialise cases
in mkfs, and stops. Fix these by initialising one of the variables in
question, and using the correct error-handling paths for the
remainder.
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Smart gcc noticed use of uninitialized warning when compiled
with -O0 flags:
mkfs.c:1291: error: 'file' may be used uninitialized in this function
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
found by valgrind:
==2559== 16 bytes in 1 blocks are definitely lost in loss record 3 of 19
==2559== at 0x4C2720E: malloc (vg_replace_malloc.c:236)
==2559== by 0x412F7E: pretty_sizes (utils.c:1054)
==2559== by 0x4179E9: main (mkfs.c:1395)
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Found by valgrind:
==8968== Use of uninitialised value of size 8
==8968== at 0x41CE7D: crc32c_le (crc32c.c:98)
==8968== by 0x40A1D0: csum_tree_block_size (disk-io.c:82)
==8968== by 0x40A2D4: csum_tree_block (disk-io.c:105)
==8968== by 0x40A7D6: write_tree_block (disk-io.c:241)
==8968== by 0x40ACEE: __commit_transaction (disk-io.c:354)
==8968== by 0x40AE9E: btrfs_commit_transaction (disk-io.c:385)
==8968== by 0x42CF66: make_image (mkfs.c:1061)
==8968== by 0x42DE63: main (mkfs.c:1410)
==8968== Uninitialised value was created by a stack allocation
==8968== at 0x42B5FB: add_inode_items (mkfs.c:493)
1. On-disk inode format has reserved (and thus, random at alloc time) fields:
btrfs_inode_item: __le64 reserved[4]
2. Sometimes extents are created on disk without writing data there.
(Or at least not all data is written there). Kernel code always had
it kzalloc'ed.
Zero them all.
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Found by valgrind:
==8968== Use of uninitialised value of size 8
==8968== at 0x41CE7D: crc32c_le (crc32c.c:98)
==8968== by 0x40A1D0: csum_tree_block_size (disk-io.c:82)
==8968== by 0x40A2D4: csum_tree_block (disk-io.c:105)
==8968== by 0x40A7D6: write_tree_block (disk-io.c:241)
==8968== by 0x40ACEE: __commit_transaction (disk-io.c:354)
==8968== by 0x40AE9E: btrfs_commit_transaction (disk-io.c:385)
==8968== by 0x42CF66: make_image (mkfs.c:1061)
==8968== by 0x42DE63: main (mkfs.c:1410)
==8968== Uninitialised value was created by a stack allocation
==8968== at 0x42B5FB: add_inode_items (mkfs.c:493)
readlink(2) does not write '\0' for us, so make it manually.
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
mkfs.btrfs does not handle relative pathnames for now. When
they are passed to it it creates empty image. So first time
I thought it does not work at all.
This patch adds error handling for scandir(). With patch it behaves
this way:
$ mkfs.btrfs -r ./root
...
fs created label (null) on output.img
nodesize 4096 leafsize 4096 sectorsize 4096 size 256.00MB
Btrfs v0.19-52-g438c5ff-dirty
scandir for ./root failed: No such file or directory
unable to traverse_directory
Making image is aborted.
mkfs.btrfs: mkfs.c:1402: main: Assertion `!(ret)' failed.
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
gcc-4.6 (as shipped in Fedora) turns on -Wunused-but-set-variable by
default, which breaks the build when combined with -Wall, e.g.:
debug-tree.c: In function ‘print_extent_leaf’:
debug-tree.c:45:13: error: variable ‘last_len’ set but not used [-Werror=unused-but-set-variable]
debug-tree.c:44:13: error: variable ‘last’ set but not used [-Werror=unused-but-set-variable]
debug-tree.c:41:21: error: variable ‘item’ set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors
This patch fixes the errors by removing the unused variables.
Signed-off-by: Chris Ball <cjb@laptop.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Changes from V1 to V2:
- support extended attributes
- move btrfs_alloc_data_chunk function to volumes.c
- fix an execution error when additional useless parameters are specified
- fix traverse_directory function so that the insertion functions for the common items are invoked in a single point
The extended attributes is implemented through llistxattr and getxattr function calls.
Thanks
Signed-off-by: Donggeun Kim <dg77.kim@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
So alot of crazy people (I'm looking at you Meego) want to use btrfs on phones
and such with small devices. Unfortunately the way we split out metadata/data
chunks it makes space usage inefficient for volumes that are smaller than
1gigabyte. So add a -M option for mixing metadata+data, and default to this
mixed mode if the filesystem is less than or equal to 1 gigabyte. I've tested
this with xfstests on a 100mb filesystem and everything is a-ok.
Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.
The new back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer by
searching the tree. The shortcoming of the new back ref is that it only works
for pointers in tree blocks referenced by their owner trees.
This is mostly a problem for snapshots, where resolving one of these fuzzy back
references would be O(number_of_snapshots) and quite slow. The solution used
here is to use the fuzzy back references in the common case where a given tree
block is only referenced by one root, and use the full back references when
multiple roots have a reference
There are still some warnings of the form:
format '%llu' expects type 'long long unsigned int' but argument has type 'u64'
In conjunction with -Werror, this is causing some build failures.
Now they're properly casted, avoiding compiler warnings.
Signed-off-by: Luca Bruno <lucab@debian.org>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
mkfs.btrfs now prints its version when invoked with -V|--version
and exits without error.
All other mkfs.* tools provide this feature and follow this
implicit argument naming convention, as it is commonly used to
check for helper tools presence.
The corrisponding manual already mentions this option, no need to
touch it.
Signed-off-by: Luca Bruno <lucab@debian.org>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
This patch updates the ext3 to btrfs converter for the new
disk format. This mainly involves changing the convert's
data relocation and free space management code. This patch
also ports some functions from kernel module to btrfs-progs.
Thank you,
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
This patch updates btrfs-progs for superblock duplication.
Note: I didn't make this patch as complete as the one for
kernel since updating the converter requires changing the
code again. Thank you,
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
Btrfs stores checksums for each data block. Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block. This means that when we read the inode,
we've probably read in at least some checksums as well.
But, this has a few problems:
* The checksums are indexed by logical offset in the file. When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data. It would be faster if we could checksum
the compressed data instead.
* If we implement encryption, we'll be checksumming the plain text and
storing that on disk. This is significantly less secure.
* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct. This makes the raid
layer balancing and extent moving much more expensive.
* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.
* There is potentitally one copy of the checksum in each subvolume
referencing an extent.
The solution used here is to store the extent checksums in a dedicated
tree. This allows us to index the checksums by phyiscal extent
start and length. It means:
* The checksum is against the data stored on disk, after any compression
or encryption is done.
* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.
This makes compression significantly faster by reducing the amount of
data that needs to be checksummed. It will also allow much faster
raid management code in general.
The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent. This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
This patch adds transaction IDs to root tree pointers.
Transaction IDs in tree pointers are compared with the
generation numbers in block headers when reading root
blocks of trees. This can detect some types of IO errors.
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
The new space balancing code needs a subvol to store the
temporary inode for data extent relocation.
Signed-off-by: Chris Mason <chris.mason@oracle.com>
This patch makes the back reference system to explicit record the
location of parent node for all types of extents. The location of
parent node is placed into the offset field of backref key. Every
time a tree block is balanced, the back references for the affected
lower level extents are updated.
