750 lines
14 KiB
ReStructuredText
750 lines
14 KiB
ReStructuredText
Send stream format
|
|
==================
|
|
|
|
Send stream format represents a linear sequence of commands describing actions
|
|
to be performed on the target filesystem (receive side), created on the source
|
|
filesystem (send side). The stream is currently used in two ways: to generate a
|
|
stream representing a standalone subvolume (full mode) or a difference between
|
|
two snapshots of the same subvolume (incremental mode). The stream can be
|
|
generated using a set of other subvolumes to look for extent references that
|
|
could lead to a more efficient stream by transferring only the references and
|
|
not full data.
|
|
|
|
The stream format is abstracted from on-disk structures (though it may share
|
|
some BTRFS specifics), the stream instructions could be generated by other
|
|
means than the send ioctl.
|
|
|
|
Data types
|
|
----------
|
|
|
|
Raw data types. Integer values are stored in little endian byte order.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Meaning
|
|
- Size
|
|
- Name
|
|
* - unsigned int
|
|
- 8 bit
|
|
- u8
|
|
* - unsigned int
|
|
- 16 bit
|
|
- u16
|
|
* - unsigned int
|
|
- 32 bit
|
|
- u32
|
|
* - unsigned int
|
|
- 64 bit
|
|
- u64
|
|
* - variable length binary data
|
|
- variable
|
|
- data
|
|
* - variable length string
|
|
- variable
|
|
- string
|
|
* - UUID
|
|
- 16 bytes
|
|
- uuid
|
|
* - time specification
|
|
- 64bit seconds, 32bit nanoseconds
|
|
- timespec
|
|
|
|
Stream structure
|
|
----------------
|
|
|
|
The stream starts with a descriptor bytes ``btrfs-stream`` followed by version
|
|
stored as little endian u32. Then the sequence of commands starts. The expected
|
|
start is a subvolume or snapshot followed by commands to change the data.
|
|
|
|
Command structure
|
|
-----------------
|
|
|
|
- u32 length
|
|
- u16 command type
|
|
- u32 checksum (CRC32C)
|
|
|
|
Note: the checksum initial seed is not 0xFFFFFFFF but 0x0. That is a slight
|
|
mistake and not the recommended way, overlooked when the protocol was
|
|
implemented. This does not have a big impact though.
|
|
|
|
Command data are structured as sequence of type-length-value (TLV):
|
|
|
|
- TLV type
|
|
- TLV length of the data
|
|
- raw data
|
|
|
|
The TLVs are stored in the command data sequentially and the order is not
|
|
mandatory, expected data for each command is looked up based on the type.
|
|
Unrecognized TLV type may be ignored on the receive side. In the following
|
|
documentation the TLVs are listed in the order as they're emitted by kernel
|
|
side generating the stream.
|
|
|
|
Subvolumes from the source and target filesystems are matched by the UUIDs.
|
|
|
|
All file or directory changes are matched by path, inode numbers may be different
|
|
on the source and target filesystems but are present in the TLVs.
|
|
|
|
Attributes, TLV types
|
|
---------------------
|
|
|
|
The TLV types are also called attributes of the command. The raw data type is
|
|
one of the Data types listed above, the actual named type may be a generic one
|
|
(like PATH) or specific for set of command (like CLONE_OFFSET).
|
|
|
|
The attributes represent the same raw data type when used in various commands,
|
|
though this is not strictly necessary.
|
|
|
|
Version 1
|
|
^^^^^^^^^
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Name
|
|
- Number
|
|
- Type
|
|
* - BTRFS_SEND_A_UNSPEC
|
|
- 0
|
|
- invalid
|
|
* - BTRFS_SEND_A_UUID
|
|
- 1
|
|
- uuid
|
|
* - BTRFS_SEND_A_CTRANSID
|
|
- 2
|
|
- u64
|
|
* - BTRFS_SEND_A_INO
|
|
- 3
|
|
- u64
|
|
* - BTRFS_SEND_A_SIZE
|
|
- 4
|
|
- u64
|
|
* - BTRFS_SEND_A_MODE
|
|
- 5
|
|
- u64
|
|
* - BTRFS_SEND_A_UID
|
|
- 6
|
|
- u64
|
|
* - BTRFS_SEND_A_GID
|
|
- 7
|
|
- u64
|
|
* - BTRFS_SEND_A_RDEV
|
|
- 8
|
|
- u64
|
|
* - BTRFS_SEND_A_CTIME
|
|
- 9
|
|
- timespec
|
|
* - BTRFS_SEND_A_MTIME
|
|
- 10
|
|
- timespec
|
|
* - BTRFS_SEND_A_ATIME
|
|
- 11
|
|
- timespec
|
|
* - BTRFS_SEND_A_OTIME
|
|
- 12
|
|
- timespec
|
|
* - BTRFS_SEND_A_XATTR_NAME
|
|
- 13
|
|
- string
|
|
* - BTRFS_SEND_A_XATTR_DATA
|
|
- 14
|
|
- data
|
|
* - BTRFS_SEND_A_PATH
|
|
- 15
|
|
- string
|
|
* - BTRFS_SEND_A_PATH_TO
|
|
- 16
|
|
- string
|
|
* - BTRFS_SEND_A_PATH_LINK
|
|
- 17
|
|
- string
|
|
* - BTRFS_SEND_A_FILE_OFFSET
|
|
- 18
|
|
- u64
|
|
* - BTRFS_SEND_A_DATA
|
|
- 19
|
|
- data
|
|
* - BTRFS_SEND_A_CLONE_UUID
|
|
- 20
|
|
- uuid
|
|
* - BTRFS_SEND_A_CLONE_CTRANSID
|
|
- 21
|
|
- u64
|
|
* - BTRFS_SEND_A_CLONE_PATH
|
|
- 22
|
|
- string
|
|
* - BTRFS_SEND_A_CLONE_OFFSET
|
|
- 23
|
|
- u64
|
|
* - BTRFS_SEND_A_CLONE_LEN
|
|
- 24
|
|
- u64
|
|
|
|
Version 2
|
|
^^^^^^^^^
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Name
|
|
- Number
|
|
- Type
|
|
* - BTRFS_SEND_A_FALLOCATE_MODE
|
|
- 25
|
|
- u32
|
|
* - BTRFS_SEND_A_FILEATTR
|
|
- 26
|
|
- u64
|
|
* - BTRFS_SEND_A_UNENCODED_FILE_LEN
|
|
- 27
|
|
- u64
|
|
* - BTRFS_SEND_A_UNENCODED_LEN
|
|
- 28
|
|
- u64
|
|
* - BTRFS_SEND_A_UNENCODED_OFFSET
|
|
- 29
|
|
- u64
|
|
* - BTRFS_SEND_A_COMPRESSION
|
|
- 30
|
|
- u32
|
|
* - BTRFS_SEND_A_ENCRYPTION
|
|
- 31
|
|
- u32
|
|
|
|
Special cases
|
|
-------------
|
|
|
|
File creation (MKFILE) is done in two phases, where first a file with special
|
|
name like *o-123-45678* is created and then renamed (RENAME) to the target
|
|
location.
|
|
|
|
Parent directory where file changes are currently done gets update of times
|
|
(UTIMES) after each operation that affects it. This is not optimal as only the
|
|
last one would be needed.
|
|
|
|
Raw data type is processed in a different way in protocol version 1 and 2. In 1
|
|
the maximum size of the TLV is 64KiB as the size is stored in u16. This is not
|
|
sufficient for encoded write (ENCODED_WRITE) command. In 2 the data length is
|
|
up to 4GiB (using the type u32) but the TLV must be last and the actual
|
|
length is calculated as the delta between the whole command and the TLV (i.e.
|
|
ignoring the TLV header length).
|
|
|
|
Stream version 1
|
|
----------------
|
|
|
|
BTRFS_SEND_C_UNSPEC (0)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Placeholder, invalid or ignored command.
|
|
|
|
BTRFS_SEND_C_SUBVOL (1)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative path of the subvolume
|
|
* - uuid
|
|
- uuid
|
|
- UUID of the sent subvolume
|
|
* - u64
|
|
- ctransid
|
|
- creation transaction
|
|
|
|
BTRFS_SEND_C_SNAPSHOT (2)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Start of commands of a given snapshot.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative path of the subvolume
|
|
* - uuid
|
|
- uuid
|
|
- UUID of the sent subvolume
|
|
* - u64
|
|
- ctransid
|
|
- creation transaction
|
|
* - uuid
|
|
- clone_uuid
|
|
-
|
|
* - u64
|
|
- clone_ctransid
|
|
-
|
|
|
|
BTRFS_SEND_C_MKFILE (3)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create regular file. See also section Special cases.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path to create
|
|
* - u64
|
|
- ino
|
|
- inode number
|
|
|
|
BTRFS_SEND_C_MKDIR (4)
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create a directory.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative directory path to create
|
|
* - u64
|
|
- ino
|
|
- inode number
|
|
|
|
File creation is done in two commands, the first one contains a special file name
|
|
that is later renamed to the final name. (WHY)
|
|
|
|
BTRFS_SEND_C_MKNOD (5)
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create a special file of type device node (mknod).
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path to create
|
|
* - u64
|
|
- mode
|
|
- file mode parameter of mknod(2)
|
|
* - u64
|
|
- rdev
|
|
- rdev parameter of mknod(2)
|
|
|
|
BTRFS_SEND_C_MKFIFO (6)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create a special file of type FIFO (mkfifo).
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path to create
|
|
* - u64
|
|
- ino
|
|
- inode number
|
|
|
|
BTRFS_SEND_C_MKSOCK (7)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create a special file of type socket (mknod S_IFSOCK).
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path to create
|
|
* - u64
|
|
- ino
|
|
- inode number
|
|
|
|
BTRFS_SEND_C_SYMLINK (8)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create a symlink.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative symlink path to create
|
|
* - u64
|
|
- ino
|
|
- inode number
|
|
* - string
|
|
- path_link
|
|
- target of the symlink
|
|
|
|
BTRFS_SEND_C_RENAME (9)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Rename file path.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative source file path
|
|
* - string
|
|
- path_to
|
|
- relative target file path
|
|
|
|
BTRFS_SEND_C_LINK (10)
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Create a file hardlink.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative source file path
|
|
* - string
|
|
- path_link
|
|
- relative target file path to link to
|
|
|
|
BTRFS_SEND_C_UNLINK (11)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Unlink file.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
|
|
BTRFS_SEND_C_RMDIR (12)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Remove directory.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative directory path
|
|
|
|
BTRFS_SEND_C_SET_XATTR (13)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Set a value of extended attribute.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - string
|
|
- xattr_name
|
|
- name of the extended attribute
|
|
* - data
|
|
- xattr_data
|
|
- value of the extended attribute
|
|
|
|
BTRFS_SEND_C_REMOVE_XATTR (14)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Remove an extended attribute.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - string
|
|
- xattr_name
|
|
- name of the extended attribute
|
|
|
|
BTRFS_SEND_C_WRITE (15)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Write file data to a given file offset.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- file_offset
|
|
- where to write data
|
|
* - data
|
|
- data
|
|
- raw file data (variable length)
|
|
|
|
BTRFS_SEND_C_CLONE (16)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Clone extents from another file.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- file_offset
|
|
- offset in the source file to clone from
|
|
* - u64
|
|
- clone_len
|
|
- length of cloned data
|
|
* - uuid
|
|
- clone_uuid
|
|
-
|
|
* - u64
|
|
- clone_ctransid
|
|
-
|
|
* - string
|
|
- clone_path
|
|
- clone target relative file path
|
|
* - u64
|
|
- clone_offset
|
|
- clone offset in the target file
|
|
|
|
BTRFS_SEND_C_TRUNCATE (17)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Truncate file to a given length.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- size
|
|
- truncate to given size
|
|
|
|
BTRFS_SEND_C_CHMOD (18)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Chmod a file or directory.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- mode
|
|
- new mode
|
|
|
|
BTRFS_SEND_C_CHOWN (19)
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Change file owner (uid) and group (gid), specified by numeric id. The uid/gid
|
|
must exist on the target filesystem, no mapping is done.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- uid
|
|
- numeric used id
|
|
* - u64
|
|
- gid
|
|
- numeric group id
|
|
|
|
BTRFS_SEND_C_UTIMES (20)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Change file atime and mtime, nanosecond precision. While the ctime is also sent
|
|
it's not possible to change it using *utimensat*. The creation time is sent
|
|
since protocol version 2 but cannot be changed on the target filesystem.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - timespec
|
|
- atime
|
|
- file atime
|
|
* - timespec
|
|
- mtime
|
|
- file mtime
|
|
* - timespec
|
|
- ctime
|
|
- file ctime
|
|
* - timespec
|
|
- otime
|
|
- (since v2) file otime (creation time)
|
|
|
|
BTRFS_SEND_C_END (21)
|
|
^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Special command to denote end of one logical stream inside the whole stream
|
|
sequence. May or may not be processed by receiver.
|
|
|
|
BTRFS_SEND_C_UPDATE_EXTENT (22)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
When send is generated without data (BTRFS_SEND_FLAG_NO_FILE_DATA), this command
|
|
informs about changed extent but does not send the actual data.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- file_offset
|
|
- file offset where data were updated
|
|
* - u64
|
|
- size
|
|
- length of the data
|
|
|
|
Stream version 2
|
|
----------------
|
|
|
|
BTRFS_SEND_C_FALLOCATE (23)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Change file extents to preallocated, punch hole or zero fill.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u32
|
|
- fallocate_mode
|
|
- which fallocate operation to do
|
|
* - u64
|
|
- file_offset
|
|
- file offset where to apply the operation
|
|
* - u64
|
|
- size
|
|
- length of the range
|
|
|
|
BTRFS_SEND_C_FILEATTR (24)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
File attributes, representing various flags (SETFLAGS ioctl, XFLAGS, BTRFS
|
|
specific inode flags). The value is set from BTRFS inode bits and the stream
|
|
format inherits that. Note that some flags like IMMUTABLE or APPEND may affect
|
|
ability to change other flags and that for some flags there's ready interface
|
|
to set them.
|
|
|
|
BTRFS_SEND_C_ENCODED_WRITE (25)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
File data encoded by the source filesystem and written directly to the target
|
|
filesystem, without any other transformation. The data can be compressed or
|
|
encrypted and the payload depends on presence of the TLVs.
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
|
|
* - Type
|
|
- Name
|
|
- Description
|
|
* - string
|
|
- path
|
|
- relative file path
|
|
* - u64
|
|
- file_offset
|
|
- file offset where to write the data
|
|
* - u64
|
|
- unencoded_file_len
|
|
-
|
|
* - u64
|
|
- unencoded_len
|
|
-
|
|
* - u64
|
|
- unencoded_offset
|
|
-
|
|
* - u32
|
|
- compression
|
|
- (optional) compression type
|
|
* - u32
|
|
- encryption
|
|
- (optional) encryption type
|
|
* - data
|
|
- data
|
|
- encoded payload
|