btrfs-progs/Documentation/dev/dev-send-stream.rst

Send stream format
==================

Send stream format represents a linear sequence of commands describing actions
to be performed on the target filesystem (receive side), created on the source
filesystem (send side). The stream is currently used in two ways: to generate a
stream representing a standalone subvolume (full mode) or a difference between
two snapshots of the same subvolume (incremental mode). The stream can be
generated using a set of other subvolumes to look for extent references that
could lead to a more efficient stream by transferring only the references and
not full data.

The stream format is abstracted from on-disk structures (though it may share
some BTRFS specifics), the stream instructions could be generated by other
means than the send ioctl.

Data types
----------

Raw data types. Integer values are stored in little endian byte order.

.. list-table::
   :header-rows: 1

   * - Meaning
     - Size
     - Name
   * - unsigned int
     - 8 bit
     - u8
   * - unsigned int
     - 16 bit
     - u16
   * - unsigned int
     - 32 bit
     - u32
   * - unsigned int
     - 64 bit
     - u64
   * - variable length binary data
     - variable
     - data
   * - variable length string
     - variable
     - string
   * - UUID
     - 16 bytes
     - uuid
   * - time specification
     - 64bit seconds, 32bit nanoseconds
     - timespec

Stream structure
----------------

The stream starts with a descriptor bytes ``btrfs-stream`` followed by version
stored as little endian u32. Then the sequence of commands starts. The expected
start is a subvolume or snapshot followed by commands to change the data.

Command structure
-----------------

- u32 length
- u16 command type
- u32 checksum (CRC32C)

Note: the checksum initial seed is not 0xFFFFFFFF but 0x0. That is a slight
mistake and not the recommended way, overlooked when the protocol was
implemented. This does not have a big impact though.

Command data are structured as sequence of type-length-value (TLV):

- TLV type
- TLV length of the data
- raw data

The TLVs are stored in the command data sequentially and the order is not
mandatory, expected data for each command is looked up based on the type.
Unrecognized TLV type may be ignored on the receive side. In the following
documentation the TLVs are listed in the order as they're emitted by kernel
side generating the stream.

Subvolumes from the source and target filesystems are matched by the UUIDs.

All file or directory changes are matched by path, inode numbers may be different
on the source and target filesystems but are present in the TLVs.

Attributes, TLV types
---------------------

The TLV types are also called attributes of the command. The raw data type is
one of the Data types listed above, the actual named type may be a generic one
(like PATH) or specific for set of command (like CLONE_OFFSET).

The attributes represent the same raw data type when used in various commands,
though this is not strictly necessary.

Version 1
^^^^^^^^^

.. list-table::
   :header-rows: 1

   * - Name
     - Number
     - Type
   * - BTRFS_SEND_A_UNSPEC
     - 0
     - invalid
   * - BTRFS_SEND_A_UUID
     - 1
     - uuid
   * - BTRFS_SEND_A_CTRANSID
     - 2
     - u64
   * - BTRFS_SEND_A_INO
     - 3
     - u64
   * - BTRFS_SEND_A_SIZE
     - 4
     - u64
   * - BTRFS_SEND_A_MODE
     - 5
     - u64
   * - BTRFS_SEND_A_UID
     - 6
     - u64
   * - BTRFS_SEND_A_GID
     - 7
     - u64
   * - BTRFS_SEND_A_RDEV
     - 8
     - u64
   * - BTRFS_SEND_A_CTIME
     - 9
     - timespec
   * - BTRFS_SEND_A_MTIME
     - 10
     - timespec
   * - BTRFS_SEND_A_ATIME
     - 11
     - timespec
   * - BTRFS_SEND_A_OTIME
     - 12
     - timespec
   * - BTRFS_SEND_A_XATTR_NAME
     - 13
     - string
   * - BTRFS_SEND_A_XATTR_DATA
     - 14
     - data
   * - BTRFS_SEND_A_PATH
     - 15
     - string
   * - BTRFS_SEND_A_PATH_TO
     - 16
     - string
   * - BTRFS_SEND_A_PATH_LINK
     - 17
     - string
   * - BTRFS_SEND_A_FILE_OFFSET
     - 18
     - u64
   * - BTRFS_SEND_A_DATA
     - 19
     - data
   * - BTRFS_SEND_A_CLONE_UUID
     - 20
     - uuid
   * - BTRFS_SEND_A_CLONE_CTRANSID
     - 21
     - u64
   * - BTRFS_SEND_A_CLONE_PATH
     - 22
     - string
   * - BTRFS_SEND_A_CLONE_OFFSET
     - 23
     - u64
   * - BTRFS_SEND_A_CLONE_LEN
     - 24
     - u64

Version 2
^^^^^^^^^

.. list-table::
   :header-rows: 1

   * - Name
     - Number
     - Type
   * - BTRFS_SEND_A_FALLOCATE_MODE
     - 25
     - u32
   * - BTRFS_SEND_A_FILEATTR
     - 26
     - u64
   * - BTRFS_SEND_A_UNENCODED_FILE_LEN
     - 27
     - u64
   * - BTRFS_SEND_A_UNENCODED_LEN
     - 28
     - u64
   * - BTRFS_SEND_A_UNENCODED_OFFSET
     - 29
     - u64
   * - BTRFS_SEND_A_COMPRESSION
     - 30
     - u32
   * - BTRFS_SEND_A_ENCRYPTION
     - 31
     - u32

Special cases
-------------

File creation (MKFILE) is done in two phases, where first a file with special
name like *o-123-45678* is created and then renamed (RENAME) to the target
location.

Parent directory where file changes are currently done gets update of times
(UTIMES) after each operation that affects it. This is not optimal as only the
last one would be needed.

Raw data type is processed in a different way in protocol version 1 and 2. In 1
the maximum size of the TLV is 64KiB as the size is stored in u16. This is not
sufficient for encoded write (ENCODED_WRITE) command. In 2 the data length is
up to 4GiB (using the type u32) but the TLV must be last and the actual
length is calculated as the delta between the whole command and the TLV (i.e.
ignoring the TLV header length).

Stream version 1
----------------

BTRFS_SEND_C_UNSPEC (0)
^^^^^^^^^^^^^^^^^^^^^^^

Placeholder, invalid or ignored command.

BTRFS_SEND_C_SUBVOL (1)
^^^^^^^^^^^^^^^^^^^^^^^

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative path of the subvolume
   * - uuid
     - uuid
     - UUID of the sent subvolume
   * - u64
     - ctransid
     - creation transaction

BTRFS_SEND_C_SNAPSHOT (2)
^^^^^^^^^^^^^^^^^^^^^^^^^

Start of commands of a given snapshot.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative path of the subvolume
   * - uuid
     - uuid
     - UUID of the sent subvolume
   * - u64
     - ctransid
     - creation transaction
   * - uuid
     - clone_uuid
     -
   * - u64
     - clone_ctransid
     -

BTRFS_SEND_C_MKFILE (3)
^^^^^^^^^^^^^^^^^^^^^^^

Create regular file. See also section Special cases.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path to create
   * - u64
     - ino
     - inode number

BTRFS_SEND_C_MKDIR (4)
^^^^^^^^^^^^^^^^^^^^^^

Create a directory.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative directory path to create
   * - u64
     - ino
     - inode number

File creation is done in two commands, the first one contains a special file name
that is later renamed to the final name. (WHY)

BTRFS_SEND_C_MKNOD (5)
^^^^^^^^^^^^^^^^^^^^^^

Create a special file of type device node (mknod).

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path to create
   * - u64
     - mode
     - file mode parameter of mknod(2)
   * - u64
     - rdev
     - rdev parameter of mknod(2)

BTRFS_SEND_C_MKFIFO (6)
^^^^^^^^^^^^^^^^^^^^^^^

Create a special file of type FIFO (mkfifo).

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path to create
   * - u64
     - ino
     - inode number

BTRFS_SEND_C_MKSOCK (7)
^^^^^^^^^^^^^^^^^^^^^^^

Create a special file of type socket (mknod S_IFSOCK).

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path to create
   * - u64
     - ino
     - inode number

BTRFS_SEND_C_SYMLINK (8)
^^^^^^^^^^^^^^^^^^^^^^^^

Create a symlink.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative symlink path to create
   * - u64
     - ino
     - inode number
   * - string
     - path_link
     - target of the symlink

BTRFS_SEND_C_RENAME (9)
^^^^^^^^^^^^^^^^^^^^^^^

Rename file path.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative source file path
   * - string
     - path_to
     - relative target file path

BTRFS_SEND_C_LINK (10)
^^^^^^^^^^^^^^^^^^^^^^

Create a file hardlink.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative source file path
   * - string
     - path_link
     - relative target file path to link to

BTRFS_SEND_C_UNLINK (11)
^^^^^^^^^^^^^^^^^^^^^^^^

Unlink file.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path

BTRFS_SEND_C_RMDIR (12)
^^^^^^^^^^^^^^^^^^^^^^^

Remove directory.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative directory path

BTRFS_SEND_C_SET_XATTR (13)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Set a value of extended attribute.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - string
     - xattr_name
     - name of the extended attribute
   * - data
     - xattr_data
     - value of the extended attribute

BTRFS_SEND_C_REMOVE_XATTR (14)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Remove an extended attribute.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - string
     - xattr_name
     - name of the extended attribute

BTRFS_SEND_C_WRITE (15)
^^^^^^^^^^^^^^^^^^^^^^^

Write file data to a given file offset.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - file_offset
     - where to write data
   * - data
     - data
     - raw file data (variable length)

BTRFS_SEND_C_CLONE (16)
^^^^^^^^^^^^^^^^^^^^^^^

Clone extents from another file.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - file_offset
     - offset in the source file to clone from
   * - u64
     - clone_len
     - length of cloned data
   * - uuid
     - clone_uuid
     -
   * - u64
     - clone_ctransid
     -
   * - string
     - clone_path
     - clone target relative file path
   * - u64
     - clone_offset
     - clone offset in the target file

BTRFS_SEND_C_TRUNCATE (17)
^^^^^^^^^^^^^^^^^^^^^^^^^^

Truncate file to a given length.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - size
     - truncate to given size

BTRFS_SEND_C_CHMOD (18)
^^^^^^^^^^^^^^^^^^^^^^^

Chmod a file or directory.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - mode
     - new mode

BTRFS_SEND_C_CHOWN (19)
^^^^^^^^^^^^^^^^^^^^^^^

Change file owner (uid) and group (gid), specified by numeric id. The uid/gid
must exist on the target filesystem, no mapping is done.
  
.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - uid
     - numeric used id
   * - u64
     - gid
     - numeric group id

BTRFS_SEND_C_UTIMES (20)
^^^^^^^^^^^^^^^^^^^^^^^^

Change file atime and mtime, nanosecond precision. While the ctime is also sent
it's not possible to change it using *utimensat*. The creation time is sent
since protocol version 2 but cannot be changed on the target filesystem.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - timespec
     - atime
     - file atime
   * - timespec
     - mtime
     - file mtime
   * - timespec
     - ctime
     - file ctime
   * - timespec
     - otime
     - (since v2) file otime (creation time)

BTRFS_SEND_C_END (21)
^^^^^^^^^^^^^^^^^^^^^

Special command to denote end of one logical stream inside the whole stream
sequence. May or may not be processed by receiver.

BTRFS_SEND_C_UPDATE_EXTENT (22)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When send is generated without data (BTRFS_SEND_FLAG_NO_FILE_DATA), this command
informs about changed extent but does not send the actual data.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - file_offset
     - file offset where data were updated
   * - u64
     - size
     - length of the data

Stream version 2
----------------

BTRFS_SEND_C_FALLOCATE (23)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Change file extents to preallocated, punch hole or zero fill.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u32
     - fallocate_mode
     - which fallocate operation to do
   * - u64
     - file_offset
     - file offset where to apply the operation
   * - u64
     - size
     - length of the range

BTRFS_SEND_C_FILEATTR (24)
^^^^^^^^^^^^^^^^^^^^^^^^^^

File attributes, representing various flags (SETFLAGS ioctl, XFLAGS, BTRFS
specific inode flags). The value is set from BTRFS inode bits and the stream
format inherits that. Note that some flags like IMMUTABLE or APPEND may affect
ability to change other flags and that for some flags there's ready interface
to set them.

BTRFS_SEND_C_ENCODED_WRITE (25)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File data encoded by the source filesystem and written directly to the target
filesystem, without any other transformation. The data can be compressed or
encrypted and the payload depends on presence of the TLVs.

.. list-table::
   :header-rows: 1

   * - Type
     - Name
     - Description
   * - string
     - path
     - relative file path
   * - u64
     - file_offset
     - file offset where to write the data
   * - u64
     - unencoded_file_len
     -
   * - u64
     - unencoded_len
     -
   * - u64
     - unencoded_offset
     -
   * - u32
     - compression
     - (optional) compression type
   * - u32
     - encryption
     - (optional) encryption type
   * - data
     - data
     - encoded payload