btrfs-progs: docs: update scrub io limiting

[ci skip]

Issue: #402
Signed-off-by: David Sterba <dsterba@suse.com>

Documentation/btrfs-scrub.rst

@@ -23,6 +23,8 @@ cancel <path>|<device>
         The progress is saved in the status file so :command:`btrfs scrub resume` can
         continue from the last position.
 
+.. _man-scrub-limit:
+
 limit [options] <path>
 	Show scrub limits set on devices of the given filesystem.
 
@@ -90,10 +92,11 @@ start [-BdrRf] <path>|<device>
         ``Deprecated options``
 
         -c <ioprio_class>
-                set IO priority class (see ``ionice(1)`` manpage) if the IO
+                set IO priority class (see ``ionice(1)`` manual page) if the IO
                 scheduler configured for the device supports ionice. This is
-                not supported byg BFQ or Kyber but is *not* supported by
-                mq-deadline.
+                only supported by BFQ or Kyber but is *not* supported by
+                mq-deadline. Please read the section about
+                :ref:`IO limiting<scrub-io-limiting>`.
         -n <ioprio_classdata>
                 set IO priority classdata (see ``ionice(1)`` manpage)
         -q
@@ -172,7 +175,8 @@ status [options] <path>|<device>
         It's possible to set a per-device limit via file
         :file:`sysfs/fs/btrfs/FSID/devinfo/scrub_speed_max`. In that case
         the limit is printed on the *Rate:* line if option *-d* is specified,
-        or without it on a single-device filesystem.
+        or without it on a single-device filesystem.  Read more about tat in
+        section about :ref:`scrub IO limiting<scrub-io-limiting>`.
 
         .. code-block:: none
 

Documentation/ch-scrub-intro.rst

@@ -7,15 +7,12 @@ the damaged one is repaired. All copies of the replicated profiles are validated
    structural damage in the filesystem. It really only checks checksums of data
    and tree blocks, it doesn't ensure the content of tree blocks is valid and
    consistent. There's some validation performed when metadata blocks are read
-   from disk but it's not extensive and cannot substitute full :command:`btrfs check`
-   run.
+   from disk (:doc:`Tree-checker`) but it's not extensive and cannot substitute
+   full :doc:`btrfs-check` run.
 
 The user is supposed to run it manually or via a periodic system service. The
-recommended period is a month but could be less. The estimated device bandwidth
-utilization is about 80% on an idle filesystem. The IO priority class is by
-default *idle* so background scrub should not significantly interfere with
-normal filesystem operation. The IO scheduler set for the device(s) might not
-support the priority classes though.
+recommended period is a month but it could be less. The estimated device bandwidth
+utilization is about 80% on an idle filesystem.
 
 The scrubbing status is recorded in :file:`/var/lib/btrfs/` in textual files named
 *scrub.status.UUID* for a filesystem identified by the given UUID. (Progress
@@ -25,3 +22,48 @@ will continue from the last saved position.
 
 Scrub can be started only on a mounted filesystem, though it's possible to
 scrub only a selected device. See :ref:`btrfs scrub start<man-scrub-start>` for more.
+
+.. _scrub-io-limiting:
+
+Bandwidth and IO limiting
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. note::
+   The ``ionice(1)`` may not be generally supported by all IO schedulers and
+   the options to :command:`btrfs scrub start` may not work as expected.
+
+In the past when the `CFQ IO scheduler
+<https://en.wikipedia.org/wiki/Completely_fair_queueing>`__ was generally used
+the ``ionice(1)`` syscalls set the priority to *idle* so the IO would not
+interfere with regular IO. Since the kernel 5.0 the CFQ is not available.
+
+The IO scheduler known to support that is `BFQ
+<https://docs.kernel.org/block/bfq-iosched.html>`__, but first read the
+documentation before using it!
+
+For other commonly used schedulers like `mq-deadline
+<https://docs.kernel.org/block/blk-mq.html>`__ it's recommended to use
+*cgroup2 IO controller* which could be managed by e.g. *systemd*
+(documented in ``systemd.resource-control``). However, starting scrub like that
+is not yet completely straightforward. The IO controller must know the physical
+device of the filesystem and create a slice so all processes started from that
+belong to the same accounting group.
+
+.. code-block:: bash
+
+   $ systemd-run -p "IOBandwidthReadMax=/dev/sdx 10M" btrfs scrub start -B /
+
+Since linux 5.14 it's possible to set the per-device bandwidth limits in a
+BTRFS-specific way using files :file:`/sys/fs/btrfs/FSID/devinfo/DEVID/scrub_speed_max`.
+This setting is not persistent, lasts until the filesystem is unmounted.
+Currently set limits can be displayed by command :ref:`btrfs scrub
+limit<man-scrub-limit>`.
+
+.. code-block:: bash
+
+   $ echo 100m > /sys/fs/btrfs/9b5fd16e-1b64-4f9b-904a-74e74c0bbadc/devinfo/1/scrub_speed_max
+   $ btrfs scrub limit /
+   UUID: 9b5fd16e-1b64-4f9b-904a-74e74c0bbadc
+   Id      Limit      Path
+   --  ---------  --------
+    1  100.00MiB  /dev/sdx