mirror of
https://github.com/ceph/ceph
synced 2024-12-28 06:23:08 +00:00
8679e0c2eb
The quota.max_bytes must be aligned to 4MB if greater than or equal to 4MB, otherwise must align to 4KB. This will make the statfs report correct block size and total blocks, then the `df` could show expected capacity. Fixes: https://tracker.ceph.com/issues/56397 Signed-off-by: Xiubo Li <xiubli@redhat.com>
123 lines
5.3 KiB
ReStructuredText
123 lines
5.3 KiB
ReStructuredText
CephFS Quotas
|
|
=============
|
|
|
|
CephFS allows quotas to be set on any directory in the file system. The
|
|
quota can restrict the number of *bytes* or the number of *files*
|
|
stored beneath that point in the directory hierarchy.
|
|
|
|
Like most other things in CephFS, quotas are configured using virtual
|
|
extended attributes:
|
|
|
|
* ``ceph.quota.max_files`` -- file limit
|
|
* ``ceph.quota.max_bytes`` -- byte limit
|
|
|
|
If the extended attributes appear on a directory that means a quota is
|
|
configured there. If they are not present then no quota is set on that
|
|
directory (although one may still be configured on a parent directory).
|
|
|
|
The value of ``ceph.quota.max_bytes`` must be aligned to 4MB if greater
|
|
than or equal to 4MB, otherwise it must be aligned to 4KB.
|
|
|
|
To set a quota, set the extended attribute on a CephFS directory with a
|
|
value::
|
|
|
|
setfattr -n ceph.quota.max_bytes -v 104857600 /some/dir # 100 MB
|
|
setfattr -n ceph.quota.max_files -v 10000 /some/dir # 10,000 files
|
|
|
|
``ceph.quota.max_bytes`` can also be set using human-friendly units::
|
|
|
|
setfattr -n ceph.quota.max_bytes -v 100K /some/dir # 100 KiB
|
|
setfattr -n ceph.quota.max_bytes -v 5Gi /some/dir # 5 GiB
|
|
|
|
.. note:: Values will be strictly cast to IEC units even when SI units
|
|
are input, e.g. 1K to 1024 bytes.
|
|
|
|
To view quota limit::
|
|
|
|
$ getfattr -n ceph.quota.max_bytes /some/dir
|
|
# file: dir1/
|
|
ceph.quota.max_bytes="104857600"
|
|
$
|
|
$ getfattr -n ceph.quota.max_files /some/dir
|
|
# file: dir1/
|
|
ceph.quota.max_files="10000"
|
|
|
|
.. note:: Running ``getfattr /some/dir -d -m -`` for a CephFS directory will
|
|
print none of the CephFS extended attributes. This is because the CephFS
|
|
kernel and FUSE clients hide this information from the ``listxattr(2)``
|
|
system call. Instead, a specific CephFS extended attribute can be viewed by
|
|
running ``getfattr /some/dir -n ceph.<some-xattr>``.
|
|
|
|
To remove a quota, set the value of extended attribute to ``0``::
|
|
|
|
$ setfattr -n ceph.quota.max_bytes -v 0 /some/dir
|
|
$ getfattr /some/dir -n ceph.quota.max_bytes
|
|
dir1/: ceph.quota.max_bytes: No such attribute
|
|
$
|
|
$ setfattr -n ceph.quota.max_files -v 0 /some/dir
|
|
$ getfattr dir1/ -n ceph.quota.max_files
|
|
dir1/: ceph.quota.max_files: No such attribute
|
|
|
|
Space Usage Reporting and CephFS Quotas
|
|
---------------------------------------
|
|
When the root directory of the CephFS mount has quota set on it, the available
|
|
space on the CephFS reported by space usage report tools (like ``df``) is
|
|
based on quota limit. That is, ``available space = quota limit - used space``
|
|
instead of ``available space = total space - used space``.
|
|
|
|
This behaviour can be disabled by setting following option in client section
|
|
of ``ceph.conf``::
|
|
|
|
client quota df = false
|
|
|
|
Limitations
|
|
-----------
|
|
|
|
#. *Quotas are cooperative and non-adversarial.* CephFS quotas rely on
|
|
the cooperation of the client who is mounting the file system to
|
|
stop writers when a limit is reached. A modified or adversarial
|
|
client cannot be prevented from writing as much data as it needs.
|
|
Quotas should not be relied on to prevent filling the system in
|
|
environments where the clients are fully untrusted.
|
|
|
|
#. *Quotas are imprecise.* Processes that are writing to the file
|
|
system will be stopped a short time after the quota limit is
|
|
reached. They will inevitably be allowed to write some amount of
|
|
data over the configured limit. How far over the quota they are
|
|
able to go depends primarily on the amount of time, not the amount
|
|
of data. Generally speaking writers will be stopped within 10s of
|
|
seconds of crossing the configured limit.
|
|
|
|
#. *Quotas are implemented in the kernel client 4.17 and higher.*
|
|
Quotas are supported by the userspace client (libcephfs, ceph-fuse).
|
|
Linux kernel clients >= 4.17 support CephFS quotas but only on
|
|
mimic+ clusters. Kernel clients (even recent versions) will fail
|
|
to handle quotas on older clusters, even if they may be able to set
|
|
the quotas extended attributes.
|
|
|
|
#. *Quotas must be configured carefully when used with path-based
|
|
mount restrictions.* The client needs to have access to the
|
|
directory inode on which quotas are configured in order to enforce
|
|
them. If the client has restricted access to a specific path
|
|
(e.g., ``/home/user``) based on the MDS capability, and a quota is
|
|
configured on an ancestor directory they do not have access to
|
|
(e.g., ``/home``), the client will not enforce it. When using
|
|
path-based access restrictions be sure to configure the quota on
|
|
the directory the client is restricted too (e.g., ``/home/user``)
|
|
or something nested beneath it.
|
|
|
|
In case of a kernel client, it needs to have access to the parent
|
|
of the directory inode on which quotas are configured in order to
|
|
enforce them. If quota is configured on a directory path
|
|
(e.g., ``/home/volumes/group``), the kclient needs to have access
|
|
to the parent (e.g., ``/home/volumes``).
|
|
|
|
An example command to create such an user is as below::
|
|
|
|
$ ceph auth get-or-create client.guest mds 'allow r path=/home/volumes, allow rw path=/home/volumes/group' mgr 'allow rw' osd 'allow rw tag cephfs metadata=*' mon 'allow r'
|
|
|
|
See also: https://tracker.ceph.com/issues/55090
|
|
|
|
#. *Snapshot file data which has since been deleted or changed does not count
|
|
towards the quota.* See also: http://tracker.ceph.com/issues/24284
|