Merge pull request #41334 from vshankar/wip-kcephfs-new-mount-syntax

mount: introduce new device mount syntax

Reviewed-by: Venky Shankar <vshankar@redhat.com>
This commit is contained in:
Venky Shankar 2021-12-01 09:33:29 +05:30 committed by GitHub
commit 3b213be189
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 590 additions and 176 deletions

View File

@ -6,6 +6,12 @@ The CephFS kernel driver is part of the Linux kernel. It allows mounting
CephFS as a regular file system with native kernel performance. It is the
client of choice for most use-cases.
.. note:: CephFS mount device string now uses a new (v2) syntax. The mount
helper (and the kernel) is backward compatible with the old syntax.
This means that the old syntax can still be used for mounting with
newer mount helpers and kernel. However, it is recommended to use
the new syntax whenever possible.
Prerequisites
=============
@ -50,51 +56,52 @@ Synopsis
========
In general, the command to mount CephFS via kernel driver looks like this::
mount -t ceph {device-string}:{path-to-mounted} {mount-point} -o {key-value-args} {other-args}
mount -t ceph {device-string}={path-to-mounted} {mount-point} -o {key-value-args} {other-args}
Mounting CephFS
===============
On Ceph clusters, CephX is enabled by default. Use ``mount`` command to
mount CephFS with the kernel driver::
mkdir /mnt/mycephfs
mount -t ceph :/ /mnt/mycephfs -o name=foo
mkdir /mnt/mycephfs
mount -t ceph <name>@<fsid>.<fs_name>=/ /mnt/mycephfs
The key-value argument right after option ``-o`` is CephX credential;
``name`` is the username of the CephX user we are using to mount CephFS. The
default value for ``name`` is ``guest``.
``name`` is the username of the CephX user we are using to mount CephFS.
``fsid`` is the FSID of the ceph cluster which can be found using
``ceph fsid`` command. ``fs_name`` is the file system to mount. The kernel
driver requires MON's socket and the secret key for the CephX user, e.g.::
The kernel driver also requires MON's socket and the secret key for the CephX
user. In case of the above command, ``mount.ceph`` helper figures out these
details automatically by finding and reading Ceph conf file and keyring. In
case you don't have these files on the host where you're running mount
command, you can pass these details yourself too::
mount -t ceph cephuser@b3acfc0d-575f-41d3-9c91-0e7ed3dbb3fa.cephfs=/ -o mon_addr=192.168.0.1:6789,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
mount -t ceph 192.168.0.1:6789,192.168.0.2:6789:/ /mnt/mycephfs -o name=foo,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
When using the mount helper, monitor hosts and FSID are optional. ``mount.ceph``
helper figures out these details automatically by finding and reading ceph conf
file, .e.g::
Passing a single MON socket in above command works too. A potential problem
with the command above is that the secret key is left in your shell's command
history. To prevent that you can copy the secret key inside a file and pass
the file by using the option ``secretfile`` instead of ``secret``::
mount -t ceph cephuser@.cephfs=/ -o secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
mount -t ceph :/ /mnt/mycephfs -o name=foo,secretfile=/etc/ceph/foo.secret
.. note:: Note that the dot (``.``) still needs to be a part of the device string.
Ensure the permissions on the secret key file are appropriate (preferably,
``600``).
A potential problem with the above command is that the secret key is left in your
shell's command history. To prevent that you can copy the secret key inside a file
and pass the file by using the option ``secretfile`` instead of ``secret``::
In case CephX is disabled, you can omit ``-o`` and the list of key-value
arguments that follow it::
mount -t ceph cephuser@.cephfs=/ /mnt/mycephfs -o secretfile=/etc/ceph/cephuser.secret
mount -t ceph :/ /mnt/mycephfs
Ensure the permissions on the secret key file are appropriate (preferably, ``600``).
Multiple monitor hosts can be passed by separating each address with a ``/``::
mount -t ceph cephuser@.cephfs=/ /mnt/mycephfs -o mon_addr=192.168.0.1:6789/192.168.0.2:6789,secretfile=/etc/ceph/cephuser.secret
In case CephX is disabled, you can omit any credential related options::
mount -t ceph cephuser@.cephfs=/ /mnt/mycephfs
.. note:: The ceph user name still needs to be passed as part of the device string.
To mount a subtree of the CephFS root, append the path to the device string::
mount -t ceph :/subvolume/dir1/dir2 /mnt/mycephfs -o name=fs
If you have more than one file system on your Ceph cluster, you can mount the
non-default FS as follows::
mount -t ceph :/ /mnt/mycephfs2 -o name=fs,fs=mycephfs2
mount -t ceph cephuser@.cephfs=/subvolume/dir1/dir2 /mnt/mycephfs -o secretfile=/etc/ceph/cephuser.secret
Unmounting CephFS
=================
@ -111,15 +118,14 @@ Persistent Mounts
To mount CephFS in your file systems table as a kernel driver, add the
following to ``/etc/fstab``::
[{ipaddress}:{port}]:/ {mount}/{mountpoint} ceph [name=username,secret=secretkey|secretfile=/path/to/secretfile],[{mount.options}]
{name}@.{fs_name}=/ {mount}/{mountpoint} ceph [mon_addr={ipaddress},secret=secretkey|secretfile=/path/to/secretfile],[{mount.options}] {fs_freq} {fs_passno}
For example::
:/ /mnt/ceph ceph name=admin,noatime,_netdev 0 2
cephuser@.cephfs=/ /mnt/ceph ceph mon_addr=192.168.0.1:6789,noatime,_netdev 0 0
The default for the ``name=`` parameter is ``guest``. If the ``secret`` or
``secretfile`` options are not specified then the mount helper will attempt to
find a secret for the given ``name`` in one of the configured keyrings.
If the ``secret`` or ``secretfile`` options are not specified then the mount helper
will attempt to find a secret for the given ``name`` in one of the configured keyrings.
See `User Management`_ for details on CephX user management and mount.ceph_
manual for more options it can take. For troubleshooting, see

View File

@ -9,8 +9,7 @@
Synopsis
========
| **mount.ceph** [*mon1_socket*\ ,\ *mon2_socket*\ ,...]:/[*subdir*] *dir* [
-o *options* ]
| **mount.ceph** *name*@*fsid*.*fs_name*=/[*subdir*] *dir* [-o *options* ]
Description
@ -19,31 +18,39 @@ Description
**mount.ceph** is a helper for mounting the Ceph file system on a Linux host.
It serves to resolve monitor hostname(s) into IP addresses and read
authentication keys from disk; the Linux kernel client component does most of
the real work. In fact, it is possible to mount a non-authenticated Ceph file
system without mount.ceph by specifying monitor address(es) by IP::
the real work. To mount a Ceph file system use::
mount -t ceph 1.2.3.4:/ /mnt/mycephfs
mount.ceph name@07fe3187-00d9-42a3-814b-72a4d5e7d5be.fs_name=/ /mnt/mycephfs -o mon_addr=1.2.3.4
The first argument is the device part of the mount command. It includes host's
socket and path within CephFS that will be mounted at the mount point. The
socket, obviously, takes the form ip_address[:port]. If the port is not
specified, the Ceph default of 6789 is assumed. Multiple monitor addresses can
be passed by separating them by commas. Only one monitor is needed to mount
successfully; the client will learn about all monitors from any responsive
monitor. However, it is a good idea to specify more than one in case the one
happens to be down at the time of mount.
Mount helper can fill in the cluster FSID by reading the ceph configuration file.
Its recommeded to call the mount helper via mount(8) as per::
If the host portion of the device is left blank, then **mount.ceph** will
attempt to determine monitor addresses using local configuration files
and/or DNS SRV records. In similar way, if authentication is enabled on Ceph
cluster (which is done using CephX) and options ``secret`` and ``secretfile``
are not specified in the command, the mount helper will spawn a child process
that will use the standard Ceph library routines to find a keyring and fetch
the secret from it.
mount -t ceph name@.fs_name=/ /mnt/mycephfs -o mon_addr=1.2.3.4
Note that the dot ``.`` still needs to be a part of the device string in this case.
The first argument is the device part of the mount command. It includes the
RADOS user for authentication, the file system name and a path within CephFS
that will be mounted at the mount point.
Monitor addresses can be passed using ``mon_addr`` mount option. Multiple monitor
addresses can be passed by separating addresses with a slash (`/`). Only one
monitor is needed to mount successfully; the client will learn about all monitors
from any responsive monitor. However, it is a good idea to specify more than one
in case the one happens to be down at the time of mount. Monitor addresses takes
the form ip_address[:port]. If the port is not specified, the Ceph default of 6789
is assumed.
If monitor addresses are not specified, then **mount.ceph** will attempt to determine
monitor addresses using local configuration files and/or DNS SRV records. In similar
way, if authentication is enabled on Ceph cluster (which is done using CephX) and
options ``secret`` and ``secretfile`` are not specified in the command, the mount
helper will spawn a child process that will use the standard Ceph library routines
to find a keyring and fetch the secret from it (including the monitor address and
FSID if those not specified).
A sub-directory of the file system can be mounted by specifying the (absolute)
path to the sub-directory right after ":" after the socket in the device part
of the mount command.
path to the sub-directory right after "=" in the device part of the mount command.
Mount helper application conventions dictate that the first two options are
device to be mounted and the mountpoint for that device. Options must be
@ -61,13 +68,6 @@ Basic
for autodiscovery of monitor addresses and auth secrets. The default is
to use the standard search path for ceph.conf files.
:command: `fs=<fs-name>`
Specify the non-default file system to be mounted. Not passing this
option mounts the default file system.
:command: `mds_namespace=<fs-name>`
A synonym of "fs=" and its use is deprecated.
:command:`mount_timeout`
int (seconds), Default: 60
@ -85,8 +85,11 @@ Basic
- ``prefer-secure``: secure mode, if denied agree to crc mode
:command:`name`
RADOS user to authenticate as when using CephX. Default: guest
:command:`mon_addr`
Monitor address of the cluster in the form of ip_address[:port]
:command:`fsid`
Cluster FSID. This can be found using `ceph fsid` command.
:command:`secret`
secret key for use with CephX. This option is insecure because it exposes
@ -192,50 +195,36 @@ Examples
Mount the full file system::
mount.ceph :/ /mnt/mycephfs
Assuming mount.ceph is installed properly, it should be automatically invoked
by mount(8)::
mount -t ceph :/ /mnt/mycephfs
mount -t ceph fs_user@.mycephfs2=/ /mnt/mycephfs
Mount only part of the namespace/file system::
mount.ceph :/some/directory/in/cephfs /mnt/mycephfs
Mount non-default FS, in case cluster has multiple FSs::
mount -t ceph :/ /mnt/mycephfs2 -o fs=mycephfs2
or
mount -t ceph :/ /mnt/mycephfs2 -o mds_namespace=mycephfs2 # This option name is deprecated.
mount.ceph fs_user@.mycephfs2=/some/directory/in/cephfs /mnt/mycephfs
Pass the monitor host's IP address, optionally::
mount.ceph 192.168.0.1:/ /mnt/mycephfs
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o mon_addr=192.168.0.1
Pass the port along with IP address if it's running on a non-standard port::
mount.ceph 192.168.0.1:7000:/ /mnt/mycephfs
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o mon_addr=192.168.0.1:7000
If there are multiple monitors, passes addresses separated by a comma::
If there are multiple monitors, pass each address separated by a `/`::
mount.ceph 192.168.0.1,192.168.0.2,192.168.0.3:/ /mnt/mycephfs
If authentication is enabled on Ceph cluster::
mount.ceph :/ /mnt/mycephfs -o name=fs_username
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o mon_addr=192.168.0.1/192.168.0.2/192.168.0.3
Pass secret key for CephX user optionally::
mount.ceph :/ /mnt/mycephfs -o name=fs_username,secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
Pass file containing secret key to avoid leaving secret key in shell's command
history::
mount.ceph :/ /mnt/mycephfs -o name=fs_username,secretfile=/etc/ceph/fs_username.secret
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o secretfile=/etc/ceph/fs_username.secret
If authentication is disabled on Ceph cluster, omit the credential related option::
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs
Availability
============

View File

View File

@ -0,0 +1,3 @@
overrides:
kclient:
syntax: 'v1'

View File

@ -0,0 +1,3 @@
overrides:
kclient:
syntax: 'v2'

View File

@ -0,0 +1,3 @@
overrides:
kclient:
syntax: 'v1'

View File

@ -30,6 +30,7 @@ class KernelMount(CephFSMount):
self.dynamic_debug = config.get('dynamic_debug', False)
self.rbytes = config.get('rbytes', False)
self.syntax_style = config.get('syntax', 'v2')
self.inst = None
self.addr = None
self._mount_bin = ['adjust-ulimits', 'ceph-coverage', self.test_dir +\
@ -43,6 +44,8 @@ class KernelMount(CephFSMount):
if not self.cephfs_mntpt:
self.cephfs_mntpt = '/'
if not self.cephfs_name:
self.cephfs_name = 'cephfs'
self._create_mntpt()
@ -77,28 +80,44 @@ class KernelMount(CephFSMount):
mountcmd_stderr.getvalue())
log.info('mount command passed')
def _make_mount_cmd_old_or_new_style(self):
optd = {}
mnt_stx = ''
if self.syntax_style == 'v1':
mnt_stx = f':{self.cephfs_mntpt}'
if self.client_id:
optd['name'] = self.client_id
if self.cephfs_name:
optd['mds_namespace'] = self.cephfs_name
elif self.syntax_style == 'v2':
mnt_stx = f'{self.client_id}@.{self.cephfs_name}={self.cephfs_mntpt}'
else:
assert 0, f'invalid syntax style: {self.syntax_style}'
return (mnt_stx, optd)
def _get_mount_cmd(self, mntopts):
opts = 'norequire_active_mds'
if self.client_id:
opts += ',name=' + self.client_id
if self.client_keyring_path and self.client_id:
opts += ',secret=' + self.get_key_from_keyfile()
if self.config_path:
opts += ',conf=' + self.config_path
if self.cephfs_name:
opts += ",mds_namespace=" + self.cephfs_name
if self.rbytes:
opts += ",rbytes"
else:
opts += ",norbytes"
if mntopts:
opts += ',' + ','.join(mntopts)
mount_cmd = ['sudo'] + self._nsenter_args
mount_dev = ':' + self.cephfs_mntpt
mount_cmd += self._mount_bin + [mount_dev, self.hostfs_mntpt, '-v',
stx_opt = self._make_mount_cmd_old_or_new_style()
for opt_name, opt_val in stx_opt[1].items():
opts += f',{opt_name}={opt_val}'
if mntopts:
opts += ',' + ','.join(mntopts)
log.info(f'mounting using device: {stx_opt[0]}')
# do not fall-back to old-style mount (catch new-style
# mount syntax bugs in the kernel).
opts += ",nofallback"
mount_cmd += self._mount_bin + [stx_opt[0], self.hostfs_mntpt, '-v',
'-o', opts]
return mount_cmd
def umount(self, force=False):

View File

@ -46,6 +46,9 @@ extern "C" void mount_ceph_get_config_info(const char *config_file,
conf.parse_env(cct->get_module_type()); // environment variables override
conf.apply_changes(nullptr);
auto fsid = conf.get_val<uuid_d>("fsid");
fsid.print(cci->cci_fsid);
ceph::async::io_context_pool ioc(1);
MonClient monc = MonClient(cct.get(), ioc);
err = monc.build_initial_monmap();

View File

@ -20,24 +20,70 @@
bool verboseflag = false;
bool skip_mtab_flag = false;
bool v2_addrs = false;
bool no_fallback = false;
bool ms_mode_specified = false;
bool mon_addr_specified = false;
static const char * const EMPTY_STRING = "";
/* TODO duplicates logic from kernel */
#define CEPH_AUTH_NAME_DEFAULT "guest"
/* path to sysfs for ceph */
#define CEPH_SYS_FS_PATH "/sys/module/ceph/"
#define CEPH_SYS_FS_PARAM_PATH CEPH_SYS_FS_PATH"/parameters"
/*
* mount support hint from kernel -- we only need to check
* v2 support for catching bugs.
*/
#define CEPH_V2_MOUNT_SUPPORT_PATH CEPH_SYS_FS_PARAM_PATH"/mount_syntax_v2"
#define CEPH_DEFAULT_V2_MS_MODE "prefer-crc"
#include "mtab.c"
enum mount_dev_format {
MOUNT_DEV_FORMAT_OLD = 0,
MOUNT_DEV_FORMAT_NEW = 1,
};
struct ceph_mount_info {
unsigned long cmi_flags;
char *cmi_name;
char *cmi_fsname;
char *cmi_fsid;
char *cmi_path;
char *cmi_mons;
char *cmi_conf;
char *cmi_opts;
int cmi_opts_len;
char cmi_secret[SECRET_BUFSIZE];
/* mount dev syntax format */
enum mount_dev_format format;
};
static void mon_addr_as_resolve_param(char *mon_addr)
{
for (; *mon_addr; ++mon_addr)
if (*mon_addr == '/')
*mon_addr = ',';
}
static void resolved_mon_addr_as_mount_opt(char *mon_addr)
{
for (; *mon_addr; ++mon_addr)
if (*mon_addr == ',')
*mon_addr = '/';
}
static void resolved_mon_addr_as_mount_dev(char *mon_addr)
{
for (; *mon_addr; ++mon_addr)
if (*mon_addr == '/')
*mon_addr = ',';
}
static void block_signals (int how)
{
sigset_t sigs;
@ -59,20 +105,101 @@ void mount_ceph_debug(const char *fmt, ...)
}
}
static int parse_src(const char *orig_str, struct ceph_mount_info *cmi)
/*
* append a key value pair option to option string.
*/
static void append_opt(const char *key, const char *value,
struct ceph_mount_info *cmi, int *pos)
{
if (*pos != 0)
*pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, *pos, ",");
if (value) {
*pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, *pos, key);
*pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, *pos, "=");
*pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, *pos, value);
} else {
*pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, *pos, key);
}
}
/*
* remove a key value pair from option string. caller should ensure that the
* key value pair is separated by "=".
*/
static int remove_opt(struct ceph_mount_info *cmi, const char *key, char **value)
{
char *key_start = strstr(cmi->cmi_opts, key);
if (!key_start) {
return -ENOENT;
}
/* key present -- try to split */
char *key_sep = strstr(key_start, "=");
if (!key_sep) {
return -ENOENT;
}
if (strncmp(key, key_start, key_sep - key_start) != 0) {
return -ENOENT;
}
++key_sep;
char *value_end = strstr(key_sep, ",");
if (!value_end)
value_end = key_sep + strlen(key_sep);
if (value_end != key_sep && value) {
size_t len1 = value_end - key_sep;
*value = strndup(key_sep, len1+1);
if (!*value)
return -ENOMEM;
(*value)[len1] = '\0';
}
/* purge it */
size_t len2 = strlen(value_end);
if (len2) {
++value_end;
memmove(key_start, value_end, len2);
} else {
/* last kv pair - swallow the comma */
--key_start;
*key_start = '\0';
}
return 0;
}
static void record_name(const char *name, struct ceph_mount_info *cmi)
{
int name_pos = 0;
int name_len = 0;
name_pos = safe_cat(&cmi->cmi_name, &name_len, name_pos, "client.");
name_pos = safe_cat(&cmi->cmi_name, &name_len, name_pos, name);
}
/*
* parse old device string of format: <mon_addr>:/<path>
*/
static int parse_old_dev(const char *dev_str, struct ceph_mount_info *cmi,
int *opt_pos)
{
size_t len;
char *mount_path;
mount_path = strstr(orig_str, ":/");
mount_path = strstr(dev_str, ":/");
if (!mount_path) {
fprintf(stderr, "source mount path was not specified\n");
return -EINVAL;
}
len = mount_path - orig_str;
len = mount_path - dev_str;
if (len != 0) {
cmi->cmi_mons = strndup(orig_str, len);
free(cmi->cmi_mons);
/* overrides mon_addr passed via mount option (if any) */
cmi->cmi_mons = strndup(dev_str, len);
if (!cmi->cmi_mons)
return -ENOMEM;
}
@ -81,23 +208,138 @@ static int parse_src(const char *orig_str, struct ceph_mount_info *cmi)
cmi->cmi_path = strdup(mount_path);
if (!cmi->cmi_path)
return -ENOMEM;
if (!cmi->cmi_name)
record_name(CEPH_AUTH_NAME_DEFAULT, cmi);
cmi->format = MOUNT_DEV_FORMAT_OLD;
return 0;
}
static char *finalize_src(struct ceph_mount_info *cmi)
/*
* parse new device string of format: name@<fsid>.fs_name=/path
*/
static int parse_new_dev(const char *dev_str, struct ceph_mount_info *cmi,
int *opt_pos)
{
size_t len;
char *name;
char *name_end;
char *dot;
char *fs_name;
name_end = strstr(dev_str, "@");
if (!name_end) {
mount_ceph_debug("invalid new device string format\n");
return -ENODEV;
}
len = name_end - dev_str;
if (!len) {
fprintf(stderr, "missing <name> in device\n");
return -EINVAL;
}
name = (char *)alloca(len+1);
memcpy(name, dev_str, len);
name[len] = '\0';
if (cmi->cmi_name && strcmp(cmi->cmi_name, name)) {
fprintf(stderr, "mismatching ceph user in mount option and device string\n");
return -EINVAL;
}
/* record name and store in option string */
if (!cmi->cmi_name) {
record_name(name, cmi);
append_opt("name", name, cmi, opt_pos);
}
++name_end;
/* check if an fsid is included in the device string */
dot = strstr(name_end, ".");
if (!dot) {
fprintf(stderr, "invalid device string format\n");
return -EINVAL;
}
len = dot - name_end;
if (len) {
/* check if this _looks_ like a UUID */
if (len != CLUSTER_FSID_LEN - 1) {
fprintf(stderr, "invalid device string format\n");
return -EINVAL;
}
cmi->cmi_fsid = strndup(name_end, len);
if (!cmi->cmi_fsid)
return -ENOMEM;
}
++dot;
fs_name = strstr(dot, "=");
if (!fs_name) {
fprintf(stderr, "invalid device string format\n");
return -EINVAL;
}
len = fs_name - dot;
if (!len) {
fprintf(stderr, "missing <fs_name> in device\n");
return -EINVAL;
}
cmi->cmi_fsname = strndup(dot, len);
if (!cmi->cmi_fsname)
return -ENOMEM;
++fs_name;
if (strlen(fs_name)) {
cmi->cmi_path = strdup(fs_name);
if (!cmi->cmi_path)
return -ENOMEM;
}
/* new-style dev - force using v2 addrs first */
if (!ms_mode_specified && !mon_addr_specified) {
v2_addrs = true;
append_opt("ms_mode", CEPH_DEFAULT_V2_MS_MODE, cmi,
opt_pos);
}
cmi->format = MOUNT_DEV_FORMAT_NEW;
return 0;
}
static int parse_dev(const char *dev_str, struct ceph_mount_info *cmi,
int *opt_pos)
{
int ret;
ret = parse_new_dev(dev_str, cmi, opt_pos);
if (ret < 0 && ret != -ENODEV)
return -EINVAL;
if (ret)
ret = parse_old_dev(dev_str, cmi, opt_pos);
if (ret < 0)
fprintf(stderr, "error parsing device string\n");
return ret;
}
/* resolve monitor host and record in option string */
static int finalize_src(struct ceph_mount_info *cmi, int *opt_pos)
{
int pos, len;
char *src;
size_t len = strlen(cmi->cmi_mons);
char *addr = alloca(len+1);
src = resolve_addrs(cmi->cmi_mons);
memcpy(addr, cmi->cmi_mons, len+1);
mon_addr_as_resolve_param(addr);
src = resolve_addrs(addr);
if (!src)
return NULL;
return -1;
len = strlen(src);
pos = safe_cat(&src, &len, len, ":");
safe_cat(&src, &len, pos, cmi->cmi_path);
return src;
resolved_mon_addr_as_mount_opt(src);
append_opt("mon_addr", src, cmi, opt_pos);
free(src);
return 0;
}
static int
@ -133,7 +375,7 @@ static int fetch_config_info(struct ceph_mount_info *cmi)
struct ceph_config_info *cci;
/* Don't do anything if we already have requisite info */
if (cmi->cmi_secret[0] && cmi->cmi_mons)
if (cmi->cmi_secret[0] && cmi->cmi_mons && cmi->cmi_fsid)
return 0;
cci = mmap((void *)0, sizeof(*cci), PROT_READ | PROT_WRITE,
@ -192,6 +434,11 @@ static int fetch_config_info(struct ceph_mount_info *cmi)
if (len < MON_LIST_BUFSIZE)
cmi->cmi_mons = strndup(cci->cci_mons, len + 1);
}
if (!cmi->cmi_fsid) {
len = strnlen(cci->cci_fsid, CLUSTER_FSID_LEN);
if (len < CLUSTER_FSID_LEN)
cmi->cmi_fsid = strndup(cci->cci_fsid, len + 1);
}
}
out:
munmap(cci, sizeof(*cci));
@ -201,13 +448,11 @@ out:
/*
* this one is partially based on parse_options() from cifs.mount.c
*/
static int parse_options(const char *data, struct ceph_mount_info *cmi)
static int parse_options(const char *data, struct ceph_mount_info *cmi,
int *opt_pos)
{
char * next_keyword = NULL;
int pos = 0;
char *name = NULL;
int name_len = 0;
int name_pos = 0;
if (data == EMPTY_STRING)
goto out;
@ -279,6 +524,8 @@ static int parse_options(const char *data, struct ceph_mount_info *cmi)
}
data = "mds_namespace";
skip = false;
} else if (strcmp(data, "nofallback") == 0) {
no_fallback = true;
} else if (strcmp(data, "secretfile") == 0) {
int ret;
@ -327,35 +574,38 @@ static int parse_options(const char *data, struct ceph_mount_info *cmi)
/* Only legacy ms_mode needs v1 addrs */
v2_addrs = strcmp(value, "legacy");
skip = false;
ms_mode_specified = true;
} else if (strcmp(data, "mon_addr") == 0) {
/* monitor address to use for mounting */
if (!value || !*value) {
fprintf(stderr, "mount option mon_addr requires a value.\n");
return -EINVAL;
}
cmi->cmi_mons = strdup(value);
if (!cmi->cmi_mons)
return -ENOMEM;
mon_addr_specified = true;
} else {
/* unrecognized mount options, passing to kernel */
skip = false;
}
/* Copy (possibly modified) option to out */
if (!skip) {
if (pos)
pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, pos, ",");
if (value) {
pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, pos, data);
pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, pos, "=");
pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, pos, value);
} else {
pos = safe_cat(&cmi->cmi_opts, &cmi->cmi_opts_len, pos, data);
}
}
if (!skip)
append_opt(data, value, cmi, opt_pos);
data = next_keyword;
} while (data);
out:
name_pos = safe_cat(&cmi->cmi_name, &name_len, name_pos, "client.");
name_pos = safe_cat(&cmi->cmi_name, &name_len, name_pos,
name ? name : CEPH_AUTH_NAME_DEFAULT);
/*
* set ->cmi_name conditionally -- this gets checked when parsing new
* device format. for old device format, ->cmi_name is set to default
* user name when name option is not passed in.
*/
if (name)
record_name(name, cmi);
if (cmi->cmi_opts)
mount_ceph_debug("mount.ceph: options \"%s\" will pass to kernel.\n",
cmi->cmi_opts);
mount_ceph_debug("mount.ceph: options \"%s\".\n", cmi->cmi_opts);
if (!cmi->cmi_opts) {
cmi->cmi_opts = strdup(EMPTY_STRING);
@ -402,8 +652,7 @@ static int parse_arguments(int argc, char *const *const argv,
return -EINVAL;
}
*opts = argv[i];
}
else {
} else {
fprintf(stderr, "Can't understand option: '%s'\n\n", argv[i]);
return -EINVAL;
}
@ -442,11 +691,175 @@ static void ceph_mount_info_free(struct ceph_mount_info *cmi)
{
free(cmi->cmi_opts);
free(cmi->cmi_name);
free(cmi->cmi_fsname);
free(cmi->cmi_fsid);
free(cmi->cmi_path);
free(cmi->cmi_mons);
free(cmi->cmi_conf);
}
static int mount_new_device_format(const char *node, struct ceph_mount_info *cmi)
{
int r;
char *rsrc = NULL;
int pos = 0;
int len = 0;
if (!cmi->cmi_fsid) {
fprintf(stderr, "missing ceph cluster-id");
return -EINVAL;
}
pos = safe_cat(&rsrc, &len, pos, cmi->cmi_name);
pos = safe_cat(&rsrc, &len, pos, "@");
pos = safe_cat(&rsrc, &len, pos, cmi->cmi_fsid);
pos = safe_cat(&rsrc, &len, pos, ".");
pos = safe_cat(&rsrc, &len, pos, cmi->cmi_fsname);
pos = safe_cat(&rsrc, &len, pos, "=");
if (cmi->cmi_path)
safe_cat(&rsrc, &len, pos, cmi->cmi_path);
mount_ceph_debug("mount.ceph: trying mount with new device syntax: %s\n",
rsrc);
if (cmi->cmi_opts)
mount_ceph_debug("mount.ceph: options \"%s\" will pass to kernel\n",
cmi->cmi_opts);
r = mount(rsrc, node, "ceph", cmi->cmi_flags, cmi->cmi_opts);
if (r)
r = -errno;
free(rsrc);
return r;
}
static int mount_old_device_format(const char *node, struct ceph_mount_info *cmi)
{
int r;
int len = 0;
int pos = 0;
char *mon_addr;
char *rsrc = NULL;
r = remove_opt(cmi, "mon_addr", &mon_addr);
if (r) {
fprintf(stderr, "failed to switch using old device format\n");
return -EINVAL;
}
pos = strlen(cmi->cmi_opts);
if (cmi->cmi_fsname)
append_opt("mds_namespace", cmi->cmi_fsname, cmi, &pos);
if (cmi->cmi_fsid)
append_opt("fsid", cmi->cmi_fsid, cmi, &pos);
pos = 0;
resolved_mon_addr_as_mount_dev(mon_addr);
pos = safe_cat(&rsrc, &len, pos, mon_addr);
pos = safe_cat(&rsrc, &len, pos, ":");
if (cmi->cmi_path)
safe_cat(&rsrc, &len, pos, cmi->cmi_path);
mount_ceph_debug("mount.ceph: trying mount with old device syntax: %s\n",
rsrc);
if (cmi->cmi_opts)
mount_ceph_debug("mount.ceph: options \"%s\" will pass to kernel\n",
cmi->cmi_opts);
r = mount(rsrc, node, "ceph", cmi->cmi_flags, cmi->cmi_opts);
free(mon_addr);
free(rsrc);
return r;
}
/*
* check whether to fall-back to using old-style mount syntax (called
* when new-style mount syntax fails). this is mostly to catch any
* new-style (v2) implementation bugs in the kernel and is primarly
* used in teuthology tests.
*/
static bool should_fallback()
{
int ret;
struct stat stbuf;
if (!no_fallback)
return true;
ret = stat(CEPH_V2_MOUNT_SUPPORT_PATH, &stbuf);
if (ret) {
mount_ceph_debug("mount.ceph: v2 mount support check returned %d\n",
errno);
if (errno == ENOENT)
mount_ceph_debug("mount.ceph: kernel does not support v2"
" syntax\n");
/* fallback on *all* errors */
return true;
}
fprintf(stderr, "mount.ceph: kernel BUG!\n");
return false;
}
static int do_mount(const char *dev, const char *node,
struct ceph_mount_info *cmi) {
int pos = 0;
int retval= -EINVAL;
bool fallback = true;
/* no v2 addresses available via config - try v1 addresses */
if (!cmi->cmi_mons &&
!ms_mode_specified &&
!mon_addr_specified &&
cmi->format == MOUNT_DEV_FORMAT_NEW) {
mount_ceph_debug("mount.ceph: switching to using v1 address\n");
v2_addrs = false;
fetch_config_info(cmi);
remove_opt(cmi, "ms_mode", NULL);
}
if (!cmi->cmi_mons) {
fprintf(stderr, "unable to determine mon addresses\n");
return -EINVAL;
}
pos = strlen(cmi->cmi_opts);
retval = finalize_src(cmi, &pos);
if (retval) {
fprintf(stderr, "failed to resolve source\n");
return -EINVAL;
}
retval = -1;
if (cmi->format == MOUNT_DEV_FORMAT_NEW) {
retval = mount_new_device_format(node, cmi);
if (retval)
fallback = (should_fallback() && retval == -EINVAL && cmi->cmi_fsid);
}
/* pass-through or fallback to old-style mount device */
if (retval && fallback)
retval = mount_old_device_format(node, cmi);
if (retval) {
retval = EX_FAIL;
switch (errno) {
case ENODEV:
fprintf(stderr, "mount error: ceph filesystem not supported by the system\n");
break;
case EHOSTUNREACH:
fprintf(stderr, "mount error: no mds server is up or the cluster is laggy\n");
break;
default:
fprintf(stderr, "mount error %d = %s\n", errno, strerror(errno));
}
}
if (!retval && !skip_mtab_flag) {
update_mtab_entry(dev, node, "ceph", cmi->cmi_opts, cmi->cmi_flags, 0, 0);
}
return retval;
}
static int append_key_or_secret_option(struct ceph_mount_info *cmi)
{
int pos = strlen(cmi->cmi_opts);
@ -485,48 +898,39 @@ static int append_key_or_secret_option(struct ceph_mount_info *cmi)
int main(int argc, char *argv[])
{
const char *src, *node, *opts;
char *rsrc = NULL;
int opt_pos = 0;
const char *dev, *node, *opts;
int retval;
struct ceph_mount_info cmi = { 0 };
retval = parse_arguments(argc, argv, &src, &node, &opts);
retval = parse_arguments(argc, argv, &dev, &node, &opts);
if (retval) {
usage(argv[0]);
retval = (retval > 0) ? 0 : EX_USAGE;
goto out;
}
retval = parse_options(opts, &cmi);
retval = parse_options(opts, &cmi, &opt_pos);
if (retval) {
fprintf(stderr, "failed to parse ceph_options: %d\n", retval);
retval = EX_USAGE;
goto out;
}
retval = parse_src(src, &cmi);
retval = parse_dev(dev, &cmi, &opt_pos);
if (retval) {
fprintf(stderr, "unable to parse mount source: %d\n", retval);
fprintf(stderr, "unable to parse mount device string: %d\n", retval);
retval = EX_USAGE;
goto out;
}
/* We don't care if this errors out, since this is best-effort */
/*
* We don't care if this errors out, since this is best-effort.
* note that this fetches v1 or v2 addr depending on @v2_addr
* flag.
*/
fetch_config_info(&cmi);
if (!cmi.cmi_mons) {
fprintf(stderr, "unable to determine mon addresses\n");
retval = EX_USAGE;
goto out;
}
rsrc = finalize_src(&cmi);
if (!rsrc) {
fprintf(stderr, "failed to resolve source\n");
retval = EX_USAGE;
goto out;
}
/* Ensure the ceph key_type is available */
modprobe();
@ -538,29 +942,10 @@ int main(int argc, char *argv[])
}
block_signals(SIG_BLOCK);
if (mount(rsrc, node, "ceph", cmi.cmi_flags, cmi.cmi_opts)) {
retval = EX_FAIL;
switch (errno) {
case ENODEV:
fprintf(stderr, "mount error: ceph filesystem not supported by the system\n");
break;
case EHOSTUNREACH:
fprintf(stderr, "mount error: no mds server is up or the cluster is laggy\n");
break;
default:
fprintf(stderr, "mount error %d = %s\n",errno,strerror(errno));
}
} else {
if (!skip_mtab_flag) {
update_mtab_entry(rsrc, node, "ceph", cmi.cmi_opts, cmi.cmi_flags, 0, 0);
}
}
retval = do_mount(dev, node, &cmi);
block_signals(SIG_UNBLOCK);
out:
ceph_mount_info_free(&cmi);
free(rsrc);
return retval;
}

View File

@ -24,11 +24,14 @@ extern "C" {
/* 2k should be enough for anyone? */
#define MON_LIST_BUFSIZE 2048
#define CLUSTER_FSID_LEN 37
void mount_ceph_debug(const char *fmt, ...);
struct ceph_config_info {
char cci_secret[SECRET_BUFSIZE]; // auth secret
char cci_mons[MON_LIST_BUFSIZE]; // monitor addrs
char cci_fsid[CLUSTER_FSID_LEN]; // cluster fsid
};
void mount_ceph_get_config_info(const char *config_file, const char *name,