Introduce XfsFileStoreBackend class, currently the only filestore
backend implementing SETALLOCHINT op. This commit adds a build-time
dependency on libxfs as xfs-specific ioctl (XFS_IOC_FSSETXATTR /
XFS_XFLAG_EXTSIZE) is used to implement the new set_alloc_hint()
method.
Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
python-ceph does not require requests, but ceph-common does (for ceph-brag).
Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 9a0ef6a181)
It's tedious to try to keep the list of test files up to date in the RPM
spec file. Use a wildcard to match them all.
Reported-by: huangjun <hjwsm1989@gmail.com>
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
The binaries file name have changed and need to be updated in the
packaging files for deb and rpm. Fix a few leftovers as well.
Fixing 1a588f18ba
Signed-off-by: Loic Dachary <loic@dachary.org>
It loads a designated erasure-code plugin and calls its
methods. It is convenient to figure out and tune the number of data
chunks, the size of an aligned chunk etc. For instance:
ceph_erasure_code \
--parameter erasure-code-plugin=jerasure \
--parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=reed_sol_van \
--parameter erasure-code-k=2 \
--parameter erasure-code-m=2 \
--all
displays the chunk size when encoding an object of 1024 bytes.
get_chunk_size(1024) 512
get_data_chunk_count 2
get_chunk_count 4
Signed-off-by: Loic Dachary <loic@dachary.org>
In the RPM spec file there is a test to deploy the uuid hack udev rules
for older udev operating systems. This includes CentOS and RHEL, but the
check currently only is for CentOS, causing RHEL clients to get a bogus
osd rules file.
Adjust the conditional to apply to RHEL as well as CentOS. (The %{rhel}
macro is defined in both platforms' redhat-rpm-config package.)
Fixes http://tracker.ceph.com/issues/7245
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
librados.hpp uses std::tr1::shared_ptr which may not be available such
as in libc++. This switches the use to ceph::shared_ptr and as a result
also ships include/memory.h for the definition.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Automake puts ceph_common.sh into libdir/ceph, but the Red Hat packaging
was not capturing this file.
Add the libdir/ceph location to the RPM packaging.
Fixes#7117
Two tests were inadvertantly missing from the RPM packaging:
ceph_test_c_headers was added to ceph in
ffdd30efc8
ceph_test_get_blkdev_size was added to ceph in
324dd54534
Add both of these to the packaging.
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
Install the plugins in /usr/lib/ceph/erasure-code instead of
/usr/lib/erasure-code to comply with FHS : "Applications may use a
single subdirectory under /usr/lib."
http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html
The debian package is modified to install the plugins as part of the
ceph package which also ships rados-classes.
Signed-off-by: Loic Dachary <loic@dachary.org>
Because the gitbuilder build environment does not read the build depends
from the packages and needs to be updated independantly.
Signed-off-by: Loic Dachary <loic@dachary.org>
make check runs vstart.sh to setup a cluster from source and it misses
the python-argparse library as well as uuidgen otherwise it will fail
with:
./vstart.sh: 460: uuidgen: not found
...
Error: Import No module named argparse
Signed-off-by: Loic Dachary <loic@dachary.org>
Currently the way 'rbd unmap' translates a user-provided block device
into an rbd id is it matches the major number of the specified device
against /sys/bus/rbd/devices/<id>/major for each rbd mapping and
declares success on the first match. This works for both entire disks
and partitions, because under the current device number allocation
scheme, each mapping means a new major number.
In preparation for support for single-major device number allocation
scheme, which would require matching both major and minor numbers, make
sure to always match against entire disk device numbers, by converting
the specified device major:minor pair into wholdedisk major:minor pair.
To achive that, use the libblkid library, which accomplishes this goal
by walking stable sysfs structures.
Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
On a virgin centos-6.4, after yum-builddep ceph and following
http://ceph.com/docs/next/install/building-ceph/ instructions to:
cd ceph
./autogen.sh
./configure
make
it fails because make is not installed. It probably is not a problem for
most people because there are few developers who did not install make.
Signed-off-by: Loic Dachary <loic@dachary.org>
Instead of telling configure to put things in /sbin, explicitly put the
two important items (mkcephfs and mount.fuse.ceph) in /sbin via an
automake rule. This unbreaks FreeBSD 9.1 and probably others.
Based on patches originally from Alan Somers <asomers@gmail.com>, modified
for the current Makefile structure and applied to the specfile too.
Fixes: #6456
Signed-off-by: Sage Weil <sage@inktank.com>
Tested-by: Alan Somers <asomers@gmail.com>
The RPM spec file currently lists `junit` as a dependency on all
platforms except recent versions of OpenSUSE. This is problematic
because `junit` refers to version 3.x, which doesn't satisfy the check
in the configure script:
configure: Cannot find junit4.jar (apt-get install junit4)
As a result, rpmbuild doesn't build `libcephfs-test.jar` and fails:
http://tracker.ceph.com/issues/6216
So let's depend on `junit4` in all cases. This package is provided by
all supported platforms (el6, fc17, fc18, fc19, opensuse12.2,
opensuse12, sles11), and my testing on CentOS 6 shows that rpmbuild
now succeeds and produces the expected `libcephfs_jni1` RPM.
Run parse_json_funcsigs against the builtin commands found
in mon/MonCommands.h. Although it does not provide for a full
validation, it will detect some degenerate cases.
It is expected to raise if a space is missing at the end of an option
specification ( see https://github.com/ceph/ceph/pull/585 ) and this
case is tested. New tests of the same kind can be added whenever an
undetected error is found. Ideally a validation function is implemented
and it would be updated ( with an associated test ) when a new
pathological case is found.
The json string given to parse_json_funcsigs is obtained from
the support program get_command_descriptions.
The python-nose dependencies are added to the build requirements in
debian/control and ceph.spec.in because make check should always be run
at built time.
http://tracker.ceph.com/issues/6274 refs #6274
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
The in-tree Hadoop shim was a combination of libcephfs wrapper, and the
bits to support Hadoop. This has been replaced by src/java that
implements generic libcephfs wrappers, and externally, the hadoop shim
(see docs).
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
If the redhat-rpm-config package is installed, the debuginfo rpms will
be built by default. The build will fail when the package installed
and the specfile also invokes the macro.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
Use sftp to upload to a directory that only this user and ceph devs can
access.
Distribute an ssh key to connect to the account. This will let us revoke
the key in the future if we feel the need. Also distribute a known_hosts
file so that users have some confidence that they are connecting to the
real ceph drop account and not some third party.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Merge back what we have in the (open)SUSE ceph spec file for JUnit.
Add missing Requires and the package is named junit4 on some SUSE
versions.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
This patch adds two buildrequires to the ceph.spec file, that are needed
to build the rpms under Fedora. Danny Al-Gaaf commented that the
snappy-devel dependency should actually be added to the leveldb-devel
package. I will try to get that fixed too, in the mean time, this patch
does make sure Ceph builds on Fedora.
Signed-off-by: Erik Logtenberg <erik@logtenberg.eu>
The ceph-libs package existed only on Redhat based distro,
there was e.g. never such a package on SUSE. Therefore: make
sure the 'Obsoletes' is only set on these affected distros.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
* wip-wsgi:
ceph-rest-api: separate into module and front-end for WSGI deploy
ceph-rest-api: make main program be "shell" around WSGI guts
Reviewed-by: Sage Weil <sage@inktank.com>
To deploy ceph-rest-api within a WSGI server (apache/mod_wsgi,
nginx/uwsgi, etc.), there needs to be an importable (.py) module
that performs all init/config when imported. ceph-rest-api was
close, but it needs to be named properly, and there's no argument
passing, so it needs to get args from a fixed file or the env.
Separate most of ceph-rest-api into pybind/ceph_rest_api.py, and make
its arguments come from the environment, and init errors be
ImportError exceptions. Recase ceph-rest-api as a thin layer that
does the usual setup and arg parsing, and then sets args into the
environment and imports ceph_rest_api.py, catching exceptions and
reporting errors. This allows standalone execution as usual.
ceph-rest-api grabs a few module globals (addr/port and the flask.app)
to use after it imports.
Accept cluster name, and do the ceph.conf search using cluster name
in the appropriate places in the searched-for files.
Also ceph_rest_api.py gets a little cleanup (fewer global variables,
cleaner conf file search algorithm, better error reporting on conf
load)
Also: doc updates, packaging updates to include ceph_rest_api.py
Signed-off-by: Dan Mick <dan.mick@inktank.com>
ceph-rest-api is a Python WSGI module for accessing the Ceph cluster.
It supports most of the commands supported by the ceph CLI,
appropriately translated to HTTP GET/PUT requests. It is not a
truly RESTful interface.
Not supported at this moment: "tell", "pg <pgid>", and "daemon"
commands.
Configuration options are specified in ceph.conf, specified with
-c/--conf or obtained from $CEPH_CONF, /etc/ceph/ceph.conf,
~/.ceph/ceph.conf, or ./ceph.conf.
-n/--name specifies the client name, used for the cluster
authentication key and for the ceph.conf section name (default
is client.restapi).
restapi keyring = <keyring file>
restapi public addr = listenIP:port (default 0.0.0.0:5000)
restapi base url = <base path> (default /api/v0.1)
restapi log level = (error, warning, info, debug)
restapi log file = (default /var/log/ceph/<clientname>.log)
Primitive human-level command discovery is supported; GET from
BASEURL (say, http://localhost:5000/api/v0.1) will show an HTML
table of all commands and arguments, method supported, and help strings.
Signed-off-by: Dan Mick <dan.mick@inktank.com>
The %ghost %dir ... line will make this get cleaned up but won't install
it.
Reported-by: Derek Yarnell <derek@umiacs.umd.edu>
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
This was commented out almost years ago in commit 9baf5ef4 but it is not
clear to me that it was correct to do so. In any case, we are not
installing the rc.d links for ceph, which means it does not start up after
a reboot.
Signed-off-by: Sage Weil <sage@inktank.com>
Remove ceph-legacy-cli
Include ceph in proper places to get it built/installed/packaged
Make ceph-common (deb) or ceph (rpm) pkg depend on python-ceph pkg
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Add installed, but not packaged files to ceph-test (ceph-monstore-tool,
ceph-osdomap-tool) rpm file section.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Add installed, but not packaged files to ceph-test (ceph-monstore-tool,
ceph-osdomap-tool) rpm file section.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
This is needed when creating new OSDs (via ceph-disk). At least for most
people. Eventually we'll want to include btrfs here.
Signed-off-by: Sage Weil <sage@inktank.com>
Two fixes for Centos 6.3 and other systems with udev versions
prior to 172. The disk peristant name using the GPT UUID does
not exist, so use the by_path persistent name instead for the
journal symlink.
The gpt label fields are not available for use in udev rules. Add
ceph-disk-udev wrapper script that extracts the partition
type guid from the label and calls ceph-disk-activate if it is
a ceph guid type. (Bug #4632)
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
Add libboost-system-dev (bug #4725).
Add hdparm to rpm installation requirements. The hdparm
command is used to determin if write-caching is enabled on
the journal device.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
Add libboost-system-dev (bug #4725).
Add hdparm to rpm installation requirements. The hdparm
command is used to determin if write-caching is enabled on
the journal device.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
Commit ae71b576a7 change some paths
again to absolute path to /usr/sbin. Use the correct macro %{_sbindir}
again as done in 4b90a958d1.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Added init-radosgw.sys file for rpm based systems, added it to
the tarball list in the makefile, and updated the specfile to
install it. Also added the a dependency in ceph since it uses
utility routes from that package (On debian systems these are
packaged in ceph-common). Incorporated review comments from
Alex. (Bug #4571)
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
Reviewed-by: Alexandre Marangone <alexandre.marangone@inktank.com>
Move 50-rbd.rules into the ceph base package since the related
ceph-rbdnamer binary is part of this package. Use correct install
pattern.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Reorder file list of ceph package. Fix handling of placeholder
directories, make use of directories marcros like %{_localstatedir}
for /var.
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
The ceph-create-keys, ceph-disk, ceph-disk-activate, and
ceph-disk-prepare scripts are built in sbin, but debian installs
them into usr/bin, and several utilities look for them there.
This commit changes the RPM to install them in /usr/bin. (Bug #3921)
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
We should clobber configuration and log data, but *not* user data. Leave
/var/lib/ceph alone.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
