If the mimecap RPM or mime-support DEB is not installed, then the
/etc/mime.types file is not present on the system. RGW attempts to read
this file during startup, and if the file is not present, RGW logs an
error:
ext_mime_map_init(): failed to open file=/etc/mime.types ret=-2
Make the radosgw package depend on the mailcap/mime-support packages so
that /etc/mime.types is always available on RGW systems.
http://tracker.ceph.com/issues/11864Fixes: #11864
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
The postrotate script in src/logrotate.conf uses the which and find utilities
to do its work. Although logrotate itself is only a Recommends, I think which
and find are so ubiquitous that it makes sense to have them as hard
dependencies.
Also, I checked and the package names which and findutils are the same on all
the RPM distros we are currently targeting in the spec file.
In Debian, find is also in a package called findutils while the package
containing 'which' is called debianutils.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Before this patch, the command 'logrotate -f /etc/logrotate.d/ceph'
was generating an error "Failed to reload ceph.target: Job type reload is not
applicable for unit ceph.target".
Before we issue systemctl reload, check that there is at least
one active ceph-* service. (The hyphen is significant.)
Since we use grep, make the grep package a dependency.
http://tracker.ceph.com/issues/12173Fixes: #12173
Signed-off-by: Tim Serong <tserong@suse.com>
Signed-off-by: Lars Marowsky-Bree <lmb@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>
Given ceph's reputation for voluminous logs, it makes sense for
logrotate to be a hard dependency for all distros.
Signed-off-by: Nathan Cutler <ncutler@suse.com>
The semantic and interface of get_pg are the same, that avoids
duplication and the ceph-helpers.sh version is tested and documented.
Make the ceph-test package dependent on xmlstarlet because it is
needed by ceph-helpers.sh.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
When ceph-objectstore-tool was moved from ceph-test to
ceph by 61cf5da0b5, the ceph package in
debian/control was updated accordingly, as recommended by
https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces
The same must be done for the ceph-dbg package because
/usr/lib/debug/usr/bin/ceph-objectstore-too is no longer in
ceph-test-dbg.
Although the change was merged may 6th, 2015
8f23382064, teuthology jobs were not
always failing because packages were not systematically upgraded during
the installation. The missing dependencies that were responsible for
this upgrade problem were fixed by
f898ec1e4e on may 18th, 2015 and all
upgrade tests relying on ceph-*-dbg packages started to fail
systematically after this date.
http://tracker.ceph.com/issues/11546Fixes: #11546
Signed-off-by: Loic Dachary <ldachary@redhat.com>
Prior to this commit, if a user installed the "ceph-common" Debian
package without installing "ceph", then /usr/bin/ceph would crash
because it was missing the ceph_argparse library.
Ship the ceph_argparse library in "ceph-common" instead of "ceph". (This
was the intention of the original commit that moved argparse to "ceph",
2a23eac549)
http://tracker.ceph.com/issues/11388 Refs: #11388
Reported-by: Jens Rosenboom <j.rosenboom@x-ion.de>
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
Debian's debug packages ought to depend on their respective binary
packages. This was the case for many of our ceph packages, but it was
not the case for ceph-test-dbg or rest-bench-dbg.
Add the dependencies on the relevant binary packages, pinned to
"= ${binary:Version}" per convention.
http://tracker.ceph.com/issues/11673Fixes: #11673
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
This change ensures that the ceph-objectstore-tool utility is present on
all OSDs. This makes it easier for users to run this tool to do manual
debugging/recovery in some scenarios.
http://tracker.ceph.com/issues/11376 Refs: #11376
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
This can be done better in a separate script, which puts these in
CEPH_EXTRA_CONFIGURE_ARGS. In particular, this lets us enable
lttng for gitbuilder builds, but not release builds.
Fixes: #11333
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
(cherry picked from commit a294daff0b)
This can be done better in a separate script, which puts these in
CEPH_EXTRA_CONFIGURE_ARGS. In particular, this lets us enable
lttng for gitbuilder builds, but not release builds.
Fixes: #11333
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
* move rst files from doc/man/8 into man/8
* extract man_pages setting for sphinx from doc/conf.py to man/conf.py
* generate all man pages in `make install`
* add python-sphinx to Build-Depends
* check for sphinx-build in `configure`
* run changed rst files through sphinx-build in `make check`
Signed-off-by: Kefu Chai <kchai@redhat.com>
On Debian, ceph-mds was split out into its own package in
9d6013e0db, but the /var/lib/ceph/mds
directory was not moved along with the rest of the mds pieces.
The /var/lib/ceph/mds directory is only necessary if a user has
installed ceph-mds. Move it to the ceph-mds subpackage.
http://tracker.ceph.com/issues/10587 Refs: #10587
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
The /usr/bin/ceph_perf_objectstore file is installed by default. Prior
to this commit it was missing from the packaging. This caused the RPM to
fail to build in mock.
Add ceph_perf_objectstore to the "ceph-test" RPM and Debian package.
If we end up developing further ceph_perf_* utilities, it would make
sense to glob them all with a wildcard, similar to what we are doing
with all the ceph_test_* utilities in ceph-test.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
The zlib1g-dev is installed indirectly for Ubuntu 12.04 or Ubuntu 14.04
but it is only suggested in Debian/jessie. Adding it to the
Build-depends is redundant and harmless for Ubuntu and resolves the
missing dependency for Debian.
http://tracker.ceph.com/issues/11068Fixes: #11068
Signed-off-by: Loic Dachary <ldachary@redhat.com>
This is inspired by dstat and scripts/perf-watch.py, to
give a convenient live view of an interesting subset
of the performance counters from a Ceph daemon.
Signed-off-by: John Spray <john.spray@redhat.com>
Most of the ceph tree is LGPLv2.1, but there are some files that are
under the full GPLv2.
Add a copy of the GNU General Public License (version 2) to the
distribution. This file was copied verbatim from
https://www.gnu.org/licenses/gpl-2.0.txt
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
We should ship the RBD udev rules in the same package that ships
/usr/bin/rbd. This package happens to be ceph-common, so move the udev
rules there.
The udev rules rely on the ceph-rbdnamer utility, so move that utility
and its man page as well.
http://tracker.ceph.com/issues/10864 Refs: #10864
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
- move argparse to ceph-common
- split out rados, rbd, and cephfs bindings into their own packages
- keep python-ceph as a metapackage
Signed-off-by: Sage Weil <sage@redhat.com>
There really should be a better way than this to ensure the right things are installed for make check.
The existing list in ceph.spec.in has been re-sorted, just as the debian/control is
Andrew Bartlett
It was unnatural to shoehorn resetting tables
into the journaltool. This new tool initially
can simply dump or reset the session/snap/ino
tables, and would also be a place for any
more complex operations in future.
Signed-off-by: John Spray <john.spray@redhat.com>
python-virtualenv is required to run make check and xmlstarlet is useful
to develop shell base tests using ceph --format xml osd dump.
Signed-off-by: Loic Dachary <ldachary@redhat.com>
These binaries haven't landed in Ubuntu, but they are in
sid and jessie for arm64. On Saucy I've installed them by
hand from ports.ubuntu.com and things seems pretty swell.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
The ceph-disk man page was added in
a450cab2b8, but this was not added to the
RPM or DEB packaging. Add it here.
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
simplify ObjectStore interface
sage-2014-11-11_08:26:01-rados-wip-sage-testing-distro-basic-multi
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
with unified ceph_objectstore_tool
Move list-lost-objects and fix-lost-objects features from
ceph_filestore_tool to ceph_objectstore_tool as list-lost, fix-lost
Change --type to --op for info, log, export...operations
Add --type for the ObjectStore type (defaults to filestore)
Change --filestore-path to --data-path
Update installation, Makefile.am, and .gitignore
Fix and rename test case to match
Add some additional invalid option checks
Signed-off-by: David Zafman <david.zafman@inktank.com>
This patch adds systemd service files. It is possible to start and
enable multiple instances (per monid, osdid, mds name), e.g.
# systemctl start ceph-mon@node01
# systemctl enable ceph-mon@node01
# systemctl start ceph-osd@0
# systemctl enable ceph-osd@0
The ceph cluster can be set in the system config file:
/etc/sysconfig/ceph
adding or editing the CLUSTER environment variable.
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Add libkrbd libtool convenience library to provide an interface for
mapping and unmapping rbd images programmatically. This will be used
by the rbd binary itself and the librbd_fsx testing tool.
libkrbd takes care of the kernel module stuff (common/module.h) and
makes use of libudev to be able to properly wait for block device
creation and deletion and tell which block device got assigned by the
kernel to the newly created mapping.
Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
Clients can make use of these directories, and ceph-common is required by
ceph, so nothing should break here.
Change the purge postrm script to be for ceph-common (it does nothing else).
Signed-off-by: Sage Weil <sage@inktank.com>
The older versions of ceph-common (ceph CLI, in particular) can't talk to
newer clusters. The primary change happened with dumpling when the new
CLI and rest-api changes were made. Although in reality ceph doesn't
care what version of ceph-common is installed, in practice this forces
ceph-common to get upgraded along with ceph and avoids some user pain.
Fixes: #7641
Signed-off-by: Sage Weil <sage@inktank.com>
Introduce XfsFileStoreBackend class, currently the only filestore
backend implementing SETALLOCHINT op. This commit adds a build-time
dependency on libxfs as xfs-specific ioctl (XFS_IOC_FSSETXATTR /
XFS_XFLAG_EXTSIZE) is used to implement the new set_alloc_hint()
method.
Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
python-ceph does not require requests, but ceph-common does (for ceph-brag).
Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 9a0ef6a181)
The binaries file name have changed and need to be updated in the
packaging files for deb and rpm. Fix a few leftovers as well.
Fixing 1a588f18ba
Signed-off-by: Loic Dachary <loic@dachary.org>
It loads a designated erasure-code plugin and calls its
methods. It is convenient to figure out and tune the number of data
chunks, the size of an aligned chunk etc. For instance:
ceph_erasure_code \
--parameter erasure-code-plugin=jerasure \
--parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=reed_sol_van \
--parameter erasure-code-k=2 \
--parameter erasure-code-m=2 \
--all
displays the chunk size when encoding an object of 1024 bytes.
get_chunk_size(1024) 512
get_data_chunk_count 2
get_chunk_count 4
Signed-off-by: Loic Dachary <loic@dachary.org>
librados.hpp uses std::tr1::shared_ptr which may not be available such
as in libc++. This switches the use to ceph::shared_ptr and as a result
also ships include/memory.h for the definition.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Install the plugins in /usr/lib/ceph/erasure-code instead of
/usr/lib/erasure-code to comply with FHS : "Applications may use a
single subdirectory under /usr/lib."
http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html
The debian package is modified to install the plugins as part of the
ceph package which also ships rados-classes.
Signed-off-by: Loic Dachary <loic@dachary.org>
Because the gitbuilder build environment does not read the build depends
from the packages and needs to be updated independantly.
Signed-off-by: Loic Dachary <loic@dachary.org>
make check runs vstart.sh to setup a cluster from source and it misses
the python-argparse library as well as uuidgen otherwise it will fail
with:
./vstart.sh: 460: uuidgen: not found
...
Error: Import No module named argparse
Signed-off-by: Loic Dachary <loic@dachary.org>
Upstart script for mapping / unmapping rbd device based on /etc/ceph/rbdmap file.
It does not mount or unmount filesystem, this part should be performed by _netdev option in fstab.
Signed-off-by: Laurent Barbe <laurent@ksperis.com>
Currently the way 'rbd unmap' translates a user-provided block device
into an rbd id is it matches the major number of the specified device
against /sys/bus/rbd/devices/<id>/major for each rbd mapping and
declares success on the first match. This works for both entire disks
and partitions, because under the current device number allocation
scheme, each mapping means a new major number.
In preparation for support for single-major device number allocation
scheme, which would require matching both major and minor numbers, make
sure to always match against entire disk device numbers, by converting
the specified device major:minor pair into wholdedisk major:minor pair.
To achive that, use the libblkid library, which accomplishes this goal
by walking stable sysfs structures.
Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
Mark RADOS class libraries as module so that they will be automatically
ignored when stripping binaries.
Signed-off-by: James Page <james.page@ubuntu.com>
Instead of telling configure to put things in /sbin, explicitly put the
two important items (mkcephfs and mount.fuse.ceph) in /sbin via an
automake rule. This unbreaks FreeBSD 9.1 and probably others.
Based on patches originally from Alan Somers <asomers@gmail.com>, modified
for the current Makefile structure and applied to the specfile too.
Fixes: #6456
Signed-off-by: Sage Weil <sage@inktank.com>
Tested-by: Alan Somers <asomers@gmail.com>
Run parse_json_funcsigs against the builtin commands found
in mon/MonCommands.h. Although it does not provide for a full
validation, it will detect some degenerate cases.
It is expected to raise if a space is missing at the end of an option
specification ( see https://github.com/ceph/ceph/pull/585 ) and this
case is tested. New tests of the same kind can be added whenever an
undetected error is found. Ideally a validation function is implemented
and it would be updated ( with an associated test ) when a new
pathological case is found.
The json string given to parse_json_funcsigs is obtained from
the support program get_command_descriptions.
The python-nose dependencies are added to the build requirements in
debian/control and ceph.spec.in because make check should always be run
at built time.
http://tracker.ceph.com/issues/6274 refs #6274
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
The in-tree Hadoop shim was a combination of libcephfs wrapper, and the
bits to support Hadoop. This has been replaced by src/java that
implements generic libcephfs wrappers, and externally, the hadoop shim
(see docs).
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Use sftp to upload to a directory that only this user and ceph devs can
access.
Distribute an ssh key to connect to the account. This will let us revoke
the key in the future if we feel the need. Also distribute a known_hosts
file so that users have some confidence that they are connecting to the
real ceph drop account and not some third party.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
All past authors for rbd.cc have consented to relicensing from GPL to
LGPL2 via email:
---
Date: Sat, 27 Jul 2013 01:59:36 +0200
From: Sylvain Munaut <s.munaut@whatever-company.com>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I hereby consent to the relicensing of any contribution I made to the
aforementioned rbd.cc file from GPL to LGPL2.1.
(I hope that'll be impressive enough, I did my best :p)
btw, tnt@246tNt.com and s.munaut@whatever-company.com are both me.
Cheers,
Sylvain
---
Date: Fri, 26 Jul 2013 17:00:48 -0700
From: Yehuda Sadeh <yehuda@inktank.com>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I consent.
---
Date: Fri, 26 Jul 2013 17:02:24 -0700
From: Josh Durgin <josh.durgin@inktank.com>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I consent.
---
Date: Fri, 26 Jul 2013 18:17:46 -0700
From: Stanislav Sedov <stas@freebsd.org>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I consent.
Thanks for taking care of it!
---
Date: Fri, 26 Jul 2013 18:24:15 -0700
From: Colin McCabe <cmccabe@alumni.cmu.edu>
I consent.
cheers,
Colin
---
Date: Sat, 27 Jul 2013 07:08:12 +0200
From: Christian Brunner <christian@brunner-muc.de>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I consent
Christian
---
Date: Sat, 27 Jul 2013 12:17:34 +0300
From: Stratos Psomadakis <psomas@grnet.gr>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
Hi,
I consent with the GPL -> LGL2.1 re-licensing.
Thanks
Stratos
---
Date: Sat, 27 Jul 2013 16:13:13 +0200
From: Wido den Hollander <wido@42on.com>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I consent!
You have my permission to re-license the code I wrote for rbd.cc to LGPL2.1
---
Date: Sun, 11 Aug 2013 10:40:32 +0200
From: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Subject: Re: btw
Hi Sage,
I agree to switch the license of ceph_argparse.py and rbd.cc from GPL2
to LGPL2.
Regards
Danny Al-Gaaf
---
Date: Tue, 13 Aug 2013 17:15:24 -0700
From: Dan Mick <dan.mick@inktank.com>
Subject: Re: Ceph rbd.cc GPL -> LGPL2 license change
I consent to relicense any contributed code that I wrote under LGPL2.1 license.
---
...and I consent too. Drop the exception from COPYING and debian/copyright
files.
Signed-off-by: Sage Weil <sage@inktank.com>
ceph-rest-api is a Python WSGI module for accessing the Ceph cluster.
It supports most of the commands supported by the ceph CLI,
appropriately translated to HTTP GET/PUT requests. It is not a
truly RESTful interface.
Not supported at this moment: "tell", "pg <pgid>", and "daemon"
commands.
Configuration options are specified in ceph.conf, specified with
-c/--conf or obtained from $CEPH_CONF, /etc/ceph/ceph.conf,
~/.ceph/ceph.conf, or ./ceph.conf.
-n/--name specifies the client name, used for the cluster
authentication key and for the ceph.conf section name (default
is client.restapi).
restapi keyring = <keyring file>
restapi public addr = listenIP:port (default 0.0.0.0:5000)
restapi base url = <base path> (default /api/v0.1)
restapi log level = (error, warning, info, debug)
restapi log file = (default /var/log/ceph/<clientname>.log)
Primitive human-level command discovery is supported; GET from
BASEURL (say, http://localhost:5000/api/v0.1) will show an HTML
table of all commands and arguments, method supported, and help strings.
Signed-off-by: Dan Mick <dan.mick@inktank.com>
The fuse-utils package was deprecated a while ago.
Switch the primary dependency for fuse tools to use
the preferred 'fuse' package.
Signed-off-by: James Page <james.page@ubuntu.com>
Remove ceph-legacy-cli
Include ceph in proper places to get it built/installed/packaged
Make ceph-common (deb) or ceph (rpm) pkg depend on python-ceph pkg
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Upstart configurations and sysv init scripts should be started and stopped
using invoke-rc.d in maintainer scripts.
This ensures the correct behaviour across both Debian (sysv) and Ubuntu (upstart)
and in restricted environments such as schroot where start/stop of services from
maintainer scripts is normally disabled.
Signed-off-by: James Page <james.page@ubuntu.com>
Wheezy's udev (175-7.2) has broken rules for the /dev/disk/by-partuuid/
symlinks that ceph-disk relies on. Install parallel rules that work. On
new udev, this is harmless; old older udev, this will make life better.
Fixes: #4865
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
...indirectly via ceph-common. We get bad behavior when they diverge, I
think because of libcommon.la being linked both statically and dynamically.
Fixes: #4997
Backport: cuttlefish, bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
Squeeze requires the cryptsetup package which has been renamed
cryptsetup-bin in later versions. Allow either package to
satisfy the dependency.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
(cherry picked from commit 83bbae415d)
Since all currently supported platforms have tcmalloc
available and it is now the default, remove broken check code
that turns it off if the package is not listed in build-depends.
Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
When Build-Depends was split into multiple lines (in commit
8f5c665744), the grep for
libgoogle-perftools-dev broke. Replace grep with perl for multiline
matching.
Fixes: #4818
Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 89692e099f)
