ceph-volume: Use a delimited CLI output parser instead of JSON
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Add a set of new tests for the case when public_addr and public_bind_addr
are different for a mon. In order to test this properly I had to employ
port forwarding with socat. This helps simulate what would happen in a
environment like Kubernetes. socat is now a build dependency.
Also, moved jq_success to ceph-helpers.sh and refactored run_mon to enable
creating the mons without creating the rbd pool immediately.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
Ensure that dh_systemd_* debhelpers are executed after dh_install
has installed the systemd unit and target definitions.
This ensures that targets are enabled by default once installed,
resolving issues with startup of ceph daemons on server reboot.
Fixes: http://tracker.ceph.com/issues/19585
Signed-off-by: James Page <james.page@ubuntu.com>
'wrap-and-sort -a -t' debian folder to apply consistent
ordering/formatting to all files.
This makes synchronization of debian package changes
between Ceph packages and Ubuntu packages easier to
manage.
Signed-off-by: James Page <james.page@ubuntu.com>
This implements a create-self-signed-cert command for the ceph-mgr
restful interface.
It is designed so that it will try to restart the module once the cert
is created.
Signed-off-by: Boris Ranto <branto@redhat.com>
both Ubuntu and Debian put the logrotate script into
ceph-common, to ensure that radosgw logs are rotated as
well.
to prevent duplicate logrotate scripts handling the same log
files, and to minimize the delta between upstream and
downstream packaging, sync this change back upstream.
Fixes: http://tracker.ceph.com/issues/19938
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Modify cmake files to take advantage or re-organization of dmclock's
cmake set-up, specifically not having dmclock's cmake files call
add_test. Remove dmclock tests from being dependencies on ceph's
"test" target. Make change so dmclock tests are not built unless
WITH_TESTS and WITH_DMCLOCK_TESTS are both set. This is so openSUSE
Leap will build correctly.
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
This is the simplest way to generate the keys and probably the least
likely to cause trouble in the future.
Signed-off-by: Boris Ranto <branto@redhat.com>
This allows users to create their own certificates that are signed by a
CA. If the keys are not present then a self-signed certificate will be
created and distributed amongst all the mgr nodes.
It also allows us to get rid of the pyOpenSSL dependency.
Signed-off-by: Boris Ranto <branto@redhat.com>
The new rest API uses pecan for the restful functionality and simplifies
the code significantly. It should be mostly equivalent in functionality
to the django-based rest API. The api is self-documenting via /doc
endpoint.
Signed-off-by: Boris Ranto <branto@redhat.com>
This commit moves the radosgw-admin command from
the radosgw package to ceph-common.
This allows for a better user expierence since the
radosgw-admin command can be run on any node.
Fixes: http://tracker.ceph.com/issues/19577
Signed-off-by: Ali Maredia <amaredia@redhat.com>
mgr/pybind/fsstatus already required this, but ceph.in does now.
Add as a build dependency as well to handle tests.
Signed-off-by: Dan Mick <dan.mick@redhat.com>
Creates an installable version of "src/include/rados/objclass.h" that allows
object classes to be built outside of the Ceph tree. cls_sdk is an example
of such an object class.
Signed-off-by: Neha Ojha <nojha@redhat.com>
currently, only plugin based on isa-l is installed. archs other than
amd64 will not have this directory or the plugin(s) residing in it.
hence dh_install will fail when trying to copy nonexistence file/dir.
* debian/ceph-common.install: chmod +x, and only install crypto on amd64
so dh_install can filter the install list using dh-exec
* debian/control: depends on dh-exec now. dh-exec v0.13 introduces support
for filtering based on architecture. see dh-exec's changelog for more
details. but trusty only offers dh-exec v0.12. so do not require ">=
0.13) at this moment.
Signed-off-by: Kefu Chai <kchai@redhat.com>
Allow boost build during toplevel cmake from Debian package build
to benefit from multiple processors. Should speed build a lot
on many-proc machines (say, arm64). Use argument passed to
debhelper.
Signed-off-by: Dan Mick <dan.mick@redhat.com>
DEB packaging builds happen with LTTNG enabled but are missing a few
files.
* libosd_tp.so*, libos_tp.so* are needed to trace OSD
* librados_tp.so, librbd_tp.so are needed along with the other files
for trace visibility within lttng tool.
Signed-off-by: Ganesh Mahalingam <ganesh.mahalingam@intel.com>
when packaging debian packages, dpkg-shlibdeps analyzes the linked
shared libraries of the binaries in given package by looking at their
names. but if the name does not include any useful versioning
information, it complains. we have no intention of versioning
libceph-common, as it is merely an internal shared library used by ceph
packages only. but there is no simple way to disable dpkg-shlibdeps'
warning of
dpkg-shlibdeps: warning: can't extract name and version from library
name 'libceph-common.so'
other than skipping the dpkg-shlibdeps for the whole package. so let's
just version it anyway.
Signed-off-by: Kefu Chai <kchai@redhat.com>
prior to this change, libcommon is a convenient library which gets
linked into librados, librbd and libcephfs and all ceph executables.
this incurs some problems:
- double dose of libcommon in memory space and HDD: waste of memory
and disk space.
- if an application links to two libraries including libcommon at the
same time. take librados and libcephfs as an example, they could
interfere with each other by changing the other guy's status.
after this change, libcommon is tuned into a shared library and
renamed to libceph-common. it will be installed into $prefix/lib/ceph,
and packaged in librados2.
ceph.spec.in,debian/librados2.install: package libceph-common in
librados2.
CMakeLists.txt:
- do not link against libboost-* if not necessary.
- s/common/ceph-common/g
- install libceph-common into $prefix/lib/ceph
- set rpath to $prefix/lib/ceph
- link against ceph-common if an executable needs access to non public
symbols in ceph.
Signed-off-by: Kefu Chai <kchai@redhat.com>
so the subvar of ${python:Depends} and ${python3:Depends} can be set
properly. also this silences the warnings like
warning: dpkg-gencontrol: Depends field of package python3-rgw:
unknown substitution variable ${python3:Depends}
Signed-off-by: Kefu Chai <kchai@redhat.com>
libcephd is a library that contains ceph daemon code
that can be statically linked in other applications.
Added MergeStaticLibraries.cmake that can merge static libraries
to form a bigger one. This approach avoids the need to mess with
STATIC libraries all over the code base.
Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
With modern releases we should be able to make do with the call to
os_release_parse only which uses /etc/os-release which should be available on
most (all?) releases we currently support. this then allows us to remove the
runtime dependency which pulls in several other packages and would be nice to
avoid.
Fixes: http://tracker.ceph.com/issues/17425
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
...at least, try to! This is a copy-paste of the ceph-mds
packaging with a search and replace mds to mgr.
Signed-off-by: John Spray <john.spray@redhat.com>
python3 is not a hard requirement to build ceph, so make it optional.
add an option named "WITH_PYTHON3" which accepts ON, OFF, or CHECK.
Fixes: http://tracker.ceph.com/issues/17103
Signed-off-by: Kefu Chai <kchai@redhat.com>
find_package(lttng-ust REQUIRED) fails to find the lttng library without
this change. because find_path(LTTNG_LIBRARY_DIR ...) does not search
in the default library paths. and the second mode of
FIND_PACKAGE_HANDLE_STANDARD_ARGS() does not stop the cmake with a fatal
message, even some of the required vars are missing. so use the
implemetantion from cmake upstream.
Signed-off-by: Kefu Chai <kchai@redhat.com>
New interfaces for fetching extended (and selective) stat information.
Additionally, applications can specify AT_NO_ATTR_SYNC in the flags to
indicate that they want to do a "lazy" statx that just hands out the
inode info from the cache, or AT_SYMLINK_NOFOLLOW to avoid following
symlinks when walking the path.
Signed-off-by: Jeff Layton <jlayton@redhat.com>
DSA keys are being deprecated: http://www.openssh.com/legacy.html
drop.ceph.com will continue to allow the old DSA key but eventually,
users submitting logs using ceph-post-file will run into issues when
OpenSSH completely drops support for the algorithm.
Fixes: http://tracker.ceph.com/issues/14267
Signed-off-by: David Galloway <dgallowa@redhat.com>
and bump up the dependency of python version to 2.7.
py2.6 support was only necessary for older (now unsupported) distro
versions.
Signed-off-by: Kefu Chai <kchai@redhat.com>
since we are not using subvar of ${shlibs:Depends} in python3-* packages,
just exclude them in dh_shlibdeps.
this silences warnings like
```
warning: dpkg-gencontrol: package python3-cephfs: unused substitution
variable ${shlibs:Depends}
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
so we can use subvars like ${python3:Depends} in debian/control.
this silences the warnings like:
```
warning: dpkg-gencontrol: Depends field of package python3-cephfs:
unknown substitution variable ${python3:Depends}
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
Stop shipping /etc/init.d/rbdmap in the Debian packages. Ship the
rbdmap.service systemd unit file instead.
The corresponding change has already been made for RPMs, in
9224ac2ad2.
For Upstart-based systems (eg Ubuntu Trusty), the Debian packages
already contain rbdmap.conf.
(This gets us a tiny bit closer to being able to remove the rbdmap SysV
script from our tree entirely.)
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
* ceph-base: use ${python:Depends} instead of listing the python
dependencies manually, dh_python2 will scan the requirements
of ceph-detect-init. and fill the subst var for us.
* ceph-common: add ${python:Depends}, as it packages ceph,
and ceph-brag client.
* ceph-osd: it packages ceph-disk, so should add ${python:Depends}
as its dependencies.
dh_python2 will figure them out.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* debian/control:
as we have listed the linked libraries in Depends section, for example,
python-rados depends on librados. and we don't need `dpkg-shlibdeps` to
help figure out shared library substvar dependencies for us. by removing
them, we can silence the warnings of
```
warning: dpkg-shlibdeps: package could avoid a useless dependency if
debian/python-rados/usr/lib/python2.7/dist-packages/rados.x86_64-linux-gnu.so
was not linked against libpthread.so.0 (it uses none of the library's
symbols)
```
-lpthread is introduced by `python-config --ldflags` but it turns out we
are not using any symbols from pthread in the extension directly. and
pthread is included in glibc. so this does not added any extra
dependency to python-* pacakges. but it's desirable to have less
warnings.
* debian/rules: exclude python-* packages from dh_shlibdeps, as we will
not use it to prepare the shlib deps substvars for these packages any
more.
Signed-off-by: Kefu Chai <kchai@redhat.com>
some packages do not package python modules or scripts. so override
dh_python2 to exclude them.
this change silences warnings like:
```
warning: dpkg-gencontrol: package ceph-mon: unused substitution
variable ${python:Provides}
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
dh_auto_configure set -DCMAKE_BUILD_TYPE=None so the default does not
take effect at all. this is on purpose, see [1].
and dpkg-buildflags is able to produce the suggested compiling flags by
debian policy. dh_* can pass these flags to cmake only if it works at
compat>=9, see [2].
and we are safe to move to compat 9, as jewel's supported debian based
distro is jessie (with debhelper 9.20150101), ubuntu trusty (with
debhelper 9.20131227ubuntu1)
---
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701233#35
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653916
Fixes: http://tracker.ceph.com/issues/16744
Signed-off-by: Kefu Chai <kchai@redhat.com>
This has been deprecated for quite some time, in favour
of using proper vxattr and libcephfs interfaces.
Fixes: http://tracker.ceph.com/issues/16035
Signed-off-by: John Spray <john.spray@redhat.com>
this helps to keep the build rule simpler, and easier to customize
* -j$(NUMJOBS) is taken care of by 'dh --parallel'
* use 'autoreconf' dh add-on to autoconf cleanup
* add dh-autoreconf to Build-Depends
* bump debhelper compatibility level to 8, as jewel's supported
debian based distro is jessie (with debhelper 9.20150101),
ubuntu trusty (with debhelper 9.20131227ubuntu1)
Signed-off-by: Kefu Chai <kchai@redhat.com>
ceph-disk activate-all walks /dev/disk/by-parttypeuuid at boot time. It
is not necessary when udev fires ADD event for each partition and
95-ceph-osd.rules gets a chance to activate a ceph disk or journal.
There are various reasons why udev ADD events may not be fired at
boot (for instance Debian Jessi 8.5 never does it and CentOS 7.2 seems
to be racy in that regard when a LVM root is being used).
Populating /dev/disk/by-parttypeuuid fixes ceph-disk activate-all that
would not work without it. And it guarantees disks are activated at boot
time regardless of wether udev fires ADD events at the right time (or at
all).
The new udev file is a partial resurection of the
60-ceph-partuuid-workaround-rules that was removed by
9f77244b8e. It is given a name that
reflects its new purpose.
Fixes http://tracker.ceph.com/issues/16351
Signed-off-by: Loic Dachary <loic@dachary.org>
plugins do have unresolved symbols. so exclude them from
dpkg-shlibdeps's scan list. this silences warnings like:
```
dpkg-shlibdeps: warning:
debian/ceph-base/usr/lib/ceph/erasure-code/libec_shec_generic.so.1.0.0
contains an unresolvable reference to symbol
_ZN4ceph6buffer3ptrC1EPNS0_3rawE: it's probably a plugin
```
Signed-off-by: Kefu Chai <kchai@redhat.com>
Prior to this change, ceph-base required the "parted" package, which
meant that any installation of ceph-osd, ceph-mon, or ceph-mds would
pull in the parted package.
Move the parted dependency to ceph-osd, since ceph-disk is the only
thing that uses parted.
The advantage of this change is that Mon and MDS systems will not need
to install the parted package.
Fixes: http://tracker.ceph.com/issues/16095
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
