Commit Graph

88864 Commits

Author SHA1 Message Date
Sage Weil
8e36f18cde pybind/rados/rados: do not pass prval from stack
The prval is a pointer to an int to write the final completion code of
the rados op.  This can't be on the stack since we immediately leave the
current scope after preparing the op (looong before we do the rados op).

We keep the tuple return value to avoid breaking users of this API
(devicehealth module, gnocchi at a minimum).

Fixes: http://tracker.ceph.com/issues/25175
Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-31 09:41:05 -05:00
Alfredo Deza
96e7576400
Merge pull request #23348 from ceph/wip-rm24957
ceph-volume: adds test for `ceph-volume lvm list /dev/sda`

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-07-31 09:56:05 -04:00
Andrew Schoen
ef10886f1e ceph-volume: adds a unit test for lvm list /dev/sda
This test is to prove that the issue from
http://tracker.ceph.com/issues/24957 was fixed
by http://tracker.ceph.com/issues/24784

When running lvm list against a raw device it should handle
gracefully the situation where there are multiple PVs with the
name of the given device.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-07-31 08:50:28 -05:00
Andrew Schoen
37ed1be08b ceph-volume: move pvolumes fixture into conftest.py
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2018-07-31 08:50:27 -05:00
xie xingguo
d9123158d1 osd/OSD: fix HeartbeatInfo.is_healthy() check
Delay to declared to be healthy until we have received the first
replies from both front and back connections.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-07-31 20:58:53 +08:00
xie xingguo
aba603736c osd/OSD: use first_tx to calculate failed_for
If we never hear any replies from a heartbeat peer, use first_tx
to calculdate failed_for, which is more accurate.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-07-31 20:58:52 +08:00
xie xingguo
477774ceee osd: refactor heartbeat health check
The original logic will reuse the timestamp which we send pings to
the specific heartbeat peer to update the last_rx_front[back] field
on receiving the corresponding replies, which later shall be honoured
as the exact time we succeed in getting the corresponding replies and
is used to calculate the heartbeat latency and determine whether the
relevant peer is dead.

However this is not accurate enough as there may be a delay between
we receive a reply and call heartbeat_check(). We can eliminate
the delay by introducing a map to track the ping-history here,
each entry of which consists of three elements:

1. "tx_time", worked as the map key, indicates the exact timestamp
   we send pings.
2. "deadline", indicates we shall receive all replies by then,
   otherwise we consider this peer as "dead".
3. "unacknowledged", indicates how many pings for the corresponding
   ping are still unacknowledged. The initial value is 2(as we send
   two pings from the front and back side for each peer).

We insert an item into the map on every time we sending out a ping, and
decrease the "unacknowledged" counter by 1 each time we get a reply from
the tracked ping. If "unacknowledged" drops to 0, we know all the replies
have been successfully collected and we can safely erase the relevant
item from the map as well as the earlier sent ones,  if there is any.

By comparing the current timestamp with the oldest deadline, we can now
make a much accurate decision about whether the corresponding peer is
healthy or not. And by setting last_rx_* to the timestamp we receiving
the reply, the lower bound when we can no longer hear a reply from the
corresponding connection is also much clear now.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-07-31 20:58:52 +08:00
Kefu Chai
cec5a23f69
Merge pull request #23336 from noahdesu/vstart-dashboard-no-rbd
vstart: disable dashboard when rbd not built

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-31 11:05:02 +08:00
Neha Ojha
1b6dafb351 osd/PGLog.cc: use lgeneric_subdout instead of generic_dout
Signed-off-by: Neha Ojha <nojha@redhat.com>
2018-07-30 16:42:55 -07:00
Patrick Donnelly
33910303cb
ceph_volume_client: use integer division for pg_num
Otherwise a float is sent to the manager which is not the invalid format.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-30 16:12:48 -07:00
Rishabh Dave
c1efd59f61
packages.yaml: install py3 compatible cephfs and rados bindings...
on Ubuntu and CentOS for upstream testing.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-07-30 16:12:48 -07:00
Patrick Donnelly
a5e3b0512c
qa: test py2 and py3 for volume client
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-30 16:12:48 -07:00
Rishabh Dave
f28274dc70
qa: make test_volume_client.py py3 compatible
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-07-30 16:12:48 -07:00
Rishabh Dave
2f35ade05f
qa/ceph-volume: allow executing python pyloads using python3
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-07-30 16:12:48 -07:00
Rishabh Dave
5106582fc7
cephfs.pyx: add py3 compatibility
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-07-30 16:12:47 -07:00
Rishabh Dave
efdf1af41d
ceph-volume-client: add py3 compatibility
Fixes: http://tracker.ceph.com/issues/17230
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-07-30 16:12:47 -07:00
Patrick Donnelly
55e60ab17d
Merge PR #23297 into master
* refs/pull/23297/head:
	ceph_volume_client: add delay for MDSMap to be distributed

Reviewed-by: Ramana Raja <rraja@redhat.com>
2018-07-30 16:11:21 -07:00
Patrick Donnelly
957bdb4abe
Merge PR #23308 into master
* refs/pull/23308/head:
	doc: s/Ceph FS/CephFS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-30 16:06:39 -07:00
Noah Watkins
34c390a096 Revert "mgr/pybind: expose erasure code profiles"
The erasure code profiles are exposed through the OSD dump which was
overlooked on inspection.

This reverts commit b5487abb8b.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-07-30 15:20:34 -07:00
Noah Watkins
5b9dd4c8a2 vstart: disable dashboard when rbd not built
dashboard doesn't load correctly without the rbd module, which means
vstart commands that interact with dashboard fail and vstart exits.

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-07-30 14:50:28 -07:00
Sage Weil
1ebafdb65f
Merge pull request #23292 from yuriw/wip-yuriw-25140-master
qa/tests: added 1st draft of mimic-x suite
2018-07-30 14:55:41 -05:00
Sage Weil
c6dd193f45
Merge pull request #23302 from yuriw/wip-yuriw-crontab-master
qa/tests: added mimic-x to the schedule
2018-07-30 14:55:27 -05:00
Alfredo Deza
81df5d18c3
Merge pull request #23321 from cernceph/dvanders_enable
ceph-volume: enable the ceph-osd during lvm activation

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-07-30 12:54:24 -04:00
Andrew Schoen
4a043de4b7
Merge pull request #23332 from alfredodeza/wip-rm25171
ceph-volume add a __release__ string, to help version-conditional calls

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-07-30 16:14:01 +00:00
Yuri Weinstein
baa4d0ea78
Merge pull request #23305 from smithfarm/wip-cleanup-upgrade
qa/upgrade: cleanup for nautilus

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-30 09:01:03 -07:00
Yuri Weinstein
e6f21c1aa3 qa/tests: added 1st draft of mimic-x suite
Fixes: https://tracker.ceph.com/issues/25140
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-07-30 08:41:18 -07:00
Alfredo Deza
5bd0c27f9d ceph-volume add a __release__ string, to help version-conditional calls
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-07-30 11:39:43 -04:00
John Spray
1447d3ddce ceph.in: friendlier message on EPERM
When we get an EPERM, we can make an informed
guess at what the problem is.  This is a reasonably
common user error if they have e.g. "mds allow"
instead of "mds allow *", or "osd allow rwx" instead
of "osd allow *".

Fixes: http://tracker.ceph.com/issues/25172
Signed-off-by: John Spray <john.spray@redhat.com>
2018-07-30 16:32:49 +01:00
Kefu Chai
ab6a032519 common/assert: mark assert helpers with [[gnu::cold]]
for better performance,
see
https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/Common-Function-Attributes.html#Common-Function-Attributes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-30 23:23:25 +08:00
Kefu Chai
df2196dbf8
Merge pull request #23276 from tchaikov/wip-config-diff-lock
common/config: fix the lock in ConfigProxy::diff()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-30 22:24:54 +08:00
Kefu Chai
1bb7be365e
Merge pull request #23251 from neha-ojha/wip-25112
osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2018-07-30 22:23:18 +08:00
Kefu Chai
b88596d93c
Merge pull request #23249 from liewegas/wip-mon-cx-nautilus
osd/OSDMap: fix CEPHX_V2 osd requirement to nautilus, not mimic

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-30 22:22:40 +08:00
Kefu Chai
53951a84b6
Merge pull request #23229 from rjfd/wip-dashboard-query-params-bug
mgr/dashboard: fix query parameters in task annotated endpoints

Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-07-30 22:21:43 +08:00
Sage Weil
c13aa38175 Merge PR #22607 into master
* refs/pull/22607/head:
	common/options: convert many TYPE_[U]INT -> TYPE_SIZE
	common/options: remove journal_max_corrupt_search

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-30 08:45:17 -05:00
Dan van der Ster
261d8ac94d ceph-volume: enable ceph-osd during lvm activation
Enable the ceph-osd@<id> unit during lvm activate to link these
units to the ceph-osd.target.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: http://tracker.ceph.com/issues/24152
2018-07-30 15:07:54 +02:00
Dan van der Ster
3e6f387be1 ceph-volume: optional systemd enable --runtime
Allow units to be enabled but not persisted across a reboot,
and use this when enabling osds.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2018-07-30 14:57:03 +02:00
Lenz Grimmer
a8dc0e593f
Merge pull request #23287 from Devp00l/wip-duplicate-error-messages
mgr/dashboard: Fix duplicate error messages

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-07-30 11:39:05 +02:00
Ricardo Dias
00ec05abbd
Merge pull request #22669 from votdev/feature_24574
mgr/dashboard: Cleanup RGW config checks

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-07-30 09:53:16 +01:00
Yan, Zheng
c6d0c035b2 client: LAZY_IO support
'client_force_lazyio' option enables LAZY_IO gloablly for libcephfs
and ceph-fuse mount.

'ceph_lazyio(...)' and 'ceph_ll_lazyio(...)' enable LAZY_IO for open
file in libcephfs.

'ioctl(fd, CEPH_IOC_LAZYIO, 1UL)' enables LAZY_IO for open fils in
ceph-fuse mount.

Fixes: http://tracker.ceph.com/issues/20598
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-07-30 16:04:07 +08:00
Nathan Cutler
17d9b5be4d qa/upgrade: cleanup for nautilus
Drop unused suites, which ATM means all of them except upgrade/luminous-x
which recently got a cleanup in https://github.com/ceph/ceph/pull/23162

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-29 19:56:53 +02:00
Nathan Cutler
cecbf3e5dd
Merge pull request #23162 from smithfarm/wip-upgrade-cleanup
tests: upgrade/luminous-x: fix order of final-workload directory

Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-29 18:08:48 +02:00
Kefu Chai
2e01287031 cmake: do not link libs against ${ALLOC_LIBS}
let libcommon or libceph-common take care of this.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-29 21:25:52 +08:00
Kefu Chai
a6c73b6ac1 cmake,make-dist: build gperftools if WITH_STATIC_LIBSTDCXX
we could create a mini project to build a shared library, and use
try_compile() to test if the found gperftools is compiled with -fPIC.
but as we are targeting mostly xenial when enabling
WITH_STATIC_LIBSTDCXX, and google-perftools on xenial by default
is built without -fPIC. so let's keep it simple.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-29 21:25:52 +08:00
Kefu Chai
13b870cc0b cmake: do not link against common_crc_aarch64
it's included by libcrc32 already

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-29 20:25:32 +08:00
Kefu Chai
e6c7e0f521 cmake: extract std::filesystem linkage checking into module
see
https://libcxx.llvm.org/docs/UsingLibcxx.html#using-libc-experimental-and-experimental,

> Note that as of libc++ 7.0 using the <experimental/filesystem>
> requires linking libc++fs instead of libc++experimental.

do not build ceph_test_admin_socket_output if we are not able to find
the library for std::experimental::filesystem . it is a workaround of
https://reviews.freebsd.org/D10840 where FreeBSD 11.2 does not ship
libc++experimental.a .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-29 20:25:32 +08:00
Pavani Rajula
46057cefc1
Provided API to change umask
Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>
2018-07-28 16:22:58 -07:00
Yuri Weinstein
bba79cdb9f qa/tests: added mimic-x to the schedule
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-07-28 14:42:39 -07:00
Sage Weil
922bfc5f3b common/options: convert many TYPE_[U]INT -> TYPE_SIZE
Note that the _cost options are in fact in units of bytes.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-28 15:11:05 -05:00
Kefu Chai
a61493062c cmake: link against libsnappy.a if WITH_STATIC_LIBSTDCXX
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-29 01:44:18 +08:00
Kefu Chai
125b5da096 rocksdb: pickup change to link against libsnappy.a
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-29 01:44:18 +08:00