Commit Graph

88282 Commits

Author SHA1 Message Date
Patrick Donnelly
4d3060b990
qa: use root to find client admin socket
Permissions on /var/run/ceph changed post-Mimic.

Fixes: http://tracker.ceph.com/issues/24872

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-11 11:28:20 -07:00
Sage Weil
860296c030 Merge PR #22908 into master
* refs/pull/22908/head:
	common: move ConfigValues out of md_config_t
	common,rbd,rgw,osd: extract config values into ConfigValues

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-11 09:50:46 -05:00
Sage Weil
2df6366c22 Merge PR #22953 into master
* refs/pull/22953/head:
	ceph-bluestore-tool: remove path_* support from prime-osd-dir

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-07-11 09:50:40 -05:00
Sage Weil
4de6fda0dc Merge PR #22968 into master
* refs/pull/22968/head:
	messages/MMDSMap: fix reencoding for older clients

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-11 09:50:35 -05:00
Sage Weil
1d9d61b784 Merge PR #22969 into master
* refs/pull/22969/head:
	qa/mgr: delete devicehealth pool after selftest

Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-11 09:50:27 -05:00
Abhishek Lekshmanan
ad28a23ec9 doc: releases: release notes for v10.2.11
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-07-11 16:39:03 +02:00
Sage Weil
267d22639e messages/MMDSMap: fix reencoding for older clients
This was broken by ea1481d08d.

Fixes: http://tracker.ceph.com/issues/24819
Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-11 08:44:23 -05:00
Nathan Cutler
12ef59fdb0 rpm: move make check-specific build deps to appropriate conditional block
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-11 13:56:49 +02:00
Nathan Cutler
3a30dd13a3 rpm: remove universal requires from distro-specific conditional blocks
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-11 13:51:04 +02:00
Lenz Grimmer
33d0f57eeb
Merge pull request #22593 from wido/mgr-dashboard-ssl
mgr/dashboard: Add option to disable SSL

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-07-11 11:25:16 +02:00
Wido den Hollander
21fbfc9c3a
mgr/dashboard: Add option to disable SSL
Although is preferred and should be enabled by default users might
want to disable SSL as the dashboard might be running behind a proxy
which terminates the SSL.

Fixes: https://tracker.ceph.com/issues/24674

Signed-off-by: Wido den Hollander <wido@42on.com>
2018-07-11 10:49:22 +02:00
Radoslaw Zarzynski
d1b16749c7 os/bluestore: fix races on SharedBlob::coll in ~SharedBlob.
Example scenario:
  ```
  A: BlueStore::Collection::split_cache(src, dest)
  A: std::lock(src->cache->lock, dest->cache->lock)

  B: SharedBlob::~SharedBlob
  B: waits coll->cache->lock // coll == src

  A: sb->coll := dest
  A: unlocks both src and dest's cache mutexes

  C: // any locked operation on dest's cache
  C: acquires dest->cache->lock
  C: begins the op

  B: // with the src's cache mutex acquired
  B: BufferSpace::_clear(coll->cache) // coll == dest
  B: // oops, B operates on cache already locked by C
  ```

Fixes: http://tracker.ceph.com/issues/24859
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-07-11 02:01:45 +02:00
Patrick Donnelly
c05a7473d0
Merge PR #22784 into master
* refs/pull/22784/head:
	doc: fix Ceph FS
	doc: new label and glossary url
	doc: drop linking to itself

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-10 14:23:53 -07:00
Patrick Donnelly
934bd3b856
Merge PR #22144 into master
* refs/pull/22144/head:
	cephfs-journal-tool: Fix purging when importing an zero-length journal.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-10 13:24:33 -07:00
Patrick Donnelly
bb7fbdb96b
Merge PR #22455 into master
* refs/pull/22455/head:
	qa/ceph-volume: add a test for put_object_versioned()
	ceph-volume-client: allow atomic updates for RADOS objects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-10 13:24:27 -07:00
Patrick Donnelly
15733f0a1f
Merge PR #22543 into master
* refs/pull/22543/head:
	qa/tasks/cephfs: add test for ceph_abort_conn()
	libcephfs: introduce ceph_get_instance_id()
	client: introduce ceph_abort_conn()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-10 13:24:20 -07:00
Patrick Donnelly
fdb1e142c1
Merge PR #22961 into master
* refs/pull/22961/head:
	doc/cephfs: make scary DR bits less prominent

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-10 12:55:42 -07:00
John Spray
d7601c546f qa/mgr: delete devicehealth pool after selftest
This prevents tests getting hung up on the health
warnings from its very low pg count.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-07-10 12:54:52 -04:00
vasukulkarni
9731dff850
Merge pull request #22921 from ceph/wip-default-ceph-volume
qa/tests: remove ceph-disk tests from ceph-deploy and default all tests to use ceph-volume
2018-07-10 09:37:34 -07:00
Radoslaw Zarzynski
871cbf0f61 common: fix races when visiting a TrackedOp.
Before the commit following race can happen:
  ```
  A           : OpTracker::visit_ops_in_flight(..., callable leaking TrackedOpRef outside)
  A           : Mutex::Locker::Locker(sdata->ops_in_flight_lock_sharded)
  A with lock : (nref > 0) == true

  B           : TrackedOp::put(), nref := 0 // updating the counter is done without the lock
  B           : OpTracker::unregister_inflight_op()
  B           : Mutex::Locker::Locker(sdata->ops_in_flight_lock_sharded)

  A with lock : visit() -> TrackedOp::get(), nref := 1
  A with lock : Mutex::Locker::~Locker()

  B with lock : boost::intrusive::list::iterator_to(op)
  B with lock : boost::intrusive::list::erase(iter)
  B with lock : Mutex::Locker::~Locker()

  A           : TrackedOp::put(), nref := 0
  A           : OpTracker::unregister_inflight_op()
  A           : Mutex::Locker::Locker(sdata->ops_in_flight_lock_sharded)
  A with lock : boost::intrusive::list::iterator_to(op) // oops as op doesn't belong to the list anymore
  ```

Fixes: https://tracker.ceph.com/issues/24664
Related-To: https://tracker.ceph.com/issues/24037
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-07-10 18:12:39 +02:00
Kefu Chai
f528475d92 common: move ConfigValues out of md_config_t
by moving ConfigValues out of md_config_t, we are able to point
md_config_t to a new ConfigValues instance at runtime, without
destroying the old one.

also remove ConfigReader, because all md_config_obs_t will be updated
with ConfigProxy now. as md_config_t is not able to offer the *data*
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-10 23:42:59 +08:00
Kefu Chai
4718b7cb2f common,rbd,rgw,osd: extract config values into ConfigValues
this change introduce three classes: ConfigValues, ConfigProxy and
ConfigReader. in seastar port of OSD, each CPU shard will hold its own
reference of configuration, and upon changes of settings, each
shard will be updated with the new setting in async. so this forces us
to be able to keep two set of configuration at the same time. so we
need to extract the changeable part of md_config_t out. so we can
replace the old one with new one on demand, and let different shards
share the same unchanged part, amon the other things, the Options map
and the lookup tables. that's why we need ConfigValues. we will add
a policy template for this class, so we can specialize for Seastar
implementation to allow different ConfigProxy instances to point
md_config_impl<> to different ConfigValues.

because the observer interface is still using md_config_t, to minimise
the impact of this change, handle_conf_change() and
handle_subsys_change() are not changed. but as it accepts a `const
md_config_t`, which cannot be used to create/reference the ConfigProxy
holding it, we need to introduce ConfigReader for reading the updated
setting from md_config_t in a simpler way, without exposing the
internal "values" member variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-10 22:51:22 +08:00
Sage Weil
737b9e3ef8 Merge PR #22940 into master
* refs/pull/22940/head:
	mon/OSDMonitor: add 'osd repeer <pgid>' command

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-10 08:42:27 -05:00
Sage Weil
c63be04dee Merge PR #22949 into master
* refs/pull/22949/head:
	cmake: cleanup FindCython.cmake

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2018-07-10 08:42:22 -05:00
Sage Weil
92e162c961 Merge PR #22954 into master
* refs/pull/22954/head:
	global/global_init: fix stdout/stderr/stdin closing for daemonization

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-07-10 08:42:17 -05:00
Kefu Chai
94453fc137
Merge pull request #22931 from tchaikov/wip-find-py-interp-for-gtest-parallel
cmake: find a python2 interpreter for gtest-parallel

Reviewed-by: Erwan Velu <erwan@redhat.com>
2018-07-10 21:29:54 +08:00
Kefu Chai
b94e3790c8 install-deps.sh: import ubuntu-toolchain-r's key without keyserver
we are running into following error recently

gpg: requesting key BA9EF27F from hkp server keyserver.ubuntu.com
gpgkeys: key 1E9377A2BA9EF27F can't be retrieved
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
gpg: keyserver communications error: keyserver helper general error
gpg: keyserver communications error: unknown pubkey algorithm
gpg: keyserver receive failed: unknown pubkey algorithm

so use the local copy of the pubkey instead

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-10 20:24:02 +08:00
John Spray
82f9960162 doc/cephfs: make scary DR bits less prominent
I'm sure people will still find them, but let's at least
force people to click through one more time to get to the
commands that can damage your cluster.

Also, the ".. danger" directive at the top of the page
wasn't actually getting special formatting, so I changed
it to a ".. warning" which is red.

Signed-off-by: John Spray <john.spray@redhat.com>
2018-07-10 10:52:52 +01:00
Lenz Grimmer
d63a8e5034
Merge pull request #22944 from falcon78921/wip-doc-rbdcmds
doc: fixed grammar in restore rbd image section

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-07-10 10:09:21 +02:00
Sage Weil
95b2281f9d mon/OSDMonitor: add 'osd repeer <pgid>' command
Selecting force peering on a single PG.  In reality this probably induces
*2* interval changes.

Note that in the case of a single OSD cluster we can't actually force a
repeer on a single PG because the pg_temp code is pretty robust about
filtering out redundant or meaningless changes, so we can't pg_temp our
way into a new interval if there are no other OSDs to switch to and the
code also prevents an empty pg_temp.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-09 16:33:11 -05:00
Lenz Grimmer
6350750cb7
Merge pull request #22711 from fghaas/wip-doc-mimic-upgrade-prereq
doc: Mention PURGED_SNAPDIRS and RECOVERY_DELETES in Mimic release notes

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-09 20:56:34 +02:00
Sage Weil
3c2d91ed11 global/global_init: fix stdout/stderr/stdin closing for daemonization
The global_init_postfork/prefork helpers close stdout/stdin/stderr on
fork and reopen /dev/null in their place.  This ensures that if later
code writes to those descriptors (e.g., a stray cout or cerr usage) the
output/input will go nowhere instead of interfering with some other open
fd.

However, with the use of preforker, there are other threads running when
these helpers are run, which means we can race with, say, filestore
opening an object file and end up sending log output there.

Fix by atomically replacing the fds with the dup2(2) syscall, which
will implicitly close and reopen the target fd in an atomic fashion.  This
behavior is present on both Linux and FreeBSD.

Fixes: http://tracker.ceph.com/issues/23492
Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-09 13:26:39 -05:00
Kefu Chai
16976bd42a cmake: should link against libatomic if libcxx/libstdc++ does not offer atomic ops
for instance,  GCC-8 on riscv64 does not offer atomic ops like
__atomic_fetch_or_1, so we need to link against libatomic to get access
to these symbols.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-10 01:37:00 +08:00
Sage Weil
1e11ef8bc7 ceph-bluestore-tool: remove path_* support from prime-osd-dir
The path_ fields have been removed, and ceph-volume populates these.

See ef510e43f0.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-09 12:31:28 -05:00
Sage Weil
975528f632 Merge remote-tracking branch 'private/wip-mon-snap-caps' 2018-07-09 08:29:32 -05:00
Nathan Cutler
dd413ad619
Merge pull request #22867 from smithfarm/wip-ccache-tweak
run-make-check.sh: ccache goodness for everyone

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Erwan Velu <erwan@redhat.com>
2018-07-09 15:20:01 +02:00
Sage Weil
f95f6aa3cf Merge remote-tracking branch 'gh/master' 2018-07-09 08:18:14 -05:00
Kefu Chai
423ca5764e cmake: find python2 interpreter for gtest-parallel
gtest-parallel only supports python2 at this moment, so find it
a python2 interpreter if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-09 21:09:18 +08:00
Sage Weil
6564adc878 Merge remote-tracking branch 'private/wip-cx'
Conflicts:
	src/include/ceph_features.h
	src/msg/async/AsyncConnection.cc
	src/msg/async/AsyncConnection.h

- SERVER_NAUTLIUS feature recently added to default
- msgr2 bool
- ceph::shared_ptr -> std::shared_ptr
2018-07-09 08:07:29 -05:00
Jason Dillaman
74ec64b1da
Merge pull request #22778 from trociny/wip-mock-warnings
test/librbd: fix gmock warning in TestMockIoImageRequestWQ.AcquireLockError

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-07-09 09:04:20 -04:00
Lenz Grimmer
ebbc3bc241
Merge pull request #22903 from tspmelo/fix-e2e-script
mgr/dashboard: Fix e2e script

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-07-09 14:37:32 +02:00
Nathan Cutler
693beaa8f3 run-make-check.sh: fill in missing DRY_RUN and add doc hint
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:05 +02:00
Nathan Cutler
458a0a6e62 run-make-check.sh: reset trap after sourcing install-deps.sh
Since we source install-deps.sh, it replaces our trap with its own,
so we have to reinstate our trap after it finishes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
b79b463f90 cleanup: run-make-check.sh: no space between -D and name of cmake option
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
3d6909b0f3 run-make-check.sh: gtest-parallel only with Python 2
Fixes: http://tracker.ceph.com/issues/24817
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
f9c0691f84 run-make-check.sh: inform user of make -j setting
This can help the user figure out why the build is so slow.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
43fe00e90c doc: cleanup: wordsmith a comment in run-make-check.sh
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
2315928acd run-make-check.sh: ccache goodness for everyone
Since run-make-check.sh already ensures that ccache is installed,
it makes sense to let everyone benefit from the ccache
tweaks introduced by 4cb5a59053

Note 1: The previous solution using "date" would cause build tools to reset
their timestamps after 24 hours, on subsequent runs of run-make-check.sh.
In order to maximize ccache effectiveness, this commit sets SOURCE_DATE_EPOCH
to a fixed value: the number of seconds elapsed since the Unix epoch as at
January 1, 2000 (chosen to commemorate Y2K armageddon).

Note 2: this commit introduces "set -e". This was actually in effect
before, via "source install-deps.sh". Better to make it explicit.

Fixes: http://tracker.ceph.com/issues/24777
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:01 +02:00
Nathan Cutler
c8691cd9a3 run-make-check.sh: drop workaround for undeclared jq build dependency
jq build dependency is properly declared now.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:25:40 +02:00
Kefu Chai
2a6096d8c8
Merge pull request #22898 from majianpeng/bluestore-remove-redundant-fault-range
os/bluestore: remove redundant fault_range.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-09 19:38:11 +08:00