Commit Graph

88192 Commits

Author SHA1 Message Date
Kefu Chai
4718b7cb2f common,rbd,rgw,osd: extract config values into ConfigValues
this change introduce three classes: ConfigValues, ConfigProxy and
ConfigReader. in seastar port of OSD, each CPU shard will hold its own
reference of configuration, and upon changes of settings, each
shard will be updated with the new setting in async. so this forces us
to be able to keep two set of configuration at the same time. so we
need to extract the changeable part of md_config_t out. so we can
replace the old one with new one on demand, and let different shards
share the same unchanged part, amon the other things, the Options map
and the lookup tables. that's why we need ConfigValues. we will add
a policy template for this class, so we can specialize for Seastar
implementation to allow different ConfigProxy instances to point
md_config_impl<> to different ConfigValues.

because the observer interface is still using md_config_t, to minimise
the impact of this change, handle_conf_change() and
handle_subsys_change() are not changed. but as it accepts a `const
md_config_t`, which cannot be used to create/reference the ConfigProxy
holding it, we need to introduce ConfigReader for reading the updated
setting from md_config_t in a simpler way, without exposing the
internal "values" member variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-10 22:51:22 +08:00
Sage Weil
737b9e3ef8 Merge PR #22940 into master
* refs/pull/22940/head:
	mon/OSDMonitor: add 'osd repeer <pgid>' command

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-10 08:42:27 -05:00
Sage Weil
c63be04dee Merge PR #22949 into master
* refs/pull/22949/head:
	cmake: cleanup FindCython.cmake

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2018-07-10 08:42:22 -05:00
Sage Weil
92e162c961 Merge PR #22954 into master
* refs/pull/22954/head:
	global/global_init: fix stdout/stderr/stdin closing for daemonization

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-07-10 08:42:17 -05:00
Kefu Chai
94453fc137
Merge pull request #22931 from tchaikov/wip-find-py-interp-for-gtest-parallel
cmake: find a python2 interpreter for gtest-parallel

Reviewed-by: Erwan Velu <erwan@redhat.com>
2018-07-10 21:29:54 +08:00
Lenz Grimmer
d63a8e5034
Merge pull request #22944 from falcon78921/wip-doc-rbdcmds
doc: fixed grammar in restore rbd image section

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-07-10 10:09:21 +02:00
Sage Weil
95b2281f9d mon/OSDMonitor: add 'osd repeer <pgid>' command
Selecting force peering on a single PG.  In reality this probably induces
*2* interval changes.

Note that in the case of a single OSD cluster we can't actually force a
repeer on a single PG because the pg_temp code is pretty robust about
filtering out redundant or meaningless changes, so we can't pg_temp our
way into a new interval if there are no other OSDs to switch to and the
code also prevents an empty pg_temp.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-09 16:33:11 -05:00
Lenz Grimmer
6350750cb7
Merge pull request #22711 from fghaas/wip-doc-mimic-upgrade-prereq
doc: Mention PURGED_SNAPDIRS and RECOVERY_DELETES in Mimic release notes

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-09 20:56:34 +02:00
Sage Weil
3c2d91ed11 global/global_init: fix stdout/stderr/stdin closing for daemonization
The global_init_postfork/prefork helpers close stdout/stdin/stderr on
fork and reopen /dev/null in their place.  This ensures that if later
code writes to those descriptors (e.g., a stray cout or cerr usage) the
output/input will go nowhere instead of interfering with some other open
fd.

However, with the use of preforker, there are other threads running when
these helpers are run, which means we can race with, say, filestore
opening an object file and end up sending log output there.

Fix by atomically replacing the fds with the dup2(2) syscall, which
will implicitly close and reopen the target fd in an atomic fashion.  This
behavior is present on both Linux and FreeBSD.

Fixes: http://tracker.ceph.com/issues/23492
Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-09 13:26:39 -05:00
Sage Weil
975528f632 Merge remote-tracking branch 'private/wip-mon-snap-caps' 2018-07-09 08:29:32 -05:00
Nathan Cutler
dd413ad619
Merge pull request #22867 from smithfarm/wip-ccache-tweak
run-make-check.sh: ccache goodness for everyone

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Erwan Velu <erwan@redhat.com>
2018-07-09 15:20:01 +02:00
Sage Weil
f95f6aa3cf Merge remote-tracking branch 'gh/master' 2018-07-09 08:18:14 -05:00
Kefu Chai
423ca5764e cmake: find python2 interpreter for gtest-parallel
gtest-parallel only supports python2 at this moment, so find it
a python2 interpreter if it is enabled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-09 21:09:18 +08:00
Sage Weil
6564adc878 Merge remote-tracking branch 'private/wip-cx'
Conflicts:
	src/include/ceph_features.h
	src/msg/async/AsyncConnection.cc
	src/msg/async/AsyncConnection.h

- SERVER_NAUTLIUS feature recently added to default
- msgr2 bool
- ceph::shared_ptr -> std::shared_ptr
2018-07-09 08:07:29 -05:00
Jason Dillaman
74ec64b1da
Merge pull request #22778 from trociny/wip-mock-warnings
test/librbd: fix gmock warning in TestMockIoImageRequestWQ.AcquireLockError

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-07-09 09:04:20 -04:00
Lenz Grimmer
ebbc3bc241
Merge pull request #22903 from tspmelo/fix-e2e-script
mgr/dashboard: Fix e2e script

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-07-09 14:37:32 +02:00
Nathan Cutler
693beaa8f3 run-make-check.sh: fill in missing DRY_RUN and add doc hint
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:05 +02:00
Nathan Cutler
458a0a6e62 run-make-check.sh: reset trap after sourcing install-deps.sh
Since we source install-deps.sh, it replaces our trap with its own,
so we have to reinstate our trap after it finishes.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
b79b463f90 cleanup: run-make-check.sh: no space between -D and name of cmake option
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
3d6909b0f3 run-make-check.sh: gtest-parallel only with Python 2
Fixes: http://tracker.ceph.com/issues/24817
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
f9c0691f84 run-make-check.sh: inform user of make -j setting
This can help the user figure out why the build is so slow.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
43fe00e90c doc: cleanup: wordsmith a comment in run-make-check.sh
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:04 +02:00
Nathan Cutler
2315928acd run-make-check.sh: ccache goodness for everyone
Since run-make-check.sh already ensures that ccache is installed,
it makes sense to let everyone benefit from the ccache
tweaks introduced by 4cb5a59053

Note 1: The previous solution using "date" would cause build tools to reset
their timestamps after 24 hours, on subsequent runs of run-make-check.sh.
In order to maximize ccache effectiveness, this commit sets SOURCE_DATE_EPOCH
to a fixed value: the number of seconds elapsed since the Unix epoch as at
January 1, 2000 (chosen to commemorate Y2K armageddon).

Note 2: this commit introduces "set -e". This was actually in effect
before, via "source install-deps.sh". Better to make it explicit.

Fixes: http://tracker.ceph.com/issues/24777
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:29:01 +02:00
Nathan Cutler
c8691cd9a3 run-make-check.sh: drop workaround for undeclared jq build dependency
jq build dependency is properly declared now.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-09 14:25:40 +02:00
Kefu Chai
2a6096d8c8
Merge pull request #22898 from majianpeng/bluestore-remove-redundant-fault-range
os/bluestore: remove redundant fault_range.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-09 19:38:11 +08:00
Kefu Chai
0d03be4f26
Merge pull request #22893 from jcsp/wip-misc
mgr: Miscellaneous small mgr fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-09 19:36:50 +08:00
Kefu Chai
06afa90b28
Merge pull request #21962 from socketpair/perf
osd: Additional fields for osd "bench" command

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-09 19:36:02 +08:00
Kefu Chai
9888cac80e cmake: cleanup FindCython.cmake
print out error message if cython module is not found.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-09 17:13:56 +08:00
Patrick Donnelly
b59d2ee04e
Merge PR #22836 into master
* refs/pull/22836/head:
	qa: remove unused distro dir

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2018-07-08 21:07:31 -07:00
Patrick Donnelly
0918f8fd4f
Merge PR #22741 into master
* refs/pull/22741/head:
	qa: fix ffsb cd argument
	qa: fix cd argument during iogen unpack

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-07-08 21:06:26 -07:00
Patrick Donnelly
e18c126044
githubmap: update contributors
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-08 21:06:25 -07:00
Patrick Donnelly
4a38cd5954
Merge PR #22766 into master
* refs/pull/22766/head:
	mds: refactor const ref return to avoid mem alloc

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2018-07-08 21:05:29 -07:00
Patrick Donnelly
ac6f789e88
githubmap: update contributors
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-07-08 21:05:28 -07:00
Patrick Donnelly
86c817febc
Merge PR #22740 into master
* refs/pull/22740/head:
	qa: create common conf for all cephfs suites
	qa: remove wrongly created random distro conf

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-07-08 20:56:09 -07:00
James McClune
575a5af0e0
doc: fixed grammar in restore rbd image section
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-07-08 22:53:59 -04:00
Nathan Cutler
a1d4f9d57c
Merge pull request #22922 from smithfarm/wip-argparse-py3
tests: make test_ceph_argparse.py pass on py3-only systems

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-07 09:52:04 +02:00
Kefu Chai
71e4255f3b
Merge pull request #22912 from tchaikov/wip-cmake-cleanup
cmake: remove workarounds for supporting cmake 2.x

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-07-07 14:38:54 +08:00
Nathan Cutler
5ed075d77f tests: make test_ceph_argparse.py pass on py3-only systems
On systems that do not have Python 2 installed, this test fails to
import StringIO.

Fixes: http://tracker.ceph.com/issues/24816
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-07-07 00:47:34 +02:00
Florian Haas
c30ede0b3a doc: Fix minor documentation issues in Mimic release notes
Small grammar glitches, typos, minor style issues.

Signed-off-by: Florian Haas <florian@citynetwork.eu>
2018-07-06 23:28:09 +02:00
Florian Haas
63f93d524c doc: Mention PURGED_SNAPDIRS and RECOVERY_DELETES in Mimic release notes
71d1dcbb98 started enforcing the
PURGED_SNAPDIRS osdmap flag before am upgraded Mon could join the
quorum.

9b80b14783 extended that check to also
include RECOVERY_DELETES. The Mimic release notes do not mention these
flags as prerequisites for an upgrade beyond Luminous. That creates an
obvious issue for users who skipped Luminous in production, and now
want to upgrade from Jewel to Mimic in, say, a weekend.

Update the release notes to include those flags as prerequisites for a
Luminous to Mimic upgrade, explain how users can make sure that they
are set, and also give users a one-liner to fix up their PGs in a
pinch, if they need to.

Signed-off-by: Florian Haas <florian@citynetwork.eu>
2018-07-06 23:28:09 +02:00
Sage Weil
d5468145b3 Merge PR #22863 into master
* refs/pull/22863/head:
	rgw: fix gc may cause a large number of read traffic

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2018-07-06 13:51:30 -05:00
Yuri Weinstein
cb467fccb9
Merge pull request #20779 from oritwas/wip-rgw-23257
rgw: radogw-admin reshard status command should print text for reshard status

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-07-06 11:49:00 -07:00
Yuri Weinstein
6e156812d8
Merge pull request #22721 from linuxbox2/wip-rgw-23099
rgw: escape markers in RGWOp_Metadata_List::execute

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-07-06 11:46:13 -07:00
Sage Weil
884a0d24ab Merge PR #22745 into master
* refs/pull/22745/head:
	debian/control: require sudo
	ceph.spec: require sudo
	ceph.spec: install etc/sudoers.d/ceph-osd-smartctl
	debian: install etc/sudoers.d/ceph-osd-smartctl
	sudoers.d/ceph-osd-smartctl: allow user ceph to run smartctl as root

Reviewed-by: Erwan Velu <erwan@redhat.com>
2018-07-06 13:43:15 -05:00
Sage Weil
f5df6ddeb2 Merge PR #22858 into master
* refs/pull/22858/head:
	osd/PrimaryLogPG: rebuild attrs from clients

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-07-06 13:43:03 -05:00
Sage Weil
bc4381691a Merge PR #22870 into master
* refs/pull/22870/head:
	mgr: replace iteritems with six.iteritems (mgr_module.py)
	mgr: replace iteritems with six.iteritems (crash)
	mgr: replace iteritems with six.iteritems (telegraf)
	mgr: replace iteritems with six.iteritems (status)
	mgr: replace iteritems with six.iteritems (restful)
	mgr: replace iteritems with six.iteritems (influx)
	mgr: replace iteritems with six.iteritems (balancer)

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Wido den Hollander <wido@42on.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-07-06 13:42:58 -05:00
Sage Weil
dc87c3c580 githubmap: update contributors
Signed-off-by: Sage Weil <sage@redhat.com>
2018-07-06 13:42:57 -05:00
Sage Weil
15732cdfc1 Merge PR #22871 into master
* refs/pull/22871/head:
	deb/rpm: add python-six as build-time and run-time dependency

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-07-06 13:42:38 -05:00
Sage Weil
7d287ff30a Merge PR #22873 into master
* refs/pull/22873/head:
	os/bluestore: fix incomplete faulty range marking when doing compression

Reviewed-by: Sage Weil <sage@redhat.com>
2018-07-06 13:42:34 -05:00
Kefu Chai
34f1ee7897 cmake: remove workarounds for supporting cmake 2.x
and bump up the required cmake version to 3.5.1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-07-07 01:24:02 +08:00