Commit Graph

102711 Commits

Author SHA1 Message Date
Samuel Just
8986d0cc06 fio_ceph_objectstore: clean up formatted output, add option to write to file
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-09-27 15:55:18 -07:00
Kefu Chai
417e9318ee
Merge pull request #30526 from uweigand/endian-private
include,os: Make ceph_le member private

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-28 02:05:29 +08:00
Ulrich Weigand
7e2b15374c bloom-filter: Remove POD overloads for insert and contains
These are not used in Ceph code currently, and should not
be used in the future either, since any use will introduce
byte-order dependent behavior.  Remove them to prevent
accidental use.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-27 19:39:51 +02:00
Ulrich Weigand
ef17972992 bloom-filter: Improve test cases
The integer bloom filter test cases do not really match typical usage
of the bloom filter in actual Ceph code.  In particular:

- the tests use consecutive integer ranges, while Ceph code uses
  hash values uniformly distributed over the uint32_t space;

- the tests pass "int" to the insert and contains functions, which
  causes the generic C++ POD type overload to be selected instead
  of the uint32_t overload that is used by Ceph code.  The POD
  overload is dependent on host byte order, and behaves actually
  different from the uint32_t overload on little-endian systems.

To fix these issues, this patch changes the integer tests to
always pass in uint32_t (instead of int), and to use results
of a pseudo-random number generator instead of consecutive
sequences.  (We assume the period of the generator is long
enough that all values generated within one test instance
are distinct.)

This not only makes the test pass on both big- and little-endian
hosts now, but it also allows tightening of the allowable actual
false positive rates, as they now match much closer the expected
values.

Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
2019-09-27 19:39:51 +02:00
Sage Weil
dff5697464 Merge PR #30525 into master
* refs/pull/30525/head:
	qa/tasks/ceph.conf.template: disable power-of-2 warning
	qa/standalone/mon/health-mute: use power of 2 for pg_num
	osd/OSDMap: remove remaining g_conf() usage
	PendingReleaseNotes: add note for 14.2.5 so we can backport this
	osd/OSDMap: health alert for non-power-of-two pg_num

Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-27 12:08:37 -05:00
Sage Weil
964bed7e2f Merge PR #30431 into master
* refs/pull/30431/head:
	pybind/ceph_argparse: add :int or :float to numerical args
	pybind/ceph_argparse: simplify osd name and target types
	pybind/ceph_argparse: prefer field names to types in help output
	pybind/ceph_argparse: more concise n=N '...'
	pybind/ceph_argparse: [] (not {}) around optional args

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-09-27 10:11:58 -05:00
Lenz Grimmer
2a143bfd1f
mgr/dashboard: Support minimum password complexity rules (#29532)
mgr/dashboard: Support minimum password complexity rules

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-09-27 13:54:17 +00:00
Lenz Grimmer
1892a4e7a5
mgr/dashboard: CephFS snapshots/quota REST API (#30256)
mgr/dashboard: CephFS snapshots/quota REST API

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2019-09-27 13:45:28 +00:00
Sage Weil
4b8574059f Merge PR #30577 into master
* refs/pull/30577/head:
	qa/suites/rados/thrash-old-clients: tolerate MON_DOWN

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-09-27 08:22:27 -05:00
Patrick Seidensal
87136e61b2 mgr/dashboard: add smart self report summary to details of osd-smart-list
Fixes: https://tracker.ceph.com/issues/41774

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2019-09-27 14:58:53 +02:00
Lenz Grimmer
2ad8e2eed5
Merge pull request #30599 from votdev/issue_41797_url_prefix
mgr/dashboard: Configuring an URL prefix does not work as expected

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2019-09-27 12:31:32 +00:00
Jeff Layton
5b36293ab8 vstart_runner: allow the use of it with kernel mounts
Add a new command-line switch to allow it to use the kernel client
instead, and add all of the machinery to handle local kcephfs mounts.

Document this in the developer guide, along with the appropriate scary
warnings about using this on a machine that you care about. While we're
in there, also correct a typo about FUSE configuration.

Fixes: https://tracker.ceph.com/issues/41910
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-27 08:23:07 -04:00
Igor Fedotov
3f79b19768 os/bluestore: fix invalid stray shared blob detection in fsck.
This is a regression, caused by https://github.com/ceph/ceph/pull/30264

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-09-27 13:51:54 +03:00
Nathan Cutler
517337e1b1
Merge pull request #30597 from smithfarm/wip-opensuse-15.1
qa/distros/all: add openSUSE 15.1, drop openSUSE 12.2

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
2019-09-27 11:59:10 +02:00
Nathan Cutler
b4cec5a121
Merge pull request #30430 from smithfarm/wip-release-schedule-14.2.4
doc/releases/schedule.rst: add 14.2.3, 14.2.4, 15.0.0 and drop dumpling

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-09-27 11:58:32 +02:00
Jos Collin
a58a6cde31
doc: protection for 'fs volume rm' command
Fixes: https://github.com/ceph/ceph/pull/30381#discussion_r324866953
Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-09-27 15:17:39 +05:30
Jos Collin
f1b9e90d7f
qa/tasks: add/update tests for --yes-i-really-mean-it
Fixes: https://tracker.ceph.com/issues/41841
Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-09-27 15:17:34 +05:30
Jos Collin
59cf9beb31
mgr/volumes: protection for 'fs volume rm' command
Fixes: https://tracker.ceph.com/issues/41841
Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-09-27 15:17:27 +05:30
Lenz Grimmer
2f4c2440e8
Merge pull request #30478 from votdev/issue_41930_standby_dashboard
mgr/dashboard: Standby Dashboards don't handle all requests properly

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2019-09-27 09:37:37 +00:00
Sebastian Wagner
489d5b685b python-common: Add small Readme
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-09-27 11:16:47 +02:00
Kefu Chai
11c31c0995
Merge pull request #30501 from liewegas/bugfix-41913
ceph-objectstore-tool: better error message if pgid and object do not match

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-09-27 16:16:37 +08:00
Venky Shankar
0a5032138b doc: document mds journaling
Fixes: http://tracker.ceph.com/issues/41783
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2019-09-27 12:51:30 +05:30
Kefu Chai
a6da8559ff
Merge pull request #30518 from liewegas/wip-mon-less-snap-metadata
mon/OSDMonitor: clean up removed_snap keys

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-27 15:04:38 +08:00
Kefu Chai
c1ab6d436e
Merge pull request #30530 from dzafman/wip-42015
osd: Remove unused osdmap flags full, nearfull from output

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-27 15:01:39 +08:00
Patrick Donnelly
7baf44b89c
Merge PR #30406 into master
* refs/pull/30406/head:
	doc: move MDS deployment from main quick start to CephFS quick start
	doc: update CephFS Quick Start doc

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-09-26 23:51:15 -07:00
Patrick Donnelly
fa1ee5b4f6
Merge PR #30561 into master
* refs/pull/30561/head:
	doc/ceph-fuse: mention -k option in ceph-fuse man page

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2019-09-26 23:11:05 -07:00
Kefu Chai
a6c08712e7
Merge pull request #30404 from shun-s/cleanup-failure-pending
osd: make osd recover more smoothly by avoiding failure peer info to resent

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-27 13:50:17 +08:00
Kefu Chai
7ae9ffff65
Merge pull request #30517 from tchaikov/wip-test/dashboard/scheme-allow-unknown
qa/tasks/mgr/dashboard/test_health: add "allow_unknown" in mgr_map

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-09-27 13:36:23 +08:00
Patrick Donnelly
719cb3f5b4
Merge PR #30297 into master
* refs/pull/30297/head:
	test_cephfs_shell: add note on how to run this testsuite
	cephfs-shell: rewrite help text for put and get commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-09-26 22:34:28 -07:00
Kefu Chai
f1ffba31a5
Merge pull request #30533 from cxytt/fix-complete-to-not-reset
osd/PGLog: reset log.complete_to when recover obect failed

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-27 13:32:19 +08:00
Kefu Chai
de44ad700c
Merge pull request #30538 from tchaikov/wip/test/ceph_argparse/yet-another-test
mon.test: improve validation and add a test for "osd pool create"

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-27 13:31:38 +08:00
Kefu Chai
866eac0834
Merge pull request #30504 from Songweibin/wip-app-meta-get
pybind/rados: add application_metadata_get

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-27 13:24:56 +08:00
Kefu Chai
120dd3367d
Merge pull request #30421 from sebastian-philipp/prometheus-always-on-modules
mgr/prometheus: Fix KeyError in get_mgr_status

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-09-27 13:21:10 +08:00
Kefu Chai
3fcaa765cd
Merge pull request #30200 from NancySu05/dev
mon/OSDMonitor: trim not-longer-exist failure reporters

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-27 13:18:19 +08:00
Patrick Donnelly
07db3c12a1
Merge PR #29906 into master
* refs/pull/29906/head:
	vstart_runner: name booleans for options differently
	qa/vstart_runner.py: add an option to remove old log
	qa/vstart_runner.py: make log initialization code reusable
	qa/vstart_runner.py: make printing of stdout of ps optional
	qa/vstart_runner.py: add timeout for vstart.sh and stop.sh
	qa/vstart_runner.py: add an option to teardown cluster

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-09-26 22:12:03 -07:00
Kefu Chai
0308ffd7ab
Merge pull request #30612 from tchaikov/wip-install-deps.sh/do-not-fail-if-already-installed
install-deps: do not install if rpm already installed

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-09-27 11:24:09 +08:00
Kefu Chai
45a4173bcf install-deps: do not install if rpm already installed
there is chance that the system already have epel-release-latest-7.noarch.rpm
installed, in that case, install-deps.sh should not fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-27 11:07:30 +08:00
Xie Xingguo
2914402345
Merge pull request #30466 from xiexingguo/wip-41924
osd/PGLog: persist num_objects_missing for replicas when peering is done

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-09-27 10:18:33 +08:00
Xie Xingguo
d502a41b7a
Merge pull request #30576 from xiexingguo/wip-42052
osd/OSDMap: do not trust partially simplified pg_upmap_item

Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-27 10:17:54 +08:00
Xie Xingguo
5044a14c9a
Merge pull request #30560 from xiexingguo/wip-balancer-upmap-cli
mgr/balancer: enable pg_upmap cli for future use

Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-27 10:17:09 +08:00
Sage Weil
239c8e8564 Merge PR #30223 into master
* refs/pull/30223/head:
	msg/async: do not register lossy client connections
	msg/async: allow anonymous client-side connections

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-09-26 19:28:51 -05:00
Sage Weil
ca80c3fc21 osd: send ops back to primary if replica is not readable
This is the simplest strategy--much simpler than queueing them and
waking them up again later.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:22:22 -05:00
Sage Weil
1d0c6f5319 osd/PeeringState: set WAIT state and block ops to wait for prior readable_until
If we start a new interval and the prior interval may have OSDs that
are still readable, set the WAIT state bit and block operations until
sufficient time has elapsed.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:22:22 -05:00
Sage Weil
7aec060e0a osd: add CheckReadable pg event, queue_check_readable()
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:22:22 -05:00
Sage Weil
6119f693d6 osd/PeeringState: update prior_readable_until_ub on new interval
If a new interval starts, apply the old interval's readable_until_ub to
our prior.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:21:53 -05:00
Sage Weil
6945cc52fd osd/PeeringState: refresh prior_readable_until_ub in pg_history_t on share
Before we share pg_history_t, refresh the prior_readable_until_ub to be
a simple duration from *now*, so that it is completely clock-independent.
The receiver can interpret it based on the receive time for the message,
which loses a bit of precision but is safe since this is an upper bound.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:21:53 -05:00
Sage Weil
3bb8a7210a osd: requeue ops when PG is no longer laggy
When we get a lease_ack, leave the laggy state.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:21:53 -05:00
Sage Weil
9c4d780c5d osd/PrimaryLogPG: put ops on waitlist when PG is laggy
Move PG to laggy state if we aren't readable

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:21:53 -05:00
Sage Weil
c6358390e3 crimson/osd/pg: add schedule_renew_lease stub
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:21:53 -05:00
Sage Weil
1e87f1494e osd/PeeringState: do full on_new_interval on new peering interval
This looks like a bug from the PeeringState transition.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-26 12:21:53 -05:00