Commit Graph

92499 Commits

Author SHA1 Message Date
Sage Weil
9f35081ae2 osd: requeue waiting peering events from deleted slots in unprime_split_children
These child slots we are deleting may have queued events that should be
requeued and reexamined.  For example, a pg query may be waiting for the
split and then discarded, instead of being reexamined and processed via
handle_pg_query_nopg().

Fixes: http://tracker.ceph.com/issues/37525
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-04 13:57:47 -06:00
Kefu Chai
c0e4fedee4
Merge pull request #25128 from dragonylffly/rb
tools/rados: allow reuse object for write test

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 23:41:22 +08:00
Kefu Chai
1cc9b48c88
Merge pull request #25317 from tchaikov/wip-37444
mgr: don't write to output if EOPNOTSUPP

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-29 23:38:12 +08:00
Kefu Chai
752098d72b
Merge pull request #25318 from javacruft/collections.abc
qa,pybind,tools: Correct usage of collections.abc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 23:36:44 +08:00
Kefu Chai
709e7ff405
Merge pull request #24822 from Rjerk/wip-fix-devicehealth-typo-error
mgr/devicehealth: fix is_valid_daemon_name typo error

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 23:34:05 +08:00
Sebastian Wagner
9c515d85db
Merge pull request #25323 from sebastian-philipp/orchestrator-utf8
mgr/orchestrator: remove unicode whitespaces

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2018-11-29 15:11:31 +01:00
Sebastian Wagner
0eda9989da mgr/orchestrator: remove unicode whitespaces
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-11-29 13:50:58 +01:00
Kefu Chai
04fb511d56
Merge pull request #25311 from wjwithagen/wjw-fix-cores-EXPECT_DEATH
test: suppress core dumping in there tests as well

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 19:26:45 +08:00
James Page
20aa3ad17b Correct usage of collections.abc
Some classes should still be imported directly from collections;
only OrderedDict, Iterable and Callable (in the context of the
ceph codebase) are found in collections.abc.

The current code works due to the fallback support for Python 2.

Signed-off-by: James Page <james.page@ubuntu.com>
2018-11-29 09:48:43 +00:00
Willem Jan Withagen
99ea7fdb02 test: suppress core dumping in there tests as well
On FreeBSD coredumps are enabled by default.
And thus tests in teardown() in ceph-helpers.sh can fail because of
unwaranted cores.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-11-29 10:25:01 +01:00
Li Wang
7f141e756d tools/rados: allow reuse object for write test
Currently, for rados bench write test, it always
creates new objects for testing. Create operation
refers to non-neglectable metadata overhead, especially
for small write performance. This patch allows to
reuse objects for write testing

Signed-off-by: Li Wang <laurence.liwang@gmail.com>
2018-11-29 09:13:28 +00:00
Kefu Chai
3733f82f33
Merge pull request #20952 from shangfufei/wip-async-external-event
msg/async: Skip the duplicated processing of the same link

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 17:12:15 +08:00
Kefu Chai
87bb80e866
Merge pull request #25302 from wjwithagen/wjw-wip-build-kv_store_2
include: define errnos if not defined for better portablity

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 17:11:15 +08:00
Kefu Chai
f53b893272
Merge pull request #25264 from tchaikov/wip-mds-mem-frag
denc: fix internal fragmentation when decoding ptr in bl

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-29 17:10:25 +08:00
Kefu Chai
78fb1d79a2
Merge pull request #24878 from trociny/wip-36656
mgr: race between daemon state and service map in 'service status'

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 17:09:25 +08:00
Kefu Chai
8d42e80941
Merge pull request #24733 from tchaikov/wip-19348
pybind: simplify timeout handling in run_in_thread()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2018-11-29 17:05:41 +08:00
Kefu Chai
8010d91960 mgr: don't write to output if EOPNOTSUPP
if process_pg_map_command() fails to fulfill the request, we should keep
the odata intact. and let the module take care of it.

before this change, we always write to odata even if
process_pg_map_command() returns -EOPNOTSUPP, this leaves unnecessary
leftover in the output, like pg_info,pg_ready.

after this change, we won't touch odata if process_pg_map_command()
returns -EOPNOTSUPP. and odata will be filled with whatever the python
module returns.

Fixes: http://tracker.ceph.com/issues/37444
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-29 15:45:37 +08:00
shangfufei
921f8f3b94 msg/async: Skip the duplicated processing of the same link.
Update Event.cc

Signed-off-by: shangfufei <shangfufei@inspur.com>
2018-11-29 14:53:16 +08:00
Patrick Donnelly
590babe826
Merge PR #24580 into master
* refs/pull/24580/head:
	mds: bump mds protocol version
	mds: handle fragment notify race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-28 19:54:13 -08:00
Sage Weil
706197a7c7 Merge PR #25272 into master
* refs/pull/25272/head:
	qa: add simple test-volumes.sh workunit and run it from fs/basic_functional
	vstart.sh: create default fs via 'fs volume create'
	mgr/volumes: fix oremote
	mon/MgrMonitor: enable volumes module by default
	mgr: create `volumes` module
	mgr: cleaner constructor for CommandResult
	mgr: block for latest osdmap after command execution
	mgr: add MgrModule.mon_command helper
	ceph_volume_client: enable using existing rados inst
	mon: give ceph-mgr access to 'fs' commands

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-11-28 11:34:57 -06:00
Casey Bodley
84f077f6e6
Merge pull request #25160 from theanalyst/rgw-empty-zg-fix
rgw: handle cases around zone deletion

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-28 10:24:13 -05:00
Matt Benjamin
7ea09b7d9e
Merge pull request #24662 from theanalyst/reshard-stale-cleanup
rgw: reshard stale instance cleanup
2018-11-28 10:22:46 -05:00
Sage Weil
43bf12e12d qa: add simple test-volumes.sh workunit and run it from fs/basic_functional
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-28 08:54:29 -06:00
Willem Jan Withagen
dbbe633687 build: FreeBSD allow kv_store tools to be build
This requires defining
`EUCLEAN`
`EKEYREJECTED`
which are defined in Linux asm-generic/error.h

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-11-28 14:49:02 +01:00
Lenz Grimmer
4db915d06d
Merge pull request #25290 from noahdesu/dashboard-tox-workdir
mgr/dashboard: use dedicated tox working dir

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-11-28 12:17:39 +01:00
Lenz Grimmer
ebd9a1632a
Merge pull request #25179 from tspmelo/wip-check-i18n
mgr/dashboard: Add i18n validation script

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-11-28 11:19:06 +01:00
Lenz Grimmer
720e0d4bfd
Merge pull request #24900 from zmc/wip-minimal-health
mgr/dashboard: Replace dashboard service

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-11-28 10:56:40 +01:00
Kefu Chai
57160722b4
Merge pull request #25280 from tchaikov/wip-ceph-py3
ceph.in: write bytes to stdout in raw_write()

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2018-11-28 13:14:19 +08:00
Kefu Chai
3fb560a422
Merge pull request #25261 from sebastian-philipp/mgr-handle-command-result
mgr: Add `HandleCommandResult` namedtuple
2018-11-28 13:12:05 +08:00
Kefu Chai
73e92ab7fa
Merge pull request #25255 from tchaikov/wip-ceph-dencoder-without-cephfs
tools/ceph-dencoder: conditionally link against mds
2018-11-28 13:10:05 +08:00
Kefu Chai
1c97ce4aad
Merge pull request #25244 from trociny/wip-crash-procname
mgr/crash: add process name to crash metadata

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-28 13:08:53 +08:00
Kefu Chai
56142dfb23
Merge pull request #24990 from dragonylffly/wip-fix-bench
tools/rados: always call rados.shutdown() before exit()

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-28 13:07:59 +08:00
Kefu Chai
2a6d98127d
Merge pull request #24939 from marcosps/simplify_examples
examples/librados: Remove not needed else clauses

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-28 13:04:07 +08:00
Sage Weil
cdcc5b4521 Merge PR #24502 into master
* refs/pull/24502/head:
	crushtool: implement --rebuild-class-roots command
	crushtool: make --reweight re-sum choose_args weight-sets too
	crushtool: --reweight should only reweight nonshadow roots
	crush/CrushWrapper: reclassify: use default parent for created buckets
	crush/CrushWrapper: reclassify: handle to-be-created buckets that we need twice
	test/cli/crushtool/reclassify: add second gabe test case
	crushtool: add --set-subtree-class; do not set class via --reclassify-root
	test/cli/crushtool/reclassify: add reclassify test cases
	doc/rados/operations/crush*: document reclassify
	doc/rados/operations/crush: remove instructions for separate crush trees for ssd
	crushtool: add --compare command
	crushtool: implement --reclassify
	crush/CrushCompiler: fix id scan to include class ids

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-27 21:32:04 -06:00
Sage Weil
6fe1ea0dd5 crushtool: implement --rebuild-class-roots command
This simply rebuilds the class roots.  Normally this should create no
change in the map since whatever was making changes to the map before
should have rebuild the shadow roots at that point.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
0f6a36e1e4 crushtool: make --reweight re-sum choose_args weight-sets too
This ensures that the weights add us for each weight-set (and each
position).  Note that since we don't have anything that actually
creates positional weight-sets, the behavior here might not be what we
want in the end, but for the compat weight-sets (no position), we *do*
keep the weights as a properly summing tree.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
c9921dc7e9 crushtool: --reweight should only reweight nonshadow roots
The shadow roots will be then be rebuilt accordingly.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
dec38c69cb crush/CrushWrapper: reclassify: use default parent for created buckets
Note that these last 2 fixes also fix the 'h' test case.  \o/

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
c2211e851d crush/CrushWrapper: reclassify: handle to-be-created buckets that we need twice
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
1028bf832a test/cli/crushtool/reclassify: add second gabe test case
New adjusted test map from Dan.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
7a80f993f4 crushtool: add --set-subtree-class; do not set class via --reclassify-root
Sometimes we don't want the --reclassify-root to set the class of every
device because a small number of them are (correctly) a different class.
Allow both behaviors by adding a new, separate command to set the class
of all devices beneath a point in the hierarchy and do not implicitly do
that relabeling as part of --reclassify-root.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
cb327b8f42 test/cli/crushtool/reclassify: add reclassify test cases
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Sage Weil
505050befd doc/rados/operations/crush*: document reclassify
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-27 20:36:43 -06:00
Zack Cerza
50b7d42fe5 mgr/dashboard: Replace dashboard service
This splits out the collection of health and log data from the
/api/dashboard/health controller into /api/health/{full,minimal} and
/api/logs/all.

/health/full contains all the data (minus logs) that /dashboard/health
did, whereas /health/minimal contains only what is needed for the health
component to function. /logs/all contains exactly what the logs portion
of /dashboard/health did.

By using /health/minimal, on a vstart cluster we pull ~1.4KB of data
every 5s, where we used to pull ~6KB; those numbers would get larger
with larger clusters. Once we split out log data, that will drop to
~0.4KB.

Fixes: http://tracker.ceph.com/issues/36675

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 16:08:53 -07:00
Patrick Donnelly
54640f7cd8
Merge PR #24923 into master
* refs/pull/24923/head:
	mds: properly update freeze_tree_state when fragmenting dirfrags

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-27 14:47:03 -08:00
Noah Watkins
9b125b087a mgr/dashboard: use dedicated tox working dir
Avoids conflicting with other test environments.

Fixes: https://tracker.ceph.com/issues/37405

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-11-27 14:28:47 -08:00
Patrick Donnelly
3358e3a2d6
Merge PR #24849 into master
* refs/pull/24849/head:
	client: remove redundant abort logic during dentry invalidation test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-11-27 14:23:20 -08:00
Patrick Donnelly
b76f14569d
Merge PR #24886 into master
* refs/pull/24886/head:
	qa: fix delay type config name

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-11-27 13:58:26 -08:00
Patrick Donnelly
302e4f4af2
Merge PR #24839 into master
* refs/pull/24839/head:
	ceph-volume-client: allow setting mode of CephFS volumes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-27 13:53:50 -08:00
Zack Cerza
a90b5ac7c7 Allow specifying a schema for unknown JObj keys
Signed-off-by: Zack Cerza <zack@redhat.com>
2018-11-27 12:47:26 -07:00