Commit Graph

92594 Commits

Author SHA1 Message Date
Casey Bodley
3f8b580c80
Merge pull request #25123 from yehudasa/wip-37281
rgw: change the way sysobj filters raw attributes, fix bucket sync state xattrs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-29 14:21:56 -05:00
Casey Bodley
52bdc3738e
Merge pull request #25078 from mkoderer/rgw-ux
[rgw]: Fix help of radosgw-admin user info in case no uid

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2018-11-29 14:20:58 -05:00
Casey Bodley
bbe075fc3c
Merge pull request #25068 from cbodley/wip-rgw-rados-yield
rgw: RADOS::Obj::operate takes optional_yield

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-11-29 14:18:54 -05:00
Casey Bodley
c4cb665708
Merge pull request #25310 from cbodley/wip-37448
rgw: data sync accepts ERR_PRECONDITION_FAILED on remove_object()

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2018-11-29 14:18:12 -05:00
Casey Bodley
1c4d44d98e
Merge pull request #25081 from cbodley/wip-37091
rgw: catch exceptions from librados::NObjectIterator

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-29 14:17:32 -05:00
jtlayton
3e90ec9ccc
Merge pull request #25329 from jtlayton/wip-orchestrator
orchestrator_cli: fix HandleCommandResult invocations in _status()
2018-11-29 12:52:53 -05:00
Kefu Chai
8f98c6e5e2
Merge pull request #25314 from tchaikov/wip-37432
qa/suites/rados/upgrade: set require-osd-release to nautilus

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-11-30 01:49:40 +08:00
Jeff Layton
2606abf91c mgr/volumes: fix "perm" field on ceph fs volume commands
They're all currently set as "r" but most of them should be "rw".

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-11-29 12:29:36 -05:00
Jeff Layton
c742c85a30 mgr/volumes: fix description text for ceph fs volume create
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-11-29 12:29:31 -05:00
Jeff Layton
bda6697862 mgr/volumes: call correct method to wait for the orchestrator
We need to use _orchestrator_wait here.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-11-29 12:29:26 -05:00
Jeff Layton
af80dbbeaa orchestrator_cli: fix HandleCommandResult invocations in _status()
These strings were being passed in as positional parameters, but we
need to pass them in as odata=.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-11-29 10:52:08 -05:00
Kefu Chai
c0e4fedee4
Merge pull request #25128 from dragonylffly/rb
tools/rados: allow reuse object for write test

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 23:41:22 +08:00
Kefu Chai
1cc9b48c88
Merge pull request #25317 from tchaikov/wip-37444
mgr: don't write to output if EOPNOTSUPP

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-29 23:38:12 +08:00
Kefu Chai
752098d72b
Merge pull request #25318 from javacruft/collections.abc
qa,pybind,tools: Correct usage of collections.abc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 23:36:44 +08:00
Kefu Chai
709e7ff405
Merge pull request #24822 from Rjerk/wip-fix-devicehealth-typo-error
mgr/devicehealth: fix is_valid_daemon_name typo error

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 23:34:05 +08:00
Ricardo Dias
088fbff7df
spec: fix cython package version to less than 0.29
Cython version 0.29 removed the support for python subinterpreters,
which completely breaks ceph-mgr funcionality.

See cython repo commit:
7e27c7cd51

Fixes: http://tracker.ceph.com/issues/37472

Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-11-29 15:31:16 +00:00
Sebastian Wagner
9c515d85db
Merge pull request #25323 from sebastian-philipp/orchestrator-utf8
mgr/orchestrator: remove unicode whitespaces

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2018-11-29 15:11:31 +01:00
Juan Miguel Olmo Martínez
526cdcdf01
mgr/ansible: Ansible orchestrator module
A Ceph Manager Orchestrator that uses a external REST API service to execute Ansible playbooks.

get_inventory implementation

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

Document how to use CLI through Orchestrator CLI

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2018-11-29 14:54:43 +01:00
Sebastian Wagner
0eda9989da mgr/orchestrator: remove unicode whitespaces
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-11-29 13:50:58 +01:00
Kefu Chai
04fb511d56
Merge pull request #25311 from wjwithagen/wjw-fix-cores-EXPECT_DEATH
test: suppress core dumping in there tests as well

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 19:26:45 +08:00
James Page
20aa3ad17b Correct usage of collections.abc
Some classes should still be imported directly from collections;
only OrderedDict, Iterable and Callable (in the context of the
ceph codebase) are found in collections.abc.

The current code works due to the fallback support for Python 2.

Signed-off-by: James Page <james.page@ubuntu.com>
2018-11-29 09:48:43 +00:00
Igor Fedotov
328ab2d9f0 osd/OSD: OSD::mkfs asserts when reusing disk with existing superblock.
Fixes: http://tracker.ceph.com/issues/37404

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-11-29 12:48:23 +03:00
Igor Fedotov
d07c10dfc0 os/bluestore: add main device expand capability.
One can do that via ceph-bluestore-tool's bluefs-bdev-expand command

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-11-29 12:48:20 +03:00
Willem Jan Withagen
99ea7fdb02 test: suppress core dumping in there tests as well
On FreeBSD coredumps are enabled by default.
And thus tests in teardown() in ceph-helpers.sh can fail because of
unwaranted cores.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-11-29 10:25:01 +01:00
Li Wang
7f141e756d tools/rados: allow reuse object for write test
Currently, for rados bench write test, it always
creates new objects for testing. Create operation
refers to non-neglectable metadata overhead, especially
for small write performance. This patch allows to
reuse objects for write testing

Signed-off-by: Li Wang <laurence.liwang@gmail.com>
2018-11-29 09:13:28 +00:00
Kefu Chai
3733f82f33
Merge pull request #20952 from shangfufei/wip-async-external-event
msg/async: Skip the duplicated processing of the same link

Reviewed-by: Haomai Wang <haomai@xsky.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 17:12:15 +08:00
Kefu Chai
87bb80e866
Merge pull request #25302 from wjwithagen/wjw-wip-build-kv_store_2
include: define errnos if not defined for better portablity

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 17:11:15 +08:00
Kefu Chai
f53b893272
Merge pull request #25264 from tchaikov/wip-mds-mem-frag
denc: fix internal fragmentation when decoding ptr in bl

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-29 17:10:25 +08:00
Kefu Chai
78fb1d79a2
Merge pull request #24878 from trociny/wip-36656
mgr: race between daemon state and service map in 'service status'

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-11-29 17:09:25 +08:00
Kefu Chai
8d42e80941
Merge pull request #24733 from tchaikov/wip-19348
pybind: simplify timeout handling in run_in_thread()

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2018-11-29 17:05:41 +08:00
Kefu Chai
8010d91960 mgr: don't write to output if EOPNOTSUPP
if process_pg_map_command() fails to fulfill the request, we should keep
the odata intact. and let the module take care of it.

before this change, we always write to odata even if
process_pg_map_command() returns -EOPNOTSUPP, this leaves unnecessary
leftover in the output, like pg_info,pg_ready.

after this change, we won't touch odata if process_pg_map_command()
returns -EOPNOTSUPP. and odata will be filled with whatever the python
module returns.

Fixes: http://tracker.ceph.com/issues/37444
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-29 15:45:37 +08:00
shangfufei
921f8f3b94 msg/async: Skip the duplicated processing of the same link.
Update Event.cc

Signed-off-by: shangfufei <shangfufei@inspur.com>
2018-11-29 14:53:16 +08:00
Kefu Chai
105ca218ee qa/suites/rados/upgrade: set require-osd-release to nautilus
* add qa/releases/nautilus.yaml so it can be reused.
* use releases/nautilus.yaml in luminous-x upgrade test, so
  test_librbd_python.sh is able to use the feature introduced in
  nautilus.

Fixes: http://tracker.ceph.com/issues/37432
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-29 14:35:17 +08:00
Patrick Donnelly
590babe826
Merge PR #24580 into master
* refs/pull/24580/head:
	mds: bump mds protocol version
	mds: handle fragment notify race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-11-28 19:54:13 -08:00
Casey Bodley
56021eab95 qa/rgw: add credentials to zone 'remove' in test_zonegroup_remove()
zone credentials are required to 'period update --commit' from
--rgw-zone remove

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Conflicts:
	src/test/rgw/rgw_multi/tests.py
2018-11-29 08:12:22 +08:00
yuliyang
6ecaec926f rgw: get or set realm zonegroup zone need check user's caps
fix: https://tracker.ceph.com/issues/37352

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
2018-11-29 08:11:09 +08:00
Casey Bodley
f9d0ccf006 rgw: data sync accepts ERR_PRECONDITION_FAILED on remove_object()
sync of deletes uses an If-UnModified-Since precondition, but does not
handle the corresponding ERR_PRECONDITION_FAILED error. treating this as
a failure means that we'll keep retrying the delete which will never
succeed. break this loop by treating ERR_PRECONDITION_FAILED as a
success

Fixes: http://tracker.ceph.com/issues/37448

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-11-28 13:56:52 -05:00
Sage Weil
706197a7c7 Merge PR #25272 into master
* refs/pull/25272/head:
	qa: add simple test-volumes.sh workunit and run it from fs/basic_functional
	vstart.sh: create default fs via 'fs volume create'
	mgr/volumes: fix oremote
	mon/MgrMonitor: enable volumes module by default
	mgr: create `volumes` module
	mgr: cleaner constructor for CommandResult
	mgr: block for latest osdmap after command execution
	mgr: add MgrModule.mon_command helper
	ceph_volume_client: enable using existing rados inst
	mon: give ceph-mgr access to 'fs' commands

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-11-28 11:34:57 -06:00
Casey Bodley
abfefe5f27 qa/rgw: fix invalid syntax error in radosgw_admin_rest.py
Fixes: http://tracker.ceph.com/issues/37440

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-11-28 10:53:23 -05:00
Casey Bodley
84f077f6e6
Merge pull request #25160 from theanalyst/rgw-empty-zg-fix
rgw: handle cases around zone deletion

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-28 10:24:13 -05:00
Matt Benjamin
7ea09b7d9e
Merge pull request #24662 from theanalyst/reshard-stale-cleanup
rgw: reshard stale instance cleanup
2018-11-28 10:22:46 -05:00
Sage Weil
43bf12e12d qa: add simple test-volumes.sh workunit and run it from fs/basic_functional
Signed-off-by: Sage Weil <sage@redhat.com>
2018-11-28 08:54:29 -06:00
Willem Jan Withagen
dbbe633687 build: FreeBSD allow kv_store tools to be build
This requires defining
`EUCLEAN`
`EKEYREJECTED`
which are defined in Linux asm-generic/error.h

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-11-28 14:49:02 +01:00
Mykola Golub
8a65b2a76d mgr/DaemonState: fix get_config_defaults locking
A fixup for 1c2bc53df7.

Fixes: http://tracker.ceph.com/issues/36590
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-11-28 15:27:52 +02:00
Lenz Grimmer
4db915d06d
Merge pull request #25290 from noahdesu/dashboard-tox-workdir
mgr/dashboard: use dedicated tox working dir

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2018-11-28 12:17:39 +01:00
Lenz Grimmer
ebd9a1632a
Merge pull request #25179 from tspmelo/wip-check-i18n
mgr/dashboard: Add i18n validation script

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-11-28 11:19:06 +01:00
Lenz Grimmer
720e0d4bfd
Merge pull request #24900 from zmc/wip-minimal-health
mgr/dashboard: Replace dashboard service

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-11-28 10:56:40 +01:00
Jan Fajerski
58316e3f82 ceph-volume: add test for disk.udevadm_property
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-11-28 10:21:36 +01:00
Jan Fajerski
41da5e9a43 ceph-volume: add device_id field to inventory output
This intends to mimic the C++ implementation in src/common/blkdev.cc.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2018-11-28 10:21:26 +01:00
xie xingguo
114c65fc0b osd: fix heartbeat brain-split behaviour
Yet another similar issue as 8d8e8a359c.
To reproduce, construct a cluster with 3 hosts, each containing a single osd only:
- cut off osd.1's cluster network, waiting osd.1 to be marked as down
- cut off both osd.2 & osd.3's cluster network

It is possible we'll get __two__ down osds (e.g., both osd.1 & osd.2 are down)
now and then restore osd.1 and osd.2's cluster network won't change anything.

The root cause is that by default we always call for at least 1/3 active heartbeat
connections with all current __up__ osds to bring a previously dead (unhealthy)
osd back to life. However, it is possible that the __up__ set could be the
minority part that has been cut off from the rest of the cluster entirely and hence
cause brain-split behaviour as demonstrated above.

The simplest way to fix is to try to re-activate an unhealthy osd whenever
we are still safe to do so. Also please keep in mind that frequent up-to-down
transitions will kill off the osd process entirely, and that is why the
```osd_markdown_log``` related checking is needed here..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-11-28 17:03:58 +08:00