Commit Graph

108777 Commits

Author SHA1 Message Date
Casey Bodley
ef8eeb59e2
Merge pull request #33628 from cbodley/wip-44353
rgw: fix sharded bucket listing with prefix/delimiter

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-03-02 09:12:34 -05:00
Lenz Grimmer
04e2058c6a
Merge pull request #31922 from rhcs-dashboard/42094-bucket-mfa-delete
mgr/dashboard: enable/disable MFA Delete on RGW bucket

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-02 13:55:52 +00:00
Lenz Grimmer
816fe8dee7
Merge pull request #33178 from rhcs-dashboard/42475-rgw-api-keys-allowed
mgr/dashboard: do not show RGW API keys if only read-only privileges

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-03-02 13:54:31 +00:00
Radoslaw Zarzynski
623dadc368 osd: drop unnecessary includes of messages/MOSDPGTrim.h.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-03-02 14:12:03 +01:00
Radoslaw Zarzynski
da862449a0 osd: peering updates peer_last_complete_ondisk via setter.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-03-02 14:05:48 +01:00
Volker Theile
ba9047eded mgr/dashboard: Not able to restrict bucket creation for new user
Hide the different meanings of max_buckets from the user by improving the UI.

Fixes: https://tracker.ceph.com/issues/44322

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-03-02 13:39:35 +01:00
luo rixin
02ae7abf16 test/test_socket: fix dispatch_sockets() unexpected exception
In the test test_unexpected_down(), there is client running on CPU0
and server runing on CPU1, the server maybe finished before the client
(client and other workloads share CPU0), so client couldn't read data
and throws "dispatch_sockets(): cb_client() got unexpected exception
std::system_error (error crimson::net:4, read eof)"

Signed-off-by: luo rixin <luorixin@huawei.com>
2020-03-02 20:16:24 +08:00
Adam Kupczyk
b426e52d04 common/mempool: Add test for mempool in btree_map
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2020-03-02 12:18:48 +01:00
Kefu Chai
28c08615e5
Merge pull request #33638 from tchaikov/wip-btree
include/cpp-btree: use the same type when allocate/deallocate

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2020-03-02 19:16:09 +08:00
Kefu Chai
59546f6a49
Merge pull request #33645 from liewegas/fix-mon-down-cephtool-test
qa/suites/rados/singleton-bluestore/cephtool: whitelist MON_DOWN

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 19:11:42 +08:00
Kefu Chai
e1ee209087
Merge pull request #33536 from yuvalif/sync_object_write_with_return_value_op
librados: allow passing flags to operate sync APIs

Reviewed-By: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-02 19:10:29 +08:00
Kefu Chai
fdb99d6b18
Merge pull request #33583 from rzarzynski/wip-recovery-cleanup1
osd: tiny clean-ups around the backfill

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 19:01:29 +08:00
Kefu Chai
c6088bdd26
Merge pull request #33593 from dzafman/wip-cot-fix
test: Fix failing ceph_objectstore_tool.py test

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 18:58:19 +08:00
Patrick Seidensal
1794b55e64 monitoring: restore lost pool full alert
Fixes: https://tracker.ceph.com/issues/44366

Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
2020-03-02 11:43:03 +01:00
Kefu Chai
3ae08245a1
Merge pull request #33373 from rzarzynski/wip-bl-bumpup_api_verion
common, include: bump the version of ceph::buffer's C++ API.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 18:25:21 +08:00
Lenz Grimmer
341a76e500
Merge pull request #33572 from p-se/wip-pse-grafana-unit-issue
mgr/dashboard: fix stray requests/error in Grafana unit test

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-03-02 10:24:30 +00:00
Kefu Chai
e50ff2c89f
Merge pull request #31041 from liu-chunmei/ceph_seastar_alien_blue_store
crimson:: add alien blue store

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 18:15:24 +08:00
Kefu Chai
5af852c800
Merge pull request #33595 from cyx1231st/wip-crimson-osd-fixes
crimson/osd: misc fixes for OSD reboot-ability

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 17:36:10 +08:00
Kefu Chai
cc0905b396
Merge pull request #33597 from optimistyzy/228_improve_sync_read
NVMEDevice: Remove the unnecessary aio_wait in sync read

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 17:30:34 +08:00
Kefu Chai
6b74b68f66 include/cpp-btree: use the same type when allocate/deallocate
btree_set<> by default uses `std::allocator<Key>`, and btree_map by
default uses `std::allocator<std::pair<Key, Value>>`.

before this change, btree uses the allocator directly for allocating n
elements where element is `Key` or `std::pair<Key, Value>` respectively,
while "n" is actually supposed to be the number of bytes used by each
node which is being allocated.
but, what we need to allocate is actually a "node_type" for holding
multiple slots, and each slot holds an element. in addition to the
slots, a node also keeps track of metadata for btree itself. in short,
what we allocate now is (in bytes):

  alignof(sizeof(node_type)) * sizeof(element)

but what we should allocate is (in bytes):

  alignof(sizeof(node_type))

in this change:

* always rebind the allocator to the correct aligned type with given
  alignment
* extract the allocator related helpers into a template class

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-02 16:27:30 +08:00
Kefu Chai
db80838260
Merge pull request #33650 from joscollin/wip-unused-nvme
test: silence warning unused variable nvme

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-02 13:26:00 +08:00
Jos Collin
fb3dabdb21
test: silence warning unused variable nvme
Signed-off-by: Jos Collin <jcollin@redhat.com>
2020-03-02 09:18:20 +05:30
Sage Weil
3c3df70733 Merge PR #33512 into master
* refs/pull/33512/head:
	os/bluestore/BlueFS: less verbose about alloc adjustments

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-03-01 19:53:35 -06:00
Sage Weil
4ad485d285 Merge PR #33575 into master
* refs/pull/33575/head:
	mgr/telemetry: fix and document proxy usage

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2020-03-01 19:53:25 -06:00
Jianpeng Ma
866acb724c osd/PrimaryLogPG: simple debug message.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2020-03-02 09:05:41 +08:00
Sage Weil
461a80689e Merge PR #33644 into master
* refs/pull/33644/head:
	qa/suites/rados/verity/tasks/mon_recovery: whitelist SLOW_OPS

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-01 17:15:29 -06:00
Sage Weil
d3305e94d1 Merge PR #33541 into master
* refs/pull/33541/head:
	mgr/test_orchestrator: add daemon_action
	qa/tasksmgr/test_orchestrator_cli: fix test_service_action
	qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
	mgr/test_orchestrator: update_foo -> apply_foo
	mgr/test_orchestrator: add force flag to remove_daemons
	qa/tasks/mgr/test_orchestrator_cli: update
	qa/tasks/mgr/test_orchestrator_cli: fix test_load_data
	mgr/test_orchestrator: fix list_daemons

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-01 16:55:06 -06:00
Sage Weil
ee76d796f0 Merge PR #33627 into master
* refs/pull/33627/head:
	doc/cephadm/administration: clarify log gathering

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-01 14:26:09 -06:00
Sage Weil
436df900b4 mgr/orch: ServiceSpec: drop 'count'
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 14:19:35 -06:00
Sage Weil
157e4eb832 mgr/rook: use spec.placement.count (instead of spec.count)
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 14:19:35 -06:00
Sage Weil
f6e289ec79 mgr/cephadm: make HostAssignment make sense
- if hosts are passed, use those.
- if all_hosts=true, use all hosts.

Otherwise, build a set of hosts based on the labels--either explicit or
implied.  If there's no label, use all hosts.

If there is a count, use a subset of candidate hosts.  If there was no
label and there is no count, fail.

If count and hosts are both provided, then we either (1) use the hosts
as the candidate list and place among them (if len(hosts) >= count), or
(2) ensure that any result includes *at least* the provided hosts.

Add tests that (I'm pretty sure) cover cases!

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 14:19:35 -06:00
Sage Weil
71d8d06f82 mgr/orch: PlacementSpec: do not combine all_hosts with anything else
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 14:19:35 -06:00
Sage Weil
7de93c1253 mgr/orch: use PlacementSpec.from_strings() for all CLI commands
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 14:19:35 -06:00
Sage Weil
e9eb094538 Merge PR #33553 into master
* refs/pull/33553/head:
	mgr/cephadm: orch ls: include specs with no daemons
	mgr/cephadm: orch ls: show spec size
	mgr/orch: remove unused fields in RGWSpec
	mgr/orch: fix ServiceSpec fields
	mgr/cephadm: simplify spec apply
	pybind/mgr/mgr_module: revert PersistentStoreDict seperator
	mgr/cephadm: apply services after refreshing inventory
	mgr/cephadm: catch and log exceptions from apply
	mgr/orch: no extra whitespace in stored json specs
	mgr/cephadm: drop daemon_type arg to _apply_service
	mgr/cephadm: use _apply() helper for all apply_ methods
	mgr/cephadm: replace PersistentStoreDict with SpecStore
	mgr/cephadm: do not remove service spec when removing a daemon
	mgr/cephadm: rename completion variables&cleanup
	mgr/cephadm: leverage service specs

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-01 14:18:39 -06:00
Sage Weil
d90cff8cf0 mgr/test_orchestrator: add daemon_action
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 14:18:16 -06:00
Sage Weil
114ef48739 qa/tasksmgr/test_orchestrator_cli: fix test_service_action
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 12:24:04 -06:00
Sage Weil
ae2b28bd0a vstart.sh: fix install of cephadm ssh keys from ~/.ssh
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 12:06:19 -06:00
Sage Weil
9639acfefe qa/tasks/test_orchestrator_cli: disable test_osd_create exception check
I'm not sure what behavior we want here...

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 11:15:51 -06:00
Sage Weil
785348203b mgr/cephadm: orch ls: include specs with no daemons
This way you see entries like

NAME           RUNNING  REFRESHED  IMAGE NAME                                      IMAGE ID        SPEC
mds.foo            0/1  -          <unknown>                                       <unknown>     present

before any daemons have been created.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 09:05:58 -06:00
Sage Weil
6ad2c2fa94 mgr/cephadm: orch ls: show spec size
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 09:05:54 -06:00
Sage Weil
72e91c8e6a mgr/orch: remove unused fields in RGWSpec
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 09:05:50 -06:00
Sage Weil
bfb9b4e8c4 mgr/orch: fix ServiceSpec fields
- service_type is required.  Make it the first position arg to the ctor.
- service_id is the id *only* and optional.
- service_name() is the full service name (no change)

The old 'name' was previously used as the id only, so it was poorly named,
and optional, but in this series was changed to include the type, breaking
naming for a bunch of things (e.g., daemons called mds.mds.fsname.xyz).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 09:04:18 -06:00
Sage Weil
05035638f8 mgr/cephadm: simplify spec apply
- Teach _apply_service how to pick the create (and config) functions, so
  that we don't need any weird wrappers in the callers.
- Replace trigger_deploy() and _apply_services() with a simpler
  _apply_all_services()
- Drop all of the per-type _apply_foo() methods.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
c6af0b4ac9 pybind/mgr/mgr_module: revert PersistentStoreDict seperator
This fixes the self-test.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
d00186cbb5 mgr/cephadm: apply services after refreshing inventory
Ensure we have an inventory to apply against.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
1d4e683b6c mgr/cephadm: catch and log exceptions from apply
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
45f944aa33 mgr/orch: no extra whitespace in stored json specs
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
8b5fa1454f mgr/cephadm: drop daemon_type arg to _apply_service
This is part of ServiceSpec now.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
4580eaac65 mgr/cephadm: use _apply() helper for all apply_ methods
Clean up a few that were missed (due to racing PRs, presumably).

'creation' -> 'update', since an apply doesn't necessarily create (it may
destroy!).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00
Sage Weil
1980250ab8 mgr/cephadm: replace PersistentStoreDict with SpecStore
Explicit implementation of the dict of specs.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-01 08:10:54 -06:00