Commit Graph

106379 Commits

Author SHA1 Message Date
Radoslaw Zarzynski
6d590f617e msg/async: use dedicated method for clearing conn's priv up.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-01-20 12:19:32 +01:00
Radoslaw Zarzynski
c2f61b95f5 msg/async: mark down local_connection before draining the stack.
`AsyncMessenger` has `local_connection` – an instance of
`AsyncConnection` used to send messages to itself (loop back).
Connections are handled by `EventCenter` in its dedicated
thread. When shutting down a messenger, it must be ensured
there is no task left in the `EventCenter`'s queue. Otherwise
a use-after-free can happen. That's the reason why `shutdown()`
of `AsyncMessenger` does `mark_down_all()` on connections
**before** draining the stack. The latter actually injects a task
into all `EventCenter` instances and waits for its completion
(synchronization barrier).

However, that's not the case for `local_connection`. Without
the patch it's marked down by the messenger's destructor
far **after** the synchronization barrier. This behavior is
dangerous when an implementation of `mark_down()` creates
a new task to be executed inside the boundaries of corresponding
`EventCenter` instance.

The fix unifies handling of `local_connection` with other
connections in the aspect of the shutdown phase.

Fixes: https://tracker.ceph.com/issues/43667
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-01-20 12:15:22 +01:00
Abhishek L
af0e482a3f
Merge pull request #32367 from capri1989/doc-gc-update
doc/radosgw/config-ref: paragraph to explain the gc settings

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-01-17 10:08:29 +01:00
Sebastian Wagner
bbd18f1986
Merge pull request #32049 from mgfritch/wip-cd-timeout
cephadm: add ability to specify a timeout

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-01-17 08:32:06 +01:00
Kefu Chai
d3435c671e
Merge pull request #32691 from rosinL/wip-txpkt
msg/async/dpdk: Fix infinite loop when sending packets

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-17 14:32:28 +08:00
Chunsong Feng
eea0b39fce msg/async/dpdk: Fix infinite loop when sending packets
tx_pkt_burst loops until the next pointer is null, set next pointer of
the last segment nullptr to fix this issue.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2020-01-17 11:55:51 +08:00
Patrick Donnelly
5608724818
Merge PR #32114 into master
* refs/pull/32114/head:
	doc: split index.rst of dev guide
	doc: tweak few section titles in dev guide
	doc: don't display contents twice in dev guide
	doc: add guide for running tests with teuthology

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-16 14:35:38 -08:00
J. Eric Ivancich
edf25e0179
Merge pull request #32513 from IlsooByun/loop_rgw_none
rgw: move forward marker even in case of many rgw.none indexes

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-01-16 12:32:28 -05:00
Casey Bodley
290da6b57c
Merge pull request #31878 from yuvalif/wip-yuval-tag-filter
rgw/pubsub: notification filtering by object tags

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-01-16 11:04:19 -05:00
Daniel Gryniewicz
46209a180e
Merge pull request #32596 from linuxbox2/wip-rgwfile-pfront
rgw_file: avoid string::front() on empty path
2020-01-16 10:12:51 -05:00
Kefu Chai
bcb6bf2bec
Merge pull request #32453 from majianpeng/kv-rocksdbstore-fix-option-work
kv/RocksDBStore: make option: compaction_threads/disableWAL/flusher_t…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-01-16 23:11:26 +08:00
Kefu Chai
59713e3324
Merge pull request #32443 from tchaikov/wip-btree
include/cpp-btree: cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-01-16 23:09:52 +08:00
Sage Weil
8bc975fb8f Merge PR #32670 into master
* refs/pull/32670/head:
	doc: remove prod cluster examples from hardware recs

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-16 08:07:02 -06:00
Laura Paduano
197388f3bb
Merge pull request #31923 from tspmelo/wip-vertical-navigation
mgr/dashboard: Add vertical menu

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-01-16 14:43:19 +01:00
Sebastian Wagner
c9904ce762
Merge pull request #32654 from liewegas/fix-43273
mgr/rook: provide full path for devices names in inventory

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-01-16 10:18:23 +01:00
Sebastian Wagner
7c15378120
Merge pull request #32372 from sebastian-philipp/cephadm-progress
mgr/cephadm: Add progress to update_mgr()

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-01-16 09:55:41 +01:00
Zac Dover
dc46a8e90a doc: remove prod cluster examples from hardware recs
This PR removes the now-very-old (2012) production cluster
examples from the hardware recommendations page.

Sage requested this change in IRC on 14 Jan 2020. There is no issue link.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2020-01-16 13:14:50 +10:00
Jason Dillaman
fd0b6be7b7
Merge pull request #32497 from zhengchengyao/snap_exists
rbd: add snap_exists method API

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-01-15 20:15:39 -05:00
Patrick Donnelly
0e178461df
Merge PR #32541 into master
* refs/pull/32541/head:
	mds: add comment in feature_bitset_t::empty

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-15 12:40:05 -08:00
Sage Weil
8464955861 Merge PR #32619 into master
* refs/pull/32619/head:
	kv/RocksDBStore: break out of compaction thread early on shutdown
	kv/RocksDBStore: debug async compaction

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-15 12:57:42 -06:00
Laura Paduano
d225e182f1
Merge pull request #32569 from votdev/feature_toggle_output
mgr/dashboard: Cleanup feature toggle status output

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-01-15 16:47:13 +01:00
Laura Paduano
e726bcd33c
Merge pull request #32635 from Devp00l/wip-43594
mgr/dashbaord: Fix E2E pools page failure

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-01-15 15:57:42 +01:00
Sage Weil
23f5159487 mgr/rook: provide full path for devices names in inventory
This matches the cephadm behavior.

Fixes: https://tracker.ceph.com/issues/43273
Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-15 08:53:03 -06:00
Sage Weil
4b7f5de235 Merge PR #32411 into master
* refs/pull/32411/head:
	qa/tasks/ceph_manager: enable ceph-objectstore-tool via cephadm
	qa/tasks/cephadm: support crush_tunables config option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-01-15 08:39:48 -06:00
Sage Weil
2739ac4a60 qa/tasks/ceph_manager: enable ceph-objectstore-tool via cephadm
- drop support for keyvaluestore
- leave a few paths non-cephadm specific (filestore, upgrade workaround)

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-15 07:53:01 -06:00
Sage Weil
6a25e27992 Merge PR #32643 into master
* refs/pull/32643/head:
	doc/bootstrap: fixed default --keyring target

Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-15 07:47:24 -06:00
Jason Dillaman
010b99bd30
Merge pull request #32567 from trociny/wip-mirror-snap-promote
librbd: fix some edge cases for snapshot mirror mode promote

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-01-15 08:09:31 -05:00
Stephan Müller
04d5da7eee mgr/dashbaord: Fix E2E pools page failure
The problem was that pg autoscaling is set to "on" by default. Therefore
the pg_num field is hidden, which caused the error. Now the test sets
the pg autoscaling to "off" revealing the pg_num field to make the tests
pass.

Fixes: https://tracker.ceph.com/issues/43594
Fixes: https://tracker.ceph.com/issues/43102
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-01-15 09:40:59 +01:00
Stephan Müller
e93e51de15 mgr/dashboard: PageHelper.selectOption
This helper method selects an option inside a select element.
It will also expect that the option was set.

Fixes: https://tracker.ceph.com/issues/43594
Signed-off-by: Stephan Müller <smueller@suse.com>
2020-01-15 09:40:59 +01:00
Kefu Chai
b2631ba462
Merge pull request #32584 from aclamk/fix-bluefs-vselector
bluestore/bluefs: make accounting resiliant to unlock()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-01-15 12:38:57 +08:00
Kefu Chai
d8a3c605cb
Merge pull request #32625 from mgfritch/cephadm-workunit-test-cleanup
qa/workunits/cephadm/test_cephadm: workunit test cleanup

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-01-15 12:33:52 +08:00
Patrick Donnelly
dc018d8b79
Merge PR #31763 into master
* refs/pull/31763/head:
	test: auto-upgrade subvolume test
	mgr/volumes: tie everything together to implement versioned subvolumes
	mgr/volumes: provide subvolume create/remove/open APIs
	mgr/volumes: implement subvolume based on subvolume template
	mgr/volumes: implement subvolume group based on group template
	mgr/volumes: implement trash as a subvolume group
	mgr/volumes: snapshot util module
	mgr/volumes: template for implementing groups and subvolumes
	mgr/volumes: implement filesystem volume module
	mgr/volumes: lock module to serialize volume operations
	mgr/volumes: introduce volume specification module
	mgr/volumes: add fs_util helper module

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 20:32:44 -08:00
Adam Kupczyk
82fa73a411 bluestore/bluefs: make accounting resiliant to unlock()
Moved vselector.sub_usage, so sub_usage and add_usage will not cross
_flush_bdev_safely(), which contains unlock().

Fixes: https://tracker.ceph.com/issues/43538
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
2020-01-15 12:21:46 +08:00
Kefu Chai
a75b7564d6
Merge pull request #32615 from liewegas/fix-43580
osd/PG: clean up fastinfo key when last_update does not increase

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-15 12:18:42 +08:00
Kefu Chai
cc4797321a
Merge pull request #30985 from chenerqi/master
mgr: ceph fs status support json format

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-01-15 12:12:26 +08:00
Kefu Chai
ea542763c5 include/cpp-btree: use "constexpr inline" for constants
in C++17, it's a more general way to define constants. and these
constants are correctly typed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-15 11:11:54 +08:00
Kefu Chai
a960391ddc test/common/test_interval_set: use ASSERT_EQ()
ASSERT_EQ() is better than ASSERT_TRUE() when checking for equality, as
it is able to print the unmatched variables.

also, in ASSERT_EQ(), the first parameter is the expected value. so
transpose the parameters for better error messages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-15 11:11:54 +08:00
Kefu Chai
1cbde6daca include/interval_set: use prefix operator for non-trivial iterator
for better performance. no need to create a temporary instance of
iterator and throw it away

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-15 11:11:54 +08:00
Kefu Chai
73f2d70fc8 include/cpp-btree: adapt from abseil::btree
adapt based on btree implementation from Abseil Common Libraries.

with following changes:

* remove abseil dependencies. as abseil libraries do support C++11,
  they are written in C++11 compatible way. and use its homebrew
  alternatives for the facilities offered by C++14 and C++17. but
  Ceph is in C++17, so replace those facilities using the ones
  from the C++ standard library
* change `key_compare_to_adapter` to adapt to arithmetic types, and
  those who have `compare()` defined.
* move SlotPolicy into *_params types, simpler this way.
* continue using *_fields types. the layout facilities is nice. but
  we don't need this flexibility at this moment.
* continue using `empty_base_handle` for EBCO, CompressedTuple is
  nice, but it involves more dependencies which we don't want at
  this moment.
* node handle support is dropped. it's part of C++17 standard for
  node-based containers. we don't use it at this moment. but it's
  relatively simple if we want to add it back.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-15 11:11:54 +08:00
Kefu Chai
460a7849c1
Merge pull request #32630 from tchaikov/wip-43306
common/util: use ifstream to read from /proc files

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-15 10:57:13 +08:00
yaarith
5216945c55 doc/bootstrap: fixed default --keyring target
Changed target from 'ceph.keyring' to 'ceph.client.admin.keyring'.

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2020-01-14 21:56:43 -05:00
Kefu Chai
dbecfa0370 cpp-btree: import abseil-cpp btree implementation
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-15 10:55:43 +08:00
Kefu Chai
a44bb9f5bf osd/osd_types: add pg_t::compare()
it's equivalent to the `<=>` operator in C++20. it helps to simplify the
definition of other equality operators.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-15 10:55:43 +08:00
Patrick Donnelly
1970b0ceda
Merge PR #32315 into master
* refs/pull/32315/head:
	client: disallow changing fuse_default_permissions option at runtime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-01-14 18:15:34 -08:00
Patrick Donnelly
4386ce56ff
Merge PR #32322 into master
* refs/pull/32322/head:
	mds: Reorganize class members in SimpleLock header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 18:14:45 -08:00
Patrick Donnelly
aca58a45a6
Merge PR #32348 into master
* refs/pull/32348/head:
	mds: Move data members of struct SnapRealm to private.
	mds: Reorganize structure members in SnapRealm header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 18:14:06 -08:00
Patrick Donnelly
bf080fc69b
Merge PR #32416 into master
* refs/pull/32416/head:
	mds: Reorganize structure members in flock header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 18:13:25 -08:00
Patrick Donnelly
d5c0b88a7e
Merge PR #32431 into master
* refs/pull/32431/head:
	mds: Reorganize structure members in inode_backtrace header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 18:12:05 -08:00
zhengyin
3c8b0ef359 librbd: add snap_exists method API
Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>
2020-01-15 09:54:22 +08:00
Sage Weil
b5d145fcf9 Merge PR #32585 into master
* refs/pull/32585/head:
	MDSMonitor: handle standby already without fscid

Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-14 16:37:31 -06:00