Commit Graph

118205 Commits

Author SHA1 Message Date
Kefu Chai
362ccec7a6 vstart: format the customized options in ceph.conf
better formatted in ceph.conf, no extra spaces and tabs anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-01-04 15:02:00 +08:00
Kefu Chai
fd2f92fd44
Merge pull request #38702 from mkogan1/wip-fix-vstart-o
vstart: fix -o ceph.conf options processing

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-24 19:50:50 +08:00
Kefu Chai
97bc181a2d
Merge pull request #38711 from Rethan/fix-osdmap-message-max-bytes-comments
option: update comments of option osd_map_message_max_bytes
2020-12-24 19:47:07 +08:00
haoyixing
82c1d375d4 option: update comments of option osd_map_message_max_bytes
Option osd_map_message_max_bytes indicates the max bytes of MOSDMap message, not maps count
as osd_map_message_max.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
2020-12-24 18:22:36 +08:00
Kefu Chai
dfe599e6b3
Merge pull request #38700 from tchaikov/wip-seastar-msgr-test-timeout
test/crimson: add abort timer to messenger test

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-12-24 14:23:32 +08:00
Kefu Chai
4f355022c4
Merge pull request #38576 from myoungwon/wip-fix-seg-fault-manifest-count
osd : prevent accessing deleted reference

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-12-24 14:10:03 +08:00
Kefu Chai
bb22041c61
Merge pull request #38662 from Songweibin/wip-121918
osd/ReplicatedBackend: handle tailing zeros of recovering objects nicer

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-24 14:08:48 +08:00
Kefu Chai
2c8fc32a50
Merge pull request #38665 from tchaikov/wip-arm32-cleanups
erasure-code, test, pybind: various arm32 related cleanups.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-24 14:06:19 +08:00
Kefu Chai
642a72ec66
Merge pull request #38677 from ilanddev/mgr-deadlock-fix
mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-24 14:02:48 +08:00
Kefu Chai
6793756f45
Merge pull request #38678 from tchaikov/wip-denc
include/denc: use pair<const K,V> in range-based for loop

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-24 14:00:42 +08:00
Patrick Donnelly
6e59b6825a
Merge PR #37428 into master
* refs/pull/37428/head:
	mds: move duplicated inner loop out
	mds: try to replicate hot dir to restarted MDS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-12-23 20:35:39 -08:00
Patrick Donnelly
c7bd8fd38f
Merge PR #36597 into master
* refs/pull/36597/head:
	client: make the send_to_auth display of mds_requests more accurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:34:25 -08:00
Patrick Donnelly
ac84e8277e
Merge PR #38037 into master
* refs/pull/38037/head:
	qa: test_readahead add kernel client support

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:33:47 -08:00
Patrick Donnelly
fb065b7625
Merge PR #38631 into master
* refs/pull/38631/head:
	MDSLogContext: fix typo in MDSLogContextBase and add more comments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:33:05 -08:00
Patrick Donnelly
ca634e0fda
Merge PR #38672 into master
* refs/pull/38672/head:
	mds: increase the l_mds_openino_backtrace_fetch when fetching the backtrace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:32:34 -08:00
Patrick Donnelly
9571f715e0
Merge PR #38692 into master
* refs/pull/38692/head:
	pybind/cephfs: pass command as size-1 array of strings

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-23 20:30:35 -08:00
Patrick Donnelly
71d64c8112
Merge PR #38668 into master
* refs/pull/38668/head:
	osdc: Fix return type of map waiting functions
	monc: Use post rather than defer
	common/async: Hold lock in constructor of blocked_result
	osdc: Don't keep wrapping completion in wait_for_latest_osdmap
	neorados: Replace unsafe uses of dispatch with post

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:20:09 -08:00
Yuri Weinstein
5ab1e53221
Merge pull request #38707 from ceph/wip-yuriw-octopus-x-4-master
qa/tests: initial check in for octopus-x/parallel tests (try 2)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-23 13:09:51 -08:00
Josh Durgin
c79fa6d780 qa/tasks/cephadm: include first mgr host when applying mgrs
This prevents the first mgr from being shut down due to lack of
appropriate placements.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-12-23 11:30:54 -08:00
Mark Kogan
fc2d5e522f vstart: fix -o ceph.conf options processing
options passed to vstart.sh via the -o param and bunched on the same
line without line termination between them
hence not being parsed.

ex:
```
cat ./ceph.conf
...
debug_ms=0 debug_objecter=0 bluestore_debug_enforce_settings=ssd
bluestore_block_size=214748364800
...
```

Fixes: https://tracker.ceph.com/issues/48708

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2020-12-23 13:59:44 +02:00
Kefu Chai
3f930bbeae test/crimson: add abort timer to messenger test
Use a 120s timer to force abort the messenger test which should normally
succeeds in 60s, in order to:
* Fail the make check to print logs about where it is blocking;
* Free the listening IP address for the next make check;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-23 17:25:30 +08:00
Yuri Weinstein
a6e601b7f2 qa/tests: add light octopus-x/copy (cephadm based)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-12-22 16:20:15 -08:00
Patrick Donnelly
37d0e85d10
Merge PR #37721 into master
* refs/pull/37721/head:
	test: add tests for new snapshot APIs
	pybind/cephfs: python bindings for new snapshot APIs
	client: Snapshot cephfs APIs
	mds: include snapshot metadata in trace reply to client
	mds: include snap metadata on-disk on snapshot creation
	mds: include metadata in SnapInfo structure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-22 12:58:11 -08:00
Harish Munjulur
f38a727c6b
Merge pull request #38468 from IlsooByun/fix_lc_mp_iter
rgw: lc: fix multipart iteration problem
2020-12-22 11:15:05 -08:00
Patrick Donnelly
db6a0cdfe8
Merge PR #38611 into master
* refs/pull/38611/head:
	qa/cephfs/nfs: Add test for readonly export

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-22 10:25:54 -08:00
Patrick Donnelly
ebc5b6136a
pybind/cephfs: pass command as size-1 array of strings
Fixes: https://tracker.ceph.com/issues/48701
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-22 09:28:58 -08:00
zdover23
d8675ecf80
Merge pull request #38187 from zdover23/wip-doc-dev-unit-tests-5-of-7
doc/dev: update unit-tests tox section

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-12-23 02:07:59 +10:00
Sebastian Wagner
eea9a41e7d
Merge pull request #38414 from pcuzner/fix-drivegroup-example
doc: Update drivegroup spec examples

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2020-12-22 13:30:54 +01:00
Zac Dover
eb99594437 doc/dev: update unit-tests tox section
This commit
- alters the structure of the
  unit-test section of the Developer Guide
  that discusses the tox tool, which is
  used to run certain unit tests
- corrects some grammar
- updates the list of locations of tox.ini
  files
- This also incorporates N. Cutler's suggestions.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2020-12-22 22:24:19 +10:00
zdover23
867f910941
Merge pull request #38621 from anthonyeleven/anthonyeleven/bluestore-doc-cleanup
doc/rados/configuration: clean up BlueStore config and enhance DB sizing

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-12-22 22:05:17 +10:00
Varsha Rao
4c83f93f83 qa/cephfs/nfs: Add test for readonly export
Fixes: https://tracker.ceph.com/issues/48622
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-12-22 11:47:38 +00:00
Igor Fedotov
d5a2611106
Merge pull request #38499 from aclamk/wip-bs-onode-prune-deleted
os/bluestore: Prune deleted onodes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-22 11:42:38 +03:00
Sebastian Wagner
af9dac1ff5
Merge pull request #38650 from dmick/wip-dmick-docker-mirror
qa/distros: configure podman to pull from docker-mirror

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-12-22 01:08:18 +01:00
Kefu Chai
c828ce2940 include/denc: use pair<const K,V> in range-based for loop
map<K,V>::value_type is pair<const K, V>, so if we use range-based for
loop when iterating through a map, we should use pair<const K,V> instead
of pair<K,V>, the latter also compiles, but it might create a temporary
object of pair<K,V> from pair<const K,V>. GCC-11 complains at seeing
this:

../src/include/denc.h:1002:21: warning: loop variable ‘e’ of type ‘const T&’ {aka ‘const std::pair<OSDPerfMetricQuery, OSDPerfMetricReport>&’} binds to a tem\
porary constructed from type ‘const std::pair<const OSDPerfMetricQuery, OSDPerfMetricReport>’ [-Wrange-loop-constru
ct]
 1002 |       for (const T& e : s) {
      |                     ^

this change

* use the value_type of container in `maplike_details<Container>`,
  so we can avoid the overhead of creating temporay objects when
  encoding a map
* define denc_traits for std::pair<const A, B> as well, so the elements
  of a map can be encoded using denc facility

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-22 01:17:28 +08:00
Patrick Donnelly
b8e369ec58
Merge PR #38666 into master
* refs/pull/38666/head:
	doc: Fix mds capabilities for manual deployment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-21 09:06:00 -08:00
Kefu Chai
1458774299
Merge pull request #37156 from CloudFerro/wip-allow-dynamic-levels
os/bluestore: allow dynamic levels

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2020-12-22 01:04:48 +08:00
Patrick Donnelly
77b8a7f8cc
Merge PR #38669 into master
* refs/pull/38669/head:
	mds: fix typo in debug logs

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-12-21 09:01:53 -08:00
Casey Bodley
b235c9afaf
Merge pull request #38674 from mkogan1/rgw-fix-t48688
rgw: fix 'this->ondisk_version' will always evaluate to 'true'

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-12-21 11:26:33 -05:00
Cory Snyder
0601b31a53 mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock
A thread that holds the GIL while attempting to acquire a mutex will cause a deadlock
if another thread owns the mutex and is waiting on the GIL. The GIL must not be treated
like an ordinary mutex since it may be preempted at any time or released when doing
blocking I/O. Such deadlocks are severe since they starve all threads from access to the
GIL and therefore prevent any Python code from running until the mgr process is restarted.

Fixes: https://tracker.ceph.com/issues/39264
Signed-off-by: Cory Snyder <csnyder@iland.com>
2020-12-21 09:57:03 -05:00
Mykola Golub
1e880b195c
Merge pull request #38653 from dillaman/wip-48647
librbd: clear implicitly enabled feature bits when creating images

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-12-21 16:56:33 +02:00
Jason Dillaman
b885dfe455
Merge pull request #38670 from Rethan/doc-fix-max_part-description
doc/rbd-nbd: fix description for command rbd-nbd's parameter --max_part

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-12-21 09:13:37 -05:00
Mark Kogan
6fd551b3bf rgw: fix 'this->ondisk_version' will always evaluate to 'true'
fix:
```
src/rgw/CMakeFiles/radosgw.dir/rgw_rest_metadata.cc.o
../src/rgw/rgw_rest_metadata.cc:267:13: warning: address of
'this->ondisk_version' will always evaluate to 'true'
[-Wpointer-bool-conversion]
                                       &ondisk_version);
```
Fixes: https://tracker.ceph.com/issues/48688

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2020-12-21 13:36:16 +02:00
Ernesto Puerta
23f2604d6f
Merge pull request #38259 from rhcs-dashboard/fix-jwt-token
mgr/dashboard: Use secure cookies to store JWT Token

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2020-12-21 11:23:53 +01:00
Xiubo Li
b116717702 mds: increase the l_mds_openino_backtrace_fetch when fetching the backtrace
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-12-21 17:58:33 +08:00
songweibin
9e541a3b2b osd/ReplicatedBackend: handle tailing zeros of recovering objects nicer
E.g.,
(1) recovery_info.copy_subset = [0, 4194304]
(2) copy_subset(fiemap) = [0, 8192]
(3) out_op->data_included.span_of(copy_subset, 0, 4194304) = [0, 8192]

We'll set new_progress.data_recovered_to = 8192 as a result, and
fail to complete the push in one go.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-12-21 17:27:20 +08:00
haoyixing
90e3740582 doc/rbd-nbd: fix description for command rbd-nbd's parameter --max_part
The parameter --max_part should actually override module param max_part,
not as nbds_max as it said.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
2020-12-21 16:46:45 +08:00
Xiubo Li
6f199f0d2f mds: fix typo in debug logs
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-12-21 15:49:52 +08:00
Adam C. Emerson
0165966200 osdc: Fix return type of map waiting functions
Fixes: https://tracker.ceph.com/issues/47294
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-21 01:12:24 -05:00
Kefu Chai
8638101531
Merge pull request #38573 from batrick/i48600
osd: remove invalid put on message

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-21 11:40:36 +08:00
Adam C. Emerson
a2ddcebb30 monc: Use post rather than defer
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-20 16:37:29 -05:00