Commit Graph

118194 Commits

Author SHA1 Message Date
Kefu Chai
2c8fc32a50
Merge pull request #38665 from tchaikov/wip-arm32-cleanups
erasure-code, test, pybind: various arm32 related cleanups.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-24 14:06:19 +08:00
Kefu Chai
642a72ec66
Merge pull request #38677 from ilanddev/mgr-deadlock-fix
mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-24 14:02:48 +08:00
Kefu Chai
6793756f45
Merge pull request #38678 from tchaikov/wip-denc
include/denc: use pair<const K,V> in range-based for loop

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-24 14:00:42 +08:00
Patrick Donnelly
6e59b6825a
Merge PR #37428 into master
* refs/pull/37428/head:
	mds: move duplicated inner loop out
	mds: try to replicate hot dir to restarted MDS

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-12-23 20:35:39 -08:00
Patrick Donnelly
c7bd8fd38f
Merge PR #36597 into master
* refs/pull/36597/head:
	client: make the send_to_auth display of mds_requests more accurate

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:34:25 -08:00
Patrick Donnelly
ac84e8277e
Merge PR #38037 into master
* refs/pull/38037/head:
	qa: test_readahead add kernel client support

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:33:47 -08:00
Patrick Donnelly
fb065b7625
Merge PR #38631 into master
* refs/pull/38631/head:
	MDSLogContext: fix typo in MDSLogContextBase and add more comments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:33:05 -08:00
Patrick Donnelly
ca634e0fda
Merge PR #38672 into master
* refs/pull/38672/head:
	mds: increase the l_mds_openino_backtrace_fetch when fetching the backtrace

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:32:34 -08:00
Patrick Donnelly
9571f715e0
Merge PR #38692 into master
* refs/pull/38692/head:
	pybind/cephfs: pass command as size-1 array of strings

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-23 20:30:35 -08:00
Patrick Donnelly
71d64c8112
Merge PR #38668 into master
* refs/pull/38668/head:
	osdc: Fix return type of map waiting functions
	monc: Use post rather than defer
	common/async: Hold lock in constructor of blocked_result
	osdc: Don't keep wrapping completion in wait_for_latest_osdmap
	neorados: Replace unsafe uses of dispatch with post

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-23 20:20:09 -08:00
Yuri Weinstein
5ab1e53221
Merge pull request #38707 from ceph/wip-yuriw-octopus-x-4-master
qa/tests: initial check in for octopus-x/parallel tests (try 2)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-23 13:09:51 -08:00
Josh Durgin
c79fa6d780 qa/tasks/cephadm: include first mgr host when applying mgrs
This prevents the first mgr from being shut down due to lack of
appropriate placements.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-12-23 11:30:54 -08:00
Yuri Weinstein
a6e601b7f2 qa/tests: add light octopus-x/copy (cephadm based)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-12-22 16:20:15 -08:00
Patrick Donnelly
37d0e85d10
Merge PR #37721 into master
* refs/pull/37721/head:
	test: add tests for new snapshot APIs
	pybind/cephfs: python bindings for new snapshot APIs
	client: Snapshot cephfs APIs
	mds: include snapshot metadata in trace reply to client
	mds: include snap metadata on-disk on snapshot creation
	mds: include metadata in SnapInfo structure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-22 12:58:11 -08:00
Harish Munjulur
f38a727c6b
Merge pull request #38468 from IlsooByun/fix_lc_mp_iter
rgw: lc: fix multipart iteration problem
2020-12-22 11:15:05 -08:00
Patrick Donnelly
db6a0cdfe8
Merge PR #38611 into master
* refs/pull/38611/head:
	qa/cephfs/nfs: Add test for readonly export

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-22 10:25:54 -08:00
Patrick Donnelly
ebc5b6136a
pybind/cephfs: pass command as size-1 array of strings
Fixes: https://tracker.ceph.com/issues/48701
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-22 09:28:58 -08:00
zdover23
d8675ecf80
Merge pull request #38187 from zdover23/wip-doc-dev-unit-tests-5-of-7
doc/dev: update unit-tests tox section

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2020-12-23 02:07:59 +10:00
Sebastian Wagner
eea9a41e7d
Merge pull request #38414 from pcuzner/fix-drivegroup-example
doc: Update drivegroup spec examples

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2020-12-22 13:30:54 +01:00
Zac Dover
eb99594437 doc/dev: update unit-tests tox section
This commit
- alters the structure of the
  unit-test section of the Developer Guide
  that discusses the tox tool, which is
  used to run certain unit tests
- corrects some grammar
- updates the list of locations of tox.ini
  files
- This also incorporates N. Cutler's suggestions.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2020-12-22 22:24:19 +10:00
zdover23
867f910941
Merge pull request #38621 from anthonyeleven/anthonyeleven/bluestore-doc-cleanup
doc/rados/configuration: clean up BlueStore config and enhance DB sizing

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-12-22 22:05:17 +10:00
Varsha Rao
4c83f93f83 qa/cephfs/nfs: Add test for readonly export
Fixes: https://tracker.ceph.com/issues/48622
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-12-22 11:47:38 +00:00
Igor Fedotov
d5a2611106
Merge pull request #38499 from aclamk/wip-bs-onode-prune-deleted
os/bluestore: Prune deleted onodes

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-12-22 11:42:38 +03:00
Sebastian Wagner
af9dac1ff5
Merge pull request #38650 from dmick/wip-dmick-docker-mirror
qa/distros: configure podman to pull from docker-mirror

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-12-22 01:08:18 +01:00
Kefu Chai
c828ce2940 include/denc: use pair<const K,V> in range-based for loop
map<K,V>::value_type is pair<const K, V>, so if we use range-based for
loop when iterating through a map, we should use pair<const K,V> instead
of pair<K,V>, the latter also compiles, but it might create a temporary
object of pair<K,V> from pair<const K,V>. GCC-11 complains at seeing
this:

../src/include/denc.h:1002:21: warning: loop variable ‘e’ of type ‘const T&’ {aka ‘const std::pair<OSDPerfMetricQuery, OSDPerfMetricReport>&’} binds to a tem\
porary constructed from type ‘const std::pair<const OSDPerfMetricQuery, OSDPerfMetricReport>’ [-Wrange-loop-constru
ct]
 1002 |       for (const T& e : s) {
      |                     ^

this change

* use the value_type of container in `maplike_details<Container>`,
  so we can avoid the overhead of creating temporay objects when
  encoding a map
* define denc_traits for std::pair<const A, B> as well, so the elements
  of a map can be encoded using denc facility

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-22 01:17:28 +08:00
Patrick Donnelly
b8e369ec58
Merge PR #38666 into master
* refs/pull/38666/head:
	doc: Fix mds capabilities for manual deployment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-21 09:06:00 -08:00
Kefu Chai
1458774299
Merge pull request #37156 from CloudFerro/wip-allow-dynamic-levels
os/bluestore: allow dynamic levels

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2020-12-22 01:04:48 +08:00
Patrick Donnelly
77b8a7f8cc
Merge PR #38669 into master
* refs/pull/38669/head:
	mds: fix typo in debug logs

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-12-21 09:01:53 -08:00
Casey Bodley
b235c9afaf
Merge pull request #38674 from mkogan1/rgw-fix-t48688
rgw: fix 'this->ondisk_version' will always evaluate to 'true'

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-12-21 11:26:33 -05:00
Cory Snyder
0601b31a53 mgr/ActivePyModules.cc: always release GIL before attempting to acquire a lock
A thread that holds the GIL while attempting to acquire a mutex will cause a deadlock
if another thread owns the mutex and is waiting on the GIL. The GIL must not be treated
like an ordinary mutex since it may be preempted at any time or released when doing
blocking I/O. Such deadlocks are severe since they starve all threads from access to the
GIL and therefore prevent any Python code from running until the mgr process is restarted.

Fixes: https://tracker.ceph.com/issues/39264
Signed-off-by: Cory Snyder <csnyder@iland.com>
2020-12-21 09:57:03 -05:00
Mykola Golub
1e880b195c
Merge pull request #38653 from dillaman/wip-48647
librbd: clear implicitly enabled feature bits when creating images

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-12-21 16:56:33 +02:00
Jason Dillaman
b885dfe455
Merge pull request #38670 from Rethan/doc-fix-max_part-description
doc/rbd-nbd: fix description for command rbd-nbd's parameter --max_part

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-12-21 09:13:37 -05:00
Mark Kogan
6fd551b3bf rgw: fix 'this->ondisk_version' will always evaluate to 'true'
fix:
```
src/rgw/CMakeFiles/radosgw.dir/rgw_rest_metadata.cc.o
../src/rgw/rgw_rest_metadata.cc:267:13: warning: address of
'this->ondisk_version' will always evaluate to 'true'
[-Wpointer-bool-conversion]
                                       &ondisk_version);
```
Fixes: https://tracker.ceph.com/issues/48688

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2020-12-21 13:36:16 +02:00
Ernesto Puerta
23f2604d6f
Merge pull request #38259 from rhcs-dashboard/fix-jwt-token
mgr/dashboard: Use secure cookies to store JWT Token

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2020-12-21 11:23:53 +01:00
Xiubo Li
b116717702 mds: increase the l_mds_openino_backtrace_fetch when fetching the backtrace
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-12-21 17:58:33 +08:00
haoyixing
90e3740582 doc/rbd-nbd: fix description for command rbd-nbd's parameter --max_part
The parameter --max_part should actually override module param max_part,
not as nbds_max as it said.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
2020-12-21 16:46:45 +08:00
Xiubo Li
6f199f0d2f mds: fix typo in debug logs
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-12-21 15:49:52 +08:00
Adam C. Emerson
0165966200 osdc: Fix return type of map waiting functions
Fixes: https://tracker.ceph.com/issues/47294
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-21 01:12:24 -05:00
Kefu Chai
8638101531
Merge pull request #38573 from batrick/i48600
osd: remove invalid put on message

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-21 11:40:36 +08:00
Adam C. Emerson
a2ddcebb30 monc: Use post rather than defer
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-20 16:37:29 -05:00
Adam C. Emerson
583da3d588 common/async: Hold lock in constructor of blocked_result
More correct, though not the cause of error.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-20 16:20:28 -05:00
Adam C. Emerson
00bf9021d3 osdc: Don't keep wrapping completion in wait_for_latest_osdmap
Also remove unsafe use of dispatch.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-20 16:20:28 -05:00
Adam C. Emerson
747f596081 neorados: Replace unsafe uses of dispatch with post
Dispatch may block the calling thread, so don't use it directly in a
called function. (In handlers is okay, though.)

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-12-20 16:20:28 -05:00
Kefu Chai
6cdf25b4ab cmake: install python scripts into /usr/sbin even if DESTDIR is empty
it's reported that on Alphine, "make install" installs
`ceph-volume-systemd` into `/usr/bin` instead of `/usr/sbin`, and this
breaks `systemd/ceph-volume@.service`. so in this change, we always
install this script into `/usr/sbin`, even if `DESTDIR` is not defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-21 01:26:46 +08:00
Kefu Chai
e59693fd96 pybind/ceph_argparse.py: use a safe value for timeout
we have reports that on arm32 machines, it timed out immediately, so
to prevent it from int overflow, use a safer value instead of
(1 << (32 - 1)) - 1.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-21 01:26:46 +08:00
Kefu Chai
6fc27237f0 pybind/ceph_argparse.py: drop unused variable
and other PEP8 compliance cleanups

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-21 01:26:46 +08:00
Kefu Chai
98e2be7f75 erasure-code: bail out early if chunks to be decoded is empty
otherwise we will read from uninitialized memory for blocksize.

update tests accordingly, as the number of decoded chunks is 0 if

* the number of available chunks is 0 and
* the number of requested chunks is not 0

after this change, because, in that case, the decode fails before any
chunk is allocated. but otherwise, the output chunks are still
allocated. so instead checking the number of output chunks, the test is
removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-21 01:26:46 +08:00
Ivan Jager
5f017bed4a doc: Fix mds capabilities for manual deployment
Now matches auth-config-ref.rst, and actually works.

Signed-off-by: Ivan Jager <aij+git@mrph.org>
2020-12-20 05:50:08 -06:00
Kefu Chai
c471f41495 test/erasure-code: fold variables with the same values
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-20 13:13:21 +08:00
Kefu Chai
9e0fe82dde test/erasure-code: define variables closer to where they are used
for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-20 13:13:21 +08:00