Commit Graph

117886 Commits

Author SHA1 Message Date
Samuel Just
739f599019
Merge pull request #38566 from tchaikov/crimson-tools
crimson/tools: use helpers from byteorder.hh

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-12-14 12:19:16 -08:00
Samuel Just
883c4b4b3d
Merge pull request #38565 from tchaikov/wip-crimson-cmake
cmake: add alien::cflags and move SubProcess.cc into crimson-alien-common

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2020-12-14 12:17:11 -08:00
Kefu Chai
4fe5665403 cmake: add "alien::cflags" library
which exposes the definitions needed for building alienized bluestore
and the shared facilities compiled with -DWITH_ALIEN and -DWITH_SEASTAR

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-15 01:55:38 +08:00
Kefu Chai
ff9cc3b44c cmake: move SubProcess.cc from crimson-common to crimson-alien-common
SubProcess is used by blkdev.cc only

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-15 01:55:38 +08:00
Ernesto Puerta
0ef9ad55b5
Merge pull request #36146 from rhcs-dashboard/wip-46577-master
mgr: add types to MODULE_OPTIONS

Reviewed-by: p-se <pseidensal@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: sebastian-philipp <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-12-14 17:35:52 +01:00
Ernesto Puerta
15395f61a1
Merge pull request #38406 from votdev/issue_48438_drop_rgw_instance
mgr/dashboard: Drop invalid RGW client instances, improve logging

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2020-12-14 16:40:05 +01:00
Mark Kogan
c945133cd8
Merge pull request #37642 from alimaredia/wip-rgw-apache-log-timestamp-change
rgw: add request time into beast log instead of current time
2020-12-14 17:06:04 +02:00
Kefu Chai
eda982e935
Merge pull request #38559 from xxhdx1985126/wip-crimson-bug-fix
crimson/osd: fix two bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-14 21:51:26 +08:00
Kefu Chai
3333ff2308 crimson/tools: use helpers from byteorder.hh
simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-14 21:19:44 +08:00
Xuehan Xu
d8293d937f crimson/osd: recovery read lock should be held across the hole object push phase
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2020-12-14 20:13:20 +08:00
Volker Theile
495ff0a2cf mgr/dashboard: Drop invalid RGW client instances, improve logging
* Log RGW settings for easier problem tracking.
* Drop RGW client instances that do not exist anymore. This happens when a RGW service is removed via the orchestrator. The Dashboard tries to access the RGW client using the previous settings which leads to an error which might be confusing. Without dropping the error message 'RGW REST API failed request ...' is displayed instead of the correct one 'No RGW found'. Dropping the RGW client instance will produce correct error messages the next time the backend tries to establish a new connection.

Fixes: https://tracker.ceph.com/issues/48586

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-12-14 12:57:14 +01:00
Ernesto Puerta
f002c6ce40
mgr: add types to MODULE_OPTIONS
Uses a typed Option class for MODULE_OPTIONS and replaces those in the
modules that are checked by mypy.

Fixes: https://tracker.ceph.com/issues/46577
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2020-12-14 10:23:59 +01:00
zdover23
851d10ed35
Merge pull request #37047 from sebastian-philipp/doc-cephadm-rm-host
doc/cephadm: Add remove host to operations

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-12-14 15:54:49 +10:00
Xuehan Xu
39c8941c5a crimson/osd: allow obc to promote lock state from none to excl
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2020-12-14 11:00:01 +08:00
Kefu Chai
684785d958
Merge pull request #38545 from tchaikov/wip-crimson-test-errorator
test/crimson: add a test to verify that errorator does not copy

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-12-13 19:26:40 +08:00
Kefu Chai
953efae652 test/crimson: add a test to verify that errorator does not copy
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-13 11:32:48 +08:00
Kefu Chai
ccaaad1d12 test/crimson: move member variable into test
so the test is more self-contained.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-13 11:32:48 +08:00
Kefu Chai
39ba718e30 test/crimson: s/do_until_test_t/errorator_test_t/
so we can reuse this test suite for more errorator related tests.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-13 11:32:31 +08:00
Kefu Chai
8bbe2e088a
Merge pull request #38554 from tchaikov/wip-admin-build-doc
admin/build-doc: consolidate the build deps into admin/doc-requirements.txt

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-12-12 22:24:52 +08:00
Sebastian Wagner
cd97c73089
Merge pull request #38547 from varshar16/wip-kubejacker-updates
script/kubejacker: Add python_common library and misc updates

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-12-12 12:55:50 +01:00
Kefu Chai
140f6c5c28 admin/build-doc: drop comments for building dummy C bindings
we don't build dummy C bindings since
2235c2db0e

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-12 16:49:21 +08:00
Kefu Chai
703ba9e7cb admin/build-doc: revert "upgrade pip setuptools first"
per Radoslow Zarzynski, the build failure was due to missing Python.h
for the python version he was using on the system.

This reverts commit bc73204812.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-12 16:29:20 +08:00
Kefu Chai
5cb9d54587 admin/build-doc: stop passing --use-feature=2020-resolver to pip
to silence the warning of

WARNING: --use-feature=2020-resolver no longer has any effect, since it is now the default dependency resolver in pip. This will become an error in pip 21.0.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-12 16:28:56 +08:00
Kefu Chai
6fb4aff5ad admin/build-doc: consolidate the build deps into admin/doc-pybind.txt
since all the python bindings are required for building the doc, extract
them into admin/doc-pybind.txt.

because the pybind python extensions require Cython python module to build,
we have to move them  into another requirement file, and install them after
Cython is installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-12-12 16:27:20 +08:00
Kefu Chai
ebdaaf88fd
Merge pull request #37433 from aclamk/wip-rocksdb-better-options
kv/RocksDBStore: Added ability to set compound options

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-12-12 12:36:54 +08:00
Kefu Chai
ea9c14c322
Merge pull request #38020 from mdw-at-linuxbox/teuth-fix-x509-san
qa/tasks/openssl_keys.py: add subjectAltName to certificates

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-12 12:18:16 +08:00
Kefu Chai
30487c755c
Merge pull request #38282 from vosdev/ceph-pool-alert
mgr/prometheus: Fix 'pool filling up' with >50% usage

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-12-12 12:10:44 +08:00
Kefu Chai
b501d8516b
Merge pull request #38290 from athanatos/sjust/wip-transaction-manager-nbd
crimson/seastore: add journaling checksumming and nbd based simple perf tester

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-12-12 11:48:41 +08:00
Patrick Donnelly
cd62a8494b
Merge PR #38322 into master
* refs/pull/38322/head:
	test_cephfs_shell: add tests for ls command

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-12-11 18:59:42 -08:00
Samuel Just
60e1d2ca92 crimson/os/seastore/journal: fix typo in comment
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:19 -08:00
Samuel Just
a99882edcb crimson/os/seastore/segment_manager: allocate page aligned in read helper
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:19 -08:00
Samuel Just
e358d24d5f crimson/os/seastore/cached_extent: always allocate page aligned
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:19 -08:00
Samuel Just
321c138ef0 crimson/os/seastore/journal: remove now unused scan_segment
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:19 -08:00
Samuel Just
5f4da5cefc crimson/os/seastore: rework jorunal::scan_extents to use scan_valid_records
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:19 -08:00
Samuel Just
b922fa8905 crimson/os/seastore/journal: replace scan_segment with scan_valid_records in replay_segment
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:19 -08:00
Samuel Just
68c17a0252 crimson/os/seastore/journal: add scan_valid_records
scan_valid_records validates record checksums and uses last_comitted
to avoid checking data crcs except on the final few records.  Will
replace scan_segment in a few patches.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 18:18:16 -08:00
Kefu Chai
b507eb2488
Merge pull request #38444 from tchaikov/wip-msgr-string-view
msg/async: drop unused member variable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-12-12 08:17:02 +08:00
Sebastian Wagner
acde2ff575
Merge pull request #38439 from adk3798/iscsi-stuff
mgr/cephadm: improve iscsi dashboard config

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-12-12 01:09:14 +01:00
Sebastian Wagner
755e66aa61
Merge pull request #38533 from sebastian-philipp/cephadm-mypy-no-untyped
mgr/cephadm: disallow_untyped_defs=True

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-12-12 01:07:12 +01:00
Kefu Chai
db3516d8f8
Merge pull request #38544 from tchaikov/wip-build-doc-auto-build
admin/build-doc: add livehtml target

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-12-12 08:05:24 +08:00
Samuel Just
ac08cfa188 crimson/os/seastore/journal: try_decode* pass bl as const &
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
d63ace5628 crimson/os/seastore/journal: validate header crc on read
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
59ff3f9735 crimson/os/seastore/lba_manager/btree/btree_range_pin: remove obsolete assert
In the event that the only two children of the root are merged, they
as well as the old root will be retired meaning that check_parent
will find no parent.

Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
6cd52cb0e6 crimson/os/seastore/journal: maintain committed_to on each record
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
2ed559fdb1 crimson/os/seastore/journal: use specified size types for header
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
56b9313fd7 crimson/os/seastore/journal: fix debugging lines
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
46e8a8db14 crimson/os/seastore/journal: add data and metadata checksums
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
1c3df6f1c7 crimson/os/seastore/.../lba_btree_node: collapse root on merge if down to one element
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
6f744a5549 crimson/os/seastore/.../lba_btree_node_impl: fix merge_entry return values
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00
Samuel Just
060fd035a5 crimson/os/seastore/journal: check segment nonce in scan_segment
Signed-off-by: Samuel Just <sjust@redhat.com>
2020-12-11 13:31:39 -08:00