Commit Graph

127213 Commits

Author SHA1 Message Date
Kefu Chai
573661de50 common/bl: move bl::prepare_iovs() to .cc file
to reduce the compilation time by having a smaller header file.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-16 07:35:25 +08:00
Anthony D'Atri
00b232a3a9
Merge pull request #43478 from anthonyeleven/anthonyeleven/fix-52830
common/options: document rgw_lc_debug_interval configuration option
2021-10-15 15:34:30 -07:00
Samuel Just
3067942607
Merge pull request #43537 from xxhdx1985126/wip-seastore-iov-max
crimson/os/seastore: deal with scenarios in which bufferlist of do_writev has more than IOV_MAX buffers

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-15 15:04:22 -07:00
Casey Bodley
e338ae1d7a
Merge pull request #43507 from cbodley/wip-rgw-log-spam
rgw: hide some unnecessary log spam

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-10-15 14:05:05 -04:00
Casey Bodley
2729b829a8
Merge pull request #43488 from cbodley/wip-52873
radosgw-admin: 'reshard list' doesn't log ENOENT errors

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-10-15 14:04:13 -04:00
Casey Bodley
fff75ecc01
Merge pull request #43428 from cfsnyder/wip-52818-random-rgw-req-id
rgw/rgw_rados: make RGW request IDs non-deterministic

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-10-15 14:03:37 -04:00
Casey Bodley
386f04fa7b
Merge pull request #42981 from mkogan1/rgw-d3n-fix-lru-exp
rgw: d3n: fix d3n cached objects on disk lru eviction entering infini…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 14:02:26 -04:00
Casey Bodley
9cbf7eb5f2
Merge pull request #42208 from BryceCao/wip-add-return-when-access-key-exist-in-subuser-create
rgw : add return when access key exist in subuser create

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 14:01:58 -04:00
Casey Bodley
1ed12cf29c
Merge pull request #40144 from hualongfeng/osd_compression_bypass
rgw: osd compression bypass after rgw compression

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 13:59:58 -04:00
Anthony D'Atri
02bec8fba4 common/options: document rgw_lc_debug_interval configuration option
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2021-10-15 09:39:48 -07:00
Kefu Chai
45862b4c2a
Merge pull request #43533 from tchaikov/wip-cmake-less-chatty
cmake: only pass --verbose when VERBOSE env variable is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-16 00:10:32 +08:00
Casey Bodley
ca8997acaf rgw: silence error message for nonexistent realm
in a normal vstart cluster without a realm or period, every command
prints this error message:

0 RGWPeriod::init failed to init realm  id  : (2) No such file or directory

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 11:57:15 -04:00
Sage Weil
a95531f37d Merge PR #43558 into master
* refs/pull/43558/head:
	doc/governance.rst: update clt membership

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-10-15 11:48:29 -04:00
Patrick Donnelly
e55381c105
Merge PR #43430 into master
* refs/pull/43430/head:
	qa: skip c-v install on pacific start

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-15 11:16:25 -04:00
Patrick Donnelly
5af9882f94
Merge PR #43426 into master
* refs/pull/43426/head:
	qa/cephfs: update xfstests_dev for centos stream

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:15:41 -04:00
Patrick Donnelly
a8e77365b1
Merge PR #43425 into master
* refs/pull/43425/head:
	qa: import CommandFailedError from exceptions not run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2021-10-15 11:13:55 -04:00
Patrick Donnelly
7aea7f48ba
Merge PR #43420 into master
* refs/pull/43420/head:
	qa: skip internal metadata directory when scanning ceph debugfs directory

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:13:05 -04:00
Patrick Donnelly
a584ffab83
Merge PR #43382 into master
* refs/pull/43382/head:
	mds: skip journaling blocklisted clients when in `replay` state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:11:48 -04:00
Daniel Gryniewicz
9b7a208241
Merge pull request #43120 from dang/wip-dang-user-load-attrs
RGW Zipper - load user attributes with user

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 11:02:18 -04:00
Kefu Chai
cae9552863
Merge pull request #31909 from rosinL/wip-fix-dpdk-cond-wait
msg/async/dpdk: exit condition waiting when DPDKStack is destructed

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-15 19:25:55 +08:00
Xuehan Xu
9f9b021087 crimson/os/seastore: deal with scenarios in which bufferlist of do_writev has more than IOV_MAX buffers
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-15 18:08:47 +08:00
Chunsong Feng
2c49202027 msg/async/dpdk: exit condition waiting when DPDKStack is destructed
exit() will call pthread_cond_destroy attempting to destroy dpdk::eal::cond
upon which other threads are currently blocked results in undefine
behavior. Link different libc version test, libc-2.17 can exit,
libc-2.27 will deadlock, the call stack is as follows:

Thread 3 (Thread 0xffff7e5749f0 (LWP 62213)):
 #0  0x0000ffff7f3c422c in futex_wait_cancelable (private=<optimized out>, expected=0,
    futex_word=0xaaaadc0e30f4 <dpdk::eal::cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 #1  __pthread_cond_wait_common (abstime=0x0, mutex=0xaaaadc0e30f8 <dpdk::eal::lock>, cond=0xaaaadc0e30c8 <dpdk::eal::cond>)
    at pthread_cond_wait.c:502
 #2  __pthread_cond_wait (cond=0xaaaadc0e30c8 <dpdk::eal::cond>, mutex=0xaaaadc0e30f8 <dpdk::eal::lock>)
    at pthread_cond_wait.c:655
 #3  0x0000ffff7f1f1f80 in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
   from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
 #4  0x0000aaaad37f5078 in dpdk::eal::<lambda()>::operator()(void) const (__closure=<optimized out>, __closure=<optimized out>)
    at ./src/msg/async/dpdk/dpdk_rte.cc:136
 #5  0x0000ffff7f1f7ed4 in ?? () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
 #6  0x0000ffff7f3be088 in start_thread (arg=0xffffe73e197f) at pthread_create.c:463
 #7  0x0000ffff7efc74ec in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

Thread 1 (Thread 0xffff7ee3b010 (LWP 62200)):
 #0  0x0000ffff7f3c3c38 in futex_wait (private=<optimized out>, expected=12, futex_word=0xaaaadc0e30ec <dpdk::eal::cond+36>)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:61
 #1  futex_wait_simple (private=<optimized out>, expected=12, futex_word=0xaaaadc0e30ec <dpdk::eal::cond+36>)
    at ../sysdeps/nptl/futex-internal.h:135
 #2  __pthread_cond_destroy (cond=0xaaaadc0e30c8 <dpdk::eal::cond>) at pthread_cond_destroy.c:54
 #3  0x0000ffff7ef2be34 in __run_exit_handlers (status=-6, listp=0xffff7f04a5a0 <__exit_funcs>, run_list_atexit=255,
    run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
 #4  0x0000ffff7ef2bf6c in __GI_exit (status=<optimized out>) at exit.c:139
 #5  0x0000ffff7ef176e4 in __libc_start_main (main=0x0, argc=0, argv=0x0, init=<optimized out>, fini=<optimized out>,
    rtld_fini=<optimized out>, stack_end=<optimized out>) at ../csu/libc-start.c:344
 #6  0x0000aaaad2939db0 in _start () at ./src/include/buffer.h:642

Fixes: https://tracker.ceph.com/issues/42890
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2021-10-15 17:17:52 +08:00
Kefu Chai
40bc172c49
Merge pull request #43555 from tchaikov/wip-dashboard-tsconfig
mgr/dashboard: s/hosts.clickHostTab/hosts.clickTab/

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-15 16:07:36 +08:00
Kefu Chai
a6bb0e9771 mgr/dashboard: s/hosts.clickHostTab/hosts.clickTab/
this change is a follow-up fix of
3276147049

otherwise we have following test failure:

[lint:tsc     ] cypress/integration/orchestrator/01-hosts.e2e-spec.ts(29,13): error TS2339: Property 'clickHostTab' does not exist on type 'HostsPageHelper'.

also change "host" to "hostname" to be more consistent

Fixes: https://tracker.ceph.com/issues/52645
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-15 14:46:40 +08:00
Xuehan Xu
b93d748d29 common: add prepare_iovs to bufferlist
Enable bufferlist that have more than IOV_MAX buffers to build a vector of iov.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-15 14:28:04 +08:00
Xuehan Xu
055dfa8d39 crimson/common: extract parallel_for_each into errorator-loop.h
otherwise, crimson::errorator<AllowedErrors...>::parallel_for_each would be
referencing incomplete crimson::parallel_for_each_state type

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-15 14:28:00 +08:00
Feng Hualong
edde516a26 rgw: osd compression bypass after rgw compression
In particular, when rgw is itself doing compression,
rgw should send a incompressible hint to OSD.
Because the OSD has little effect on the data
compressed in the condition. So there bypassing the
osd compression is to avoid repeated compression
calculation.

Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-10-15 14:20:17 +08:00
Kefu Chai
4cb57b7539
Merge pull request #43554 from athanatos/sjust/wip-block-segment-manager-sync
crimson/os/seastore/segment_manager/block: open with dsync

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-15 10:09:52 +08:00
Josh Durgin
5d80867f2e doc/governance.rst: update clt membership
This now matches the mailing list which Sage audited.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-10-14 21:30:26 -04:00
Sage Weil
ec4fbe918b Merge PR #43333 into master
* refs/pull/43333/head:
	.github/CODEOWNERS: tag rook devs on rook related PRs

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 21:00:13 -04:00
Samuel Just
d741334111 crimson/os/seastore/segment_manager/block: open with dsync
67efc4 appears to be simply incorrect, I don't see any calls
to flush(), so we do need to open with dsync until we
implement a smarter flushing scheme.

Also, refactor open_device to remove mode param -- we always
pass the same value.

Reverts: 67efc48332
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-14 14:54:11 -07:00
Casey Bodley
51b2b76408
Merge pull request #43477 from anthonyeleven/anthonyeleven/fix-23027
ceph: document rgw_multipart_min_part_size

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-14 16:13:28 -04:00
Sebastian Wagner
c19250ce3c
Merge pull request #42997 from josephsawaya/wip-mgr-rook-orch-ls
mgr/rook: orch ls displays info about OSDs

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2021-10-14 17:33:26 +02:00
Sebastian Wagner
4e28b163a2
Merge pull request #43332 from sebastian-philipp/sphinx-rgw-spec
doc/cephadm: use sphinx autoclass to document RGWSpec

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-14 16:54:18 +02:00
Kefu Chai
70b049ffdb
Merge pull request #43239 from trociny/wip-48959
osd: handle inconsistent hash info during backfill and deep scrub gracefully

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-14 22:43:16 +08:00
Kefu Chai
7aefc0ff77
Merge pull request #43498 from dheart-joe/bluestore_fix
os/bluestore: _do_write_small fix head_pad

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-10-14 22:41:52 +08:00
Kefu Chai
44e89c613c
Merge pull request #42099 from aclamk/wip-bluefs-fine-grain-locking-2
os/bluestore: BlueFS fine grain locking

Reviewed-by: Sage Weil <sage@newdream.net>
2021-10-14 22:40:20 +08:00
Kefu Chai
552ac66577
Merge pull request #43374 from ljflores/wip-remove-pool-name
mgr/telemetry: remove pool name from telemetry perf report

Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2021-10-14 22:39:14 +08:00
Kefu Chai
b5d2548ceb
Merge pull request #43463 from Zhiwei-Dai/wip-enhance-qa-python3-compatibility
qa/tasks: replace iterkeys() with keys() for Python 3

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:38:39 +08:00
Kefu Chai
94a9757ff8
Merge pull request #43405 from ljflores/wip-perfcounter-priorities
os/bluestore: update perf counter priorities

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-14 22:37:36 +08:00
Kefu Chai
cb40e8fc65
Merge pull request #43422 from ronen-fr/wip-rf-exact-time-sec
common/ceph_time: fix wrong seconds output in exact_timespan_str()

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:36:30 +08:00
Kefu Chai
84adba7588
Merge pull request #43515 from fengchunsong/dpdk-dev
msgs/async/dpdk: fix dpdk build error

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:34:56 +08:00
Kefu Chai
9fb3ded7b5
Merge pull request #43531 from tchaikov/wip-pick-addr
common/pick_address: refactor pick_addresses()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-14 22:33:39 +08:00
Sebastian Wagner
a3fc496553
Merge pull request #43073 from strenuous-life/wip-cephadm-osd-rm-status
mgr/cephadm: the display of 'orch osd rm status' is incorrect

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:29:05 +02:00
Sebastian Wagner
d4783f5a65
Merge pull request #43214 from batrick/i52654
pybind/mgr/cephadm: set allow_standby_replay during CephFS upgrade

Reviewed-by: Sage Weil <sage@newdream.net>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:28:30 +02:00
Sebastian Wagner
64c6fa2d84
Merge pull request #43253 from guits/fix_orch_ls_osd
cephadm: fix osd size reported in 'orch ls'

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 16:28:03 +02:00
Sebastian Wagner
9e68e27ce6
Merge pull request #43339 from adk3798/volume-chunk
mgr/cephadm: use json-pretty for ceph-volume calls

Reviewed-by: Sage Weil <sage@newdream.net>
2021-10-14 16:27:10 +02:00
Patrick Donnelly
6e589e5080
Merge PR #43517 into master
* refs/pull/43517/head:
	doc / cephfs: health message codes should be permalinks

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-14 10:08:18 -04:00
Kefu Chai
a948322275
Merge pull request #43538 from rzarzynski/wip-crimson-net-actual-bind-failure
crimson/osd: print the actual reason on bind failure.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-14 22:00:27 +08:00
Ernesto Puerta
90bbcab09f
Merge pull request #42557 from ceph/feature-50336-cluster-creation-wizard
mgr/dashboard: Cluster Creation/Expansion Wizard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Volker Theile <vtheile@suse.com>
2021-10-14 15:12:42 +02:00