Commit Graph

127236 Commits

Author SHA1 Message Date
Sage Weil
c576184cd8 Merge PR #43504 into master
* refs/pull/43504/head:
	mgr/nfs: add some test coverage for cluster info, cluster config
	mgr/nfs: add 'nfs cluster config get'

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-18 15:03:33 -04:00
Sage Weil
4a63e3eafd Merge PR #43510 into master
* refs/pull/43510/head:
	qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
	mgr/cephadm: make upgrade ls output structured
	mgr/cephadm: add 'orch upgrade ls' to list available versions

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-18 15:03:24 -04:00
Sage Weil
75b2cc5c1b Merge PR #43549 into master
* refs/pull/43549/head:
	mgr/rook: fix mds service list

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-18 13:56:39 -04:00
Ernesto Puerta
19535b1d0e
Merge pull request #43469 from rhcs-dashboard/hosts-grafana-dashboards
mgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-18 17:14:03 +02:00
Sage Weil
8900775841 qa/suites/orch/cephadm/upgrade: smoke test for 'orch upgrade ls'
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-18 10:05:32 -04:00
Ilya Dryomov
cf8b6dc972
Merge pull request #42760 from ideepika/wip-iscsi-testing
qa/suites/rbd: switch iscsi tests to cephadm

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-18 14:25:36 +02:00
Sebastian Wagner
cd2a476fa1
Merge pull request #43564 from sebastian-philipp/doc-cephadm-host-location
doc/cephadm: Hosts: Add "initial CRUSH location"

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-18 13:58:55 +02:00
Deepika Upadhyay
b09c6ca2cc src/test/cli-integration: correct hostname for iscsi gateway
iscsi accepts first word of hostname as local hostname[0] instead of
smithi061.front.sepia.ceph.com we use: smithi061 otherwise gateway
creation will error out with:
```The first gateway defined must be the local machine```

[0] ff3e430b70/gwcli/utils.py (L36)

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
3e9b8c8f8b src/test/cli-integration/rbd: update gwcli_* to use podman exec
* since iscsi now resides inside the container, for testing we use the
  hack:
  - find the iscsi container/
  - use found container and podman exec command to test gateway creation
    deletion from host itself

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
cbd2c71398 qa/tasks: adapt ceph_iscsi.py task to ceph_iscsi_client
* we use setup_iscsi_client.py to deploy iscsi client services,
  configuring intiator and mulitpath this is done by qa task
  ceph_iscsi_client
* qa/cephadm: adds remotes ip addresses to iscsi gateway,
* rename poolname: iscsi >> datapool, which we usually use for tests and
  expresses type of pool more clearly.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
a7952949a8 qa/suites/rbd: remove baremetal based setup needed for iscsi testing
* replace ceph baremetal deployment with cephadm based deployment

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Deepika Upadhyay
8a35516294 doc: add iSCSI initiator client configuration section for cephadm
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:43 +05:30
Kefu Chai
5db11d3988
Merge pull request #43571 from tchaikov/wip-bl-prepare-iovs
common/bl: bl::prepare_iovs() related cleanups

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-16 10:57:05 +08:00
Kefu Chai
4e81734247 common/bl: mark bl::prepare_iovs() const
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-16 08:31:08 +08:00
Kefu Chai
e9678d0b67 common/bl: simplify bl::prepare_iovs()
* consolidate the code to initialize a vector<iov>
* use an iterator to track the current vector<iovec>. less
  repeating this way.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-16 08:29:59 +08:00
Kefu Chai
f04a6cea54 common/bl: define iov_vec_t using a dedicated struct
more readable this way

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-16 07:48:58 +08:00
Kefu Chai
573661de50 common/bl: move bl::prepare_iovs() to .cc file
to reduce the compilation time by having a smaller header file.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-16 07:35:25 +08:00
Anthony D'Atri
00b232a3a9
Merge pull request #43478 from anthonyeleven/anthonyeleven/fix-52830
common/options: document rgw_lc_debug_interval configuration option
2021-10-15 15:34:30 -07:00
Samuel Just
3067942607
Merge pull request #43537 from xxhdx1985126/wip-seastore-iov-max
crimson/os/seastore: deal with scenarios in which bufferlist of do_writev has more than IOV_MAX buffers

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-15 15:04:22 -07:00
Casey Bodley
e338ae1d7a
Merge pull request #43507 from cbodley/wip-rgw-log-spam
rgw: hide some unnecessary log spam

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-10-15 14:05:05 -04:00
Casey Bodley
2729b829a8
Merge pull request #43488 from cbodley/wip-52873
radosgw-admin: 'reshard list' doesn't log ENOENT errors

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-10-15 14:04:13 -04:00
Casey Bodley
fff75ecc01
Merge pull request #43428 from cfsnyder/wip-52818-random-rgw-req-id
rgw/rgw_rados: make RGW request IDs non-deterministic

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-10-15 14:03:37 -04:00
Casey Bodley
386f04fa7b
Merge pull request #42981 from mkogan1/rgw-d3n-fix-lru-exp
rgw: d3n: fix d3n cached objects on disk lru eviction entering infini…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 14:02:26 -04:00
Casey Bodley
9cbf7eb5f2
Merge pull request #42208 from BryceCao/wip-add-return-when-access-key-exist-in-subuser-create
rgw : add return when access key exist in subuser create

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 14:01:58 -04:00
Casey Bodley
1ed12cf29c
Merge pull request #40144 from hualongfeng/osd_compression_bypass
rgw: osd compression bypass after rgw compression

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 13:59:58 -04:00
Anthony D'Atri
02bec8fba4 common/options: document rgw_lc_debug_interval configuration option
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2021-10-15 09:39:48 -07:00
Kefu Chai
45862b4c2a
Merge pull request #43533 from tchaikov/wip-cmake-less-chatty
cmake: only pass --verbose when VERBOSE env variable is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-16 00:10:32 +08:00
Casey Bodley
ca8997acaf rgw: silence error message for nonexistent realm
in a normal vstart cluster without a realm or period, every command
prints this error message:

0 RGWPeriod::init failed to init realm  id  : (2) No such file or directory

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 11:57:15 -04:00
Sage Weil
a95531f37d Merge PR #43558 into master
* refs/pull/43558/head:
	doc/governance.rst: update clt membership

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-10-15 11:48:29 -04:00
Patrick Donnelly
e55381c105
Merge PR #43430 into master
* refs/pull/43430/head:
	qa: skip c-v install on pacific start

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-15 11:16:25 -04:00
Patrick Donnelly
5af9882f94
Merge PR #43426 into master
* refs/pull/43426/head:
	qa/cephfs: update xfstests_dev for centos stream

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:15:41 -04:00
Patrick Donnelly
a8e77365b1
Merge PR #43425 into master
* refs/pull/43425/head:
	qa: import CommandFailedError from exceptions not run

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2021-10-15 11:13:55 -04:00
Patrick Donnelly
7aea7f48ba
Merge PR #43420 into master
* refs/pull/43420/head:
	qa: skip internal metadata directory when scanning ceph debugfs directory

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:13:05 -04:00
Patrick Donnelly
a584ffab83
Merge PR #43382 into master
* refs/pull/43382/head:
	mds: skip journaling blocklisted clients when in `replay` state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-15 11:11:48 -04:00
Sage Weil
be3fd05fc5 mgr/cephadm: make upgrade ls output structured
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-15 11:09:07 -04:00
Daniel Gryniewicz
9b7a208241
Merge pull request #43120 from dang/wip-dang-user-load-attrs
RGW Zipper - load user attributes with user

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-15 11:02:18 -04:00
Sebastian Wagner
8e4f9ea46c
doc/cephadm: Hosts: Add "initial CRUSH location"
Followup on d2a9a35993

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-15 15:59:08 +02:00
Kefu Chai
cae9552863
Merge pull request #31909 from rosinL/wip-fix-dpdk-cond-wait
msg/async/dpdk: exit condition waiting when DPDKStack is destructed

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-15 19:25:55 +08:00
Xuehan Xu
9f9b021087 crimson/os/seastore: deal with scenarios in which bufferlist of do_writev has more than IOV_MAX buffers
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-15 18:08:47 +08:00
Chunsong Feng
2c49202027 msg/async/dpdk: exit condition waiting when DPDKStack is destructed
exit() will call pthread_cond_destroy attempting to destroy dpdk::eal::cond
upon which other threads are currently blocked results in undefine
behavior. Link different libc version test, libc-2.17 can exit,
libc-2.27 will deadlock, the call stack is as follows:

Thread 3 (Thread 0xffff7e5749f0 (LWP 62213)):
 #0  0x0000ffff7f3c422c in futex_wait_cancelable (private=<optimized out>, expected=0,
    futex_word=0xaaaadc0e30f4 <dpdk::eal::cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
 #1  __pthread_cond_wait_common (abstime=0x0, mutex=0xaaaadc0e30f8 <dpdk::eal::lock>, cond=0xaaaadc0e30c8 <dpdk::eal::cond>)
    at pthread_cond_wait.c:502
 #2  __pthread_cond_wait (cond=0xaaaadc0e30c8 <dpdk::eal::cond>, mutex=0xaaaadc0e30f8 <dpdk::eal::lock>)
    at pthread_cond_wait.c:655
 #3  0x0000ffff7f1f1f80 in std::condition_variable::wait(std::unique_lock<std::mutex>&) ()
   from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
 #4  0x0000aaaad37f5078 in dpdk::eal::<lambda()>::operator()(void) const (__closure=<optimized out>, __closure=<optimized out>)
    at ./src/msg/async/dpdk/dpdk_rte.cc:136
 #5  0x0000ffff7f1f7ed4 in ?? () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
 #6  0x0000ffff7f3be088 in start_thread (arg=0xffffe73e197f) at pthread_create.c:463
 #7  0x0000ffff7efc74ec in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

Thread 1 (Thread 0xffff7ee3b010 (LWP 62200)):
 #0  0x0000ffff7f3c3c38 in futex_wait (private=<optimized out>, expected=12, futex_word=0xaaaadc0e30ec <dpdk::eal::cond+36>)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:61
 #1  futex_wait_simple (private=<optimized out>, expected=12, futex_word=0xaaaadc0e30ec <dpdk::eal::cond+36>)
    at ../sysdeps/nptl/futex-internal.h:135
 #2  __pthread_cond_destroy (cond=0xaaaadc0e30c8 <dpdk::eal::cond>) at pthread_cond_destroy.c:54
 #3  0x0000ffff7ef2be34 in __run_exit_handlers (status=-6, listp=0xffff7f04a5a0 <__exit_funcs>, run_list_atexit=255,
    run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
 #4  0x0000ffff7ef2bf6c in __GI_exit (status=<optimized out>) at exit.c:139
 #5  0x0000ffff7ef176e4 in __libc_start_main (main=0x0, argc=0, argv=0x0, init=<optimized out>, fini=<optimized out>,
    rtld_fini=<optimized out>, stack_end=<optimized out>) at ../csu/libc-start.c:344
 #6  0x0000aaaad2939db0 in _start () at ./src/include/buffer.h:642

Fixes: https://tracker.ceph.com/issues/42890
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2021-10-15 17:17:52 +08:00
Kefu Chai
40bc172c49
Merge pull request #43555 from tchaikov/wip-dashboard-tsconfig
mgr/dashboard: s/hosts.clickHostTab/hosts.clickTab/

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-15 16:07:36 +08:00
Kefu Chai
a6bb0e9771 mgr/dashboard: s/hosts.clickHostTab/hosts.clickTab/
this change is a follow-up fix of
3276147049

otherwise we have following test failure:

[lint:tsc     ] cypress/integration/orchestrator/01-hosts.e2e-spec.ts(29,13): error TS2339: Property 'clickHostTab' does not exist on type 'HostsPageHelper'.

also change "host" to "hostname" to be more consistent

Fixes: https://tracker.ceph.com/issues/52645
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-15 14:46:40 +08:00
Xuehan Xu
b93d748d29 common: add prepare_iovs to bufferlist
Enable bufferlist that have more than IOV_MAX buffers to build a vector of iov.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-15 14:28:04 +08:00
Xuehan Xu
055dfa8d39 crimson/common: extract parallel_for_each into errorator-loop.h
otherwise, crimson::errorator<AllowedErrors...>::parallel_for_each would be
referencing incomplete crimson::parallel_for_each_state type

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-15 14:28:00 +08:00
Feng Hualong
edde516a26 rgw: osd compression bypass after rgw compression
In particular, when rgw is itself doing compression,
rgw should send a incompressible hint to OSD.
Because the OSD has little effect on the data
compressed in the condition. So there bypassing the
osd compression is to avoid repeated compression
calculation.

Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2021-10-15 14:20:17 +08:00
Kefu Chai
4cb57b7539
Merge pull request #43554 from athanatos/sjust/wip-block-segment-manager-sync
crimson/os/seastore/segment_manager/block: open with dsync

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-15 10:09:52 +08:00
Josh Durgin
5d80867f2e doc/governance.rst: update clt membership
This now matches the mailing list which Sage audited.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-10-14 21:30:26 -04:00
Sage Weil
ec4fbe918b Merge PR #43333 into master
* refs/pull/43333/head:
	.github/CODEOWNERS: tag rook devs on rook related PRs

Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-14 21:00:13 -04:00
Sage Weil
03ba3ceb5d mgr/nfs: add some test coverage for cluster info, cluster config
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-14 20:57:41 -04:00
Sage Weil
522c184f42 mgr/nfs: add 'nfs cluster config get'
Fixes: https://tracker.ceph.com/issues/52942
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-14 20:57:38 -04:00