Commit Graph

129049 Commits

Author SHA1 Message Date
Benoît Knecht
c05a3b769d mon: Abort device health when device not found
If `store->get_devname()` returns an empty device name, it means it couldn't
determine the device that backs the monitor DB store directory.

This can happen if `ceph-mon` runs with `PrivateDevices=yes` in systemd, or
within a container where the host `/dev` isn't exposed.

This commit makes sure we abort trying to get the device health at that point,
and return an appropriate error.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2022-01-04 14:38:32 +01:00
Venky Shankar
9c659f228e
Merge pull request #44067 from MrFreezeex/dirfrags-test
qa/suites/fs: add prefetch_dirfrags false to thrasher suite

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-01-04 15:15:41 +05:30
Venky Shankar
edcc0ecf2c
Merge pull request #44255 from lxbsz/syncfs-44916
client: fix sync fs to force flush mdlog for all sessions

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-01-04 15:14:59 +05:30
Pere Diaz Bou
3cc808bac8 cephadm/box: remove unnecessary bash files
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-04 10:10:15 +01:00
Venky Shankar
a612b3cb85
Merge pull request #43618 from kotreshhr/recover-symlink
mds: Store symlink target in first data object

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-01-04 14:39:22 +05:30
Venky Shankar
53a336ab9d
Merge pull request #44246 from yongseokoh/fix-prefetch-dirfrags
mds: split _prefetch_dirfrags to avoid heartbeat timeout during up:rejoin

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-01-04 14:37:59 +05:30
Venky Shankar
4d372e9557
Merge pull request #43236 from mchangir/mgr/snap_schedule-fix-db-connection-concurrent-usage
mgr/snap_schedule: fix db connection concurrent usage

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-01-04 14:36:07 +05:30
Xiubo Li
d75164fd60 mds: remove the duplicated or incorrect respond
The try_get_auth_inode() has already responded to the client or
forwarded to auth MDS if returning NULL.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2022-01-04 10:11:17 +08:00
Sage Weil
7ae62167e3 doc/rados/operations: document pg_num_max
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:12:54 -05:00
Sage Weil
09642a8ce8 mgr: set max of 32 pgs for .mgr pool
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:12:54 -05:00
Sage Weil
094d0268f7 mgr/dashboard: expect pg_num_max property for pools
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:12:54 -05:00
Sage Weil
5945d8453b mon/OSDMonitor: add option --pg-num_max arg for create pool
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:12:54 -05:00
Sage Weil
2d8753f4fe mon/OSDMonitor: disallow setting pg_num < min or > max
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:12:54 -05:00
Sage Weil
73893f4d1c mgr/pg_autoscaler: apply pg_num_max
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:12:53 -05:00
Sage Weil
4439035c32 mon: add pg_num_max pool property
Signed-off-by: Sage Weil <sage@newdream.net>
2022-01-03 14:11:14 -05:00
dependabot[bot]
aab3b6a811
mgr/dashboard: bump simplebar-angular
Bumps simplebar-angular from 2.3.0 to 2.3.6.

---
updated-dependencies:
- dependency-name: simplebar-angular
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-01-03 16:50:17 +00:00
Alfonso Martínez
f0c8fd73a0
Merge pull request #44407 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/swagger-ui-4.1.3
build(deps): bump swagger-ui from 4.1.2 to 4.1.3 in /src/pybind/mgr/dashboard/frontend 

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-03 17:47:13 +01:00
Alfonso Martínez
7363bc3af1 mgr/dashboard: add test coverage for API docs (SwaggerUI)
Fixes: https://tracker.ceph.com/issues/53756
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2022-01-03 17:43:07 +01:00
Kefu Chai
902bb28127
Merge pull request #44354 from tchaikov/cmake-cleanup
cmake: use new CMP0127 policy

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-01-03 23:31:24 +08:00
Laura Flores
fa20cb8b63
Merge pull request #43337 from ljflores/wip-bluestore-zero-detection
os/bluestore: detect unnecessary zeroes in bufferlist when data is written
2022-01-03 09:16:32 -06:00
Alfonso Martínez
046cbec6a0
Merge pull request #44187 from rhcs-dashboard/wip-configure_dependabot-master
mgr/dashboard: configure dependabot

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-01-03 14:17:27 +01:00
Ilya Dryomov
a72c56388d
Merge pull request #44130 from LittleFox94/wip-doc-rbd-default-object-size
doc: how to change default rbd object size

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-01-03 10:06:06 +01:00
Venky Shankar
2f4060b8c4 mds: ignore unknown client op when tracking op latency
Server::handle_client_request() ignores unknown client operation
by returning -ENOTSUPP, however, Server::perf_gather_op_latency()
aborts on unknown client op, thereby causing -ENOTSUPP to never
reach the client. ceph_abort() seems unnecessary here.

Note, we could have invoked Server::perf_gather_op_latency()
when the return value to client is not -ENOTSUPP, however,
a valid client operation *might* just return -ENOTSUPP in
some cases.

@mchangir ran into this with his getvxattr op changes (PR #42001).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2022-01-03 14:22:05 +05:30
Yin Congmin
6290446b81 librbd/cache/pwl/: remove IO waited state
IO waited state variables including waited_lanes, waited_entries,
waited_buffers are only used to print logs and have no practical
effect.no_space = true is enough to indicate state.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-12-31 16:04:19 +08:00
Yin Congmin
1b4f4841bb doc/rbd: change persistent write-back cache to persistent write log cache
The full name of PWL is persistent write log, and the full name of
RWL is replica write log. Change the title to make it consistent
with the name of the feature and better reflect its design.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-12-31 16:04:19 +08:00
Yin Congmin
b1d712d29d librbd/cache/pwl: clean code and update comments
some comments are outdated, some are no longger needed, update them.
remove redundant ";". remove redundant log. remove redundant "".

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-12-31 16:04:09 +08:00
Nizamudeen A
7b9fb258c4 mgr/dashboard: fix orchestrator/02-hosts-inventory.e2e failed.
I removed the `02-hosts-inventory.e2e` file because it is a duplicate
test of one of the test in the `01-hosts.e2e` file and fixed the error
from that file.

Also, in the inventory Identify test, we test for an element to be not
visible. According to the latest cypress docs, this should be not.exist
instead of not.visible since the cd-modal will not even be present in
the DOM

Fixes: https://tracker.ceph.com/issues/53499
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-12-31 12:56:12 +05:30
Yin Congmin
265cedbf36 librbd/cache/pwl/rwl: add pmemobj_close() during error handle
After create/open pmem device, add pmemobj_close() during error handle.
Merge the same error handle to the end of function.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-12-31 14:21:12 +08:00
Yingxin Cheng
632916721b crimson/os/seastore: differentiate loggers in seastore
To enable fine-grain logger control for easier bug triage.

See common/subsys.h for the new classifications.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-31 13:32:08 +08:00
Yingxin Cheng
f3a80659ba crimson/os/seastore: clean up logging
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-31 13:32:08 +08:00
Xinyu Huang
402e3d9a71 crimson/os/seastore/cache: clean up LRU
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2021-12-31 11:16:53 +08:00
Nizamudeen A
a2f5a3c1db
Merge pull request #44398 from rhcs-dashboard/e2e-script-improvement
mgr/dashboard: dashboard cephadm e2e improvement

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2021-12-30 11:25:58 +05:30
Yingxin Cheng
12b166b60a test/crimson: leverage --default-log-level to configure the global log level
So we can have freedom to customize global/individual log levels during
manual tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-12-30 09:41:07 +08:00
Nizamudeen A
26f86f6cd3 mgr/dashboard: dashboard cephadm e2e improvement
Fixes: https://tracker.ceph.com/issues/53742
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-12-29 22:06:27 +05:30
Tongliang Deng
984c014416 test: fix bluestore unshare blob test case
Ensure async queue_transactions of remove op on cloned
object happens before KV range scan, otherwise assertion
on the number of shared blob may failed.

Signed-off-by: Tongliang Deng <dengtongliang@sensetime.com>
2021-12-29 21:34:18 +08:00
Yingxin
d70fdd74b9
Merge pull request #44403 from cyx1231st/wip-fix-crimson-args
crimson/osd: fix and improve args processing

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-29 13:01:50 +08:00
Xiubo Li
bff8136716 qa: do not use any time related suffix for *_op_timeouts
When testing the upgration, for the old version the *_op_timeouts
options' type is TYPE_FLOAT, but when parsing them by strict_strtod
it will failed with error:

  strict_strtod: garbage at end of string. got: '15m'

This commit will remove the 'm' suffix by switch the time to 900,
thus the old version still could parse it successfully.

Introduced-by: d060c9a768
https://tracker.ceph.com/issues/51705
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-12-29 12:39:32 +08:00
Chunsong Feng
f84196ac0f msg/async/dpdk:Add the TSO configuration
The TSO feature of some NICs(mlx5,iavf,sfc,hns3) is abnormal in DPDK-20.11.
The TSO feature is fiexed in DPDK-21.11. Therefore, a configure is added
for compatibility.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe  <hanfengzhe@hisilicon.com>
2021-12-29 03:19:06 +00:00
Xinyu Huang
a041322352 crimson/os/seastore/cache: trim after adding to lru
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2021-12-29 10:22:26 +08:00
Samuel Just
277e573e4a crimson/os/seastore/cache: add lru
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-12-29 10:22:22 +08:00
Kefu Chai
2925e34887
Merge pull request #44396 from cyx1231st/wip-seastore-fix-seastar-runner
crimson/test: fix SeastarRunner when app is not started

Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-29 00:31:03 +08:00
Kefu Chai
efd2002b6a
Merge pull request #44274 from fengchunsong/dpdk-dfx
msg/async/dpdk: add commands to obtain the NIC status and statistics

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-12-28 19:55:41 +08:00
Chunsong Feng
43b5f96093 msg/async/dpdk:add commands to obtain the NIC status and statistics
Commands are added to obtain the network adapter status and statistics
for debugging network adapter packet loss and mbuf insufficiency issues.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe  <hanfengzhe@hisilicon.com>
2021-12-28 02:01:25 +00:00
fengchunsong
cc55af8828
Merge pull request #44276 from fengchunsong/dpdk-affinity
common/numa: Skip the DPDK thread when setting NUMA affinity
2021-12-28 07:53:19 +08:00
Mara Sophie Grosch
5d872f3a28 doc: add docs for rbd_default_order
Adds documentation how to change default rbd object size. With the
previous option `--order` it was easy to guess the config name for the
default value, with the current option name `--object-size` thats hard
to guess.

Also extends the documentation for rbd_default_order to include
* how object-size is derived from the configured value
* allowed range of the value

In the first version of this commit I also added min and max for this
parameter (12/25, matching the object size range in `man 8
rbd`/Striping/object-size), but this made some tests fail, since some
seem to set values outside this range (and probably are fine since
included for some time already). To have this a doc-change only, I
removed the range.

Signed-off-by: Mara Sophie Grosch <littlefox@lf-net.org>
2021-12-27 19:58:56 +01:00
Patrick Donnelly
135be96971
Merge PR #44342 into master
* refs/pull/44342/head:
	mds: trigger stray reintegration when loading dentry
	qa: test that scrub causes reintegration

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-12-27 12:55:30 -05:00
Yuval Lifshitz
1f3fbf218f rgw/multisite/test: allow passing rgw parameter to multisite script
this allow changing rgw conf when sratting the test scrip. e.g.
MON=1 OSD=1 MDS=0 MGR=1 ../src/test/rgw/test-rgw-multisite.sh 2
--rgw_max_objs_per_shard=50

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2021-12-27 17:19:53 +02:00
Chunsong Feng
6411cbe2dd common/numa: Skip the DPDK thread when setting NUMA affinity
The CPU affinity of the DPDK thread has been set during DPDK initialization.
Do not modify the DPDK affinity when setting NUMA affinity.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe  <hanfengzhe@hisilicon.com>
2021-12-27 13:09:33 +00:00
Chunsong Feng
8015cb8bb4 msg/async: refactory rename_thread for DPDKStack
The thread_name of the DPDK thread has been set during DPDK initialization.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Han Fengzhe  <hanfengzhe@hisilicon.com>
2021-12-27 13:04:56 +00:00
Ilya Dryomov
8fe3a1be37
Merge pull request #44243 from ideepika/wip-deepika-iscsi
qa/rbd: update the cephadm required distro

Reviewed-by: Melissa Li <melissali@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-12-27 12:29:07 +01:00