Commit Graph

129677 Commits

Author SHA1 Message Date
Guillaume Abrioux
47c33179f9 ceph-volume: honour osd_dmcrypt_key_size option
ceph-volume doesn't honour osd_dmcrypt_key_size.
It means the default size is always applied.

It also changes the default value in `get_key_size_from_conf()`

From cryptsetup manpage:

> For XTS mode you can optionally set a key size of 512 bits with the -s option.

Using more than 512bits will end up with the following error message:

```
Key size in XTS mode must be 256 or 512 bits.
```

Fixes: https://tracker.ceph.com/issues/54006

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-02-10 02:39:03 +01:00
Yaarit Hatuka
85f4b5b839 PendingReleaseNotes: add a note about perf channel
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2022-02-09 21:48:51 +00:00
Ernesto Puerta
f5b79d7e6b
Merge pull request #44956 from rhcs-dashboard/appli-fix
mgr/dashboard: change the readFile to readFileSync

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-02-09 20:44:55 +01:00
Casey Bodley
2f6f32bd9c
Merge pull request #44939 from soumyakoduri/wip-skoduri-dbstore-tests
rgw/dbstore: Add dbstore-tests to `make check`

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-09 13:01:22 -05:00
Nizamudeen A
cbfdd551d9 mgr/dashboard: change the readFile to readFileSync
Apparently the readFile i added in #44934 is async and that's not what
we want. so changing it to the synchronous call that is readFileSync

Fixes: https://tracker.ceph.com/issues/54190
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-02-09 22:08:09 +05:30
Patrick Donnelly
e883dc3b82
Merge PR #42000 into master
* refs/pull/42000/head:
	qa: update rhel kclient to setup container tools
	qa: stop overriding distro for k-testing
	qa: only use RHEL for workload testing
	qa: convert fs:workload to use cephadm
	qa: split fs begin task
	qa/tasks/cephadm: setup CephManager when OSDs are provisioned
	qa/tasks/cephadm: setup file system if MDS are provisioned

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 09:34:49 -05:00
Judah van der Ster
bb78fb51a3 purge_queue.py: fixed typo
Signed-off-by: Judah van der Ster <jevanderster@gmail.com>
2022-02-09 15:25:58 +01:00
Ilya Dryomov
1a06950c17
Merge pull request #44937 from idryomov/wip-rbd-help-positional-optional
rbd: mark optional positional arguments as such in help output

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2022-02-09 15:08:58 +01:00
Casey Bodley
682fe1267e
Merge pull request #44492 from xijiacun/master
rgw: Erase old storage class attr when the object is rewrited using r…

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-09 08:37:35 -05:00
Sarthak0702
ea1af5438d mgr/dashboard:Directories Menu Can't Use on Ceph File System Dashboard
Added exception handling to opendir() in cephfs.py for directories with no execute permission.

Fixes: https://tracker.ceph.com/issues/51611
Signed-off-by: Sarthak0702 <sarthak.0702@gmail.com>
2022-02-09 17:29:37 +05:30
Avan Thakkar
76dcf6a881 mgr/dashboard: add snmp-gateway service e2e tests
Fixes: https://tracker.ceph.com/issues/54034
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-02-09 16:50:07 +05:30
Ernesto Puerta
97c2760d10
Merge pull request #44934 from rhcs-dashboard/applitools-baseline-branch
mgr/dashboard: set appropriate baseline branch for applitools 

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-02-09 12:05:21 +01:00
Soumya Koduri
5b91ca9d48 rgw/dbstore: Add dbstore-tests to make check
Include and run dbstore-tests as part of `make check` target

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-09 15:16:34 +05:30
shenhang
d7feb89823 mds: add an option to decide whether prefetching entire dir or not.
Accessing one single dentry could be fastened by set this option to
false, when dir is not in the memory.
Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:46 +08:00
Shen, Hang
b6a74b8ec1 mds: avoid adding null dentry to bloom filter
Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:45 +08:00
Shen, Hang
02f9720c24 mds: make open_ino(...) fetch dentries in batch
Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:45 +08:00
Shen, Hang
634082c156 mds: don't add null auth dentry to bottom lru
null dentries are the key of not fetching complete dirfrag. don't trim
them too fast

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:45 +08:00
Shen, Hang
91746db4d6 mds: fetch single dentry instead of complete dirfrag for lookup
Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:45 +08:00
Shen, Hang
9c5e6df648 mds: remove CDir::take_sub_waiting()
migrator already does the job when importing dirfrag.

Signed-off-by: "Shen, Hang" <shenhang@kuaishou.com>
2022-02-09 17:31:45 +08:00
Nizamudeen A
40c902ac59 mgr/dashboard: set appropriate baseline branch for applitools
All the dashboard PRs are checked against a baseline branch called
'default' in the visual regresstion testing. This will cause issues when
testing PRs in different branches. For eg: currently our master and
pacific has to save two different screenshots since the two of them
differ slightly.

Disabling the applitools logs as well because its too 'noisy'

Fixes: https://tracker.ceph.com/issues/54190
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-02-09 11:46:21 +05:30
Venky Shankar
e15a52997b
Merge pull request #44477 from SMIL-Infra/fix-expire_recursive-segfault
mds: fix seg fault in expire_recursive

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 11:38:58 +05:30
Venky Shankar
f33030c074
Merge pull request #44638 from yunfeiguan/wip-53911
mds: kill session state are open when mds do ms_handle_remote_reset

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 11:37:20 +05:30
Venky Shankar
b7af2a94a4
Merge pull request #42549 from ajarr/wip-add-volume-rename
mgr/volumes: Add `fs volume rename` command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 11:34:32 +05:30
Venky Shankar
047d8faede
Merge pull request #44651 from nmshelke/fake-mount-53903
mount: add option to supports fake mounts in mount.ceph

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-09 11:30:27 +05:30
zhangzhiming
1b6ca99f1c rgw: Erase old storage class attr when the object is rewrited using radosgw-admin.
Fixes: https://tracker.ceph.com/issues/53790
Signed-off-by: zhiming zhang <zhangzhm1@chinatelecom.cn>
2022-02-09 08:45:52 +08:00
Soumya Koduri
9596630073
Merge pull request #44813 from soumyakoduri/wip-skoduri-dbstore-obj-id
rgw/dbstore: Use Object ID to handle I/O concurrency
2022-02-08 22:44:45 +05:30
Casey Bodley
48315ec623
Merge pull request #44678 from kipdoudou/master
rgw: CopyObject works with x-amz-copy-source-if-* headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-02-08 10:27:01 -05:00
Soumya Koduri
65ca2f53b8 rgw/dbstore: Handle read vs delete races
Now that tail objects are associated with objectID, they are not deleted
as part of this DeleteObj operation. Such tail objects (with no head object
in *.object.table are cleaned up later by GC thread.

To avoid races between writes/reads & GC delete, mtime is maintained for each
tail object. This mtime is updated when tail object is written and also when
its corresponding head object is deleted (like here in this case).

Also done some cleanup.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-08 20:56:30 +05:30
Vicente Cheng
cfca2daf31 msg/async: fix outgoing_bl overflow and reset state_offset
- we should reset state_offset when read done.

    - check outgoing_bl before we try to write a message.

    In some environments, network would temporily block and return EAGAIN.
    For async msgr, we would callback the write event directly, but that still
    increase the outgoing_bl.

    Think about this case, the sender is in congestion or network driver
    has some problems. The data appended to outgoing_bl and outgoing_bl
    is not consumed up-to-date immediately.
    That size of outgoing_bl will increase with time then overflow.
    The wrong outgoing_bl would cause some problems so we need to wait
    for outgoing_bl before we appended another message.

Signed-off-by: Vicente Cheng <vicente_cheng@bigtera.com>
2022-02-08 14:13:14 +00:00
Laura Flores
e63f146d61
Merge pull request #44930 from rishabh-d-dave/py-req-fix 2022-02-08 07:53:02 -06:00
Venky Shankar
de489c203e
Merge pull request #44667 from dvanders/dvanders_oft
mds: mds_oft_prefetch_dirfrags default to false

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-02-08 18:36:56 +05:30
Ronen Friedman
f0eb33fd82
Merge pull request #44922 from ronen-fr/wip-rf-fix-bescrub-test
osd/scrub: fix unintended changes to scrub (cluster)logs

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2022-02-08 14:59:10 +02:00
Daniel Gryniewicz
88ce1caefb
Merge pull request #44927 from dang/wip-dang-cloud-tier
RGW - Move early return

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2022-02-08 07:29:25 -05:00
Aggelos Toumasis
11909d0d84 doc/man/8: Fix spelling in user rename example
Signed-off-by: Aggelos Toumasis toumasis.aggelos@gmail.com
2022-02-08 12:35:27 +01:00
Ronen Friedman
e6a205a2bd osd/scrub: fix unintended changes to scrub (cluster)logs
OSD logs and cluster logs are monitored by some scrub tests:
some specific strings are required to either appear or not appear in
the logs. The Scrubber backend PR has unintentionally modified some
of these logs, and here we restore the exact logs text.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-02-08 10:30:15 +00:00
Guillaume Abrioux
b252c6495c qa/orch: use hwe kernel in rook/smoke tests (ubuntu 20.04)
Use the override in ./src/qa/distros/container-hosts/ubuntu_20.04.yaml
in order to use hwe kernel for Ubuntu 20.04

This is because the ubuntu 20.04 kernel (5.4) has a bug that prevents
from using nvme-loop.

see https://lkml.org/lkml/2020/9/21/1456

Fixes: https://tracker.ceph.com/issues/54094

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-02-08 10:58:02 +01:00
Ilya Dryomov
cb0df397aa rbd: mark optional positional arguments as such in help output
Currently at least five commands have optional positional arguments.

Overloading po::value<std::string>()->default_value("") for this
is a bit sneaky but nothing better fits into the existing Shell.cc
framework.

Note that strictly speaking "[<interval>] [<start-time>]" should be
"[<interval> [<start-time>]]" but we aren't doing that here because
"ceph" command doesn't do it either.

Fixes: https://tracker.ceph.com/issues/54191
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-02-08 10:11:49 +01:00
Miaomiao Liu
47fd35d52e compressor: replace snappy and lz4 compressors with zlib for QAT based compression
current QAT hardware only supports zlib compressor

Signed-off-by: Miaomiao Liu <miaomiao.liu@intel.com>
Signed-off-by: Hualong Feng <hualong.feng@intel.com>
2022-02-08 14:06:07 +08:00
Miaomiao Liu
6b879a7478 crypto/qat: fix issues about QAT based Encryption for RGW
update the librares usage and add a namespace before ostream

Fixes: https://tracker.ceph.com/issues/54059
Signed-off-by: Miaomiao Liu <miaomiao.liu@intel.com>
Signed-off-by: Hualong Feng <hualong.feng@intel.com>
2022-02-08 14:06:07 +08:00
Miaomiao Liu
082dcf0a58 cmake: replace BuildQatDrv.cmake with FindQatDrv.cmake
because QAT driver with version v1.7.l.4.14.0 or higher cannot be dowmloaded
directly by URL, FindQatDrv.cmake can find the locally installed QAT package and libraries

Signed-off-by: Miaomiao Liu <miaomiao.liu@intel.com>
Signed-off-by: Hualong Feng <hualong.feng@intel.com>
2022-02-08 14:06:07 +08:00
Rishabh Dave
a6f5efb620 monitoring: mention PyYAML only once in requirements
Following error occurs while running "sudo install-deps.sh" -
ERROR: Double requirement given: PyYAML==6.0 (from -r requirements-lint.txt (line 5)) (already in pyyaml (from -r requirements-alerts.txt (line 1)), name='PyYAML')

PyYAML is mentioned twice as a requirement. It is mentioned once in both
the following files -
monitoring/ceph-mixin/requirements-lint.txt
monitoring/ceph-mixin/requirements-alerts.txt

These requirements were added in commits
44d3e4c264 and
4750ac0d77.

Fixes: https://tracker.ceph.com/issues/54185
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-02-08 11:19:15 +05:30
Samuel Just
21a4254d0e
Merge pull request #44921 from cyx1231st/wip-seastore-cleanup-lba
crimson/os/seastore/lba_manager: consolidate logs

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-02-07 21:37:19 -08:00
Soumya Koduri
3349c6186b
Merge pull request #44916 from joke-lee/dbstore-test-init-env
rgw: fix dbstore test init env wrong
2022-02-08 09:30:41 +05:30
Yingxin Cheng
f2a1c7027c crimson/os/seastore/lba_manager: introduce subsys seastore_lba_details
We will share the lba tree implementation, so classify the related logs
and rename to seastore_lba_details for now.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-08 10:54:05 +08:00
Yingxin Cheng
798f00e7b4 crimson/os/seastore/lba_manager: consolidate logs
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-08 10:54:05 +08:00
Laura Flores
f69cec5b70 mgr/telemetry: separate device class usage statistics into their own collection
The new collection is called `basic_usage_by_class`. This info should be separate
from `basic_pool_usage` since it doesn't involve pool statistics.

Signed-off-by: Laura Flores <lflores@redhat.com>
2022-02-08 00:45:02 +00:00
Laura Flores
c71a54ec1a mgr/telemetry: update basic_pool_usage collection desc
- Added the word "default" since we are only collecting
default pool applications

- Removed the word "data" since we are actually collecting
usage *statistics*

Signed-off-by: Laura Flores <lflores@redhat.com>
2022-02-08 00:42:37 +00:00
Yuri Weinstein
d2b53aa1f7
Merge pull request #44713 from aclamk/fix-aclamk-vselector-53906
[BlueStore] Fix problem with volume selector

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2022-02-07 14:49:05 -08:00
Samuel Just
9c3e280dee
Merge pull request #44261 from Matan-B/wip-matanb-c-msgr-network
test/crimson: Inject socket level random delays/failures msgr unit test

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-02-07 13:09:12 -08:00
Ali Maredia
8de2884e06
Merge pull request #44856 from cbodley/wip-54103
qa/rgw: install s3cmd with extra_system_packages

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-02-07 16:02:40 -05:00