Commit Graph

132468 Commits

Author SHA1 Message Date
Ilya Dryomov
0a6a70760a qa/suites/rbd: disable workunit timeout for dynamic_features_no_cache
The I/O workload in this test is xfstests (qa/run_xfstests_qemu.sh)
which isn't subjected to any timeout other than global max_job_time
limit in any other subsuite (e.g. qemu/workloads/qemu_xfstests.yaml).
But here, there is a parallel "op" workload defined as a workunit.
The workunit task has a default timeout of 3 hours which is effectively
imposed on the entire job.  In the "rbd cache = false" configuration,
it's sometimes exceeded.

Fixes: https://tracker.ceph.com/issues/48038
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-07-17 19:06:18 +02:00
Adam C. Emerson
dcc92e07b2
Merge pull request #47025 from adamemerson/wip-55765
rgw: Guard against malformed bucket URLs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-07-16 22:17:05 -04:00
Adam King
e7896a9c56
Merge pull request #46897 from rkachach/fix_issue_55808
mgr/cephadm: check for events key before accessing it

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-07-16 18:43:11 -04:00
Nizamudeen A
23eace17e0
Merge pull request #47011 from s0nea/wip-prevent-alert-redirects
mgr/dashboard: prevent alert redirect

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-07-16 19:26:26 +05:30
Ilya Dryomov
3d710cda2f
Merge pull request #44217 from CongMinYin/fix-pwl-recovery-test
qa/suites/rbd/pwl-cache: ensure recovery is actually tested

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-07-16 11:34:55 +02:00
Ilya Dryomov
2de0574382 qa/tasks: rename persistent write log cache trash task
It doesn't really thrash anything, just repeatedly restarts the
workload on top of a dirty cache file.  rbd_pwl_cache_recovery is
more on point and gets covered by existing CODEOWNERS.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-07-16 09:46:58 +02:00
zdover23
f43c7a6e5f
Merge pull request #47109 from zdover23/wip-doc-2022-07-15-start-hw-recs-cleanup-1
doc/start: update hardware recs

Reviewed-by: Anthony D'Atri
2022-07-16 12:28:43 +10:00
Neha Ojha
d4714405c8
Merge pull request #46908 from mlausch/snapshot_key_conversion
osd/SnapMapper: fix legacy key conversion in snapmapper class

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-07-15 13:50:47 -07:00
Adam C. Emerson
3ee9a3b41a rgw: Guard against malformed bucket URLs
Misplaced colons can result in radosgw thinking is has a bucket URL
but with no bucket name, leading to a crash later on.

Fixes: https://tracker.ceph.com/issues/55765
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-07-15 16:32:09 -04:00
Adam C. Emerson
b1d3e6c006 rgw: Fix rgw::sal::Bucket::empty static method signatures
`unique_ptr` overload should take by reference.

Both should be const.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-07-15 15:42:40 -04:00
Neha Ojha
a6f1a1c6c0
Merge pull request #47052 from neha-ojha/wip-cot-label
.github/labeler.yml: add core label to some tools

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-07-15 11:36:18 -07:00
Ilya Dryomov
d4bfb46878
Merge pull request #47115 from ceph/fix-mib
ceph.spec.in: fix path for mib file and properly mark in %files

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-07-15 18:48:47 +02:00
Ilya Dryomov
f54e9181a2
Merge pull request #47035 from idryomov/wip-56516
rbd-mirror: remove bogus completed_non_primary_snapshots_exist check

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-07-15 15:46:24 +02:00
Justin Caratzas
fddb7dd2b0 ceph.spec.in: fix path for mib file and properly mark in %files
Fixes typos introduced in https://github.com/ceph/ceph/pull/46918

Signed-off-by: Justin Caratzas <jcaratza@redhat.com>
2022-07-15 08:41:24 -04:00
Yingxin
1dfa11170c
Merge pull request #47028 from xxhdx1985126/wip-seastore-backref-cache-refactor
crimson/os/seastore: simplify backref cache

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-07-15 15:16:42 +08:00
Xuehan Xu
e589d0d35c crimson/os/seastore: simplify backref cache
Currently, the following transaction exec sequence would lead to
loss of backref:

1. Trans `A` merge a alloc backref for extent `X`
2. Trans `B` add a release backref for extent `X` to backref cache,
   during which it finds an in-cache alloc backref for extent `X` and
   decide not to add the release backref to cache
3. Trans `A` commit

In the above sequece, the release backref for extent `X` is lost.

This is a regression introduced when we try to optimize the backref cache.

This commit fix the issue by caching inflight backrefs in a multiset,
alloc/release ops that happen on the same paddr are queued in the order of
their happening. When doing gc, all those backrefs are merged.

Fixes: https://tracker.ceph.com/issues/56519
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-07-15 09:58:32 +08:00
Zac Dover
29c44a3bd6 doc/start: update hardware recs
This PR picks up the parts of
https://github.com/ceph/ceph/pull/44466
that were not merged back in January, when that
pull request was raised.

Matters added here:
* improved organzation of matter
* emphasis of IOPs per core over cores per OSD

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-07-15 05:29:11 +10:00
Adam King
f38cdd9cdf
Merge pull request #46546 from mgfritch/vstart-stop-mds
src/stop.sh: stop existing ceph-mds daemons

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2022-07-14 14:02:32 -04:00
Pere Diaz Bou
45ecfc6ac7
Merge pull request #46014 from vrushch/rbd_form_fix
mgr/dashboard: rbd striping setting pre-population and pop-over

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-14 18:20:24 +02:00
Adam King
2377bda27e
Merge pull request #47066 from adk3798/osd-removal-docs-update
doc/cephadm: add note about OSDs being recreated to OSD removal section

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-07-14 11:07:08 -04:00
Adam King
4144386fc0
Merge pull request #47062 from guits/update-latest-stable-default
cephadm: update LATEST_STABLE_RELEASE

Reviewed-by: Adam King <adking@redhat.com>
2022-07-14 11:03:39 -04:00
Ilya Dryomov
795daff95e
Merge pull request #46260 from CongMinYin/wip-enable-ndctl
cmake: enable ndctl when building PMDK for WITH_BLUESTORE_PMEM

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-07-14 15:21:29 +02:00
Nizamudeen A
367bf22731
Merge pull request #46644 from rhcs-dashboard/rbd-list-pagination
mgr/dashboard: rbd image pagination

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-07-14 17:04:31 +05:30
Nizamudeen A
990069d31f
Merge pull request #45981 from rhcs-dashboard/box-no-more-ultron
cephadm/box: Rootless podman box implementation

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: anthonyeleven <NOT@FOUND>
Reviewed-by: melissa-kun-li <NOT@FOUND>
2022-07-14 16:43:49 +05:30
Manuel Lausch
66bea86ab4 osd/SnapMapper: fix pacific legacy key conversion and introduce test
Octopus modified the SnapMapper key format from

  <LEGACY_MAPPING_PREFIX><snapid>_<shardid>_<hobject_t::to_str()>

to

  <MAPPING_PREFIX><pool>_<snapid>_<shardid>_<hobject_t::to_str()>

When this change was introduced, 94ebe0ea also introduced a conversion
with a crucial bug which essentially destroyed legacy keys by mapping them
to

  <MAPPING_PREFIX><poolid>_<snapid>_

without the object-unique suffix.  This commit fixes this conversion going
forward, but a fix for existing clusters still needs to be developed.

Fixes: https://tracker.ceph.com/issues/56147
Signed-off-by: Manuel Lausch <manuel.lausch@1und1.de>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-07-14 08:09:48 +00:00
Yuval Lifshitz
803eb78960
Merge pull request #46650 from faithuniterh/Modifying-boto3-examples
Modifying boto3 examples

reviewed-by: ylifshit@redhat.com
2022-07-14 11:03:20 +03:00
Yingxin
93a6fd1b02
Merge pull request #47039 from rosinL/fix-perf-cirmson-msgr
tools/crimson/perf_crimson_msgr:fix perf_crimson_msgr abort

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-07-14 14:25:53 +08:00
Samuel Just
eccd0a5faf
Merge pull request #47081 from zhscn/fix-map-exist
crimson/os/seastore: fix bug of Transaction::is_retired

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-07-13 13:59:51 -07:00
Adam King
d4a39cd046 doc/cephadm: add note about OSDs being recreated to OSD removal section
Signed-off-by: Adam King <adking@redhat.com>
2022-07-13 14:21:17 -04:00
Justin Caratzas
23a849e3d0
Merge pull request #46918 from bigjust/wip-jcaratza-mib-rpm
monitoring:package SNMP MIB file as an rpm
2022-07-13 12:56:53 -04:00
Pere Diaz Bou
80964fb520 mgr/dashboard: fix rbdconfiguration init type
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-13 17:45:37 +02:00
Tatjana Dehler
965005e078
mgr/dashboard: prevent alert redirect
Prevent Alertmanager alerts from being redirected to the active mgr
dashboard instance. There are two reasons for it:

1. It doesn't bring any additional benefit. The Alertmanager config
   includes all available mgr instances - active and passive ones. In
   case of an alert, it will be sent to all of them. It ensures that
   the active mgr dashboard will receive the alert in any case.
2. The redirect URL includes the mgr IP and NOT the FQDN. This leads
   to issues in environments where an SSL certificate is configured and
   matches the FQDNs, only.

Fixes: https://tracker.ceph.com/issues/56401
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2022-07-13 17:12:15 +02:00
Pere Diaz Bou
6f5ac1274b cephadm/box: Choose between docker or podman with --engine
With ./box.py --engine docker you can specify you want to use docker
instead of podman. With docker box.py command should be run with sudo.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-13 16:43:00 +02:00
Yuri Weinstein
f9bd8922d7
Merge pull request #46114 from myoungwon/wip-dedup-tool-object-dedup-snapshot
tool/ceph-dedup-tool: add performing dedup option on cloned object

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-07-13 07:32:32 -07:00
Yingxin
1311524218
Merge pull request #47021 from aisakaki/wip-cache-opt
crimson/os/seastore/cache: fine-grained lru cache control with GC

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-07-13 22:01:13 +08:00
Zhang Song
ee9195d09e crimson/os/seastore: fix bug of Transaction::is_retired
The retired extent may exist as a RetiredExtentPlaceholder, casting
this extent to LogicalCachedExtent will cause undefined behavior.

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2022-07-13 20:45:05 +08:00
Pere Diaz Bou
869c716e9c
Merge pull request #46898 from rhcs-dashboard/cleanup-55720-master
mgr/dashboard: don't log tracebacks on 404s

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-07-13 14:43:06 +02:00
Pere Diaz Bou
864955f489
Merge pull request #46987 from rhcs-dashboard/fix-ingress-backend-service-filter
mgr/dashboard: ingress backend service should list all supported services

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-07-13 14:41:31 +02:00
Venky Shankar
f8decfbfb7
Merge pull request #47006 from teuchert/fix_56269
mgr/snap_schedule: Use rados.Ioctx.remove_object() instead of remove().

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-07-13 14:59:30 +05:30
Yin Congmin
0eab8de3c0 qa/tasks: add thrash test for persistent write log cache
add thrash test for persistent write log cache. run rbd bench
on persistent write log cache, thrashes rbd bench, test the
recovery function of persistent write log cache.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2022-07-13 13:31:02 +08:00
Xinyu Huang
8a09b9b457 crimson/os/seastore/cache: fine-grained lru cache control with GC
GC transaction is not sourced by user behaviors, so the extent read
operations from GC transaction don’t satisfy the time locality
principle. These extents should not be added to LRU cache.

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-07-13 01:42:06 +00:00
Samuel Just
8189cf5986
Merge pull request #47064 from rzarzynski/wip-crimson-fix-client-request-leak
crimson/osd: don't leak ClientRequests on stopping OSD

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-07-12 14:10:52 -07:00
Pere Diaz Bou
08c980ede3 mgr/dashboard: fix api tests + lint
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:03 +02:00
Pere Diaz Bou
23615b3e1b mgr/dashboard: cleanups
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00
Pere Diaz Bou
5a0ead8cc8 mgr/dashboard: RbdConfiguration.list reuse image context.
Image contexts are reopen even though we pass the context as an
argument. This commit changes that so you can forget about reopening
a rbd image context again.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00
Pere Diaz Bou
c01ce41266 mgr/dashboard: change sort char to '+' and page icons
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00
Pere Diaz Bou
90869173ee mgr/dashboard: fix linting errors
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00
Pere Diaz Bou
3c308804d5 mgr/dashboard: add rbd list sorting support
Support sorting with name, pool name and namespace

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00
Pere Diaz Bou
5386dd1cb3 mgr/dashboard: update rbd list api version to 2
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00
Pere Diaz Bou
c832464998 mgr/dashboard: debounce reload data with serverSide
Whenever we use serverSide (paginate through backend) we should
debounce reloadData since it might call api calls too much times.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-07-12 19:09:02 +02:00