Commit Graph

136623 Commits

Author SHA1 Message Date
Ilya Dryomov
f4edd7728a
Merge pull request #49614 from isodude/wip-librbd-misalign-discard
librbd: Fix local rbd mirror journals growing forever

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-17 18:09:39 +01:00
Adam King
d9292afedd mgr/cephadm: fix error_ok not being passed in unit tests
Signed-off-by: Adam King <adking@redhat.com>
2023-02-17 10:37:58 -05:00
Mykola Golub
ac882006e8 mgr/cephadm: try to avoid pull when getting container image info
Fixes: https://tracker.ceph.com/issues/58051
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2023-02-17 10:37:58 -05:00
Mykola Golub
ad3f576dde mgr/cephadm: allow _run_cephadm_json to be silent on error
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2023-02-17 10:37:58 -05:00
Adam King
9027acf43b
Merge pull request #49866 from rkachach/fix_issue_58466
cephadm: using short hostname to create the initial mon and mgr

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-02-17 10:32:06 -05:00
Lucian Petrut
9de2d0949d win32_build.sh: build "ceph_test*"
At the moment, win32_build.sh builds the "tests" target, which
only includes the "unittest_*" tests.

We'll update the build script to include "ceph_test_*" targets
as well.

Note that we can't just build all the targets that were generated
by cmake since some of them are unavailable on Windows. When
doing the Windows port, we decided to avoid polluting the Cmake
files and keep the amount of Windows checks to a minimum.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-02-17 10:42:40 +00:00
Lucian Petrut
0e124ed1a9 test/libcephfs: add missing includes
Some tests will need to include "compat.h" and "fs_types.h" in
order to build for Windows.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-02-17 10:42:36 +00:00
Lucian Petrut
e200985c24
Merge pull request #49961 from stefan-chivu/test_rbd_wnbd_latency
qa: Added latency to results table in test_rbd_wnbd.py
2023-02-17 12:11:15 +02:00
myoungwon oh
8961d77f0c crimson/os/seastore: fix traversal behavior in scan_mapped_space to avoid replay-allocation conflicts
1. Traverse the allocations from backref-tree leaf-node entries;
2. Traverse the pending alloc-deltas by sequence;
3. Traverse the allocations from backref-tree internal-node entries;

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-02-17 17:25:05 +09:00
Casey Bodley
b4250ba40b qa/rgw: add tox task before ragweed
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-16 15:37:58 -05:00
Casey Bodley
fc81fe705c qa/ragweed: run ragweed tests with tox
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-16 15:37:58 -05:00
Casey Bodley
d8dcffa4ef
Merge pull request #49709 from yima77/rgw_fix_mp_upload_obj_leak_due_to_reupload
rgw: fix multipart upload object leaks due to re-upload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-02-16 12:22:41 -05:00
J. Eric Ivancich
140941cfa4 rgw/flight: don't access non-existant flight store during GetObj
The front end must be configured via ceph.conf to start up both the
flight_server and the flight_store. RGWGetObj needs to check for the
existence of a flight_store prior to trying to use it.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-02-16 10:40:47 -05:00
myoungwon oh
cfef56a69b crimson/os/seastore: In rewrite_extent, do not resolve addr if address is RANDOM_BLOCK
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-16 22:39:52 +09:00
myoungwon oh
569c480874 crimson/os/seastore: determine whether lba_backref_node can be allocated to OOL
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-16 22:39:37 +09:00
Lucian Petrut
3fdad23e63 quincy: qa: update test_rbd_wnbd.py to report latency in seconds
At the moment, the latency results are reported in nanoseconds.
In order to improve readability, we'll convert it to seconds.

While at it, we'll fix the fio duration report, which we're
wrongfully dividing by 1000 twice.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-02-16 12:11:58 +00:00
Stefan Chivu
fa62b2add6 qa: Added latency to results table in test_rbd_wnbd.py
Added data regarding completion latency (ns) to the fio results
table, such as min, max, mean, stddev, etc.

Signed-off-by: Stefan Chivu <schivu@cloudbasesolutions.com>
2023-02-16 12:11:48 +00:00
Ilya Dryomov
fcfef0a19e qa/workunits/rbd-nbd: work around "rbd feature disable" hang
"rbd feature disable" appears to reliably hang if the corresponding
remote request is proxied to rbd-nbd (because rbd-nbd happens to own
the exclusive lock after a series of blkdiscard calls) [1].  Work
around it here by enabling journaling before the image is mapped
and disabling it after the image is unmapped.

Also, don't assert on the output of "rbd journal inspect --verbose"
having a certain number of entries.  This is racy: if the script gets
delayed after the last blkdiscard call for some reason, there may be
fewer entries present in the journal or none at all.

[1] https://tracker.ceph.com/issues/58740

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-16 13:05:05 +01:00
Ilya Dryomov
34e59c4502 test/librbd: add LengthModifiedDiscardJournalAppendEnabled test
Currently nothing triggers the length_modified case in
ImageDiscardRequest::prune_object_extents() in isolation. It's only
triggered in DiscardGranularityJournalAppendEnabled test together with
the prune_required case and a bad refactoring could easily break the
length_modified logic again.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-16 13:05:05 +01:00
Nizamudeen A
7e5b2542ab
Merge pull request #50109 from rhcs-dashboard/landing-page-v3
mgr/dashboard: Landing page v3

Reviewed-by: bryanmontalvan <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-02-16 16:21:12 +05:30
Jos Collin
ddd20d4d4d
qa: add tests for cephfs-top
* Updated the existing tests
* Added new qa tests for --dump and --dumpfs options
* Updated the cluster configuration to 2 mds, 2 clients

Fixes: https://tracker.ceph.com/issues/57014
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-02-16 15:13:19 +05:30
Jos Collin
247f81e5b4
doc: updates --dump and --dumpfs
Fixes: https://tracker.ceph.com/issues/57014
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-02-16 15:13:19 +05:30
Jos Collin
6f08159a22
cephfs-top: dump values to stdout
Fixes: https://tracker.ceph.com/issues/57014
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-02-16 15:13:19 +05:30
Nizamudeen A
8ea1623541
Merge pull request #50114 from rhcs-dashboard/cephadm-prom-401-error
mgr/dashboard: ignore the rules 400 error in dashboard kcli e2e

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2023-02-16 12:20:51 +05:30
N Balachandran
c7ae0f6eb6 rbd-mirror: fix syncing_percent calculation logic in get_replay_status()
When a snapshot sync is resumed and the get_replay_status function
is called before handle_copy_image_progress, the syncing_percent
value may be greater than 100 as the m_local_object_count is still
set to zero. This commit sets the syncing_percent to 0 in such cases.

Fixes: https://tracker.ceph.com/issues/58706

Signed-off-by: N Balachandran <nibalach@redhat.com>
2023-02-16 10:27:02 +05:30
Yingxin
2b1d3fee85
Merge pull request #50111 from rzarzynski/wip-crimson-osd-ftbfs-fmtptr
crimson/osd: fix FTBFS due to smart-ptr-to-bool conversion in fmt::ptr

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-16 10:30:56 +08:00
Anthony D'Atri
86621220ff
Merge pull request #50138 from zdover23/wip-doc-2023-02-16-radosgw-realms-refinement
doc/rgw: refine "Realms" section
2023-02-15 20:55:25 -05:00
Xiubo Li
3584ecc876 client: switch to use 32 bits ext_num_fwd
The MDS will increase the forward count, and if the forward count
is less than the one saved in request in client side, that means
the MDS is old version and it was overflowed. Then just stop
forwarding.

Fixes: https://tracker.ceph.com/issues/57854
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:49:41 +08:00
Xiubo Li
4fb04172ba client: switch to use 32 bits ext_num_retry
Check the CEPHFS_FEATURE_32BITS_RETRY_FWD feature bit and if not
set, that means it's connecting to an old MDS and will limit the
max retry to 256 times.

Fixes: https://tracker.ceph.com/issues/57854
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:48:28 +08:00
Xiubo Li
cbd7e30402 ceph_fs.h: add 32 bits extended num_retry and num_fwd support
To make sure both the num_retry and num_fwd in all the structs have
the same type, which is 32 bits.

And also for old cephs and clients they still could use the 8 bits
members.

Fixes: https://tracker.ceph.com/issues/57854
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:44:29 +08:00
Xiubo Li
82d7d20941 ceph_fs.h: switch to use its own encode/decode helpers
Prepare switching num_retry and num_fwd to __le32.

Fixes: https://tracker.ceph.com/issues/57854
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:44:24 +08:00
Xiubo Li
0ee2f80367 qa: add require_kernel_mount helper support
This will simplify the code later.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:43:39 +08:00
Xiubo Li
b699307a34 qa: rename test_fscrypt to test_fscrypt_encrypt
The test_fscrypt_encrypt will only run the 'encrypt' related test
cases without 'test_dummy_encryption' option enabled. This will
test the filename and content verification.

After this I will add the whole test cases with 'test_dummy_encryption'
option.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-02-16 08:43:39 +08:00
Zac Dover
5cea2529c4 doc/rgw: refine "Realms" section
Make various syntactical and semantic improvements to "Realms" section.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-16 10:11:41 +10:00
zdover23
a9a0262b10
Merge pull request #50119 from zdover23/wip-doc-2023-02-15-multisite-zonegroup-spelling-correction
doc/rgw: s/[Zz]one [Gg]roup/zonegroup/g
2023-02-16 09:37:13 +10:00
Guillaume Abrioux
50a32f4acc
Merge pull request #50017 from guits/fix-simple-scan-dir
ceph-volume: do not raise RuntimeError in util.lsblk
2023-02-15 20:09:30 +01:00
Yixin Jin
6d5988cbda rgw: Fix segfault due to concurrent socket use at timeout
This commit fixes a potential segfault risk when
rgw timeout handler works on the socket in one
thread while it is concurrently used by another.
The details of the fix are:

1. Instead of calling socket close(), which resets
descriptor_data in boost::asio socket and risks
segfault due to concurrent use of the socket,
the timeout handler now calls cancel() to abort
all pending ops followed by shutdown() to disable
the underlying transport. The eventual closure of
the socket will be done in the socket destructor.

2. Expose the actual boost::asio socket via get_socket()
from Connection so that the timeout handler can call
cancel() and shutdown() on it, although the socket data
member is already accessible. It allows future expansion
that wants to hide the socket even though it renders the
existing close() less useful.

Fixes: https://tracker.ceph.com/issues/58670
Signed-off-by: Yixin Jin <yjin77@yahoo.ca>
2023-02-15 17:32:29 +00:00
Adam King
3f3cf5ddc2
Merge pull request #50062 from phlogistonjohn/jjm-more-cephadm-tests
cephadm: a few more utility function tests

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-02-15 11:04:38 -05:00
Adam King
d16826e6a2
Merge pull request #50084 from adk3798/cephadm-autopep8-error
cephadm: fix autopep8 arg parsing on newer tox versions

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-02-15 09:17:40 -05:00
Lucian Petrut
d8ea79e232 test: disable unsupported tests on Windows
ceph_test_c2c will run until being signaled. However, we have limited
signal support on Windows, which is why we're going to skip it for
now.

At the same time, "fork" is unavailable on Windows, so we'll have
to temporarily disable the libcephfs reclaim test.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-02-15 14:12:38 +00:00
Venky Shankar
588f1f8cf8 Merge PR #48053 into main
* refs/pull/48053/head:
	test/libcephfs: fix rebasing issues
	libcephfs: replace errno.h errors with CEPHFS_E*
	messages: avoid converting ceph errors on Windows
	client: use CEPHFS_E*
	test/libcephfs: use CEPHFS_E* errors
	libcephfs: switch to CEPHFS_E* errors
	test/libcephfs: disable flaky timestamp assertion on Windows
	client: use _setattrx when changing timestamps
	client: set nsec to 0 when converting stat struct on Windows
	test/libcephfs: skip dirent inode check on Windows
	client: avoid trimming inodes on Windows
	test/libcephfs: address windows issues
	test/libcephfs: include compat.h
	test/libcephfs: enable the tests on Windows

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-02-15 18:58:29 +05:30
Pedro Gonzalez Gomez
7f7996fa55 mgr/dashboard: dashboard-v3: cluster-utilization card
mgr/dashboard: some improvements

mgr/dashboard: updated promqls and improved tooltips

mgr/dashboard: some fixes and improvements

mgr/dashboard: added square box plugin and improved/fixed code

mgr/dashboard: added maxValue option for graphs and added tooltips to text

mgr/dashboard: removed unnecessary variables

mgr/dashboard: fix openapi

tracker: https://tracker.ceph.com/issues/57863
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:20:49 +01:00
bryanmontalvan
c63523119d mgr/dashboard: dashboard-v3: inventory card
This commit starts the inventory-card which is one of the cards which
will be located in the landing-page revamp

tracker: https://tracker.ceph.com/issues/58065
Signed-off-by: bryanmontalvan <bmontalv@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:20:40 +01:00
bryanmontalvan
79439213b5 mgr/dashboard: dashboard-v3: status card
This commit is the bare-bones work of the status card. The only logic
written in this commit is the Cluster health status icon.

tracker: https://tracker.ceph.com/issues/58728
Signed-off-by: bryanmontalvan <bmontalv@redhat.com>

mgr/dashboard: introduce active alerts to status cards

Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-15 13:20:40 +01:00
Pedro Gonzalez Gomez
95e20d27b3 xmgr/dashboard: dashboard-v3: capacity card
tracker: https://tracker.ceph.com/issues/57862
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: fix openapi tox error

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: capacity-card set interval and display bytes used

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: Landing Page v3

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:19:36 +01:00
Pedro Gonzalez Gomez
cc93d53080 mgr/dashboard: dashboard-v3: details card
mgr/dashboard: added interface, changed html table to list, using mgrModuleService now

tracker: https://tracker.ceph.com/issues/58731
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:18:08 +01:00
bryanmontalvan
6fbc9ae01a mgr/dashboard: dashboard-v3: landing-page revamp
This commit add the following
- A new dashboard component, which will exist in parallel with in the
  current landing-page
- Created a route for this dashboard `/dashboard_3`
- Created a bare-bones bootstrap grid with mock-up card components

Signed-off-by: bryanmontalvan <bmontalv@redhat.com>
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: changes to first layout

CHANGES:

 - Renamed dashboardcomponents

 - Removed unnecesary styling

 - Added unit tests

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Moved router.url logic inside html template

This commit removes the `this.router.url` logic which was located in the
`workbench-layout.component.ts` file and moved it into the HTML template
section.

Signed-off-by: bryanmontalvan <bmontalv@redhat.com>

mgr/dashboard: syntax changes from bootstrap 4 to 5

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: small fixes and improvements over all cards and layout

	- all cards placed evenly with the same height
	- increased font size on details card and adjusted margin
	- changed capacity card legend to: "Used"
	- adjusted cluster utilization card margins and increased graphs height
	- improved status card toggle
	- changed orchestror to orchestrator
	- switched IPS/OPS graph colors

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: added recovery throghput graph, improve promqls, fix inventory's card lines

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:17:16 +01:00
Ronen Friedman
4cc70b709d
Merge pull request #44749 from ronen-fr/wip-rf-large-chunk
osd/scrub: use separate chunk size configuration for shallow scrubs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2023-02-15 12:06:58 +02:00
Anthony D'Atri
daa43d1b48
Merge pull request #50120 from zdover23/wip-doc-2023-02-15-glossary-realm
doc/glossary: add "realm" to glossary
2023-02-15 04:56:35 -05:00
Zac Dover
5e8aed3367 doc/glossary: add "realm" to glossary
Add the term "Realm" to the glossary, and link to the section in the RGW
documentation that provides more information about realms.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-15 17:51:41 +10:00