Commit Graph

136334 Commits

Author SHA1 Message Date
Ilya Dryomov
b21a379c5b librbd: call apply_changes() after setting librados_thread_count
Otherwise the setting doesn't take effect.  While at it, replace
home-grown stringify() with standard to_string().

Fixes: https://tracker.ceph.com/issues/58833
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-23 12:50:45 +01:00
Ilya Dryomov
a6b610f9cc
Merge pull request #50096 from nbalacha/syncpercent-fix
rbd-mirror: fix syncing_percent calculation logic in get_replay_status()

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-17 18:11:03 +01:00
Ilya Dryomov
f4edd7728a
Merge pull request #49614 from isodude/wip-librbd-misalign-discard
librbd: Fix local rbd mirror journals growing forever

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-17 18:09:39 +01:00
Adam King
9027acf43b
Merge pull request #49866 from rkachach/fix_issue_58466
cephadm: using short hostname to create the initial mon and mgr

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-02-17 10:32:06 -05:00
Lucian Petrut
e200985c24
Merge pull request #49961 from stefan-chivu/test_rbd_wnbd_latency
qa: Added latency to results table in test_rbd_wnbd.py
2023-02-17 12:11:15 +02:00
Casey Bodley
d8dcffa4ef
Merge pull request #49709 from yima77/rgw_fix_mp_upload_obj_leak_due_to_reupload
rgw: fix multipart upload object leaks due to re-upload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-02-16 12:22:41 -05:00
Lucian Petrut
3fdad23e63 quincy: qa: update test_rbd_wnbd.py to report latency in seconds
At the moment, the latency results are reported in nanoseconds.
In order to improve readability, we'll convert it to seconds.

While at it, we'll fix the fio duration report, which we're
wrongfully dividing by 1000 twice.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-02-16 12:11:58 +00:00
Stefan Chivu
fa62b2add6 qa: Added latency to results table in test_rbd_wnbd.py
Added data regarding completion latency (ns) to the fio results
table, such as min, max, mean, stddev, etc.

Signed-off-by: Stefan Chivu <schivu@cloudbasesolutions.com>
2023-02-16 12:11:48 +00:00
Ilya Dryomov
fcfef0a19e qa/workunits/rbd-nbd: work around "rbd feature disable" hang
"rbd feature disable" appears to reliably hang if the corresponding
remote request is proxied to rbd-nbd (because rbd-nbd happens to own
the exclusive lock after a series of blkdiscard calls) [1].  Work
around it here by enabling journaling before the image is mapped
and disabling it after the image is unmapped.

Also, don't assert on the output of "rbd journal inspect --verbose"
having a certain number of entries.  This is racy: if the script gets
delayed after the last blkdiscard call for some reason, there may be
fewer entries present in the journal or none at all.

[1] https://tracker.ceph.com/issues/58740

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-16 13:05:05 +01:00
Ilya Dryomov
34e59c4502 test/librbd: add LengthModifiedDiscardJournalAppendEnabled test
Currently nothing triggers the length_modified case in
ImageDiscardRequest::prune_object_extents() in isolation. It's only
triggered in DiscardGranularityJournalAppendEnabled test together with
the prune_required case and a bad refactoring could easily break the
length_modified logic again.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-16 13:05:05 +01:00
Nizamudeen A
7e5b2542ab
Merge pull request #50109 from rhcs-dashboard/landing-page-v3
mgr/dashboard: Landing page v3

Reviewed-by: bryanmontalvan <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-02-16 16:21:12 +05:30
Nizamudeen A
8ea1623541
Merge pull request #50114 from rhcs-dashboard/cephadm-prom-401-error
mgr/dashboard: ignore the rules 400 error in dashboard kcli e2e

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2023-02-16 12:20:51 +05:30
N Balachandran
c7ae0f6eb6 rbd-mirror: fix syncing_percent calculation logic in get_replay_status()
When a snapshot sync is resumed and the get_replay_status function
is called before handle_copy_image_progress, the syncing_percent
value may be greater than 100 as the m_local_object_count is still
set to zero. This commit sets the syncing_percent to 0 in such cases.

Fixes: https://tracker.ceph.com/issues/58706

Signed-off-by: N Balachandran <nibalach@redhat.com>
2023-02-16 10:27:02 +05:30
Yingxin
2b1d3fee85
Merge pull request #50111 from rzarzynski/wip-crimson-osd-ftbfs-fmtptr
crimson/osd: fix FTBFS due to smart-ptr-to-bool conversion in fmt::ptr

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-16 10:30:56 +08:00
Anthony D'Atri
86621220ff
Merge pull request #50138 from zdover23/wip-doc-2023-02-16-radosgw-realms-refinement
doc/rgw: refine "Realms" section
2023-02-15 20:55:25 -05:00
Zac Dover
5cea2529c4 doc/rgw: refine "Realms" section
Make various syntactical and semantic improvements to "Realms" section.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-16 10:11:41 +10:00
zdover23
a9a0262b10
Merge pull request #50119 from zdover23/wip-doc-2023-02-15-multisite-zonegroup-spelling-correction
doc/rgw: s/[Zz]one [Gg]roup/zonegroup/g
2023-02-16 09:37:13 +10:00
Guillaume Abrioux
50a32f4acc
Merge pull request #50017 from guits/fix-simple-scan-dir
ceph-volume: do not raise RuntimeError in util.lsblk
2023-02-15 20:09:30 +01:00
Adam King
3f3cf5ddc2
Merge pull request #50062 from phlogistonjohn/jjm-more-cephadm-tests
cephadm: a few more utility function tests

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-02-15 11:04:38 -05:00
Adam King
d16826e6a2
Merge pull request #50084 from adk3798/cephadm-autopep8-error
cephadm: fix autopep8 arg parsing on newer tox versions

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-02-15 09:17:40 -05:00
Venky Shankar
588f1f8cf8 Merge PR #48053 into main
* refs/pull/48053/head:
	test/libcephfs: fix rebasing issues
	libcephfs: replace errno.h errors with CEPHFS_E*
	messages: avoid converting ceph errors on Windows
	client: use CEPHFS_E*
	test/libcephfs: use CEPHFS_E* errors
	libcephfs: switch to CEPHFS_E* errors
	test/libcephfs: disable flaky timestamp assertion on Windows
	client: use _setattrx when changing timestamps
	client: set nsec to 0 when converting stat struct on Windows
	test/libcephfs: skip dirent inode check on Windows
	client: avoid trimming inodes on Windows
	test/libcephfs: address windows issues
	test/libcephfs: include compat.h
	test/libcephfs: enable the tests on Windows

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-02-15 18:58:29 +05:30
Pedro Gonzalez Gomez
7f7996fa55 mgr/dashboard: dashboard-v3: cluster-utilization card
mgr/dashboard: some improvements

mgr/dashboard: updated promqls and improved tooltips

mgr/dashboard: some fixes and improvements

mgr/dashboard: added square box plugin and improved/fixed code

mgr/dashboard: added maxValue option for graphs and added tooltips to text

mgr/dashboard: removed unnecessary variables

mgr/dashboard: fix openapi

tracker: https://tracker.ceph.com/issues/57863
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:20:49 +01:00
bryanmontalvan
c63523119d mgr/dashboard: dashboard-v3: inventory card
This commit starts the inventory-card which is one of the cards which
will be located in the landing-page revamp

tracker: https://tracker.ceph.com/issues/58065
Signed-off-by: bryanmontalvan <bmontalv@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:20:40 +01:00
bryanmontalvan
79439213b5 mgr/dashboard: dashboard-v3: status card
This commit is the bare-bones work of the status card. The only logic
written in this commit is the Cluster health status icon.

tracker: https://tracker.ceph.com/issues/58728
Signed-off-by: bryanmontalvan <bmontalv@redhat.com>

mgr/dashboard: introduce active alerts to status cards

Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-15 13:20:40 +01:00
Pedro Gonzalez Gomez
95e20d27b3 xmgr/dashboard: dashboard-v3: capacity card
tracker: https://tracker.ceph.com/issues/57862
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: fix openapi tox error

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: capacity-card set interval and display bytes used

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: Landing Page v3

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:19:36 +01:00
Pedro Gonzalez Gomez
cc93d53080 mgr/dashboard: dashboard-v3: details card
mgr/dashboard: added interface, changed html table to list, using mgrModuleService now

tracker: https://tracker.ceph.com/issues/58731
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:18:08 +01:00
bryanmontalvan
6fbc9ae01a mgr/dashboard: dashboard-v3: landing-page revamp
This commit add the following
- A new dashboard component, which will exist in parallel with in the
  current landing-page
- Created a route for this dashboard `/dashboard_3`
- Created a bare-bones bootstrap grid with mock-up card components

Signed-off-by: bryanmontalvan <bmontalv@redhat.com>
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: changes to first layout

CHANGES:

 - Renamed dashboardcomponents

 - Removed unnecesary styling

 - Added unit tests

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Moved router.url logic inside html template

This commit removes the `this.router.url` logic which was located in the
`workbench-layout.component.ts` file and moved it into the HTML template
section.

Signed-off-by: bryanmontalvan <bmontalv@redhat.com>

mgr/dashboard: syntax changes from bootstrap 4 to 5

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: small fixes and improvements over all cards and layout

	- all cards placed evenly with the same height
	- increased font size on details card and adjusted margin
	- changed capacity card legend to: "Used"
	- adjusted cluster utilization card margins and increased graphs height
	- improved status card toggle
	- changed orchestror to orchestrator
	- switched IPS/OPS graph colors

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: added recovery throghput graph, improve promqls, fix inventory's card lines

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-02-15 13:17:16 +01:00
Ronen Friedman
4cc70b709d
Merge pull request #44749 from ronen-fr/wip-rf-large-chunk
osd/scrub: use separate chunk size configuration for shallow scrubs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2023-02-15 12:06:58 +02:00
Anthony D'Atri
daa43d1b48
Merge pull request #50120 from zdover23/wip-doc-2023-02-15-glossary-realm
doc/glossary: add "realm" to glossary
2023-02-15 04:56:35 -05:00
Zac Dover
5e8aed3367 doc/glossary: add "realm" to glossary
Add the term "Realm" to the glossary, and link to the section in the RGW
documentation that provides more information about realms.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-15 17:51:41 +10:00
Nizamudeen A
a9634e902a mgr/dashboard: ignore the rules 400 error in dashboard kcli e2e
Fixes: https://tracker.ceph.com/issues/58722
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-15 13:03:57 +05:30
Nizamudeen A
4601454279
Merge pull request #50057 from rhcs-dashboard/nfs-exports-return-fix
mgr/dashboard: adapt nfs export code to the new changes in nfs module 

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-02-15 12:08:01 +05:30
Zac Dover
956c54b7c9 doc/rgw: s/[Zz]one [Gg]roup/zonegroup/g
Change all instances of "Zone Group" and "Zone group" and "zone group"
to "Zonegroup" or "zonegroup".

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-15 15:30:17 +10:00
zdover23
c7e891901d
Merge pull request #50108 from zdover23/wip-doc-2023-02-13-multisite-multisite-config-reference
doc/rgw: refine "Multi-site Config Ref" (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-02-15 14:39:36 +10:00
Samuel Just
3aea58ddba
Merge pull request #50019 from rzarzynski/wip-crimson-fix-zeroed-startevent
crimson/osd: fix zeroed starting timestamp of tracked ops

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-14 19:57:02 -08:00
Yingxin
3d8b9137ff
Merge pull request #49420 from cyx1231st/wip-seastar-msgr-multi-core
crimson/net: refactors to encapsulate ProtocolV2 message read and write paths and event dispatching

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-15 09:13:06 +08:00
Zac Dover
2d45a4e5e0 doc/rgw: refine "Multi-site Config Ref" (1 of x)
Refine the syntax (and in some cases, the semantics) of the English in
"Multi-site Configuration Reference" in doc/radosgw/multisite.rst. This
is a long section, so more PRs are likely.

https://tracker.ceph.com/issues/58632

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-15 08:39:32 +10:00
Laura Flores
377738630d
Merge pull request #47555 from JoshSalomon/wip-prim-balance-score
mon, osd: Adding read (aka workload, primary) balance score
2023-02-14 15:39:09 -06:00
John Mulligan
480f7fc115 cephadm/tests: add test coverage of get_distro function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-14 10:37:21 -05:00
John Mulligan
d17b92c7c8 cephadm/tests: add test coverage of check_time_sync function
This is also effectively full coverage of `check_units` as well
as check_time_sync is the only consumer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-14 10:37:21 -05:00
Anthony D'Atri
410cb54b61
Merge pull request #50104 from cityofships/fix_docs_lvm_activate
doc/ceph-volume: fix cephadm references
2023-02-14 09:39:12 -05:00
Radoslaw Zarzynski
79e9e046e6 crimson/osd: fix FTBFS due to smart-ptr-to-bool conversion in fmt::ptr
```
../src/fmt/include/fmt/format.h: In instantiation of ‘const void* fmt::v9::ptr(T) [with T = seastar::shared_ptr<crimson::net::SocketConnection>]’:
../src/crimson/net/ProtocolV2.cc:1109:72:   required from here
../src/fmt/include/fmt/format.h:3809:37: error: static assertion failed
 3809 |   static_assert(std::is_pointer<T>::value, "");
      |                                     ^~~~~
../src/fmt/include/fmt/format.h:3809:37: note: ‘std::integral_constant<bool, false>::value’ evaluates to false
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-14 14:37:56 +00:00
Casey Bodley
cdd53d9430
Merge pull request #50102 from cbodley/wip-github-examples-rgw
examples: move rgw examples under subdirectory

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-02-14 09:11:42 -05:00
kalebskeithley
8fda937e29
Merge pull request #50099 from kalebskeithley/rgw_token.gcc13
rgw: rgw_token.cc error: reference to 'formatter' is ambiguous
2023-02-14 07:56:26 -05:00
Nizamudeen A
b9e2436f8c
Merge pull request #49801 from rhcs-dashboard/service-instances-flickering
mgr/dashboard: move service_instances logic to backend

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2023-02-14 18:14:42 +05:30
Casey Bodley
e25a7e86e9 doc/rgw: update links to examples/rgw
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-14 07:33:57 -05:00
Nizamudeen A
9db0d6bb87
Merge pull request #49834 from rhcs-dashboard/rgw-list-roles
mgr/dashboard: rgw role listing

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-02-14 18:03:28 +05:30
Nizamudeen A
dbcf28d462 mgr/dashboard: adapt nfs export code to the new changes in nfs module
when you create/edit an nfs export from dashboard it leaves this traceback and error

```
Feb 09 14:15:54 ceph-node-00 ceph-mgr[3235]: [dashboard ERROR taskexec] Error while calling Task(ns=nfs/create, md={'path': 'e2e.nfs.bucket', 'fsal': 'RGW', 'cluster_id': 'testnfs'})
                                             Traceback (most recent call last):
                                               File "/usr/share/ceph/mgr/dashboard/tools.py", line 550, in _run
                                                 val = self.task.fn(*self.task.fn_args, **self.task.fn_kwargs)  # type: ignore
                                               File "/usr/share/ceph/mgr/dashboard/controllers/nfs.py", line 148, in create
                                                 ret, _, err = export_mgr.apply_export(cluster_id, json.dumps(raw_ex))
                                             TypeError: 'AppliedExportResults' object is not iterable
Feb 09 14:15:54 ceph-node-00 ceph-mgr[3235]: [dashboard INFO taskmgr] finished Task(ns=nfs/create, md={'path': 'e2e.nfs.bucket', 'fsal': 'RGW', 'cluster_id': 'testnfs'})
Feb 09 14:15:54 ceph-node-00 ceph-mgr[3235]: [dashboard INFO request] [::ffff:192.168.100.1:43896] [POST] [500] [0.767s] [admin] [172.0B] /api/nfs-ganesha/export
```
This started after https://github.com/ceph/ceph/pull/46209, so dashboard code needs to be adapted

Fixes: https://tracker.ceph.com/issues/58681
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-14 16:41:09 +05:30
Nizamudeen A
bf724b505e mgr/dashboard: move service_instances logic to backend
Fixes: https://tracker.ceph.com/issues/58504
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-14 16:26:23 +05:30
Josh Salomon
8ed66d3087 osd, mon: add read balance score to command ceph osd pool ls detail
for replicated pools.

osd: Added workload balance score to the command
     ceph osd pool ls detail
     (different flavors for console output and json/xml output)
mon: Added command 'osd rm-primary-temp'
(developers only) rm was not working
vstart: Added osd debug messages into mon log with -d flag
        For commands that execute methonds in OSD module

Signed-off-by: Josh Salomon <jsalomon@redhat.com>
2023-02-14 12:49:56 +02:00