Commit Graph

132184 Commits

Author SHA1 Message Date
Deepika Upadhyay
3eb6c29aa5 cmake: remove git fetch and use submodule for building opentelemetry lib
We were using git fetch and an unofficial remote to fetch
opentelemetry-cpp, as it needed additional patch to include boost
headers, with the merge of

open-telemetry/opentelemetry-cpp#1100
open-telemetry/opentelemetry-cpp#1020

we do not require to maintain our own patched version. Also, we were
using git fetch as a temporary change, now that tracing is always on, it
should be right time to adapt to using opentelemetry-cpp as a submodule.

This removes compile time fetch for opentelemetry-cpp instead use
official opentelemetry-cpp lib as a submodule.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-06-22 01:06:33 +00:00
Deepika Upadhyay
2c15a80a67 .gitmodules: add opentelemetry-cpp submodule
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-06-22 00:50:57 +00:00
chunmei-liu
5be9c3754d crimson/osd: check truncate size in sparse_read
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-06-21 17:39:52 -07:00
chunmei-liu
a0c9d06bda crimson/osd: return error code to break next op in cmp_ext
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-06-21 17:14:20 -07:00
Samuel Just
3b983cccbb
Merge pull request #46616 from xxhdx1985126/wip-seastore-lba-wrong-lba-node-size
crimson/os/seastore/lba_manager: correct btree node sizes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-06-21 16:18:27 -07:00
Samuel Just
ae33f9c272
Merge pull request #46614 from xxhdx1985126/wip-backref-bug-fixes
crimson/os/seastore: bug fixes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-06-21 16:17:21 -07:00
J. Eric Ivancich
c396bf1bd0 rgw: remove unnecessary "struct" from req_state declarations
The "struct" is superfluous and makes it harder for those looking for
the definition.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2022-06-21 19:11:24 -04:00
Laura Flores
8302ba9520
Merge pull request #46786 from ljflores/wip-doc-fix
doc/dev/developer_guide: fix small typo
2022-06-21 17:58:41 -05:00
Adam King
93a144b5fc
Merge pull request #46716 from rkachach/fix_issue_54474
doc/cephadm: Add post-upgrade section

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2022-06-21 18:07:07 -04:00
Laura Flores
462bc054cb doc/dev/developer_guide: fix small typo
Introduced in 682b806efa.

Signed-off-by: Laura Flores <lflores@redhat.com>
2022-06-21 15:52:39 -05:00
David Galloway
2679a1e0f4
Merge pull request #46773 from tchaikov/wip-libpmem
install-deps.sh: do not install libpmem from chacra
2022-06-21 16:21:54 -04:00
Kamoltat
10d2d2e2dd src/mgr/DaemonServer.cc: fix typo in output gap >= max_pg_num_change
Fix typo of the output when pgnum is scaling faster
than pgp_num in the function DaemonServer::adjust_pgs().

Fixes: https://tracker.ceph.com/issues/56151

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-06-21 20:11:04 +00:00
Sarthak0702
826741f192 mgr/dashboard: dashboard help command showing wrong syntax for login-banner
Signed-off-by: Sarthak0702 <sarthak.dev.0702@gmail.com>
2022-06-22 00:02:51 +05:30
Kefu Chai
fbfb369722 install-deps.sh: do not install libpmem from chacra
this change reverts 17d2bc3707, before
we recreate a chacra repo hosting libpmem packages, we are not able
to query the repo from shaman or pull the dependencies from chacra.

in future, we should be able to get the libpmem dependencies from
offical ubuntu package repo and fedora, CentOS Stream and RHEL repos.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-06-21 23:28:25 +08:00
Pedro Gonzalez Gomez
0cce84b71d mgr/dashboard: rbd image primary ui
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-06-21 17:15:22 +02:00
Ilya Dryomov
a5cb82d353
Merge pull request #46743 from idryomov/wip-rbd-mirror-schedule-watchless
mgr/rbd_support: always rescan image mirror snapshots on refresh

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2022-06-21 16:19:35 +02:00
Zac Dover
3e6bcd7f0b doc/dev: add context note to dev guide config
This PR adds a note directing first-time cloners of
their Ceph git forks to make sure to cd into the ceph/
directory before trying to run the "git config" commands.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-06-22 00:09:05 +10:00
Prashant D
46ce0beda3 pybind/mgr/autoscaler: Donot show NEW PG_NUM value if autoscaler is not on
When noautscale is set, autoscale-status shows NEW PG_NUM
value if pool pg_num is more than 96. If autoscaler is in
off or warn mode for the pool then donot adjust the final
pg count for the pool.

Fixes: https://tracker.ceph.com/issues/56136

Signed-off-by: Prashant D <pdhange@redhat.com>
2022-06-21 07:18:39 -04:00
Ilya Dryomov
7fb4fdbed0 mgr/rbd_support: always rescan image mirror snapshots on refresh
Establishing a watch on rbd_mirroring object and skipping rescanning
image mirror snapshots on periodic refresh unless rbd_mirroring object
gets notified in the interim is flawed.  rbd_mirroring object is
notified when mirroring is enabled or disabled on some image (including
when the image is removed), but it is not notified when images are
promoted or demoted.  However, load_pool_images() discards images that
are not primary at the time of the scan.  If the image is promoted
later, no snapshots are created even if the schedule is in place.  This
happens regardless of whether the schedule is added before or after the
promotion.

This effectively reverts commit 69259c8d37 ("mgr/rbd_support: make
mirror_snapshot_schedule rescan only updated pools").  An alternative
fix could be to stop discarding non-primary images (i.e. drop

    if not info['primary']:
        continue

check added in commit d39eb283c5 ("mgr/rbd_support: mirror snapshot
schedule should skip non-primary images")), but that would clutter the
queue and therefore "rbd mirror snapshot schedule status" output with
bogus entries.  Performing a rescan roughly every 60 seconds should be
manageable: currently it amounts to a single mirror_image_status_list
request, followed by mirror_image_get, get_snapcontext and snapshot_get
requests for each snapshot-based mirroring enabled image and concluded
by a single dir_list request.  Among these, per-image get_snapcontext
and snapshot_get requests are necessary for determining primaryness.

Fixes: https://tracker.ceph.com/issues/53914
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-21 12:46:49 +02:00
Ilya Dryomov
a6ca576617
Merge pull request #46734 from idryomov/wip-rbd-schedule-load-race
mgr/rbd_support: avoid losing a schedule on load vs add race

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2022-06-21 12:43:53 +02:00
Ernesto Puerta
e580e3c681
Merge pull request #46527 from rhcs-dashboard/mirroring-workflow
mgr/dashboard: configure rbd mirroring

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-06-21 12:05:07 +02:00
Pere Diaz Bou
6585abe4e6 mgr/dashboard: fix linting issues
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-06-21 10:34:56 +02:00
Pritha Srivastava
ff673210f7 rgw: adding 'user-policy' caps to a user to enable
execution of iam tests locally.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2022-06-21 14:00:21 +05:30
Ilya Dryomov
3743de2378
Merge pull request #46760 from idryomov/wip-wnbd-main-branch
win32_deps_build.sh: master -> main for wnbd

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-06-21 09:57:37 +02:00
Ilya Dryomov
27048bf2b2 win32_deps_build.sh: master -> main for wnbd
wnbd is switching, see https://github.com/cloudbase/wnbd/pull/67.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 22:23:27 +02:00
Ilya Dryomov
2bd2050909 rbd-mirror: spell out "remote image is not primary" status correctly
There is a difference: non-primary means NON_PRIMARY promotion state,
while "not primary" can refer to any of NON_PRIMARY, ORPHAN or UNKNOWN
promotion states.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 19:47:18 +02:00
Ilya Dryomov
8ce97c8cb7 rbd-mirror: fix up "error preparing image for replay" messages
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 19:47:18 +02:00
Ilya Dryomov
b6d6a2737e rbd-mirror: fix up PrepareReplayDisconnected test case
It was botched in commit 2bca9ee96c ("rbd-mirror: consolidate
prepare local/remote image steps to bootstrap") and went unnoticed
because currently no special handling is needed for disconnected
clients -- is_disconnected() check happens to be the last step
and it doesn't generate an error.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 19:47:18 +02:00
Ilya Dryomov
adf7f1ee2e rbd-mirror: drop m_remote_promotion_state from PrepareReplayRequest
Now unused (and if it was used, the entire StateBuilder is passed in
anyway).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 19:47:18 +02:00
Ilya Dryomov
79d28e63cb rbd-mirror: generally skip replay/resync if remote image is not primary
Replay and resync should generally be skipped if the remote image is
not primary.

If this is not done for replay, snapshot-based mirroring can run into
a livelock if the primary image is demoted while a mirror snapshot is
being synced.  On the demote site, rbd-mirror would pick up the just
demoted image, grab the exclusive lock on it and idle waiting for a new
mirror snapshot to be created.  On the (still) non-primary site,
rbd-mirror would eventually finish syncing that mirror snapshot and
attempt to unlink from it on the demote site.  These attempts would
fail with EROFS due to exclusive lock being held in the "refuse proxied
maintenance operations" mode, blocking forward progress (syncing of the
demotion snapshot so that the non-primary image can be orderly promoted
to primary, etc).

If this is not done for resync, data loss can ensue as the just demoted
image would be immediately trashed, underneath the non-primary site that
is still syncing.

Currently this is done in PrepareReplayRequest only for journal-based
mirroring.  Note that it is conditional: if the local image is linked
to the remote image, proceeding is desirable.

Generalize this check, consolidate it with a related check in
PrepareRemoteImageRequest and move the result to BootstrapRequest to
cover both "local image does not exist" and "local image is unlinked"
cases for both modes.

Fixes: https://tracker.ceph.com/issues/54448
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 19:47:18 +02:00
Ilya Dryomov
c60f1d5813 rbd-mirror: strengthen is_local_primary() and is_linked()
Initialize local_promotion_state and remote_promotion_state to UNKNOWN
instead of counterintuitive PRIMARY and NON_PRIMARY -- half the time the
final values are flipped.  Then is_local_primary() and is_linked() can
be strengthened as a non-existent image should stay in UNKNOWN.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-06-20 19:47:18 +02:00
Redouane Kachach
df1aaacb7d
doc/cephadm: enhancing daemon operations documentation
Fixes: https://tracker.ceph.com/issues/54399

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-06-20 11:44:49 +02:00
Ernesto Puerta
e47dfb0edb
Merge pull request #45012 from nSedrickm/grafana-iframe-ux-update
mgr/dashboard: display helpfull message when the iframe-embedded Grafana dashboard failed to load

Reviewed-by: Sarthak Gupta <sarthak.dev.0702@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>
2022-06-20 11:34:50 +02:00
Ernesto Puerta
f9846007e9
Merge pull request #46512 from rhcs-dashboard/rbd-mirroring-force-resync
mgr/dashboard: rbd force resync from front-end

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-06-20 11:32:38 +02:00
Ernesto Puerta
3f18dc94aa
Merge pull request #46639 from rhcs-dashboard/mirroring-page-fixes
mgr/dashboard: improve edit site name action in rbd-mirroring

Reviewed-by: Sarthak Gupta <sarthak.dev.0702@gmail.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-06-20 11:17:00 +02:00
Ernesto Puerta
b3b139be58
Merge pull request #46533 from rhcs-dashboard/dictionaryIscsi
mgr/dashboard: iterate through copy of items

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-06-20 11:13:24 +02:00
Venky Shankar
df19b8dbc7
Merge pull request #44900 from nmshelke/fix-54111
mon: verify data pool is already not in use by any file system

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2022-06-20 10:02:41 +05:30
Venky Shankar
c34fd00339
Merge pull request #46295 from dparmar18/tracker3998
mds: split up mdstypes

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by:  Rishabh Dave <ridave@redhat.com>
2022-06-20 09:59:53 +05:30
Venky Shankar
61673a77d4
Merge pull request #46363 from dparmar18/tracker_55548
qa/cephfs: fallback to older way of get_op_read_count

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-20 09:57:26 +05:30
Venky Shankar
fd45602f84
Merge pull request #46412 from lxbsz/wip-55778
client: choose auth MDS for getxattr with the Xs caps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-20 09:55:06 +05:30
Venky Shankar
615f553cf4
Merge pull request #46560 from lxbsz/wip-55824
qa: wait rank 0 to become up:active state before mounting fuse client

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-06-20 09:52:32 +05:30
Venky Shankar
c88a5c832e
Merge pull request #46635 from nmshelke/fix-55822
mgr/volumes: remove incorrect 'size' from output of 'snapshot info'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by:  Anthony D'Atri <anthony.datri@gmail.com>
2022-06-20 09:50:20 +05:30
Venky Shankar
0fb05df9ec
Merge pull request #46636 from nmshelke/fix-55759
mgr/volumes: subvolume ls command crashes if groupname as '_nogroup'

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
2022-06-20 09:48:35 +05:30
Venky Shankar
771f6f7d27
Merge pull request #46699 from neesingh-rh/wip-56065
qa: TestMDSMetrics.test_delayed_metrics failure

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-06-20 09:47:09 +05:30
Xuehan Xu
ce9326bbe3 crimson/os/seastore/lba_manager: drop scan_mapped_space method
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-06-19 20:21:03 +08:00
Xuehan Xu
fa39684dcb crimson/os/seastore/omap_manager: count omap/onode tree extents
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-06-19 20:21:03 +08:00
Xuehan Xu
9d9073b80d crimson/os/seastore/btree: count fixed kv btrees' extents num
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-06-19 20:21:03 +08:00
Xuehan Xu
ef36a9c2ba crimson/os/seastore: add metrics for count btree extents
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-06-19 20:21:00 +08:00
Kefu Chai
4933b4e585
Merge pull request #45144 from dvanders/doc_overrides
common/options/osd: clarify the non-zero overrides

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-06-18 12:30:23 +08:00
Yuri Weinstein
3087698475
Merge pull request #46630 from rzarzynski/wip-pglog-trim-dups
tools: ceph-objectstore-tool is able to trim solely pg log dups' entries

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-06-17 13:47:33 -07:00