Commit Graph

130934 Commits

Author SHA1 Message Date
Laura Flores
7ecbc830f8
Merge pull request #46009 from ljflores/wip-link-submitting-backports 2022-04-27 18:01:30 -05:00
Daniel Gryniewicz
4d1a039772 RGW - Allow radosgw-admin to work on other stores
For debugging purposes, allow radosgw-admin to run with stores other
than RadosStore.  Many operations will still fail (by crashing), so care
must be taken when running this way.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-04-27 14:04:36 -04:00
Daniel Gryniewicz
0c50cecb29 RGW - Allow starting RGW/dbstore without connecting to Mons
DBStore, and some other Stores like Motr, don't need to connect to the
Mons to work.  However, startup automatically connects to the mons.
There's provision to not connect, but the split isn't quite right.  We
need to call global_pre_init() to get config from the file, to determine
which store to start, but we then need to decide before calling
global_init() whether the configured store needs to connect to the mons.

This requires a slight change to global_init() to set no_mon_config from
the new flags.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-04-27 14:04:36 -04:00
Ilya Dryomov
7d7fb643da
Merge pull request #46025 from CongMinYin/fix-endianness-issue-encoding
librbd/cache/pwl: fix bit field endianness issue

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-04-27 18:22:43 +02:00
Ramana Raja
3adb70a24d doc/mgr/nfs: Add commands to check the statuses
.. of NFS and ingress services after creating/deleting a NFS cluster.
The `nfs cluster info` command is not sufficient to show that the
NFS cluster is created/deleted as expected.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2022-04-27 12:12:36 -04:00
J. Eric Ivancich
0521c2ae83 rgw: "bucket check --fix" should delete damaged multipart uploads from bi
As one of the steps in `radosgw-admin bucket check --fix ...` it looks
for bucket index entries for incomplete multipart uploads that do not
have a corresponding ".meta" entry in the same bucket index. It then
intends to delete those entries, however the function that it calls
to perform the bucket index deletions was flawed and did not direct
the removals to the appropriate shard(s), but instead a non-existant
oid.

This commit determines the appropriate shard for each of the entries
to be removed and asynchronously issues a librados call to
omap_rm_keys.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2022-04-27 10:51:30 -04:00
Matt Benjamin
991adad473 rgw: dbstore: add mechanism to signal gc_worker to exit
This addresses a trivial hang on shutdown.

Fixes: https://tracker.ceph.com/issues/55431

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2022-04-27 09:47:36 -04:00
Matt Benjamin
c0c0eb86a1
Merge pull request #46022 from linuxbox2/wip-dbstore-nfs
librgw: teach librgw about rgw_backend_store
2022-04-27 09:43:36 -04:00
Radoslaw Zarzynski
1daa027664
Merge pull request #45954 from Matan-B/wip-matanb-c-cyanlog
crimson/os: CyanStore debug subsystem

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-04-27 15:13:15 +02:00
Guillaume Abrioux
095704ca79 ceph-volume: only warn when config file isn't found
According to [1], this should only be a warning.

[1] 0487a9ac60

Fixes: https://tracker.ceph.com/issues/47633

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-04-27 14:58:23 +02:00
Ernesto Puerta
88622d2251
Merge pull request #45811 from rhcs-dashboard/grafana-e2e
mgr/dashboard: introduce grafana frontend e2e testing 

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-04-27 14:09:29 +02:00
Rishabh Dave
2fcdbcd6f5 qa/cephfs: don't remove sudo from the command arguments
run_shell() in qa.tasks.cephfs.mount.CephFSMount prepends "sudo" to its
command arguments but it doesn't specify to the underlying method that
"sudo" shouldn't be deleted from the command arguments.

Fixes: https://tracker.ceph.com/issues/53601
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-27 16:02:36 +05:30
Rishabh Dave
d9fbf65f77 qa/vstart_runner: set LocalCephManager.cluster
Use LocalContext instance to set LocalCephManager.cluster.

Fixes: https://tracker.ceph.com/issues/53601
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-27 16:02:34 +05:30
Kotresh HR
c57a13f8be qa: Fix a few tracebacks in vstart_runner
1. CommandFailedError: Command failed with status 127: \
	['None/archive/coverage', 'rados' ...]
2. TypeError: a bytes-like object is required, not '_io.BytesIO'

Fixes: https://tracker.ceph.com/issues/53601
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-04-27 16:02:32 +05:30
Nizamudeen A
672e27cbd2 mgr/dashboard: prometheus rules internal server error
After we increase/decrease the count of the node-exporter, we get a 500
- Internal server error from api/prometheus/rules endpoint. On further
  debugging its caused by the jsonDecodder, because I guess the expected
input for the json.loads() is not a json formatted input. So to fix
that issue I can either do an error handling on the json.loads() or I
can move the json.loads() on the already existing try block. I went for
the second approach here.

Fixes: https://tracker.ceph.com/issues/54356
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-04-27 12:04:02 +05:30
Samuel Just
7abc382bfb
Merge pull request #46012 from liu-chunmei/crimson-osd_op.rval
crimson/osd: still need set error code value in osd_op.rval

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-04-26 16:22:38 -07:00
J. Eric Ivancich
41f4e83cf5 rgw: address crash and race in RGWIndexCompletionManager
An atomic int was used in a modulo operator to distribute contention
among a set of locks and to track completions. Because it was an int,
enough increments would cause it to go negative (due to
twos-complement encoding and overflow) thereby causing a
crash. Additionally, even though it was atomic, the read and increment
were separate operations, leading to a race.

This commit addresses both of these issues.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2022-04-26 18:36:19 -04:00
Adam King
11fde3303c
Merge pull request #45598 from rkachach/fix_issue_55008
mgr/cephadm: Adding image tag and date to cephadm startup messages

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 15:48:03 -04:00
Adam King
b16ff1b0a0
Merge pull request #45855 from rkachach/fix_issue_55265
mgr/cephadm: fixing call method timeout handling

Reviewed-by: Adam King <adking@redhat.com>
2022-04-26 15:11:34 -04:00
Adam King
0b7ed08d80
Merge pull request #45877 from rkachach/fix_issue_55185
mgr/cephadm: Adding support to store ceph conf per cluster fsid

Reviewed-by: Adam King <adking@redhat.com>
2022-04-26 15:10:26 -04:00
Adam King
9268e32047
Merge pull request #45899 from rkachach/fix_issue_53527
mgr/cephadm: fixing upgrade candidate versions listing

Reviewed-by: Adam King <adking@redhat.com>
2022-04-26 15:09:37 -04:00
dparmar18
eefa3f6742 doc/man/cephfs-shell.rst: add use of exclamation mark with cmds
Description: `!` -  Exclamation mark can be used to interact with local
             file system apart from Ceph File System too. This PR intends
             to document it.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2022-04-27 00:39:23 +05:30
Liav Turkia
b303ae1f11 rgw: notifications on object replication
Signed-off-by: liavt <liav.turkia@gmail.com>
2022-04-26 21:59:14 +03:00
Laura Flores
246c21d7f9 .github: remove "Submitting Patches to Ceph" comment
Ideally, this should not be in comment form so users
can click on the link.

Signed-off-by: Laura Flores <lflores@redhat.com>
2022-04-26 13:17:06 -05:00
Daniel Gryniewicz
c60a9c77ea
Merge pull request #45623 from dang/wip-dang-zipper-objctx
RGW: Zipper - zonegroup, placement tier

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-04-26 14:03:11 -04:00
Neha Ojha
cf78cf4980
Merge pull request #46010 from gregsfortytwo/wip-55419-blocklist-test
test: Fix a blocklist API test

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-04-26 10:59:07 -07:00
John Mulligan
bb2b45d703 mgr/nfs: add test for access type validation func
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
2bcd911e31 mgr/nfs: add test for squash validation func
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
3d17b9ee74 mgr/nfs: lift validate access type method to file level function
More reusable across the entire file and made private, shrinking
the public surface area of the type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
2587c29c8a mgr/nfs: lift validate squash method to file level function
More reusable across the entire file and made private, shrinking
the public surface area of the type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
42e0a9ad9f mgr/nfs: move write_block func to file level format_block
The method was an unnecessary use of staticmethod as well as a
confusingly named function. Move the file and rename it for clarity.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
1a7b7e35b2 mgr/nfs: lift _format_val function to file level
There's no advantage to having this function as a closure and it makes
reading the code (and testing it) more challenging.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
6b33cec657 mgr/nfs: simplify implementation of _indentation function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
d791d5c229 mgr/nfs: lift generic indentation function to file level
Tucking a private method away as a staticmethod makes it harder to reuse
and less obvious that it can be resused.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
John Mulligan
bc895b6a7f mgr/nfs: rename export_utils.py to ganesha_conf.py
This new name is clearer about what topic the code in the file covers.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-04-26 13:38:22 -04:00
Nizamudeen A
3d477a39bb mgr/dashboard: introduce grafana frontend e2e testing
Used the https://www.npmjs.com/package/@grafana/e2e npm packages and
followed
https://github.com/grafana/grafana/blob/main/contribute/style-guides/e2e.md
to understand the style of the grafana e2e testing.

In this PR I introduces the tests for the Hosts Overall
Performance and also RGW per Daemon and Overall Performance

Fixes: https://tracker.ceph.com/issues/54356
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-04-26 23:07:38 +05:30
Zac Dover
e3d8428dd0 doc: clean up PendingReleaseNotes for reef
This PR removes all of the release notes up to
version 17 (Quincy) inclusive.

As far as I know, for as long as I have worked
on the Ceph project, we have never cleaned up
the release notes from release to release.

Until now.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-04-27 03:15:27 +10:00
Venky Shankar
76b33670b5
Merge pull request #44958 from rishabh-d-dave/qa-cephfs-timeout
qa/cephfs: change default timeout from 900 secs to 300

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-26 22:30:43 +05:30
J. Eric Ivancich
3793a29486 rgw: use dpp in bitx logging code
This updates the bitx logging code to use the logging provided by
DoutPrefixProvider.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2022-04-26 12:50:31 -04:00
Casey Bodley
7bc93fcae7
Merge pull request #45737 from cbodley/wip-rgw-ops-log-defaults
rgw: change default ops log backend from rados to file

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Cory Snyder <csnyder@iland.com>
2022-04-26 12:43:44 -04:00
Casey Bodley
197e9ea347
Merge pull request #46007 from cbodley/wip-49302
rgw: RGWCoroutine::set_sleeping() checks for null stack

Reviewed-by: Or Friedmann <ofriedma@redhat.com>
2022-04-26 12:25:36 -04:00
Casey Bodley
bab8b96a7a
Merge pull request #46027 from cbodley/wip-55432
rgw: OpsLogFile::stop() signals under mutex

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Cory Snyder <csnyder@iland.com>
2022-04-26 12:24:35 -04:00
Casey Bodley
ece51d5a6a
Merge pull request #45784 from cbodley/wip-rgw-multisite-configurables
rgw multisite: configurable polling intervals and concurrency windows

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-04-26 12:22:43 -04:00
Rishabh Dave
ac5de474cf qa/cephfs: change default timeout from 900 secs to 300
15 minutes is unnecessarily large as a default value for timeout for a
command. Not having to wait unnecessarily on a crash of a command will
reduce teuthology's testing queue and will save individual developer's
time while running tests locally.

Whatever lines are modified for this purpose are also modified to follow
the stlye guideline, specfically wrapping at 80 characters.

Fixes: https://tracker.ceph.com/issues/54236
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-04-26 18:34:42 +05:30
Ernesto Puerta
5ba7d8520d
Merge pull request #45830 from rhcs-dashboard/cephadm-logs-save
mgr/dashboard: save mgr logs inside a folder 

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-04-26 12:50:43 +02:00
Yin Congmin
64f741eceb librbd/cache/pwl: fix endianness issue
fix endianness issue with WriteLogCacheEntry encoding. abandon the
use of bits in the union. make '&' operation with the whole union
filed(flags) to get the bit information.

Fixes: https://tracker.ceph.com/issues/55389

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2022-04-26 16:01:39 +08:00
Venky Shankar
210bae76a4
Merge pull request #45370 from lxbsz/wip-54411
client: add option to disable collecting and sending metrics

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-04-26 09:55:04 +05:30
chunmei-liu
b7401f1959 crimson/osd: still need set error code value in osd_op.rval
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-04-25 19:43:24 -07:00
Yuri Weinstein
f6d1cb6e30
Merge pull request #45904 from cfsnyder/fix_rocksdb_iter_perf
os/bluestore: set upper and lower bounds on rocksdb omap iterators

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2022-04-25 15:58:45 -07:00
Neha Ojha
11bf4e8adc
Merge pull request #46028 from ceph/revert-43413-bluestore-bufferbloat-mitigation
Revert "os/bluestore: Add CoDel to BlueStore for Bufferbloat mitigation"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2022-04-25 15:07:38 -07:00