Commit Graph

133525 Commits

Author SHA1 Message Date
Adam King
2622a42522
Merge pull request #47523 from adk3798/oracle-linux-support
cephadm: support for Oracle Linux 8

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-08-17 12:49:49 -04:00
Matan Breizman
02113c9cf6 PendingReleaseNotes: fix typo in 15.2.17
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-17 16:43:48 +00:00
Casey Bodley
77558ef2c4
Merge pull request #47422 from adamemerson/wip-coroutine-cleaver
rgw: Break Full and Incremental sync out into their own classes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-17 10:00:04 -04:00
Matan Breizman
5db85e6f45 qa/suites: Reduce rados_python time out
This test runs for few minutes,
reducing timeout from 3h to 1h to avoid hanging jobs.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-17 12:05:20 +00:00
Matan Breizman
c8b85873e0 qa/crimson-rados: add 'wait' attribute
wait_for_complete tests currently hanging with crimson-osd
and are temporary disabled in Crimson suite.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-17 12:02:09 +00:00
Neeraj Pratap Singh
af554a8b1e mgr/stats: change in structure of perf_stats o/p
*Adding fs_name as a field option in perf_stats o/p
*perf stats command incorrect output with non-existing mds_rank filter
 If `ceph fs perf stats` runs with non-existing mds_rank filter,
 even then it shows all the clients `client_metadata` and `global_metrics`

Fixes: https://tracker.ceph.com/issues/56162
Fixes: https://tracker.ceph.com/issues/56169
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-08-17 16:00:13 +05:30
Aswin Toni
2e0e684fc2 ceph-mixin: Remove jsonnet building
Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
2022-08-17 12:08:56 +02:00
Aswin Toni
5cdc1c62c5 prometheus: add multicluster support to alerts
Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
2022-08-17 12:08:56 +02:00
Xinyu Huang
5caf48396c crimson/os/seastore/omap: optimize merge policy
in some corner cases on single key erase sence,
for e.g. we have three nodes which are brothers:
1 2 3 4 5 | 6 7 8 | 9 10 11, node min size is 3.
When erasing 11 will trigger balance instead of
merge, the middle nodes will adjust to 6 7, this
will not ensure the all nodes above min size.
This corner case might not be harmful but if we
want to erase 10,11 at one time (range remove wip
need this), this will trigger balance in past
policy which will make both two nodes below min
size and we might need at least one more merge.
This is actually because we cannot ensure a node
to merge in tree is above min size.

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-08-17 08:42:35 +00:00
Xinyu Huang
f5143a2e11 crimson/os/seastore/omap: code cleanup
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-08-17 08:42:29 +00:00
Yingxin
b578d2ecb6
Merge pull request #47491 from cyx1231st/wip-seastore-fix-return-future
crimson/os/seastore/transaction_manager: return future when enoent

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-08-17 16:39:22 +08:00
Ronen Friedman
f8a0ca5d25
Merge pull request #46855 from ronen-fr/wip-rf-pg-updates
osd/scrub: Scrubbing PGs now periodically update their stats

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
2022-08-17 11:18:25 +03:00
Yingxin
c1f6beda7c
Merge pull request #47553 from cyx1231st/wip-seastore-cleanup-reclaim
crimson/os/seastore: cleanups to gc reclaim

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Zhang Song <zhangsong325@gmail.com>
2022-08-17 16:11:46 +08:00
Venky Shankar
cd38fe3c85
Merge pull request #47477 from mchangir/doc-clarify-start-time-in-snap-schedule
doc/cephfs: note regarding start time time zone

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-08-17 12:36:10 +05:30
Venky Shankar
df5bef054a
Merge pull request #47099 from joscollin/wip-B56537-wrong-rsp-wsp-values
cephfs-top: fix the rsp/wsp display

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-08-17 11:14:21 +05:30
Matt Benjamin
12a9833fd1 rgw/s3: add conditional json output for s3:ListBucket
Adapt the XML ListBucket response to contain well-formed JSON,
matching the implied schema of awscli:

$ aws --endpoint-url http://localhost:8000 s3api list-objects --bucket testbucket

suggested by Casey in review.

Fixes: https://tracker.ceph.com/issues/53455

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2022-08-16 15:05:02 -04:00
Adam King
042ca579c2
Merge pull request #47365 from adk3798/explicit-host-no-schedule
mgr/cephadm: fix handling of draining hosts with explicit placement specs

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-08-16 14:35:16 -04:00
Adam King
efcab49ea2
Merge pull request #47529 from hookak/container-args
cephadm: Remove duplicated process args in promtail and loki

Reviewed-by: Adam King <adking@redhat.com>
2022-08-16 14:32:03 -04:00
Adam King
b4ec87db95
Merge pull request #47545 from adk3798/grafana-password
mgr/cephadm: set dashboard grafana-api-password when user provides one

Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-08-16 14:27:37 -04:00
Matt Benjamin
7ad5386110 rgw/s3: align output format with client request
In general, e.g., in the S3 ListBucket response, the
output Formatter is already of the type requested in the HTTP
Accept header.

Therefore, while there are still many instances where the of
Formatter is assumed to be XML, it appears necessarily safe
and seemingly correct to extend this assumption to end_header(...),
which sends the response content-type.

Fixes: https://tracker.ceph.com/issues/55680

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2022-08-16 14:18:06 -04:00
Laura Flores
d6b46a2c77 qa/workunits/rados: pull librados files from "main" instead of "master"
Fixes: https://tracker.ceph.com/issues/57122
Signed-off-by: Laura Flores <lflores@redhat.com>
2022-08-16 18:07:51 +00:00
Matt Benjamin
9070c229ac rgw/rest: replace RGW_FORMAT_ macros with an enum class
This is a pure cleanup.  The method to print an RGWFormat object
as a MIME type is now called to_mime_type().

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2022-08-16 13:52:42 -04:00
Matt Benjamin
66d98b383a rgw/iam: thread DoutPrefixProvider into policy evaluation
Also conditionally prints policy and resource in verify_bucket_permission.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2022-08-16 13:39:11 -04:00
Matt Benjamin
2e26ede8e9 rgw: fix link dependencies when not building dbstore
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2022-08-16 13:39:04 -04:00
Daniel Gryniewicz
5211f655bd RGW - Zipper - Use zipper APIs
Some code was using old, non-zipper versions of code that existed in
zipper.  Use the zipper versions instead.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-16 12:31:27 -04:00
Daniel Gryniewicz
f93956927d RGW - Zipper - Add verify_mfa
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-16 12:31:27 -04:00
Daniel Gryniewicz
ff1b565887 RGW - Zipper - Make Zone generic
- RadosStore had a dummy implementation of Zone that just passed through
  to the Zone service to get info about the local zone.  Implement the
  Zone API for real, so that we can reference multiple zones.
- Convert get_id() to string; it was only ever used as a string anyway.
- Add get_zone_by*() to zonegroup and use it apporpriately

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-16 12:31:27 -04:00
Daniel Gryniewicz
5a6fb22707 RGW - Zipper - LC - Allow notifications on all stores
The notification API exsits in all stores, so make the useage
unconditional.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-16 12:31:27 -04:00
Daniel Gryniewicz
43ef192e7d RGW - Zipper - Move generic get_zonegroup() to Store
The get_zonegroup() that looks up by ID doesn't belong in the zone.  It
was there because of a conflation of zone as an object an zone as an
interface.  Move it to Store where it belongs.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-16 12:31:27 -04:00
Ronen Friedman
c50c20fd19 osd/scrub: PGs that are scrubbing now update stats periodically
Added periodic calls to all PGs for which the OSD is the primary,
asking for a scrub statistics update.

This allows operator queries (e.g. 'pg dump pgs') to present
up-to-date scrub duration, "scrub is blocked" duration, etc'.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-08-16 17:19:21 +03:00
Kotresh HR
aece3b9b28 qa: Validate cleaning of the stale snapshot metadata
Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 18:27:54 +05:30
Adam King
e3d4c7707e
Merge pull request #47565 from adk3798/unit-test-runtime
mgr/cephadm: mock CephadmHttpServer run in unit tests

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2022-08-16 08:50:10 -04:00
Milind Changire
5d281e9b88 doc/cephfs: note regarding start time time zone
Fixes: https://tracker.ceph.com/issues/56730
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-08-16 17:51:37 +05:30
Nizamudeen A
c7778d3328
Merge pull request #47119 from rhcs-dashboard/image-refs-bug
mgr/dashboard: fix _rbd_image_refs caching

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2022-08-16 17:29:05 +05:30
Kotresh HR
65af2d123a mgr/volumes: Remove stale snapshot user metadata
This patch adds the capability to remove the stale snapshot user
metadata while loading the subvolume if it is present. It can't
be done in 'SubvolumeBase.discover' since v1 and v2 snapshot paths
are different. This is done just after the discover before returning
the specific version object.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 17:11:33 +05:30
Kotresh HR
0687f78650 mgr/volumes: Allow forceful snapshot removal on osd full
When the osd is full, if the snapshot has metadata set, it
can't be removed as user metadata can't be removed when osd
is full. This patch provides a way to remove the snapshot
with 'force' option while keeping the corresponding metadata
which gets removed on subvolume discover when it finds space.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 17:08:47 +05:30
Kotresh HR
a64f049614 qa: Add subvolume clone and snapshot rm tests when osd is full
Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 17:08:36 +05:30
Kotresh HR
ec57215d50 mgr/volumes: Better handle config file on osd full scenario
The 'metadata_mgr.flush()' used to truncate the config file
before flushing the new config data. This could lead to an
empty config file when there is no space to write new config
data. This patch handles this scenario by writing it to
temporary file and rename it to config file. This would
retain the config file without truncating it.

Also, there are bunch of places which wasn't handling
'MetadataMgrException' because of this. Fixed those.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-08-16 17:08:16 +05:30
Kefu Chai
23b2e983ae
Merge pull request #47609 from cbodley/wip-install-deps-rhel
install-deps: remove outdated rhel8 devtool packages

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-08-16 19:19:07 +08:00
Mark Kogan
51fa31878e rgw: fix linking with tcmalloc when specified by -DALLOCATOR
Fixes: https://tracker.ceph.com/issues/57140

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2022-08-16 10:46:18 +00:00
Ilya Dryomov
61df853f8c
Merge pull request #45628 from PepperJo/comparev_and_writev_v7
librbd: add compare-and-write compare length limit, remove sector size restriction

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-08-16 11:53:05 +02:00
Ernesto Puerta
f6c88f4c30 mgr/dashboard: add backend-driven UI tables
As an example this PR displays the list of Ceph Auth users, keys and
caps.

It tries to minimize the amount of UI code required by feeding a generic
table-like component (RestTable) with backend-generated JSON data.

This is just a proof of concept and there's lot of room for improvement.

Fixes: https://tracker.ceph.com/issues/52701

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-08-16 11:25:14 +02:00
Mer.xuanyi
46f82c67f0 mgr/snap-schedule: cleanup code
Signed-off-by: Mer Xuanyi <xuanyi.meng@xtaotech.com>
2022-08-16 17:20:45 +08:00
Mer.xuanyi
ac5f64e86b mgr/snap-schedule: use the right way to check the result returned by fetchone()
Fixes: https://tracker.ceph.com/issues/57138
Signed-off-by: Mer Xuanyi <xuanyi.meng@xtaotech.com>
2022-08-16 17:18:13 +08:00
Kefu Chai
3e37ee6e56
Merge pull request #47617 from tchaikov/wip-cmake-jsonnet
cmake: set $PATH for tests using jsonnet tools

Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2022-08-16 16:59:43 +08:00
Yingxin
601f7be038
Merge pull request #47618 from xxhdx1985126/wip-journal-move
crimson/os/seastore/journal: move bufferlists to reduce the amount of memory copies

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-08-16 12:51:23 +08:00
Xuehan Xu
a531b76775 crimson/os/seastore/journal: move bufferlists to reduce the amount of memory copies
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-08-16 10:58:37 +08:00
Kefu Chai
4a3afcf277 cmake: set $PATH for tests using jsonnet tools
otherwise they would not able to find executables installed into
${CMAKE_CURRENT_BINARY_DIR}.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-16 10:53:29 +08:00
Kefu Chai
7e309f3a1f
Merge pull request #47594 from tchaikov/wip-cmake-win32
cmake: enforce gcc-10 for win32 port

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-16 09:45:54 +08:00
jinhong.kim
6141d207cb
cephadm: Remove duplicated process args in promtail and loki(config.file)
Signed-off-by: jinhong.kim <jinhong.kim0@navercorp.com>
2022-08-16 09:48:23 +09:00