Commit Graph

135363 Commits

Author SHA1 Message Date
Lucian Petrut
f07b8a7984
Merge pull request #49277 from petrutlucian94/fix_log_win32
log: fix stderr handling on Windows
2022-12-12 12:44:48 +02:00
Yingxin
388e45d345
Merge pull request #49325 from liu-chunmei/crimson-reactor_utilization
crimson/osd: dump each shard seastar metrics

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-12 15:43:12 +08:00
colemitchell
6ebdce58a4
Merge pull request #49383 from zdover23/wip-doc-2022-12-12-rados-user-management-prompts
doc/rados: add prompts to user-management.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-11 23:32:46 -05:00
Zac Dover
46f9506e4c doc/rados: add prompts to user-management.rst
Add unselectable prompts to doc/rados/operations/user-management.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-12 14:25:07 +10:00
Ronen Friedman
4d5c1bfc21
Merge pull request #48545 from ronen-fr/wip-rf-slow-replica
osd/scrub: report replicas slow to respond to scrub requests

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-11 19:05:34 +02:00
Ronen Friedman
7730baa6c5 osd/scrub: modifying some tests to ignore late replicas
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
Ronen Friedman
1295b3752c osd/scrub: earlier response to interval changes
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
Ronen Friedman
09ec8f544f osd/scrub: minor logging changes
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
Ronen Friedman
411ea10084 osd/scrub: report replicas slow to repond to scrub requests
Implemented timeouts:

1: Slow-Secondary Warning:

Once at least half of the replicas have accepted the reservation, we
start reporting any secondary that takes too long (more than <conf>
milliseconds after the previous response received) to respond to the reservation
request.
(Why? because we have encountered real-life situations where a specific
OSD was systematically very slow to respond (e.g. 5 seconds in one case) to
the reservation requests, slowing the scrub process to a crawl).

2: Reservation Process Timeout:

We now limit the total time the primary waits for the replicas to
respond to the reservation request. If we do not get all the responses
(either Grant or Reject) within <conf> milliseconds, we give up and release all the
reservations we have acquired so far.
(Why? because we have encountered instances where a reservation request
was lost - either due to a bug or due to a network issue.)

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-11 17:27:42 +02:00
colemitchell
0fb0add0a5
Merge pull request #49376 from zdover23/wip-doc-2022-12-11-install-clone-source-fix-git-command
doc/install: update clone-source.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-10 16:31:53 -05:00
Zac Dover
344ef1553e doc/install: update clone-source.rst
Beef up clone-source.rst. Repair semantics. Add internal references. Fix
a broken "git submodule foreach git clean -fdx" command.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-11 07:24:09 +10:00
Kefu Chai
c273757601
Merge pull request #49375 from adk3798/partial-revert-49359
pybind/mgr: partial reversion of #49359

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 23:57:01 +08:00
colemitchell
1f24b872a8
Merge pull request #49368 from zdover23/wip-doc-2022-12-10-rados-upmap-prompts
doc/rados: add prompts to upmap.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 23:42:48 -05:00
colemitchell
fe6033809a
Merge pull request #49367 from zdover23/wip-doc-2022-12-10-rados-stretch-mode-prompts
doc/rados: add prompts to stretch-mode.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 23:42:12 -05:00
Zac Dover
06d8ac91b2 doc/rados: add prompts to upmap.rst
Add unselectable prompts to doc/rados/operations/upmap.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-10 14:34:46 +10:00
Zac Dover
bafe76c204 doc/rados: add prompts to stretch-mode.rst
Add unselectable prompts to doc/rados/operations/stretch-mode.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-10 14:15:45 +10:00
colemitchell
018d4351bd
Merge pull request #49364 from zdover23/wip-doc-2022-12-10-rbd-rados-rbd-retrieving-image-information-typo
doc/rbd: remove typo and ill-formed command

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 22:31:57 -05:00
Zac Dover
8df894b753 doc/rbd: remove typo and ill-formed command
Ingest Ilya Dryomov's comments here https://github.com/ceph/ceph/pull/49340/files#r1044365857

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-10 13:20:17 +10:00
Adam King
a6065a8340 pybind/mgr: partial reversion of #49359
When creating #49359 I was testing on an outdated
branch and didn't realize part of what I was fixing
had already been fixed in #49321. Basically ended up
changing what a variable "mypy_args" is set to but that
variable is no longer being used. It has no actual effect
but we should remove the extraneous code.

Signed-off-by: Adam King <adking@redhat.com>
2022-12-09 14:20:47 -05:00
Adam King
9c9d0d3618
Merge pull request #49359 from adk3798/mypy-args
mgr/pybind: fix mypy and autopep8 arg parsing

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-09 13:19:10 -05:00
Casey Bodley
500a28fbe8
Merge pull request #48626 from qiuxinyidian/rgw-coroutine
rgw: modify multisite coroutine stack block

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-12-09 12:56:28 -05:00
Casey Bodley
c27987273e
Merge pull request #48559 from baergj/fix-listbuckets-truncation-main
rgw: Fix truncated ListBuckets response.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2022-12-09 12:55:21 -05:00
Casey Bodley
2e08de5b8d
Merge pull request #48554 from giubacc/rgw_dns_name_multivalue
rgw: add multivalue support to rgw_dns_name config option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-09 12:54:13 -05:00
Casey Bodley
46462ebf27
Merge pull request #48390 from cbodley/wip-57784
rgw: beast frontend checks for local_endpoint() errors

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-12-09 12:52:32 -05:00
Casey Bodley
e86e0b7de0
Merge pull request #45818 from cbodley/wip-system-rapidjson
cmake: use rapidjson headers from s3select's submodule

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-12-09 12:52:07 -05:00
Daniel Gryniewicz
2c8520c925
Merge pull request #49336 from dang/wip-dang-zipper-driver
Store -> Driver rename + header cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-09 12:41:13 -05:00
Adam King
2ecd74cdac pybind/mgr: fix tox autopep8 args
Similar to mypy, we now need a backslash to signal
a newline doesn't mean to start a new command

Signed-off-by: Adam King <adking@redhat.com>
2022-12-09 11:25:01 -05:00
Adam King
8e5840ff2c mgr/pybind: fix mypy arg parsing
On the new tox version it is treating each line as a new command
so it will do something like "mypy --config-file=../../mypy.ini"
as one command and then "-m balancer" as a totally separate command.
The first one immediately fails as it doesn't include any modules
to test. Adding backslashes to the ends of the lines gets it to
handle the lines as one long command

Signed-off-by: Adam King <adking@redhat.com>
2022-12-09 11:19:43 -05:00
Yingxin
94a142f227
Merge pull request #48691 from zhscn/wip-reserve
crimson/os/seastore: correct the behavior of reserving space

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-12-09 18:01:52 +08:00
Pere Diaz Bou
92168eac67
Merge pull request #47842 from trociny/wip-rbd-rest-api-metadata
mgr/dashboard: allow to get/update RBD image metadata via REST API

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-12-09 09:34:06 +01:00
colemitchell
7c8617423d
Merge pull request #49342 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-removing-a-block-device-semantics
doc/rbd: refine "Removing a Block Device Image"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 01:45:28 -05:00
Zac Dover
3a6284a49b doc/rbd: refine "Removing a Block Device Image"
Refine and add unselectable prompts to "Removing a Block Device Image"
in doc/rbd/rados-rbd-cmds.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 16:38:46 +10:00
colemitchell
9ccf87c165
Merge pull request #49341 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-resizing-a-block-device-image-semantics
doc/rbd: refine "Resizing a Block Device Image"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 01:17:17 -05:00
colemitchell
10505022c7
Merge pull request #49343 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-restoring-a-block-device-image-semantics
doc/rbd: refine "Restoring a Block Device Image"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 01:16:43 -05:00
Zac Dover
8fb4edb92d doc/rbd: refine "Restoring a Block Device Image"
Refine and add unselectable prompts to "Restoring a Block Device Image"
in doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 16:07:19 +10:00
Zac Dover
078bec94d0 doc/rbd: refine "Resizing a Block Device Image"
Refine and add unselectable prompts to "Resizing a Block Device Image"
in doc/rbd/rados/rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 15:19:54 +10:00
colemitchell
19426975b2
Merge pull request #49338 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-creating-a-block-device-image-semantics
doc/rbd: refine "Creating a Block Device Image"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 23:50:23 -05:00
colemitchell
7c1191da75
Merge pull request #49340 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-retrieving-image-information-semantics
doc/rbd: refine "Retrieving Image Information"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 23:50:07 -05:00
colemitchell
2d50caa884
Merge pull request #49339 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-listing-block-device-images-semantics
doc/rbd: refine "Listing Block Device Images"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 21:50:05 -05:00
Zac Dover
d418037b38 doc/rbd: refine "Retrieving Image Information"
Refine the text and prompts in "Retrieving Image Information" in
doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 10:45:19 +10:00
Kefu Chai
af7d7d386b
Merge pull request #49321 from tchaikov/wip-qa-install-with-utf-8
qa: set locale to C.UTF-8 in tox.ini

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2022-12-09 08:31:05 +08:00
chunmei-liu
2ec6ff5e39 crimson/osd: dump each shard seastar metrics
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-12-08 16:27:10 -08:00
Zac Dover
ad3c93535f doc/rbd: refine "Listing Block Device Images"
Refine the text and prompts in "Listing Block Device Images" in
doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 10:21:47 +10:00
Zac Dover
20b14437d6 doc/rbd: refine "Creating a Block Device Image"
Refine the English and prompts in "Creating a Block Device Image" in
doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 09:53:14 +10:00
Matt Benjamin
c30449b35b rgw: define a hierarchy of basic type headers
Defines a hierarchy of "simple" data types ensured to be
safe to include in any context above the zipper line, and
also from CLS.

The following headers are currently defined to contain basic
types only:

 rewrite src/rgw/rgw_basic_types.h (70%) // includes the whole hierarchy
 create mode 100644 src/rgw/rgw_acl_types.h
 create mode 100644 src/rgw/rgw_bucket_types.h
 create mode 100644 src/rgw/rgw_obj_types.h
 create mode 100644 src/rgw/rgw_placement_types.h
 create mode 100644 src/rgw/rgw_pool_types.h
 create mode 100644 src/rgw/rgw_quota_types.h
 create mode 100644 src/rgw/rgw_user_types.h
 create mode 100644 src/rgw/rgw_zone_types.h

This commit consolidates the following original commits:

* rgw: move RGWUploadPartInfo to rgw_basic_types.{h,cc}
* rgw: move rgw_obj_key to rgw_basic_types.{h,cc}
* rgw: move rgw_placement_rule to rgw_basic_types.{h,cc}
* rgw: move rgw_obj to rgw_basic_types.{h,cc}
* rgw: include rgw_compression_types.h in rgw_basic_types.{h,cc}
* rgw: move rgw_raw_obj to rgw_basic_types.{h,cc}
* rgw: rgw_multi.h: remove unused RGWMPObj forward decl

and the following cleanups from review:

* rgw: remove stray comments
* rgw: move rgw_obj_manifest.h inclusion to top of rgw_basic_types.h
* rgw: nit: indentation
* rgw: remove this line
* rgw: move rgw_bucket_shard to rgw_bucket_types.h, cleanup
* rgw: fix rgw_quota.h

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:42:34 -05:00
Matt Benjamin
9ee09fc27f rgw: setup for inclusion of rgw_basic_types in rgw cls
In downstream 4.1 version this change moved rgw_pool, rgw_bucket,
and some related types--but these have already moved on more
recent branches.

Include rgw_basic_types.h only from cls_rgw_types.h (review).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:42:34 -05:00
Daniel Gryniewicz
644a488190 RGW - Zipper - move rgw/store to rgw/driver
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:42:29 -05:00
Daniel Gryniewicz
1fac0a75ee RGW - Zipper - Rename rgw::sal::Store to rgw::sal::Driver
All along, we've had an overload of the term "store".  It was the base
class of the entrypoint, and it was the lowest layer in a stack.  This
renames the base class to Driver.  So, the primary elements of zipper
that to work are now Drivers, and they come in two different flavors:
Filters live in the middle of the stack, and do not need to implement
every API, but pass through instead.  And Stores live a the bottom of
the stack, and must implement the entire API.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:13:12 -05:00
Kefu Chai
c67106b120 pybind/mgr: do not test with py37
as we always test with ubuntu jammy, which does not provide python3.7:

py37: skipped because could not find python interpreter with spec(s): py37

so there is no point testing with python3.7.

also, in tox v4, it is not able to handle "key = value" anymore, where
value has newlines in it. so we need to find a better way passing
command line options to the test command.

this change partially reverts 2dd86c9e3e

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-09 01:07:19 +08:00
Kefu Chai
da40d84640 cmake: temporarily disable py3 test
py3 tries to import all python modules to ensure that they are
python3 compatible. but the installation fails on jenkins test node:

  Resolved https://github.com/ceph/teuthology.git to commit 4da97cf64e542f347ec47b7bdbe5eca99759f9b7
  Installing build dependencies: started
  error: subprocess-exited-with-error

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-09 01:07:19 +08:00