Commit Graph

135415 Commits

Author SHA1 Message Date
Kefu Chai
b1382d0a23 rgw/rgw_common: define FMT_HEADER_ONLY
to be consistent with other parts of rgw

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-10 09:55:05 +08:00
Adam King
a6065a8340 pybind/mgr: partial reversion of #49359
When creating #49359 I was testing on an outdated
branch and didn't realize part of what I was fixing
had already been fixed in #49321. Basically ended up
changing what a variable "mypy_args" is set to but that
variable is no longer being used. It has no actual effect
but we should remove the extraneous code.

Signed-off-by: Adam King <adking@redhat.com>
2022-12-09 14:20:47 -05:00
Adam King
9c9d0d3618
Merge pull request #49359 from adk3798/mypy-args
mgr/pybind: fix mypy and autopep8 arg parsing

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-09 13:19:10 -05:00
Casey Bodley
500a28fbe8
Merge pull request #48626 from qiuxinyidian/rgw-coroutine
rgw: modify multisite coroutine stack block

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-12-09 12:56:28 -05:00
Casey Bodley
c27987273e
Merge pull request #48559 from baergj/fix-listbuckets-truncation-main
rgw: Fix truncated ListBuckets response.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2022-12-09 12:55:21 -05:00
Casey Bodley
2e08de5b8d
Merge pull request #48554 from giubacc/rgw_dns_name_multivalue
rgw: add multivalue support to rgw_dns_name config option

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-09 12:54:13 -05:00
Casey Bodley
46462ebf27
Merge pull request #48390 from cbodley/wip-57784
rgw: beast frontend checks for local_endpoint() errors

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-12-09 12:52:32 -05:00
Casey Bodley
e86e0b7de0
Merge pull request #45818 from cbodley/wip-system-rapidjson
cmake: use rapidjson headers from s3select's submodule

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-12-09 12:52:07 -05:00
Daniel Gryniewicz
2c8520c925
Merge pull request #49336 from dang/wip-dang-zipper-driver
Store -> Driver rename + header cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-09 12:41:13 -05:00
Adam King
2ecd74cdac pybind/mgr: fix tox autopep8 args
Similar to mypy, we now need a backslash to signal
a newline doesn't mean to start a new command

Signed-off-by: Adam King <adking@redhat.com>
2022-12-09 11:25:01 -05:00
Adam King
8e5840ff2c mgr/pybind: fix mypy arg parsing
On the new tox version it is treating each line as a new command
so it will do something like "mypy --config-file=../../mypy.ini"
as one command and then "-m balancer" as a totally separate command.
The first one immediately fails as it doesn't include any modules
to test. Adding backslashes to the ends of the lines gets it to
handle the lines as one long command

Signed-off-by: Adam King <adking@redhat.com>
2022-12-09 11:19:43 -05:00
Yingxin
94a142f227
Merge pull request #48691 from zhscn/wip-reserve
crimson/os/seastore: correct the behavior of reserving space

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-12-09 18:01:52 +08:00
Pere Diaz Bou
92168eac67
Merge pull request #47842 from trociny/wip-rbd-rest-api-metadata
mgr/dashboard: allow to get/update RBD image metadata via REST API

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-12-09 09:34:06 +01:00
colemitchell
7c8617423d
Merge pull request #49342 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-removing-a-block-device-semantics
doc/rbd: refine "Removing a Block Device Image"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 01:45:28 -05:00
Zac Dover
3a6284a49b doc/rbd: refine "Removing a Block Device Image"
Refine and add unselectable prompts to "Removing a Block Device Image"
in doc/rbd/rados-rbd-cmds.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 16:38:46 +10:00
colemitchell
9ccf87c165
Merge pull request #49341 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-resizing-a-block-device-image-semantics
doc/rbd: refine "Resizing a Block Device Image"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 01:17:17 -05:00
colemitchell
10505022c7
Merge pull request #49343 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-restoring-a-block-device-image-semantics
doc/rbd: refine "Restoring a Block Device Image"

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-09 01:16:43 -05:00
Zac Dover
8fb4edb92d doc/rbd: refine "Restoring a Block Device Image"
Refine and add unselectable prompts to "Restoring a Block Device Image"
in doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 16:07:19 +10:00
Zac Dover
078bec94d0 doc/rbd: refine "Resizing a Block Device Image"
Refine and add unselectable prompts to "Resizing a Block Device Image"
in doc/rbd/rados/rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 15:19:54 +10:00
colemitchell
19426975b2
Merge pull request #49338 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-creating-a-block-device-image-semantics
doc/rbd: refine "Creating a Block Device Image"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 23:50:23 -05:00
colemitchell
7c1191da75
Merge pull request #49340 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-retrieving-image-information-semantics
doc/rbd: refine "Retrieving Image Information"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 23:50:07 -05:00
colemitchell
2d50caa884
Merge pull request #49339 from zdover23/wip-doc-2022-12-09-rbd-rados-rbd-cmds-listing-block-device-images-semantics
doc/rbd: refine "Listing Block Device Images"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 21:50:05 -05:00
Zac Dover
d418037b38 doc/rbd: refine "Retrieving Image Information"
Refine the text and prompts in "Retrieving Image Information" in
doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 10:45:19 +10:00
Kefu Chai
af7d7d386b
Merge pull request #49321 from tchaikov/wip-qa-install-with-utf-8
qa: set locale to C.UTF-8 in tox.ini

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
2022-12-09 08:31:05 +08:00
chunmei-liu
2ec6ff5e39 crimson/osd: dump each shard seastar metrics
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-12-08 16:27:10 -08:00
Zac Dover
ad3c93535f doc/rbd: refine "Listing Block Device Images"
Refine the text and prompts in "Listing Block Device Images" in
doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 10:21:47 +10:00
Zac Dover
20b14437d6 doc/rbd: refine "Creating a Block Device Image"
Refine the English and prompts in "Creating a Block Device Image" in
doc/rbd/rados-rbd-cmds.rst.

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-09 09:53:14 +10:00
Matt Benjamin
c30449b35b rgw: define a hierarchy of basic type headers
Defines a hierarchy of "simple" data types ensured to be
safe to include in any context above the zipper line, and
also from CLS.

The following headers are currently defined to contain basic
types only:

 rewrite src/rgw/rgw_basic_types.h (70%) // includes the whole hierarchy
 create mode 100644 src/rgw/rgw_acl_types.h
 create mode 100644 src/rgw/rgw_bucket_types.h
 create mode 100644 src/rgw/rgw_obj_types.h
 create mode 100644 src/rgw/rgw_placement_types.h
 create mode 100644 src/rgw/rgw_pool_types.h
 create mode 100644 src/rgw/rgw_quota_types.h
 create mode 100644 src/rgw/rgw_user_types.h
 create mode 100644 src/rgw/rgw_zone_types.h

This commit consolidates the following original commits:

* rgw: move RGWUploadPartInfo to rgw_basic_types.{h,cc}
* rgw: move rgw_obj_key to rgw_basic_types.{h,cc}
* rgw: move rgw_placement_rule to rgw_basic_types.{h,cc}
* rgw: move rgw_obj to rgw_basic_types.{h,cc}
* rgw: include rgw_compression_types.h in rgw_basic_types.{h,cc}
* rgw: move rgw_raw_obj to rgw_basic_types.{h,cc}
* rgw: rgw_multi.h: remove unused RGWMPObj forward decl

and the following cleanups from review:

* rgw: remove stray comments
* rgw: move rgw_obj_manifest.h inclusion to top of rgw_basic_types.h
* rgw: nit: indentation
* rgw: remove this line
* rgw: move rgw_bucket_shard to rgw_bucket_types.h, cleanup
* rgw: fix rgw_quota.h

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:42:34 -05:00
Matt Benjamin
9ee09fc27f rgw: setup for inclusion of rgw_basic_types in rgw cls
In downstream 4.1 version this change moved rgw_pool, rgw_bucket,
and some related types--but these have already moved on more
recent branches.

Include rgw_basic_types.h only from cls_rgw_types.h (review).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:42:34 -05:00
Daniel Gryniewicz
644a488190 RGW - Zipper - move rgw/store to rgw/driver
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:42:29 -05:00
Daniel Gryniewicz
1fac0a75ee RGW - Zipper - Rename rgw::sal::Store to rgw::sal::Driver
All along, we've had an overload of the term "store".  It was the base
class of the entrypoint, and it was the lowest layer in a stack.  This
renames the base class to Driver.  So, the primary elements of zipper
that to work are now Drivers, and they come in two different flavors:
Filters live in the middle of the stack, and do not need to implement
every API, but pass through instead.  And Stores live a the bottom of
the stack, and must implement the entire API.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 14:13:12 -05:00
Kefu Chai
c67106b120 pybind/mgr: do not test with py37
as we always test with ubuntu jammy, which does not provide python3.7:

py37: skipped because could not find python interpreter with spec(s): py37

so there is no point testing with python3.7.

also, in tox v4, it is not able to handle "key = value" anymore, where
value has newlines in it. so we need to find a better way passing
command line options to the test command.

this change partially reverts 2dd86c9e3e

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-09 01:07:19 +08:00
Kefu Chai
da40d84640 cmake: temporarily disable py3 test
py3 tries to import all python modules to ensure that they are
python3 compatible. but the installation fails on jenkins test node:

  Resolved https://github.com/ceph/teuthology.git to commit 4da97cf64e542f347ec47b7bdbe5eca99759f9b7
  Installing build dependencies: started
  error: subprocess-exited-with-error

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-09 01:07:19 +08:00
Yuri Weinstein
dc61eb4372
Merge pull request #48375 from lbausch/mgr/balancer-fix_end_weekday_being_exclusive
mgr/balancer: Fix `end_weekday` being exclusive

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-12-08 08:49:17 -08:00
Yuri Weinstein
006e228f71
Merge pull request #48318 from gregsfortytwo/wip-17170
osd: mds: suggest clock skew when failing to obtain rotating service keys

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-12-08 08:48:23 -08:00
Kefu Chai
127dd54db8 qa: add pytest to deps of py3 env
otherwise it is mising when running test, and we'd have following
failure:

py3: exit 2 (0.00 seconds) /home/jenkins-build/build/workspace/ceph-pull-requests/qa> pytest --assert=plain test_import.py

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-08 23:57:59 +08:00
Kefu Chai
bc2b551e9f qa: set locale to C.UTF-8 in tox.ini
as ansible is using UTF-8 encoded characters in the file names, so,
to avoid failures like:

  File "/home/jenkins-build/build/workspace/ceph-pull-requests/qa/.tox/py3/lib/python3.10/site-packages/pip/_internal/utils/unpacking.py", line 217, in untar_file
    with open(path, "wb") as destfp:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 137-140: ordinal not in range(256)

we have to set a locale which is able to handle UTF-8.

see also https://github.com/ceph/teuthology/pull/1671

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-08 22:39:57 +08:00
Casey Bodley
bc01d2ed31
Merge pull request #49236 from yuvalif/wip-yuval-missing-init
rgw/notification: initialize member in reservation ctor

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-08 09:24:22 -05:00
Casey Bodley
4f5c9f3a62
Merge pull request #49290 from inspur-wyq/wip-rgw-svc-doc
src/rgw/services/svc_finisher.cc: fix typos

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-08 09:04:25 -05:00
Kefu Chai
cf6442b330
Merge pull request #49322 from tchaikov/wip-allowlist_externals
*: s/whitelist_externals/allowlist_externals/

Reviewed-by: Adam King <adking@redhat.com>
2022-12-08 21:33:38 +08:00
Avan Thakkar
04cfd23122 mgr/dashboard: allow Origin for CORS if present in config cross_origin_url
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-12-08 16:54:41 +05:30
Kefu Chai
c9cc795d02 pybind/mgr: drop cython from requires
cython is not required for running tox commands.

this should address the test failure like:

ROOT: will run in automatically provisioned tox, host /home/jenkins-build/build/workspace/ceph-pull-requests/build/mgr-virtualenv/bin/python3.10 is missing [requires (has)]: cython

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-08 18:26:42 +08:00
Kefu Chai
8b20bb8ca2 cephadm: drop cython from requires
cython is not required for launching the tox environment.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-08 18:23:57 +08:00
Kefu Chai
34e2e33870 *: s/whitelist_externals/allowlist_externals/
as allowlist_externals was introduced in
tox v4.0. see
5e33fda1a4 , but
this option was backported to 3.18 as an alias of whitelist_externals, so we don't need
to specify the minversion to 4.0 in this change.

as we started using tox 4.0 and up (v4.0.2 in specific). tox complains
and fails like:

alerts-lint: failed with promtool is not allowed, use allowlist_externals to allow it
  alerts-lint: FAIL code 1 (9.25 seconds)

see https://tox.wiki/en/latest/faq.html#tox-4-removed-tox-ini-keys
and https://tox.wiki/en/latest/config.html#allowlist_externals

it'd be nice to use a more inclusive language also. so, in this change,
s/whitelist_externals/allowlist_externals/ in all tox.ini in this
project.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-08 15:07:00 +08:00
zdover23
70a1011d18
Merge pull request #49306 from zdover23/wip-doc-2022-12-08-rbd-rados-rbd-cmds-create-a-block-device-user-semantics
doc/rbd: refine "Create a Block Device User"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-12-08 15:21:24 +10:00
Anthony D'Atri
dad9f9409f
Merge pull request #49288 from BryceCao/wip-update-doc-ubuntu-deps-install
doc/start: update "apt install" command in documenting-ceph.rst
2022-12-07 23:24:42 -05:00
cao.leilc
85a11ed22c doc/start: update "apt install" command in documenting-ceph.rst
Signed-off-by: caolei <cao.leilc@inspur.com>
2022-12-08 10:07:51 +08:00
Zac Dover
b6cc04e0bc doc/rbd: refine "Create a Block Device User"
Refine "Create a Block Device User", after suggestions made by Ilya
Dryomov in https://github.com/ceph/ceph/pull/49301#pullrequestreview-1208285685

https://tracker.ceph.com/issues/57001

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-08 10:38:52 +10:00
Laura Flores
4f693d8a57
Merge pull request #48943 from yaarith/fix-telemetry-ident
qa/workunits: identify ceph test clusters
2022-12-07 16:40:03 -06:00
Casey Bodley
d597c34fbb
Merge pull request #49121 from tchaikov/wip-cmake-cares
cmake: add c-ares::cares as a global visible library

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-07 15:07:27 -05:00