Commit Graph

604 Commits

Author SHA1 Message Date
J. Eric Ivancich
43f1838c0a
Merge pull request #33053 from tpsilva/add_max_connections
rgw: make max_connections configurable in beast

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-02-13 14:13:53 -05:00
J. Eric Ivancich
63faa3e4fb
Merge pull request #33165 from clwluvw/policy-subuser
rgw: Add support bucket policy for subuser

Reviewed-By: Adam Emersen <aemerson@redhat.com>
2020-02-13 14:11:00 -05:00
Liu Lan
b6bce1c850 doc/radosgw/multisite-sync-policy.rst: fix typo
Signed-off-by: Liu Lan <rjerk.whatever@gmail.com>
2020-02-12 13:23:44 +08:00
Seena Fallah
84b96f8d4f rgw: Add support bucket policy for subuser
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2020-02-11 12:23:46 +03:30
Tiago Pasqualini
d6dada5bcb rgw: make max_connections configurable in beast
Beast frontend currently accepts a hardcoded number of connections
that is defined by boost::asio::socket_base::max_connections. This
commit makes it configurable via a 'max_connections' config option
on rgw frontend.

Fixes: https://tracker.ceph.com/issues/43952
Signed-off-by: Tiago Pasqualini <tiago.pasqualini@canonical.com>
2020-02-06 17:42:38 -03:00
Yehuda Sadeh
0c69aa00ff rgw: bucket sync: doc fixes
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2020-01-28 10:20:40 -08:00
Yehuda Sadeh
caac422445 rgw: sync policy doc
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2020-01-28 10:20:40 -08:00
Casey Bodley
4bf2f02506
Merge pull request #31185 from joke-lee/rgw-user-modify-placement
rgw: support specify user default placement and placement_tags when create or modify user

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-01-27 10:44:17 -05:00
Casey Bodley
b041c62304 doc/rgw: fix broken link to boto s3 extensions document
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-20 16:46:51 -05:00
Yuval Lifshitz
07630a8759 rgw/notification: add opaque data
opaque data may be set in topic configuration and later on sent inside
nottifications triggered by that topic.

Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-01-19 15:37:56 +02:00
Abhishek L
af0e482a3f
Merge pull request #32367 from capri1989/doc-gc-update
doc/radosgw/config-ref: paragraph to explain the gc settings

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-01-17 10:08:29 +01:00
Casey Bodley
290da6b57c
Merge pull request #31878 from yuvalif/wip-yuval-tag-filter
rgw/pubsub: notification filtering by object tags

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-01-16 11:04:19 -05:00
Yuval Lifshitz
529d5c6362 rgw/pubsub: add ssl+sasl security to kafka
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-01-08 14:11:53 +02:00
Kai Wagner
b484a92356 doc/radosgw/config-ref: added a short paragraph to explain the garbage collection settings
Signed-off-by: Kai Wagner <kwagner@suse.com>
2020-01-07 15:47:26 +01:00
Matthew Oliver
9f40b38d3e doc: Add a RGW swift auth note
Swift accounts are not like normal accounts, they are more akin to a
bank account that multile people could share. Or in the case of a cloud
it is usually mapped to the tenant.

Radosgw deals with this with a user and subuser, which is great, but a
little confusing. So this patch adds a note to those used to the Swift
API to make it more clear.

Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-01-02 00:32:01 +00:00
Yuval Lifshitz
d19474f3d7 rgw/pubsub: notification filtering by object tags
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-12-15 14:20:41 +02:00
yuliyang
bdabd3d042 rgw: support specify user default placement and
placement_tags when create or modify user

fix https://tracker.ceph.com/issues/43164

Signed-off-by: yuliyang <yuliyang@cmss.chinamobile.com>
2019-12-06 09:47:03 +08:00
Casey Bodley
f6bda33bc9
Merge pull request #31926 from yuvalif/wip-yuval-fix-pubsub-record-format
rgw/pubsub: fix records/event json format to match documentation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-12-05 13:55:31 -05:00
Yuval Lifshitz
19832a0dae rgw/pubsub: support eventId in push mode
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-12-03 20:42:20 +02:00
Seena Fallah
c13cdfd57d Fixed bug on wrong name for user_id
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2019-12-03 19:37:04 +03:30
Sergio de Carvalho
f9d7b685c1 rgw: improvements to SSE-KMS with Vault
* Minor improvements to Vault documentation
* Add teuthology tests for Transit secrets engine
* Add unit tests for KV secrets engine, minor improvements to Transit
  secrets engine
* use string_view::npos instead of string::npos

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-11-28 09:19:39 +00:00
Andrea Baglioni
cebb48bffe rgw: Vault's Transit Secrets updates and docs
* Drop polymorfism for KMS class
  * Fix issue in kms-key selection
  * Update documentation for Vault section

 Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
 Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-11-28 09:19:39 +00:00
Andrea Baglioni
a41832eb42 rgw: Add support to Vault's Transit Secrets Engine in SSE-KMS
* refactor rgw_kms.cc to support extension to multiple secret engines.
* introduced support to Vault Namesapces
* added support for Vault Agent

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-11-28 09:19:39 +00:00
Casey Bodley
8aa638b0fa
Merge pull request #30960 from yuvalif/wip-yuval-add-kafka-notif-endpoint
rgw/pubsub: add kafka notification endpoint

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-11-21 14:42:23 -05:00
Sergio de Carvalho
2650ebe8af rgw: improvements to SSE-KMS with Vault
* add 'rgw crypt vault prefix' config setting to allow restricting
  secret space in Vault where RGW can retrieve keys from
* refuse Vault token file if permissions are too open
* improve concatenation of URL paths to avoid constructing an invalid
  URL (missing or double '/')
* doc: clarify SSE-KMS keys must be 256-bit long and base64 encoded,
  document Vault policies and tokens, plus other minor doc improvements
* qa: check SHA256 signature of Vault zip download
* qa: fix teuthology tests broken by previous PR which made SSE-KMS
  backend default to Barbican

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-11-12 13:51:25 +00:00
Yuval Lifshitz
354d775e04 rgw/pubsub: add kafka notification endpoint
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-11-05 19:25:03 +02:00
J. Eric Ivancich
1d61b25dbf
Merge pull request #30795 from ivancich/wip-dynamic-reshard-adjustments
rgw: adjust allowable bucket index shard counts for dynamic resharding

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-10-27 14:46:01 -04:00
Yuval Lifshitz
1531cf5d04
Merge pull request #30600 from yuvalif/boto3_extensions_examples
rgw/examples: adding examples for boto3 extensions to AWS S3
2019-10-15 09:57:27 +03:00
Yuval Lifshitz
bd017b2b12 rgw/examples: adding info on boto3 extensions to docs
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-10-14 20:38:13 +03:00
J. Eric Ivancich
c2a4a7159a rgw: adjust allowable bucket index shard counts for dynamic resharding
This commit adds a configurable option rgw_max_dynamic_shards that
provides a maximum bucket index shard count that dynamic resharding
can take a bucket to; the default is 1999. Note: this does not limit
the number of bucket index shards when set manually.

This commit also only allows prime shard counts when
rgw_max_dynamix_shards is no larger than 1999. Once it is larger, then
it allows any shard count, including non-prime values.

Finally, this commit adds unit tests to make sure the bucket index
shard count calculations work as expected.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-10-11 15:58:45 -04:00
Ali Maredia
d4872ce97a
Merge pull request #29783 from scarvalhojr/ssevault
rgw: add SSE-KMS with Vault using token auth
2019-10-06 22:33:36 -04:00
Sergio de Carvalho
7b216bab3d rgw: add SSE-KMS with Vault using token auth
Clarify supported secret engine in the Vault documentation.

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-10-04 15:55:51 +01:00
Kefu Chai
c2adfb62c6
Merge pull request #30583 from mika/mika/typos
doc: fix typos

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-10-04 16:06:20 +08:00
Sergio de Carvalho
9b425335de rgw: add SSE-KMS with Vault using token auth
Minor fix to config documentation.

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-10-02 17:28:06 +01:00
Sergio de Carvalho
1e5b58ad50 rgw: add SSE-KMS with Vault using token auth
Extend server-side encryption functionality in Rados Gateway to support
HashiCorp Vault as a Key Management System in addition to existing
support for OpenStack Barbican.

This is the first part of this change, supporting Vault's token-based
authentication only. Agent-based authentication as well as other
features such as Vault namespaces will be added in subsequent commits.

Note that Barbican remains the default backend for SSE-KMS
(rgw crypt s3 kms backend) to avoid breaking existing deployments.

Feature: https://tracker.ceph.com/issues/41062
Notes: https://pad.ceph.com/p/rgw_sse-kms

Implemented so far:
* Move existing SSE-KMS functions from rgw_crypt.cc to rgw_kms.cc
* Vault authentication with a token read from file
* Add new ceph.conf settings for Vault
* Document new ceph.conf settings
* Update main encryption documentation page
* Add documentation page for SSE-KMS using Vault

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-10-01 19:55:23 +01:00
Liu Lan
a2d5046340 doc/radosgw: fix typo
Signed-off-by: Liu Lan <liulan@umcloud.com>
2019-09-30 11:58:57 +08:00
Michael Prokop
d98b3e36a9 doc: fix typos
s/amount of times/number of times/
s/assosciated/associated/
s/availabe/available/
s/Commiting/Committing/
s/Containter/Container/
s/dependant/dependent/
s/developement/development/
s/filesytem/filesystem/
s/guarenteed/guaranteed/
s/hiearchy/hierarchy/
s/intance/instance/
s/Interger/Integer/
s/mutiple/multiple/
s/nubmer/number/
s/occured/occurred/
s/overriden/overridden/
s/reseted/reset/
s/sytem/system/
s/unkown/unknown/

Signed-off-by: Michael Prokop <mika@grml.org>
2019-09-26 09:17:07 +02:00
Yuval Lifshitz
751c4514e0
Merge pull request #28971 from yuvalif/wip-yuval-pubsub-api2
rgw/pubsub: add S3 compliant API to master zone
2019-09-20 01:35:02 +03:00
Casey Bodley
30b3ecbcc6
Merge pull request #29909 from cbodley/wip-40935
rgw: fix default storage class for get_compression_type

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-09-19 14:22:18 -04:00
Casey Bodley
06cc5f4d3a
Merge pull request #29839 from hualongfeng/master
doc/radosgw/admin:fix how to modify subuser info

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-09-17 13:49:20 -04:00
Casey Bodley
bf49aff3a1 rgw: disable compression/encryption on Appendable objects
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-13 14:49:46 -04:00
Yuval Lifshitz
543f8f7a27 rgw/pubsub: add notification filtering
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-09-11 19:24:55 +03:00
Yuval Lifshitz
923799fd96 rgw/pubsub: support deletion markers and multipart upload
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-09-10 18:54:05 +03:00
Yuval Lifshitz
c345566831 rgw/pubsub: push notifications documentation
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-09-10 18:54:05 +03:00
Patrick Donnelly
e7a7cf429e
doc: filesystem to file system
"Filesystem" is not a word (although fairly common in use).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-09-10 08:43:28 -07:00
root
848f1de0d0 rgw: fixed "unrecognized arg" error when using "radosgw-admin zone rm".
Signed-off-by: Hongang Chen <c744402859@gmail.com>

Author:    root <root@devel.localdomain>
Date:      Mon Sep 2 11:23:45 2019 +0800
2019-09-02 18:52:47 +08:00
Casey Bodley
f12108117d
Merge pull request #28885 from Kriechi/fix-bucket-policy-spelling
docs: rgw: fix bucket operation spelling: ListBucketMultipartUploads

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-30 09:06:38 -04:00
ypdai
3b8fe9aa96 doc: modify the wrong word "defails" to "details".
Signed-off-by: ypdai <self19900924@gmail.com>
(cherry picked from commit 8cefe3de78)
2019-08-27 15:22:21 +02:00
Feng Hualong
1a8741e37e doc/radosgw/admin:fix how to modify subuser info
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2019-08-23 16:41:38 +08:00
Casey Bodley
f0575a7144
Merge pull request #26787 from soumyakoduri/bucket_name_validation
[rgw]:Validate bucket names as per revised s3 spec

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-16 10:53:54 -04:00