Commit Graph

107306 Commits

Author SHA1 Message Date
Abhishek L
e7c7e513ae
Merge pull request #30033 from theanalyst/rgw/public-buckets
rgw: add PublicAccessBlock set of APIs on buckets

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-02-04 19:01:05 +01:00
Casey Bodley
3326ded944
Merge pull request #33049 from yuvalif/wip-yuval-fix-43768
qa/rgw/pubsub: fix tests to sync from master

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-02-04 11:44:46 -05:00
Abhishek Lekshmanan
df384ea95f doc: add Pending Release Notes entry on public access config
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-04 17:28:46 +01:00
Abhishek L
e2020c7834
Merge pull request #32119 from joke-lee/rgw-sts-crash-duration-invalid
rgw: fix rgw crash when duration is invalid in sts request
2020-02-04 17:24:12 +01:00
Abhishek L
72763b2deb
Merge pull request #31987 from linuxbox2/rgw-putacls-no-body
rgw: s3: don't require a body in S3 put-object-acl
2020-02-04 17:22:35 +01:00
Abhishek L
7c1a690560
Merge pull request #30684 from theanalyst/rgw/qa/rgw-admin-user-stats
qa: radosgw_admin: validate a simple user stats output

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2020-02-04 17:21:25 +01:00
Sage Weil
19928c13ec Merge PR #33048 into master
* refs/pull/33048/head:
	cephadm: fix error handling in `command_check_host()`
	cephadm: increase default retry_max value

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-02-04 07:08:22 -06:00
Sage Weil
b9a38a0371 Merge PR #32995 into master
* refs/pull/32995/head:
	cephadm: add group 'disk' to privileged container
	cephadm: adopt: disable ceph-volume unit on host
	cephadm: refactor privileged arg handling
	cephadm: chown block symlink targets
	cephadm-adoption-corpus: add stud-mon.tgz
	cephadm: adopt: rename leveldb *.ldb -> *.sst
	cephadm: adopt: by default, pull first
	cephadm: adopt: chown data content

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-02-04 07:07:01 -06:00
Sage Weil
3a3fd5679c Merge PR #33020 into master
* refs/pull/33020/head:
	osdc/Objecter: inline pool full check
	osdc/Objecter: remove duplicated pause check code
	osdc/Objecter: only pause if respects_full()
	osdc/Objecter: move respects_full() to op_target_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-03 21:28:40 -06:00
Sage Weil
64ef3242bf Merge PR #32831 into master
* refs/pull/32831/head:
	common, include: drop the copy{_in} from bufferlist entirely.
	os/bluestore: switch copy_in() users to bufferlist::iterator.
	osdc: switch users of bufferlist::copy{_in} to iterators.
	osd: switch users of bufferlist::copy{_in} to iterators.
	rgw: switch copy{_in} users to bufferlist::iterator.
	ec: switch users of bufferlist::copy{_in} to iterators.
	cls/queue: switch users of bufferlist::copy{_in} to iterators.
	client: switch users of bufferlist::copy{_in} to iterators.
	*: switch trivial users of bufferlist::copy{_in} to iterators.
	test/bl: switch copy{_in} users to bufferlist::iterator.
	common, include: kill the bl::last_p member.
	common: encode for std::list<T> doesn't use bl::copy_in() anymore.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-03 21:28:19 -06:00
Josh Durgin
79040c2ea3
Merge pull request #32531 from zdover23/wip-doc-landing-page-update
doc: Added the crisp getting started guide to index.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-02-03 15:50:25 -08:00
Sage Weil
b5e5c753f4 cephadm: add group 'disk' to privileged container
This lets the osd read block devs that are group rw disk even after they
drop root privs.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 16:49:20 -06:00
Sage Weil
e17ffa6c11 Merge PR #32977 into master
* refs/pull/32977/head:
	qa/workunits/cephadm/test_cephadm.sh: add missing monitoring tests
	cephadm: simplify Monitoring.components structure
	cephadm: add proper tox type for monitoring components

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-02-03 16:28:04 -06:00
Sage Weil
0f61bbcdcb Merge PR #33012 into master
* refs/pull/33012/head:
	mgr/cephadm: prefix daemon ids with hostname
	cephadm: bootstrap: name mgr with $hostname.$random

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-02-03 16:27:51 -06:00
Sage Weil
1a529bf230 cephadm: adopt: disable ceph-volume unit on host
This might be a simple or lvm unit.  Disable it so that the host doesn't
try to start this OSD after a reboot.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 16:24:34 -06:00
Sage Weil
2884223817 cephadm: refactor privileged arg handling
Pass a bool if we want a privileged container instead of explicitly
passing --privileged.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 16:24:34 -06:00
Guillaume Abrioux
f67610c73c cephadm: fix error handling in command_check_host()
`find_program()` raises `ValueError` when the executable hasn't been
found. It means we need to catch `ValueError` exception in
`command_check_host()` and raise `Error` instead of `RuntimeError` since
only `Error` is caught at the end.

Typical failure:

```
INFO:cephadm:/usr/bin/ceph:stderr Error ENOENT: New host mon1 failed check: ['INFO:cephadm:podman|docker (/bin/podman) is present', 'INFO:cephadm:systemctl is present', 'Traceback (most recent call last):', '  File "<stdin>", line 2820, in <module>', '  File "<stdin>", line 2434, in command_check_host', '  File "<stdin>", line 796, in find_program', 'ValueError: lvcreate not found']
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-02-03 18:43:39 +01:00
Abhishek Lekshmanan
949aa83ae5 rgw: move public access conf to perm_state_base
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
f88a48c6c0 test: rgw_iam_policy update tests with new Get Actions
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
bc8b8abda3 rgw: iam_policy: add all the actions to actpairs map
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
2831d4876b rgw: public access: drop unused function
Also cleanup the comment to mention why we've deviated from the spec here

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
97c57f05ee rgw: rename ACL & policy IsPublic to is_public
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
b7ddec14ac rgw: PublicAccessConfiguration -> PublicAccessBlockConfiguration
also drop iam namespace

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
2c98fa754d rgw_op: get_public_access_from_attr indent fixes
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
6fd6897bca rgw_rest_s3: use formatter->dump_bool
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
451b18e08f public_access: reuse formatter->dump_bool
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
e97b7d6426 rgw: Fix IgnorePublicACLs for bucket ACLs
Currently Bucket ACLs with IgnorePublicACLs were broken this should fix that

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:34 +01:00
Abhishek Lekshmanan
ff5cadbe8f drop redundant bucket policy status in rgw_common
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
065ecd3f43 rgw: implement IgnorePublicACLs
This allows for ignoring bucket/object acls that are configured to be public

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
ab745eae21 rgw: move PublicAccessConfiguration to req_state
This allows for evaluation of more complex use cases where IgnorePublicACLs and
the like are set which need to be evaluated for GET/HEAD requests as well

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
0c594c8b86 rgw: op: Get Policy Status checks if there is a policy first
before evaluating its public nature

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
c4bc1e8e75 rgw_op: reuse function to get public access conf
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
9fc16df7ed rgw: honor PublicAccessBlockConfiguration for put object
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
11f92eab4c rgw: block public access for Policies
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
f692d042d1 rgw: enforce BlockPublicPolicy on put bucket policy
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
065b5358f7 rgw: public_access: rename the getters to reflect what the conf actually does
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
f2e1840aab rgw_op: use ldpp_dout version wherever logs are required
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
c654e709a0 rgw: move IsPublic to RGWAccessControlPolicy class
This helps reusing when evaluating for PutACLs and Put Policy

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
2ba4a0f6f5 rgw: implement get/put/delete public access block for buckets
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
824b26c7ed add RGWPublicAccess for configuring public access settings for a resource
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:31 +01:00
Abhishek Lekshmanan
36bb77d8b7 rgw: both princ and nonprinc will not coexist in a single statement
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:31 +01:00
Abhishek Lekshmanan
ff972d6956 rgw: initial implementation of a public policy tester
doesn't cover all the cases involving a nonprinc user yet

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:31 +01:00
Abhishek Lekshmanan
e0b4562c61 rgw: s3: implement GetBucketPolicyStatus API
This API returns whether the Bucket Policies/ACLs are public. There are a couple
of caveats:
- AWS currently returns PolicyNotFound error in case a bucket policy doesn't
exist, though a non existant bucket policy would mean the default ACLs apply
where the bucket is private, so error return here seems like an error
- the API spec mentions TRUE and FALSE as the response IsPublic element value,
however in practice both boto/aws clients and AWS S3 return/expect a lowercase
response.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Conflicts:
	src/rgw/rgw_rest_s3.h
merge conflict after zipper rework, dropped a spurious newline in rgw_rest_s3.h
after get_obj_op decl.
src/rgw/rgw_common.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h:
merge conflict after bucket replication merge, trivial conflicts
2020-02-03 17:53:30 +01:00
Abhishek Lekshmanan
57baa84044 rgw: acl: drop unused function & make get_group_perm const
Drop the unused RGWAccessControlPolicy::get_group_perm, make the ACL
get_group_perm as a const member function

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:30 +01:00
Guillaume Abrioux
fed96416dc cephadm: increase default retry_max value
When playing with cephadm, at multiple times, I've reached the max
number of attempt in `is_available()`

Increasing the `retry_max` helps to avoid failure like following:

```
INFO:cephadm:mgr not available, waiting (1/5)...
INFO:cephadm:mgr not available, waiting (2/5)...
INFO:cephadm:mgr not available, waiting (3/5)...
INFO:cephadm:mgr not available, waiting (4/5)...
INFO:cephadm:mgr not available, waiting (5/5)...
ERROR: mgr not available after 5 tries

```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-02-03 17:25:06 +01:00
Yuval Lifshitz
c5af4b3505
Merge pull request #32155 from yuvalif/wip-yuval-multidelete-notifications
rgw/pubsub: send notifications from multi-delete op
2020-02-03 18:02:06 +02:00
Sage Weil
1ad536ba40 mgr/cephadm: prefix daemon ids with hostname
This is friendlier to a human operator since they can immediately see
where an instance is located, as with the legacy scheme, while still
keeping the unique random suffix.  Use a . to separate so that we can
set per-host options.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 08:23:37 -06:00
Sebastian Wagner
97fffb13fe
Merge pull request #32859 from liewegas/wip-cephadm-services
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-03 14:27:38 +01:00
Abhishek L
b6aa44cad5
Merge pull request #33030 from smithfarm/wip-luminous-12.2.13-release-notes
doc: release notes for Luminous v12.2.13

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 13:32:50 +01:00
Abhishek L
f2f868c107
Merge pull request #33031 from smithfarm/wip-nautilus-14.2.7-release-notes
doc: release notes for Nautilus 14.2.7

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 13:32:05 +01:00