Commit Graph

107397 Commits

Author SHA1 Message Date
Abhishek Lekshmanan
9fc16df7ed rgw: honor PublicAccessBlockConfiguration for put object
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
11f92eab4c rgw: block public access for Policies
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
f692d042d1 rgw: enforce BlockPublicPolicy on put bucket policy
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:33 +01:00
Abhishek Lekshmanan
065b5358f7 rgw: public_access: rename the getters to reflect what the conf actually does
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
f2e1840aab rgw_op: use ldpp_dout version wherever logs are required
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
c654e709a0 rgw: move IsPublic to RGWAccessControlPolicy class
This helps reusing when evaluating for PutACLs and Put Policy

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
2ba4a0f6f5 rgw: implement get/put/delete public access block for buckets
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:32 +01:00
Abhishek Lekshmanan
824b26c7ed add RGWPublicAccess for configuring public access settings for a resource
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:31 +01:00
Abhishek Lekshmanan
36bb77d8b7 rgw: both princ and nonprinc will not coexist in a single statement
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:31 +01:00
Abhishek Lekshmanan
ff972d6956 rgw: initial implementation of a public policy tester
doesn't cover all the cases involving a nonprinc user yet

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:31 +01:00
Abhishek Lekshmanan
e0b4562c61 rgw: s3: implement GetBucketPolicyStatus API
This API returns whether the Bucket Policies/ACLs are public. There are a couple
of caveats:
- AWS currently returns PolicyNotFound error in case a bucket policy doesn't
exist, though a non existant bucket policy would mean the default ACLs apply
where the bucket is private, so error return here seems like an error
- the API spec mentions TRUE and FALSE as the response IsPublic element value,
however in practice both boto/aws clients and AWS S3 return/expect a lowercase
response.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Conflicts:
	src/rgw/rgw_rest_s3.h
merge conflict after zipper rework, dropped a spurious newline in rgw_rest_s3.h
after get_obj_op decl.
src/rgw/rgw_common.h
src/rgw/rgw_rest_s3.cc
src/rgw/rgw_rest_s3.h:
merge conflict after bucket replication merge, trivial conflicts
2020-02-03 17:53:30 +01:00
Abhishek Lekshmanan
57baa84044 rgw: acl: drop unused function & make get_group_perm const
Drop the unused RGWAccessControlPolicy::get_group_perm, make the ACL
get_group_perm as a const member function

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 17:53:30 +01:00
Sage Weil
ebca44ccaa qa/suites/rados: improve valgrind leak check
Verify we can detect leak in the osd, mon, and mgr independently.  Also
include a negative test (no leaks).

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 10:25:39 -06:00
Sage Weil
e807669b77 common/ceph_context: add an asok command to deliberately leak memory
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 10:25:39 -06:00
Guillaume Abrioux
fed96416dc cephadm: increase default retry_max value
When playing with cephadm, at multiple times, I've reached the max
number of attempt in `is_available()`

Increasing the `retry_max` helps to avoid failure like following:

```
INFO:cephadm:mgr not available, waiting (1/5)...
INFO:cephadm:mgr not available, waiting (2/5)...
INFO:cephadm:mgr not available, waiting (3/5)...
INFO:cephadm:mgr not available, waiting (4/5)...
INFO:cephadm:mgr not available, waiting (5/5)...
ERROR: mgr not available after 5 tries

```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-02-03 17:25:06 +01:00
Casey Bodley
d486b5bc45 qa/rgw: test with non-default rgw-obj-stripe-size
each job will select one of the striping strategies at random

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-02-03 11:24:52 -05:00
Yuval Lifshitz
c5af4b3505
Merge pull request #32155 from yuvalif/wip-yuval-multidelete-notifications
rgw/pubsub: send notifications from multi-delete op
2020-02-03 18:02:06 +02:00
Sage Weil
1ad536ba40 mgr/cephadm: prefix daemon ids with hostname
This is friendlier to a human operator since they can immediately see
where an instance is located, as with the legacy scheme, while still
keeping the unique random suffix.  Use a . to separate so that we can
set per-host options.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-03 08:23:37 -06:00
xie xingguo
d08c1ff199 osd/OSD: prevent down osds from immediately rejoining the culster
In 114c65fc I posted a work-around to fix a heartbeat brain-split case
but it really looks to me now like I am missing some other cases where
an immediate attempt to rejoin is bad, like when the network actually
isn't working properly rather than being predictably manipulated by an
admin.
This patch instead slows the unconditionally rejoining attempt down,
especially make sure that we don't try to immediately rejoin the culster
when an osd has just been marked down by mon.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-02-03 21:35:05 +08:00
Sebastian Wagner
97fffb13fe
Merge pull request #32859 from liewegas/wip-cephadm-services
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-02-03 14:27:38 +01:00
Abhishek L
b6aa44cad5
Merge pull request #33030 from smithfarm/wip-luminous-12.2.13-release-notes
doc: release notes for Luminous v12.2.13

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 13:32:50 +01:00
Abhishek L
f2f868c107
Merge pull request #33031 from smithfarm/wip-nautilus-14.2.7-release-notes
doc: release notes for Nautilus 14.2.7

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2020-02-03 13:32:05 +01:00
xie xingguo
59db3379ae osd/OSD: trim osd_markdown_log in tick() thread
so we don't have to do it in multiple places. Note that
we can't do it in the tick_without_osd_lock thread instead
because we we can not access it safely without the protection
of osd_lock.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-02-03 20:09:37 +08:00
Kefu Chai
80487f4a60
Merge pull request #33038 from tchaikov/wip-librados-reinterpret-cast
librados: prefer reinterpret_cast over c-style cast

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2020-02-03 19:58:13 +08:00
Kefu Chai
0ac42dc7fd
Merge pull request #32933 from DimStar77/obs-mini
ceph.spec.in: Use pkgconfig() style BuildRequires for udev/libudev-devel

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-02-03 19:45:07 +08:00
Lenz Grimmer
4c9c4229ed
Merge pull request #32553 from tspmelo/wip-ts-strict
mgr/dashboard: Enable compiler options used by Angular --strict flag

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-02-03 11:33:33 +00:00
Kefu Chai
5fcf0641cd librados: prefer reinterpret_cast over c-style cast
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-03 18:16:09 +08:00
Kefu Chai
7c1f7807d7
Merge pull request #33016 from tchaikov/wip-do-cmake
do_cmake.sh: bail out if something goes wrong

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-02-03 17:15:16 +08:00
Michael Fritch
4535216267
qa/workunits/cephadm/test_cephadm.sh: add missing monitoring tests
add tests for node-exporter, prometheus, and grafana

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-02-02 21:08:02 -07:00
Michael Fritch
c37504e81c
cephadm: simplify Monitoring.components structure
by removing the nested image/metadata dict

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-02-02 21:08:02 -07:00
Zac Dover
97df0b5274 More commits to the landing page project.
Signed-off-by: Zac Dover <zac.dover@gmail.com>
2020-02-03 11:21:34 +10:00
Yuval Lifshitz
ab26135f51 rgw/pubsub: fix tests to sync from master
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-02-02 21:03:25 +02:00
Sage Weil
128095c219 cephadm: bootstrap: name mgr with $hostname.$random
Signed-off-by: Sage Weil <sage@redhat.com>
2020-02-02 09:43:13 -06:00
Radoslaw Zarzynski
c28a802df8 include: switch mempool.h to ceph::atomic.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-02-02 16:21:34 +01:00
Patrick Donnelly
29d850fb7e
Merge PR #32570 into master
* refs/pull/32570/head:
	cephfs-shell: Add tests for setxattr, getxattr and listxattr
	cephfs-shell: Add listxattr command
	cephfs-shell: Add getxattr command
	cephfs-shell: Add setxattr command
	doc: Update about extended attributes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-02 06:56:50 -08:00
Patrick Donnelly
3a6f107331
Merge PR #31633 into master
* refs/pull/31633/head:
	cephfs-shell: Instead of assert use stat for tests in rmdir
	cephfs-shell: Add function for common rmdir test code
	cephfs-shell: Add rmdir test for non empty directory
	cephfs-shell: Add rmdir -p test for non empty directory
	cephfs-shell: Add rmdir -p test for non existing dir
	cephfs-shell: Add rmdir -p test to delete all dirs in given path
	cephfs-shell: Add rmdir -p test for root directory with empty directories
	cephfs-shell: Add rmdir test for valid file
	cephfs-shell: Add rmdir test for invalid directory
	cephfs-shell: Add rmdir test for valid directory
	cephfs-shell: Fix rmdir '-p' issues

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-02-02 06:52:23 -08:00
Kefu Chai
55953ba0da src/: s/advance/operator+=/
for better readablity

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-02 11:37:28 +08:00
Sage Weil
b66f5df514 Merge PR #32986 into master
* refs/pull/32986/head:
	qa/tasks/ceph_manager: fix movement of cot exports with cephadm

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-02-01 10:47:56 -06:00
Kefu Chai
5ab82abbbb include/buffer: add operator+=() for iterators
less type than p.advance(). please note, this does not imply that
list::iterator is a RandomAccessIterator. as the parameter type of
p.adavance() is not `ptrdiff_t`, which is a signed type.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-02-01 23:18:10 +08:00
Nathan Cutler
08615ccba2 doc: release notes for Nautilus 14.2.7
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-02-01 12:45:03 +01:00
Nathan Cutler
b18a1ca4e1 doc: release notes for Luminous v12.2.13
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-02-01 12:28:20 +01:00
Ramana Raja
b7768eca2a
Merge pull request #32030 from vshankar/wip-mgr-volumes-clone
mgr/volumes: clone from snapshot
2020-02-01 13:17:51 +05:30
Sage Weil
4813b4132b Merge PR #33014 into master
* refs/pull/33014/head:
	mgr/dashboard: pass a list of drive_group to create_osds

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-01-31 23:28:44 -06:00
Sage Weil
25e7d92026 Merge PR #32973 into master
* refs/pull/32973/head:
	mgr/insights: make 'insights prune-health-history 0' zap current state

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-01-31 21:26:46 -06:00
Sage Weil
d8a7c73a48 Merge PR #32987 into master
* refs/pull/32987/head:
	qa/tasks/ceph_manager: make fix_pgp_num behave when no pool is found

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-31 17:40:23 -06:00
Sage Weil
a5d848d206 Merge PR #32989 into master
* refs/pull/32989/head:
	qa/tasks/ceph_manager: add --log-early to raw_cluster_cmd

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-31 17:40:13 -06:00
Sage Weil
b02c3ff18b Merge PR #32830 into master
* refs/pull/32830/head:
	global: ensure CEPH_ARGS is decoded before early arg processing

Reviewed-by: Sage Weil <sage@redhat.com>
2020-01-31 17:31:44 -06:00
Sage Weil
798213b512 Merge PR #32806 into master
* refs/pull/32806/head:
	common/bl: fix accessibility of bptr's _off and _len fields.
	common/bl: drop get_raw() from the public buffer::ptr interface.
	common: drop sharing of buffer::raw outside bufferlist.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-31 17:31:31 -06:00
Sage Weil
42768600d4 qa/tasks/ceph_manager: fix movement of cot exports with cephadm
I think this will finally work...

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-31 17:26:10 -06:00
Yaarit Hatuka
be1257f324 mgr/telemetry: anonymizing smartctl report itself
smartctl JSON output contains the device's serial number in two
different keys ('serial_number' & 'output'). Serial is now obfuscated in
both.

Fixes: https://tracker.ceph.com/issues/43939
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2020-01-31 16:44:30 -05:00