Commit Graph

98703 Commits

Author SHA1 Message Date
Chang Liu
e4ab9bf31c rgw: add set_bucket_tags api
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2019-06-04 10:36:31 +08:00
Chang Liu
993d86fad8 rgw: uses 404 http code when there is NoSuchTagSetError
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2019-06-04 10:36:31 +08:00
Chang Liu
56189e2754 rgw: get bucket tagging
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2019-06-04 10:36:31 +08:00
Chang Liu
f7ea2bdbbe rgw: declares bucket tagging op classes
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2019-06-04 10:36:31 +08:00
Chang Liu
25bcddf060 rgw:new op type and error code about bucket tagging
Signed-off-by: Chang Liu <liuchang0812@gmail.com>
2019-06-04 10:36:31 +08:00
Patrick Donnelly
fd358e73c0
qa: fix kclient symlink to use latest rhel7
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-03 14:39:57 -07:00
Patrick Donnelly
39e23bb596
qa: fix latest distros and add helper symlinks
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-03 14:39:17 -07:00
Jeff Layton
3ade7c046c client: clean up error checking and return of _lookup_parent
ll_lookup_inode can end up getting back 0 from _lookup_parent,
without zeroing out the parent pointer, which ends up remaining
uninitialized.

Fix this by moving most of the sanity checks in _lookup_parent
into ll_lookup_inode, and only have it issue the call to the MDS.

This also allows us to do the checks in a more sane order.

Fixes: http://tracker.ceph.com/issues/40085
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-06-03 16:57:41 -04:00
Sage Weil
611c8949fa Merge PR #26547 into master
* refs/pull/26547/head:
	Added validation of zabbix_host to support hostnames, IPv4 and IPv6.
	mgr/zabbix: Documentation added.
	mgr/zabbix: Adds possibility to send data to multiple zabbix servers.

Reviewed-by: Wido den Hollander <wido@42on.com>
2019-06-03 15:45:42 -05:00
Sage Weil
097e8d210a Merge PR #27667 into master
* refs/pull/27667/head:
	src/test: fix creating two different objects for testing chunked object

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-03 15:44:11 -05:00
Sage Weil
898d29cb78 Merge PR #28200 into master
* refs/pull/28200/head:
	pybind/mgr: add slash to prometheus illegal metric name characters
	pybind/mgr: fix format for rbd-mirror prometheus metrics

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-03 15:43:33 -05:00
Samuel Just
f1b573c3d4
Merge pull request #28180 from athanatos/sjust/wip-crimson-peering
Integrate PeeringState into crimson, fix related bugs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-03 13:41:05 -07:00
Casey Bodley
c2cae94c68
Merge pull request #28068 from cbodley/wip-qa-rgw-s3a
qa/rgw: don't use ceph-ansible in s3a-hadoop suite

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
2019-06-03 14:29:09 -04:00
Patrick Donnelly
ffda69a4ab
Merge PR #28300 into master
* refs/pull/28300/head:
	doc: cephfs: add section on fsync error reporting to posix.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-03 11:28:47 -07:00
Jeff Layton
5c419b6475 client: drop Client::lookup_parent
Nothing calls this method.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-06-03 14:01:51 -04:00
Kefu Chai
61bd70535f cmake: link rbd_reply against rbd_replay_types
should make it `PUBLIC` because, for instance,

src/rbd_replay/actions.hpp uses
`rbd_replay::action::IoActionBase::dump()` and
`rbd_replay::action::ActionBase::dump()`

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-04 01:26:11 +08:00
Kefu Chai
68fb46d3ac cmake: link libosd against libos
apparently, libosd uses libos, but it does not use leveldb directly,
so s/leveldb/os/.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-04 01:26:11 +08:00
Kefu Chai
142be53986 cmake: link rgw tests against ${rgw_libs}
if `WITH_RADOSGW_AMQP_ENDPOINT` is enabled, we should always link rgw
applications against `RabbitMQ::RabbitMQ` or `amqp_mock`, so, to
simplify the linkage dependencies, we should just use `${rgw_libs}`
instead of using `rgw_a` directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-04 01:26:11 +08:00
Casey Bodley
8c8a706b6f qa/rgw: add dnsmasq back to s3a-hadoop
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
658e5932fb qa/rgw: rgw task can override --rgw-dns-name on the command line
the value for rgw_dns_name isn't known until a machine is assigned, so
it can't be set via 'override: conf:'. add a per-client config option
to the rgw task so it can add the endpoint's hostname and/or s3website
hostname on the radosgw command line

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
0fc2c8ecee qa/rgw: remove ceph-ansible from s3a-hadoop suite
Fixes: http://tracker.ceph.com/issues/39706

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
3ff5f0c240 qa/rgw: use default ports (80 or 443) unless overridden
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
921faebb72 qa/rgw: allow rgw client config to override port
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
dead71f5f9
Merge pull request #28146 from smanjara/wip-39551
rgw/multisite:RGWListBucketIndexesCR for data full sync pagination

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 12:53:09 -04:00
Casey Bodley
5a2661e48e
Merge pull request #26915 from dvanders/dvanders_doc_heap
doc: describe metadata_heap cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 12:51:13 -04:00
Kefu Chai
cf8c952a9b
Merge pull request #28359 from tchaikov/wip-split-ceph-dencoder
tools/ceph-dencoder: split types.h into smaller pieces

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-03 23:51:20 +08:00
Sage Weil
fd00136eb3 qa/suites/rados/multimon: skew clocks 2s (< paxos lease)
If the leader is the one with the accurate clock, it can still
form quorum, but if the leader has the skewed clock, all other mons appear
skewed from its perspective and no quorum is formed.  This leads to
intermittent failures, depending on the non-deterministic teuthology
deployment order and how the mon IPs sort.

Fix by reducing the skew.  This is enough skew to trigger a warning, but
not enough that it will break quorum.  This ensures that the parts of the
teuthology test that issue random mon commands won't fail (e.g., 'ceph osd
dump').

Fixes: http://tracker.ceph.com/issues/40112
Signed-off-by: Sage Weil <sage@redhat.com>
2019-06-03 10:49:02 -05:00
Dan van der Ster
e76604224c doc: describe metadata_heap cleanup
Fixes: http://tracker.ceph.com/issues/18174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2019-06-03 17:14:39 +02:00
Kefu Chai
8ceb11416d global/global_context: always add '\0' after strncpy()
see also f35fa1c58c

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 21:38:32 +08:00
Kefu Chai
11689c0e88 osd/ClassHandler: use std::variant for storing func
for better readablity, and less error-prone

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 21:21:52 +08:00
Casey Bodley
bdfc3bb92d
Merge pull request #28305 from rzarzynski/wip-bug-38827
qa/valgrind.supp: generalize the whiterule for aes-128-gcm to help rgw suite.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 09:09:24 -04:00
Casey Bodley
088d3ea26c
Merge pull request #25498 from cbodley/wip-27217
rgw: parse_copy_location defers url-decode

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 09:00:13 -04:00
Casey Bodley
62472c0ca3
Merge pull request #16659 from rzarzynski/wip-rgw-20797
rgw: TempURL should not allow PUTs with the X-Object-Manifest.

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 08:59:12 -04:00
Casey Bodley
8a0b2c7a67
Merge pull request #12709 from rzarzynski/wip-rgw-18364-dbgs
rgw: improve debugs on the path of RGWRados::cls_bucket_head.

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 08:57:37 -04:00
Casey Bodley
eb51579ba4
Merge pull request #12470 from rzarzynski/wip-rgw-18241
rgw: dump s3_code as the Code response element in RGWDeleteMultiObj_ObjStore_S3

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 08:57:13 -04:00
xie xingguo
a37106f0fa osd: maybe_remove_pg_upmaps -> clean_pg_upmaps
It should always be the preferred option to kill the unnecessary
or duplicated code, which is good for maintenance.
Also I've noticed there is already a clean_temps helper, so re-naming
maybe_remove_pg_upmaps to clean_pg_upmaps to at least keep pace with
that sounds to be a natural choice for me..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-03 17:13:50 +08:00
Kefu Chai
c63b33b61a cmake: link libkv against libheap_profiler
PriorityCache.cc uses `ceph_heap_*` functions, and libkv contains
`PriorityCache.cc`, so it should link against libheap_profiler.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 16:41:23 +08:00
Kefu Chai
7a9830323c osd/ClassHandler: no need to add struct before struct type
unlike C language, C++ is put struct names and other names in a single
namespace. so we don't need to add `strcut` before struct identifiers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 16:03:02 +08:00
Kefu Chai
69611dcc54 osd/ClassHandler: use ceph::mutex instead of Mutex
for better performance

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 16:03:02 +08:00
Kefu Chai
9eef9a8cdd osd/ClassHandler: remove unused function
rook does not use "embedded" ceph anymore. and we've dropped the
embedded build support, so remove this function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 16:03:01 +08:00
Kefu Chai
053906af9f osd/ClassHandler: s/string/std::string/ in header
and such-like, we cannot assume `using namespace std` or `using
std::string`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 16:03:01 +08:00
Kefu Chai
588f515a5c cmake: link libkv against common_prioritycache_obj
as both `kv/rocksdb_cache/BinnedLRUCache.cc` and
`kv/rocksdb_cache/ShardedCache.cc` uses `PriorityCache` which is in turn
offered by common_prioritycache_obj, we need to link libkv against the
latter.

libos uses `PriorityCache` as well, but since libos links against
libkv, so it's safe.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 15:12:44 +08:00
Kefu Chai
e7cf91ba16 cmake: remove kv_objs target
libkv is better than kv_objs, as it is able to bring more
properties with with it, for instance, the public include
directories, the public/private libraries it is linked against.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 14:52:52 +08:00
Kefu Chai
ed2ce0efad
Merge pull request #28339 from tchaikov/wip-cmake-asan
cmake/modules: replace ";" with " " in compile flags

Reviewed-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-06-03 12:28:32 +08:00
xie xingguo
02e5499b35 osd/OSDMap: maybe_remove_pg_upmaps - avoid do_crush twice
which is extremely time-consuming.
Half of the amount of time of calling maybe_remove_pg_upmaps
has been saved by applying this patch as a result..

Was: maybe_remove_pg_upmaps (~10000 pg_upmap_items) latency:104s
Now: maybe_remove_pg_upmaps (~10000 pg_upmap_items) latency:56s

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-03 11:17:24 +08:00
xie xingguo
d9ed406a61 osd/OSDMap: maybe_remove_pg_upmaps - s/pg_to_raw_up/pg_to_raw_upmap/
The upmap results are directly applied after calling
_pg_to_raw_osds, which means it basically has nothing to do
with the up/down status.
In other words, if a pg_upmap/pg_upmap_items remapped a pg
into some down osds and is now causing collided result,
we should still be able to detect and cancel that.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-03 11:06:29 +08:00
xie xingguo
c0ce22b8c8 test/osd: add performance test case for maybe_remove_pg_upmap
Tom Byrne reported that maybe_remove_pg_upmap might become
super inefficient for large clusters with balancer on.
To identify and resolve the problem, we need to add some good
measurements first.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-03 10:50:45 +08:00
Jan Fajerski
76204d32fb ceph.spec.in: install grafana dashboards world readable
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-06-02 13:31:27 +02:00
Kefu Chai
eaaec3d319 tools/ceph-dencoder: split types.h into smaller pieces
instead of putting all types in a single compilation unit, split them
into smaller groups, in hope to reduce the memory footprint of compiler
when compiling ceph_dencoder.cc.

sometimes, GCC just fails to compile this source file.

Fixes: http://tracker.ceph.com/issues/39595
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-01 23:12:16 +08:00
Patrick Donnelly
880468b4bf
Merge PR #28354 into master
* refs/pull/28354/head:
	doc: Add warning that the root directory cannot be fragmented

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-05-31 19:13:06 -07:00