Commit Graph

98628 Commits

Author SHA1 Message Date
Patrick Donnelly
39e23bb596
qa: fix latest distros and add helper symlinks
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-03 14:39:17 -07:00
Jeff Layton
3ade7c046c client: clean up error checking and return of _lookup_parent
ll_lookup_inode can end up getting back 0 from _lookup_parent,
without zeroing out the parent pointer, which ends up remaining
uninitialized.

Fix this by moving most of the sanity checks in _lookup_parent
into ll_lookup_inode, and only have it issue the call to the MDS.

This also allows us to do the checks in a more sane order.

Fixes: http://tracker.ceph.com/issues/40085
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-06-03 16:57:41 -04:00
Sage Weil
611c8949fa Merge PR #26547 into master
* refs/pull/26547/head:
	Added validation of zabbix_host to support hostnames, IPv4 and IPv6.
	mgr/zabbix: Documentation added.
	mgr/zabbix: Adds possibility to send data to multiple zabbix servers.

Reviewed-by: Wido den Hollander <wido@42on.com>
2019-06-03 15:45:42 -05:00
Sage Weil
097e8d210a Merge PR #27667 into master
* refs/pull/27667/head:
	src/test: fix creating two different objects for testing chunked object

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-03 15:44:11 -05:00
Sage Weil
898d29cb78 Merge PR #28200 into master
* refs/pull/28200/head:
	pybind/mgr: add slash to prometheus illegal metric name characters
	pybind/mgr: fix format for rbd-mirror prometheus metrics

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-03 15:43:33 -05:00
Samuel Just
f1b573c3d4
Merge pull request #28180 from athanatos/sjust/wip-crimson-peering
Integrate PeeringState into crimson, fix related bugs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-03 13:41:05 -07:00
Casey Bodley
c2cae94c68
Merge pull request #28068 from cbodley/wip-qa-rgw-s3a
qa/rgw: don't use ceph-ansible in s3a-hadoop suite

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
2019-06-03 14:29:09 -04:00
Patrick Donnelly
ffda69a4ab
Merge PR #28300 into master
* refs/pull/28300/head:
	doc: cephfs: add section on fsync error reporting to posix.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-03 11:28:47 -07:00
Jeff Layton
5c419b6475 client: drop Client::lookup_parent
Nothing calls this method.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-06-03 14:01:51 -04:00
Kefu Chai
61bd70535f cmake: link rbd_reply against rbd_replay_types
should make it `PUBLIC` because, for instance,

src/rbd_replay/actions.hpp uses
`rbd_replay::action::IoActionBase::dump()` and
`rbd_replay::action::ActionBase::dump()`

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-04 01:26:11 +08:00
Kefu Chai
68fb46d3ac cmake: link libosd against libos
apparently, libosd uses libos, but it does not use leveldb directly,
so s/leveldb/os/.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-04 01:26:11 +08:00
Kefu Chai
142be53986 cmake: link rgw tests against ${rgw_libs}
if `WITH_RADOSGW_AMQP_ENDPOINT` is enabled, we should always link rgw
applications against `RabbitMQ::RabbitMQ` or `amqp_mock`, so, to
simplify the linkage dependencies, we should just use `${rgw_libs}`
instead of using `rgw_a` directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-04 01:26:11 +08:00
Casey Bodley
8c8a706b6f qa/rgw: add dnsmasq back to s3a-hadoop
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
658e5932fb qa/rgw: rgw task can override --rgw-dns-name on the command line
the value for rgw_dns_name isn't known until a machine is assigned, so
it can't be set via 'override: conf:'. add a per-client config option
to the rgw task so it can add the endpoint's hostname and/or s3website
hostname on the radosgw command line

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
0fc2c8ecee qa/rgw: remove ceph-ansible from s3a-hadoop suite
Fixes: http://tracker.ceph.com/issues/39706

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
3ff5f0c240 qa/rgw: use default ports (80 or 443) unless overridden
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
921faebb72 qa/rgw: allow rgw client config to override port
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 13:07:41 -04:00
Casey Bodley
dead71f5f9
Merge pull request #28146 from smanjara/wip-39551
rgw/multisite:RGWListBucketIndexesCR for data full sync pagination

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 12:53:09 -04:00
Casey Bodley
5a2661e48e
Merge pull request #26915 from dvanders/dvanders_doc_heap
doc: describe metadata_heap cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 12:51:13 -04:00
Kefu Chai
cf8c952a9b
Merge pull request #28359 from tchaikov/wip-split-ceph-dencoder
tools/ceph-dencoder: split types.h into smaller pieces

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-03 23:51:20 +08:00
Dan van der Ster
e76604224c doc: describe metadata_heap cleanup
Fixes: http://tracker.ceph.com/issues/18174
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2019-06-03 17:14:39 +02:00
Casey Bodley
bdfc3bb92d
Merge pull request #28305 from rzarzynski/wip-bug-38827
qa/valgrind.supp: generalize the whiterule for aes-128-gcm to help rgw suite.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-03 09:09:24 -04:00
Casey Bodley
088d3ea26c
Merge pull request #25498 from cbodley/wip-27217
rgw: parse_copy_location defers url-decode

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 09:00:13 -04:00
Casey Bodley
62472c0ca3
Merge pull request #16659 from rzarzynski/wip-rgw-20797
rgw: TempURL should not allow PUTs with the X-Object-Manifest.

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 08:59:12 -04:00
Casey Bodley
8a0b2c7a67
Merge pull request #12709 from rzarzynski/wip-rgw-18364-dbgs
rgw: improve debugs on the path of RGWRados::cls_bucket_head.

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 08:57:37 -04:00
Casey Bodley
eb51579ba4
Merge pull request #12470 from rzarzynski/wip-rgw-18241
rgw: dump s3_code as the Code response element in RGWDeleteMultiObj_ObjStore_S3

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-03 08:57:13 -04:00
Kefu Chai
c63b33b61a cmake: link libkv against libheap_profiler
PriorityCache.cc uses `ceph_heap_*` functions, and libkv contains
`PriorityCache.cc`, so it should link against libheap_profiler.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 16:41:23 +08:00
Kefu Chai
588f515a5c cmake: link libkv against common_prioritycache_obj
as both `kv/rocksdb_cache/BinnedLRUCache.cc` and
`kv/rocksdb_cache/ShardedCache.cc` uses `PriorityCache` which is in turn
offered by common_prioritycache_obj, we need to link libkv against the
latter.

libos uses `PriorityCache` as well, but since libos links against
libkv, so it's safe.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 15:12:44 +08:00
Kefu Chai
e7cf91ba16 cmake: remove kv_objs target
libkv is better than kv_objs, as it is able to bring more
properties with with it, for instance, the public include
directories, the public/private libraries it is linked against.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-03 14:52:52 +08:00
Kefu Chai
ed2ce0efad
Merge pull request #28339 from tchaikov/wip-cmake-asan
cmake/modules: replace ";" with " " in compile flags

Reviewed-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-06-03 12:28:32 +08:00
Jan Fajerski
76204d32fb ceph.spec.in: install grafana dashboards world readable
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-06-02 13:31:27 +02:00
Kefu Chai
eaaec3d319 tools/ceph-dencoder: split types.h into smaller pieces
instead of putting all types in a single compilation unit, split them
into smaller groups, in hope to reduce the memory footprint of compiler
when compiling ceph_dencoder.cc.

sometimes, GCC just fails to compile this source file.

Fixes: http://tracker.ceph.com/issues/39595
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-01 23:12:16 +08:00
Patrick Donnelly
880468b4bf
Merge PR #28354 into master
* refs/pull/28354/head:
	doc: Add warning that the root directory cannot be fragmented

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-05-31 19:13:06 -07:00
Samuel Just
6d66072cc4 osd/: clarify rebuilt_missing_with_deletes and set in init
Updates PeeringState::init to set may_include_deletes_in_missing when
possible to avoid needlessly rebuilding the missing set on the next map.
The previous behavior was also subtly wrong as until the the next map
the osd would be possibly adding deletes to the missing set without the
flag having been set in the missing set on disk

This patch also renames the flag to reflect the fact that it simply
denotes wheter the may_include_deletes_in_missing omap entry needs to be
written out.  This may happen during rebuild_missing_set_with_deletes,
but it'll also happen more commonly during init now.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:08:48 -07:00
Samuel Just
8c86678ba7 crimson: hook PeeringState into pg
This patch:
1) Adds partially implemented stubs for PeeringState::PeeringListener
2) Revamps the message handling to batch the peering events from
   a single message.
3) Replaces the message handling with hand-offs to PeeringState
4) Adds support for MOSDPGCreate2 enabling creation of pgs.
5) Introduces ShardServices for exposing osd machinery to the pgs.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:45 -07:00
Samuel Just
909f4e260e crimson/osd/main: add --debug option
Really, what we need to do is examine the ceph conf and set
the right logger levels based on configured log levels, but
this'll suffice for now.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:45 -07:00
Samuel Just
1b22a8313a crimson/osd/pg_backend: clear exists in remove
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:45 -07:00
Samuel Just
8864de8164 cyan_store: map -ENOENT to 0 for _remove
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:45 -07:00
Samuel Just
c5fec5f7bd crimson/net/ProtocolV1: close connection when replaced
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:45 -07:00
Samuel Just
c489c53ce7 Revert "cmake: define BOOST_SP_DISABLE_THREADS for crimson"
For whatever reason, this patch appears to cause the reactor/peering
to hang much of the time.  Reverting until we have time to debug it.

This reverts commit a09a4f342c.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:45 -07:00
Samuel Just
395a2f6828 crimson/common/assert.cc: flush cout before abort
Seastar logger() instances don't flush the channel, so recent messages
won't be present otherwise.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
0c254023cf dout: let <= 20 correspond to debug for crimson
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
5c76f63078 vstart: add --redirect-output option to redirect output for --nodaemon
Useful for crimson which doesn't currently support writing directly
to a log file.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
e799b38aed crimson/osd: use configured ceph-osd uuid in mkfs
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
d0dbb36e2e crimson/net/Connection: add peer_is_* helpers
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
212593dac5 crimson/net/SocketMessenger: wait for accepter thread to die on shutdown
Otherwise, it ends up trying to operator<< *this after it's been
deleted.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
d07592eab7 crimson/mon/MonClient: handle mon client connection race
The first to establish a connection and close the others needs
to ensure that the others don't mess with broken state.  To that
end, leave a cleared pending_conns map while the conns are being
closed.  Also, take care to clean up outstanding promises and
fallout from conns disappearing.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
c839c2d42b crimson/os/cyan_store: run t.get_on_* contexts
PeeringState makes liberal use of those callbacks, run them
upon completion of do_transaction.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
382a649fa0 crimson/os/cyan_store: add some omap machinery
Adds transaction methods for PeeringState.

Mainly omap_set, but also a stub for collection hint and the other omap
update methods.  Also adds omap_get and debugging to dump the
transaction upon encountering a problem.

Also fixes up std and ceph::bufferlist namespace qualifications.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00
Samuel Just
201dd35656 os/cyan_collection: add pragma once
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-31 14:05:44 -07:00