Commit Graph

109753 Commits

Author SHA1 Message Date
Haomai Wang
beedd6d804 kv/RocksDBStore: using PinnableSlice to avoid extra value copy
Signed-off-by: Haomai Wang <haomai@xsky.com>
2020-03-24 12:59:29 +08:00
Kefu Chai
3a638841f9
Merge pull request #34103 from yanghonggang/perf-msgr-client
test/msgr/perf_msgr_client.cc: fix misleading total op num

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-24 12:46:55 +08:00
Yan, Zheng
ca0978a892
Merge pull request #33753 from ukernel/wip-44448
mds: fix 'if there is lock cache on dir' check
2020-03-24 10:07:36 +08:00
Sage Weil
01015f8ca3 debian: add ceph-grafana-dashboards package
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 20:13:19 -05:00
Sage Weil
f1b516d1bf ceph.spec: put prometheus alerts in vendor-neutral location
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 20:13:18 -05:00
Sage Weil
b1463998bd mgr/cephadm: include prom alerts, if present in the container
The prometheus config already looks in/etc/prometheus/alerting/*, so drop
this file into position.

Note that the file is currently only present in SUSE builds, so that needs
to be fixed still.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 20:13:08 -05:00
Sage Weil
895b856aee Merge remote-tracking branch 'gh/octopus' 2020-03-23 20:09:41 -05:00
Sage Weil
53037187c4 Merge PR #34100 into octopus
* refs/pull/34100/head:
	doc/cephadm: notes on status/stability
	doc/cephadm: some troubleshooting tips

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-23 20:01:12 -05:00
Sage Weil
e85f2410e7 Merge PR #34115 into octopus
* refs/pull/34115/head:
	doc/releases/octopus: drop stray line
	doc/releases/octopus: note about repository locations
	doc/releases: include octopus in index
	doc/install/get-packages: update package install instructions
	doc/releases/octopus: final notes

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2020-03-23 20:00:58 -05:00
Sage Weil
04e0b9c2f8 Merge PR #34126 into master
* refs/pull/34126/head:
	qa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-23 13:55:16 -05:00
Sage Weil
78acb68c01 doc/releases/octopus: drop stray line
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 13:54:12 -05:00
Sage Weil
7e71b19b3e doc/releases/octopus: note about repository locations
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 13:53:32 -05:00
Sage Weil
821d8908c4 doc/releases: include octopus in index
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 13:53:32 -05:00
Jenkins Build Slave User
dc6a0b5c3c 15.2.0 2020-03-23 17:47:45 +00:00
Sage Weil
41bc82f284 doc/install/get-packages: update package install instructions
- include cephadm method
- update to include new version URLs

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 12:40:19 -05:00
Sage Weil
659684bfb1 Merge PR #34119 into octopus
* refs/pull/34119/head:
	doc: update octopus release notes for rgw

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2020-03-23 12:38:24 -05:00
Sage Weil
5dabfc0b73 Merge PR #34122 into octopus
* refs/pull/34122/head:
	cephadm: default to v15 tag for octopus

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-23 12:38:12 -05:00
Nathan Cutler
7a4855849c script/ceph-backport.sh: update for octopus
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-23 16:49:36 +01:00
Neha
cfebec1b12 qa/*/osd-backfill-recovery-log.sh: flush_pg_stats before checking log length
It is possible for the pg dump to not be the latest when we check for newprimary
in _common_test(). This is because mgr_stats_period is 5 seconds, and we may not
have fetched the latest stats just yet. This causes the test to look at the same
stats before and after wait_for_clean.

Fixes: https://tracker.ceph.com/issues/43807 (2)
Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-03-23 15:37:12 +00:00
Sage Weil
24acf825d2 cephadm: default to v15 tag for octopus
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 10:34:16 -05:00
Casey Bodley
37c3406eb8 doc: update octopus release notes for rgw
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 984f3ff7e0)
2020-03-23 10:00:31 -05:00
Sage Weil
9daa451795 Merge PR #34028 into master
* refs/pull/34028/head:
	doc: update octopus release notes for rgw

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2020-03-23 09:59:21 -05:00
Sebastian Wagner
c840dd2204 mgr/cephadm: Add example to run when debugging ssh failures
```
$ ceph orch host add foobar
Error ENOENT: Failed to connect to foobar (foobar).  Check that the host is reachable and accepts connections using the cephadm SSH key
you may want to run:
> ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) rook@foobar
$  ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) rook@foobar
ssh: Could not resolve hostname foobar: Temporary failure in name resolution
```

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-03-23 14:48:29 +01:00
Sage Weil
de20c7bc61 Merge PR #34105 into master
* refs/pull/34105/head:
	Merge PR #34042 into octopus
	Merge PR #33959 into octopus
	Merge PR #34067 into octopus
	mgr/DaemonServer: add explicit check that acting matches for merge
	Merge pull request #34040 from dillaman/wip-44396-partial-fix
	Merge PR #34098 into octopus
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	Merge PR #34082 into octopus
	Merge PR #34068 into octopus
	cephadm: relabel /etc/ganesha mount
	Merge PR #34046 into octopus
	Merge PR #34092 into octopus
	Merge pull request #33719 from ukernel/wip-44416
	rbd-mirror: leader watcher should not cancel get locker if locker is invalid
	rbd-mirror: snapshot sync request needs to check for interruption
	librbd: request exclusive lock when moving to trash
	rbd-mirror: basic integration with sync throttling
	rbd-mirror: don't prematurely finish snapshot replay loop
	rbd-mirror: pass InstanceWatcher to snapshot Replayer
	doc/releases/octopus.rst: add note about ec recovery below min_size
	mgr/cephadm: configure rgw_frontends for rgw service
	cephadm: switch grafana image to the ceph repo
	Merge PR #34034 into octopus
	qa/suites/rados/cephadm/upgrade: update starting version
	Merge PR #33540 into octopus
	Merge PR #34023 into octopus
	Merge PR #34044 into octopus
	Merge PR #34030 into octopus
	doc/orchestrator: update rgw creation
	mgr/cephadm: clean up client.crash.* container_image settings after upgrade
	cephadm: make add-repo --release and --version independent
	cephadm: env over last used
	mgr/orch: accept port and ssl flags to 'apply rgw'
	mgr/orch: 'ceph upgrade ...' -> 'ceph orch upgrade ...'
	cephadm: fall back to default for infer_image
	cephadm: remove outdated check
	cephadm: consolidate default image logic
	remove ceph_test_rados_watch_notify
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	cephadm: only infer image for shell, run, inspect-image, pull, ceph-volume
	mgr/test_orchestrator: fix service filtering when using dummy data
	mgr/dashboard: fix adding/removing host errors
	mgr/rook: fix 'orch ps' for osds
	qa: fix all the fsx.sh-invoking yaml files to install dependencies
	mds: pass proper MutationImpl::LockOp to Locker::wrlock_start()

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-03-23 08:24:06 -05:00
Sage Weil
8d0e6f99f3 doc/releases/octopus: final notes
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 08:20:57 -05:00
Sage Weil
cf73c06184 doc/cephadm: notes on status/stability
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-23 07:55:16 -05:00
Lenz Grimmer
4a4b737cbe
Merge pull request #33741 from ricardoasmarques/iscsi-password-msg
mgr/dashboard: Improve iSCSI CHAP message

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-23 13:47:41 +01:00
Lenz Grimmer
b819847556
Merge pull request #34063 from s0nea/wip-dashboard-crush-rule-suite
mgr/dashboard: add crush rule test suite

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-23 13:11:50 +01:00
Lenz Grimmer
e45d96d23e
Merge pull request #34113 from s0nea/wip-dashboard-orch-docu-link
mgr/dashboard: correct Orchestrator documentation link

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-23 13:07:17 +01:00
Tatjana Dehler
6e91edb287 mgr/dashboard: correct Orchestrator documentation link
Fixes: https://tracker.ceph.com/issues/44708
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-03-23 11:45:53 +01:00
Kefu Chai
5132d9851d
Merge pull request #34104 from tchaikov/crimson-admin-close
crimson/admin: do not reset connected_sock before closing

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-23 17:10:40 +08:00
Kefu Chai
5305cfbc43
Merge pull request #33909 from cyx1231st/wip-seastar-msgr-fix-reset
crimson: misc fixes for writes to multiple-osd cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-23 17:05:57 +08:00
Yingxin Cheng
d78cbb3ebd crimson/net: add critical info logs to track and debug racing
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-23 13:00:23 +08:00
Yingxin Cheng
e202069f09 crimson/net: fix incorrect SocketConnection::print()
The informaction about SocketConnection::side and
SocketConnection::ephemeral_port are not up-to-date in the log, because
they are not moved with Socket during connection replacement. They are
actually socket-level information.

Also take the chance to reorder Socket members.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-23 13:00:23 +08:00
Yingxin Cheng
373e16499e crimson/osd: make send_heartbeat() atomic
The item in Heartbeat:peers could be removed/re-added during the
asynchronous operation.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-23 13:00:15 +08:00
Kefu Chai
d6ead8b9ad
Merge pull request #32171 from rosinL/wip-ec-isla-aarch64
erasure-code: enable isa-l EC for aarch64 platform

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-23 12:46:39 +08:00
Jacek Suchenia
ad4d790a35 mgr: ceph_pg_* metrics contains last value instead of sum all of them
During evaluation of pool stats metrics contains last reported value instead of sum

Fixes: https://tracker.ceph.com/issues/44590
Signed-off-by: Jacek Suchenia <jacek.suchenia@gmail.com>
2020-03-23 11:02:47 +08:00
Kefu Chai
1ce152f597 crimson/admin: do not reset connected_sock before closing
* no need to discard_result(). as `output_stream::close()` returns an
  empty future<> already
* free the connected socket after the background task finishes, because:

we should not free the connected socket before the promise referencing it is fulfilled.

otherwise we have error messages from ASan, like

==287182==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000019aa0 at pc 0x55e2ae2de882 bp 0x7fff7e2bf080 sp 0x7fff7e2bf078
READ of size 8 at 0x611000019aa0 thread T0
    #0 0x55e2ae2de881 in seastar::reactor_backend_aio::await_events(int, __sigset_t const*) ../src/seastar/src/core/reactor_backend.cc:396
    #1 0x55e2ae2dfb59 in seastar::reactor_backend_aio::reap_kernel_completions() ../src/seastar/src/core/reactor_backend.cc:428
    #2 0x55e2adbea397 in seastar::reactor::reap_kernel_completions_pollfn::poll() (/var/ssd/ceph/build/bin/crimson-osd+0x155e9397)
    #3 0x55e2adaec6d0 in seastar::reactor::poll_once() ../src/seastar/src/core/reactor.cc:2789
    #4 0x55e2adae7cf7 in operator() ../src/seastar/src/core/reactor.cc:2687
    #5 0x55e2adb7c595 in __invoke_impl<bool, seastar::reactor::run()::<lambda()>&> /usr/include/c++/10/bits/invoke.h:60
    #6 0x55e2adb699b0 in __invoke_r<bool, seastar::reactor::run()::<lambda()>&> /usr/include/c++/10/bits/invoke.h:113
    #7 0x55e2adb50222 in _M_invoke /usr/include/c++/10/bits/std_function.h:291
    #8 0x55e2adc2ba00 in std::function<bool ()>::operator()() const /usr/include/c++/10/bits/std_function.h:622
    #9 0x55e2adaea491 in seastar::reactor::run() ../src/seastar/src/core/reactor.cc:2713
    #10 0x55e2ad98f1c7 in seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) ../src/seastar/src/core/app-template.cc:199
    #11 0x55e2a9e57538 in main ../src/crimson/osd/main.cc:148
    #12 0x7fae7f20de0a in __libc_start_main ../csu/libc-start.c:308
    #13 0x55e2a9d431e9 in _start (/var/ssd/ceph/build/bin/crimson-osd+0x117421e9)

0x611000019aa0 is located 96 bytes inside of 240-byte region [0x611000019a40,0x611000019b30)
freed by thread T0 here:
    #0 0x7fae80a4e487 in operator delete(void*, unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.6+0xac487)
    #1 0x55e2ae302a0a in seastar::aio_pollable_fd_state::~aio_pollable_fd_state() ../src/seastar/src/core/reactor_backend.cc:458
    #2 0x55e2ae2e1059 in seastar::reactor_backend_aio::forget(seastar::pollable_fd_state&) ../src/seastar/src/core/reactor_backend.cc:524
    #3 0x55e2adab9b9a in seastar::pollable_fd_state::forget() ../src/seastar/src/core/reactor.cc:1396
    #4 0x55e2adab9d05 in seastar::intrusive_ptr_release(seastar::pollable_fd_state*) ../src/seastar/src/core/reactor.cc:1401
    #5 0x55e2ace1b72b in boost::intrusive_ptr<seastar::pollable_fd_state>::~intrusive_ptr() /opt/ceph/include/boost/smart_ptr/intrusive_ptr.hpp:98
    #6 0x55e2ace115a5 in seastar::pollable_fd::~pollable_fd() ../src/seastar/include/seastar/core/internal/pollable_fd.hh:109
    #7 0x55e2ae0ed35c in seastar::net::posix_server_socket_impl::~posix_server_socket_impl() ../src/seastar/include/seastar/net/posix-stack.hh:161
    #8 0x55e2ae0ed3cf in seastar::net::posix_server_socket_impl::~posix_server_socket_impl() ../src/seastar/include/seastar/net/posix-stack.hh:161
    #9 0x55e2ae0ed943 in std::default_delete<seastar::net::api_v2::server_socket_impl>::operator()(seastar::net::api_v2::server_socket_impl*) const /usr/include/c++/10/bits/unique_ptr.h:81
    #10 0x55e2ae0db357 in std::unique_ptr<seastar::net::api_v2::server_socket_impl, std::default_delete<seastar::net::api_v2::server_socket_impl> >::~unique_ptr()
	/usr/include/c++/10/bits/unique_ptr.h:357    #11 0x55e2ae1438b7 in seastar::api_v2::server_socket::~server_socket() ../src/seastar/src/net/stack.cc:195
    #12 0x55e2aa1c7656 in std::_Optional_payload_base<seastar::api_v2::server_socket>::_M_destroy() /usr/include/c++/10/optional:260
    #13 0x55e2aa16c84b in std::_Optional_payload_base<seastar::api_v2::server_socket>::_M_reset() /usr/include/c++/10/optional:280
    #14 0x55e2ac24b2b7 in std::_Optional_base_impl<seastar::api_v2::server_socket, std::_Optional_base<seastar::api_v2::server_socket, false, false> >::_M_reset() /usr/include/c++/10/optional:432
    #15 0x55e2ac23f37b in std::optional<seastar::api_v2::server_socket>::reset() /usr/include/c++/10/optional:975
    #16 0x55e2ac21a2e7 in crimson::admin::AdminSocket::stop() ../src/crimson/admin/admin_socket.cc:265
    #17 0x55e2aa099825 in operator() ../src/crimson/osd/osd.cc:450
    #18 0x55e2aa0d4e3e in apply ../src/seastar/include/seastar/core/apply.hh:36

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-23 10:44:20 +08:00
Sage Weil
243cbd6224 Merge PR #34042 into octopus
* refs/pull/34042/head:
	mgr/rook: list rgw services
	mgr/rook: tolerate timestamps that are None
	mgr/orch: add 'subcluster' property to RGWSpec
	mgr/rook: do not create radosgw pools
	mgr/rook: refactor apply/add for rgw
	mgr/cephadm: configure rgw_frontends for rgw service
	mgr/orch: accept port and ssl flags to 'apply rgw'
	python-common/ceph/deployment/service_spec: add ssl to RGWSpec
	mgr/rook: fix 'orch ps' for osds

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2020-03-22 18:32:11 -05:00
Kefu Chai
86de901d89
cephadm: init config and keyring with None
and we should not assume that both `config` and `keying` are specified
when calling this method. because, for instance, `create_daemon_dirs()`
does handle the case where `config` and/or `keyring` is not specified.

this is a follow-up fix of 245d6a5cec

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-03-22 12:17:41 -06:00
Sage Weil
2740349122 Merge PR #33959 into octopus
* refs/pull/33959/head:
	qa: fix all the fsx.sh-invoking yaml files to install dependencies

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-22 10:56:31 -05:00
Sage Weil
3b28477f0d Merge PR #34067 into octopus
* refs/pull/34067/head:
	mgr/DaemonServer: add explicit check that acting matches for merge

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2020-03-22 10:55:54 -05:00
Adam C. Emerson
1e8864092d mon: Build ceph-mon without using namespace declarations in headers
This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-22 00:28:46 -04:00
Adam C. Emerson
0b1935bc3f crypto: Build without using namespace declarations in headers
This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-22 00:28:39 -04:00
Adam C. Emerson
674a6f6309 compressor: Build without using namespace declarations in headers
This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-22 00:28:28 -04:00
Adam C. Emerson
2f56e9b6d8 global: Build without using namespace declarations in headers
This is part of a series of commits to clean up using namespace at top
level in headers.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-03-22 00:28:00 -04:00
Kefu Chai
ec362f4499
Merge pull request #34071 from badone/wip-docker-test-helper-use-podman-by-default
tests: Use podman if available

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-22 10:41:23 +08:00
Kefu Chai
961834c3b1
Merge pull request #34048 from tchaikov/wip-test-docker-fc31
tests: update Dockerfile to support fc-31

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2020-03-22 10:40:16 +08:00
Brad Hubbard
a1e8f61cb7 tests: Use podman if available
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2020-03-22 09:07:50 +10:00
Sage Weil
1700d18158 mgr/DaemonServer: add explicit check that acting matches for merge
Add an explicit check that the PG acting for the source and target
match before merging.

Fixes: https://tracker.ceph.com/issues/44684
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-21 14:17:30 -05:00