Commit Graph

120352 Commits

Author SHA1 Message Date
Kefu Chai
cb7c0b41c5
Merge pull request #39827 from xxhdx1985126/wip-crimson-minor-bugs
crimson/osd: capture error_code by value in PG::handle_failed_op

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-03-04 18:33:25 +08:00
Kefu Chai
762b27af2c
Merge pull request #39812 from rzarzynski/wip-crimson-admin-config-help
crimson/admin: add support for 'config help'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-04 17:45:16 +08:00
Xuehan Xu
1df4ab9425 crimson/osd: capture error_code by value in PG::handle_failed_op
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-03-04 17:20:28 +08:00
Ernesto Puerta
461d17c29a
Merge pull request #39801 from rhcs-dashboard/49574-fix-pyjwt
mgr/dashboard: fix issues related with PyJWT versions >=2.0.0

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-03-04 09:53:42 +01:00
Nathan Cutler
18627bfeb0
Merge pull request #39755 from smithfarm/wip-49550
rpm: use PMDK system libraries on SUSE

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-03-04 09:29:47 +01:00
Patrick Donnelly
61ea12852f
Merge PR #39799 into master
* refs/pull/39799/head:
	doc: update affiliation for myoungwon

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-03 20:05:22 -08:00
myoungwon oh
d0c56ec3ef doc: update affiliation for myoungwon
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-03-04 09:33:41 +09:00
Nathan Cutler
30df650e64 rpm: use PMDK system libraries on SUSE
As of a49d1dbb32, when the rbd_rwl_cache and
rbd_ssd_cache bconds are enabled and WITH_SYSTEM_PMDK is disabled (as it is by
default), the RPM build attempts to

    git clone https://github.com/ceph/pmdk.git

but of course that won't work in the OBS, where the build workers have no
Internet connectivity.

Fortunately, the openSUSE/SLE versions targeted by Ceph master and pacific ship
the necessary PMDK libraries as RPM packages.

Fixes: a49d1dbb32
Fixes: https://tracker.ceph.com/issues/49550
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 20:46:20 +01:00
Nathan Cutler
ef0384dc50
Merge pull request #39765 from smithfarm/wip-ncpus-for-obs
rpm: set build parallelism from memory on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Galloway <dgallowa@redhat.com>
2021-03-03 20:44:20 +01:00
Patrick Donnelly
7bb6845e87
Merge PR #39682 into master
* refs/pull/39682/head:
	vstart_runner: remove duplicate methods from LocalCephManager

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-03 10:05:35 -08:00
Patrick Donnelly
3107a1f1f8
Merge PR #39780 into master
* refs/pull/39780/head:
	qa/vstart_runner: dont log "not Ceph bin" msg too often

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-03-03 10:04:23 -08:00
Patrick Donnelly
c9c64eb66b
Merge PR #39681 into master
* refs/pull/39681/head:
	vstart_runner: define path to ceph binary and use it

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-03-03 10:02:41 -08:00
Casey Bodley
38bb60b1e0
Merge pull request #39382 from gauravsitlani/master
rgw: added missing documentation on "pubsub" in rgw_enable_apis

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-03-03 12:13:16 -05:00
Nathan Cutler
4d47f8a928
Merge pull request #39762 from smithfarm/wip-support-leap-15.3
rpm/luarocks: simplify conditional and support Leap 15.3

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
2021-03-03 18:05:07 +01:00
Sebastian Wagner
86b25bc0be
Merge pull request #39789 from zdover23/wip-doc-cephadm-convert--bash-shells-2021-Mar-03
doc/cephadm: add prompts to adoption.rst

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-03-03 16:45:00 +01:00
Casey Bodley
fc03ae8f12
Merge pull request #39794 from cbodley/wip-qa-rgw-valgrind-supp-librados-delete
valgrind: suppress operator delete[] for tcmalloc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-03-03 10:38:36 -05:00
Sebastian Wagner
089938a1af
Merge pull request #39558 from sebastian-philipp/doc-rados-rm-toctree-cephadm
doc/rados: avoid including cephadm into the toctree

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
2021-03-03 16:17:51 +01:00
Nathan Cutler
19f492014b rpm: drop old SUSE-specific OOM fix
This fix was needed some years ago, but no longer.

Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:52:04 +01:00
Nathan Cutler
7efcc72483 rpm: limit build jobs by system memory on SUSE
43b441f9a3 removed a bunch of code which the SUSE
builds were relying on to avoid OOM. This commit brings back that code in
a much-streamlined form: the SUSE-specific %limit_build macro.

This also has the advantage of not breaking the build on older RPMs which only
know about %_smp_mflags, and not the newer %_smp_build_ncpus etc. macros.

Fixes: 43b441f9a3
Fixes: https://tracker.ceph.com/issues/49556
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-03-03 15:52:04 +01:00
Radoslaw Zarzynski
47a5447bdc crimson/admin: add support for 'config help'
This commit solves a problem unveiled in teuthology testing
where `ceph --admin-daemon ... config help` were constantly
returning `EINVAL`:

```
2021-03-03T02:43:01.566 DEBUG:tasks.admin_socket:Testing config help with config {'branch': 'master'}
2021-03-03T02:43:01.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config help
2021-03-03T02:43:01.711 INFO:teuthology.orchestra.run.smithi083.stderr:no valid command found; 3 closest matches:
2021-03-03T02:43:01.712 INFO:teuthology.orchestra.run.smithi083.stderr:config get <var>
2021-03-03T02:43:01.712 INFO:teuthology.orchestra.run.smithi083.stderr:config set <var> <val>...
2021-03-03T02:43:01.712 INFO:teuthology.orchestra.run.smithi083.stderr:config show
2021-03-03T02:43:01.713 INFO:teuthology.orchestra.run.smithi083.stderr:admin_socket: invalid command
2021-03-03T02:43:01.714 DEBUG:teuthology.orchestra.run:got remote process result: 22
2021-03-03T02:43:01.714 INFO:tasks.admin_socket:ceph cli returned an error, command not registered yet?
2021-03-03T02:43:01.715 INFO:tasks.admin_socket:sleeping and retrying ...
```

The root cause is no support for `config help` in crimson.
As shown below this command hasn't been registered at all:

```
DEBUG 2021-03-03 02:42:48,242 [shard 0] osd - start: asok socket path=/var/run/ceph/ceph-osd.0.asok
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): mark_unfound_lost)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): query)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): perf dump_seastar)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): dump_pgstate_history)
INFO  2021-03-03 02:42:48,242 [shard 0] osd - register_command(): flush_pg_stats)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): send_beacon)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): status)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): injectargs)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): config show)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): config set)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): config get)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): get_command_descriptions)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): help)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): git_version)
INFO  2021-03-03 02:42:48,243 [shard 0] osd - register_command(): version)
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-03-03 14:47:29 +00:00
Alfonso Martínez
4b96bb51e8 mgr/dashboard: fix issues related with PyJWT versions >=2.0.0
Fixes: https://tracker.ceph.com/issues/49574
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2021-03-03 15:36:06 +01:00
Daniel Gryniewicz
bfde964cf0
Merge pull request #39399 from dang/wip-dang-zipper-10
RGW - Zipper 10: The Great Zippening
2021-03-03 08:56:04 -05:00
Sage Weil
5290ed3777 Merge PR #38909 into master
* refs/pull/38909/head:
	PendingReleaseNotes: document option osd_fast_shutdown_notify_mon
	osd: add osd_fast_shutdown_notify_mon option (default false)

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-03-03 08:38:58 -05:00
Sage Weil
67e842d519 Merge PR #39654 into master
* refs/pull/39654/head:
	common/options: drop ms_async_max_op_threads
	msg/async: drop Stack::num_workers
	msg/async: s/num_workers/workers.size()/
	msg/async: use range-based loop in NetworkStack
	msg/async: do not pass worker id to Stack::spawn_worker()
	async/Stack: pass Worker* to NetworkStack::add_thread()
	async/rdma: do not reference worker id in RDMAStack::spawn_worker()
	async/dpdk: do not use worker id when creating worker
	async/PosixStack: do not reference worker id in ctor
	async/rdma: initialize worker in RDMAStack::create_worker()
	async/rdma: move RDMAStack::create_worker() to .cc

Reviewed-by: luo runbing <luo.runbing@zte.com.cn>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2021-03-03 08:38:21 -05:00
Sage Weil
bdff94a09d Merge PR #39665 into master
* refs/pull/39665/head:
	qa/rados/dashbord use "random" objectore

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-03-03 08:35:51 -05:00
Sage Weil
86b824ec60 Merge PR #39689 into master
* refs/pull/39689/head:
	src/global/signal_handler.h: fix preprocessor logic for alpine

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-03 08:33:28 -05:00
Sage Weil
cb98159b12 Merge PR #39750 into master
* refs/pull/39750/head:
	mgr: create client messenger using ms_public_type

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-03 08:32:41 -05:00
Sebastian Wagner
d4292afebf
Merge pull request #39770 from liewegas/bug-46745
cephadm: add docker.service dependency in systemd units

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-03-03 10:51:47 +01:00
Mykola Golub
7c214b24ce
Merge pull request #39795 from dillaman/wip-migration-import-docs
doc/rbd: clarify QCOW2 migration support

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-03-03 10:45:35 +02:00
Kefu Chai
163f3b81b9
Merge pull request #39555 from tchaikov/wip-drop-pre-nautilus
*: drop backward compatibility with pre-nautilus versions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-03-03 16:39:46 +08:00
Kefu Chai
de049079fb
Merge pull request #39798 from tchaikov/wip-crimson-monc-copy-sent
crimson/mon: keep a copy of sent MMonCommand messages

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-03-03 12:31:39 +08:00
Kefu Chai
a977061602 crimson/mon: keep a copy of sent MMonCommand messages
as per Yingxin Cheng,

> The send process can be asynchronous (there is a conn.out_q, or if the
> underlying socket lives in a different core in the m:n model to be
> ordered there). If user really wants to reuse a message, they must be
> careful not to modify it because it may result in modifing the pending
> messages.
>
> I think the best way is to copy the message if user want to resend it,
> and keep the ceph_assert(!msg->get_seq()). It may looks good to reuse a
> message under lossy policy, but the correctness is now up to user not to
> modify it inplace.

see also https://github.com/ceph/ceph/pull/39786

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-03 11:41:38 +08:00
Jason Dillaman
68fda590cf doc/rbd: clarify QCOW2 migration support
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-03-02 17:29:41 -05:00
Casey Bodley
bf087c3f84 valgrind: suppress operator delete[] for tcmalloc
suppresses a mismatched operator delete[] from the librados client's
destructor:

<error>
  <unique>0xeef7eb</unique>
  <tid>1</tid>
  <kind>MismatchedFree</kind>
  <what>Mismatched free() / delete / delete []</what>
  <stack>
    <frame>
      <ip>0x4C32EA0</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator delete[](void*, unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>660</line>
    </frame>
    <frame>
      <ip>0x5FBEB86</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5FBEF2C</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5F7F7C5</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
      <fn>librados::v14_2_0::Rados::shutdown()</fn>
    </frame>

Fixes: https://tracker.ceph.com/issues/49501

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-03-02 15:38:41 -05:00
Sage Weil
d4a6a97f5b Merge PR #39730 into master
* refs/pull/39730/head:
	cephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-03-02 13:56:01 -05:00
Zac Dover
7eb65fe2cf doc/cephadm: add prompts to adoption.rst
This PR formats the bash prompts. It also formats the
bash output so that it appears in the correct (easily
copy-and-pasteable) format. This PR will be followed by
a grammar-improving PR, but this PR is just a
formatting PR.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-03-03 04:22:51 +10:00
Ilya Dryomov
9611104a19
Merge pull request #39781 from idryomov/wip-krbd-xfstests-fixes
qa/suites/krbd: address recent issues caused by newer kernels

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-03-02 18:43:20 +01:00
Sage Weil
0b91432170 Merge PR #39723 into master
* refs/pull/39723/head:
	mon/ConfigMonitor: make config changes via KVMonitor's pending set

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-03-02 12:08:52 -05:00
Sage Weil
3ea514c552 cephadm: fix escaping/quoting of stderr-prefix arg for ceph daemons
On some versions of podman doing

   --default-log-stderr-prefix="debug "

seems to work okay, but on others it does not, producing lines like

   /usr/bin/ceph-mon: "debug "2021-02-26T15:30:36.266+0000 7f3d8a955700  4 rocksdb:                    Options.db_write_buffer_size: 0

Quote things properly for the unit.run bash script.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-02 10:35:41 -06:00
Sage Weil
65724b1f97 Merge PR #39739 into master
* refs/pull/39739/head:
	cephadm: set CEPH_USE_RANDOM_NONCE if using --init
	msg/Messenger: use random nonce if CEPH_USE_RANDOM_NONCE or pid == 1
	Revert "Merge PR #39482 into master"

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-03-02 11:32:12 -05:00
Daniel Gryniewicz
c107f970ca RGW - Fix mock User class in Lua test
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2021-03-02 10:51:12 -05:00
Sage Weil
0b5bc8fc2e cephadm: add docker.service dependency in systemd units
This ensures that (1) we start after the docker daemon is running, and
(2) a restart of docker will also restart the ceph services.

Fixes: https://tracker.ceph.com/issues/46745
Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-02 09:50:49 -05:00
Patrick Donnelly
150868a489
Merge PR #39679 into master
* refs/pull/39679/head:
	Revert "mon/MDSMonitor: add missing frozen checks"

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2021-03-02 06:40:50 -08:00
Sebastian Wagner
c28524d301
Merge pull request #38995 from mgfritch/cephadm-check-host-errors
cephadm: remove redundant `ERROR` during check-host

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-03-02 15:33:04 +01:00
Sebastian Wagner
06258eafd1
Merge pull request #39722 from p-se/wip-pse-cephadm-SUSE-alertmanager
cephadm: `cephadm ls` broken for SUSE downstream alertmanager container

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-03-02 15:30:17 +01:00
Sebastian Wagner
5dd8c08c53
Merge pull request #38915 from Daniel-Pivonka/clientsoktostop
mgr/cephadm: add ok-to-stop functions for ceph client services

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-03-02 15:28:58 +01:00
Sebastian Wagner
0cbec89deb
Merge pull request #39352 from sebastian-philipp/orch-remove-promise
mgr/orch: Make orchestrator interface synchronous

Reviewed-by: Varsha Rao <varao@redhat.com>
2021-03-02 15:25:02 +01:00
Sebastian Wagner
a171b32a21
Merge pull request #39612 from jmolmo/fix_rook_orch_ls
mgr/orchestrator: Fix ceph orch ls in Rook

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-03-02 14:47:25 +01:00
Kefu Chai
504aaaa9e5
Merge pull request #39784 from rzarzynski/wip-crimson-monc-close-active-conn-on-reset
crimson/monc: close() active_con before destructing it on resets.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-03-02 20:48:35 +08:00
Kefu Chai
11cd800fe6
Merge pull request #39783 from tchaikov/wip-crimson-clang
crimson/os: fix clang build and cleanups.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-03-02 20:45:24 +08:00