Commit Graph

120365 Commits

Author SHA1 Message Date
Sebastian Wagner
f2c6db822f
Merge pull request #39614 from sebastian-philipp/cephadm-again-test-ubuntu_2004_podman
qa/cephadm: Test again podman on ubuntu 20.04

Reviewed-by: Sage Weil <sage@redhat.com>
2021-02-26 11:02:24 +01:00
Kefu Chai
12a8d274b2 mgr/rbd_support: fix type annotation of mirror_info
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 17:33:44 +08:00
Kefu Chai
c27e0253d6
Merge pull request #38604 from cybozu/doc-fix-the-meaning-of-min_size
doc: fix the meaning of default_min_size of pools

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-26 17:25:45 +08:00
Kefu Chai
7d737aa665
Merge pull request #39684 from xxhdx1985126/wip-alienstore-parallelism-improvement
crimson/os/alienstore: improve alienstore's write parallelism

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-26 14:06:44 +08:00
Kefu Chai
809957ab92
Merge pull request #39602 from tchaikov/wip-mgr-flux-ann
mgr: add type annotations to BaseMgrModule and influx

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-02-26 11:37:28 +08:00
Kefu Chai
ae5f38a924
Merge pull request #39371 from tchaikov/wip-mgr-rbd-support-ann
mgr/rbd_support: add type annotation and cleanups

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-02-26 11:36:15 +08:00
Xuehan Xu
14d046fd66 crimson/os/alienstore: improve alienstore's write parallelism
replace the grand per store tp_mutex with a finer grained per-collection
lock for better concurrency

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-02-26 11:32:55 +08:00
Kefu Chai
ee7184639d common/options: drop ms_async_max_op_threads
the last user of this option was DPDKStack::DPDKStack(). which used it
to initialize its func vector. but this option is not used anymore.

so let's drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
789c1bd62a msg/async: drop Stack::num_workers
no need to store it as a member variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
52044dcbc7 msg/async: s/num_workers/workers.size()/
prepare for dropping num_workers, the latter can be dropped in a later
commit.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
7319340044 msg/async: use range-based loop in NetworkStack
there is no need to reference num_workers here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
2ecaa16673 msg/async: do not pass worker id to Stack::spawn_worker()
this parameter is not used anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
2d6988feab async/Stack: pass Worker* to NetworkStack::add_thread()
so we don't need to pass worker id to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
ea7686fe99 async/rdma: do not reference worker id in RDMAStack::spawn_worker()
it's not needed when creating a new worker.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
db8d9f6641 async/dpdk: do not use worker id when creating worker
so we can drop the `i` parameter in a later change.

also restructure DPDKStack::spawn_worker() to capture variables by value
instead of by reference, we cannot assume that the variables allocated on
stack are still available when the function is scheduled on another
stack and core.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
892bfa097e async/PosixStack: do not reference worker id in ctor
as it's not needed and confusing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
103ec86cfa async/rdma: initialize worker in RDMAStack::create_worker()
in ff65c800b3, we moved create_worker()
call out of the constructor to avoid calling virtual functions in
constructor. but this created a regression where RDMAStack's constructor
tries to reference its workers not yet created.

in this change, the workers are initialized right after they are
created.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:07 +08:00
Kefu Chai
a5ddf1840b async/rdma: move RDMAStack::create_worker() to .cc
to hide the implementation in .cc file, also speed up the compilation.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-26 11:28:06 +08:00
Samuel Just
702ddbdc3c crimson/os/futurized_store: fix return comment on omap_get_values
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-02-25 17:59:37 -08:00
Samuel Just
3ec3721b87 crimson/os/seastore/omap_manager: rework to be null safe and adjust interfaces
Adjusts a few things in omap_manager/btree:
- fix to be null safe, simplify/rename iterator interface
- update interface methods to match futurized_store.h
- misc whitespace fixes

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-02-25 17:59:34 -08:00
Sage Weil
82aeb07f9d mon/PGMap: include the per-pool usage breakdown in pg dump
We started accounting for this ages ago, but we don't expose it anywhere.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-25 19:58:27 -06:00
Neha Ojha
731c20fe49
Merge pull request #39666 from zdover23/wip-doc-dev-t8y-rerunning-tests-rewrite-2021-Feb-24
doc/dev: rewrite t8y "re-running tests"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-02-25 17:14:59 -08:00
Neha Ojha
85fcd117e9
Merge pull request #39694 from zdover23/wip-doc-dev-t8y-debugging-triaging-rewrite-2021-Feb-25
doc/dev: rewrite t8y "triaging" section

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-02-25 17:09:56 -08:00
Neha Ojha
deb7d811e0
Merge pull request #39695 from zdover23/wip-doc-dev-t8y-debugging-adding-sections-2021-Feb-26
doc/dev: add t8y debug section names

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-02-25 17:08:30 -08:00
Neha Ojha
e2d53858ef
Merge pull request #39697 from zdover23/wip-doc-dev-t8y-debug-reporting-the-issue-rewrite-2021-Feb-26
doc/dev: rewrite t8y "reporting the issue"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-02-25 17:07:49 -08:00
Jason Dillaman
9e88f36e5c
Merge pull request #39707 from petrutlucian94/overlapped
rbd: fix mingw 5.0.3 compatibility issue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-02-25 19:08:31 -05:00
Patrick Donnelly
6093b3a581
qa: run fs:verify on all distros
It's believed this is no longer a problem now that we use tcmalloc.

Fixes: https://tracker.ceph.com/issues/49391
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-02-25 13:27:24 -08:00
Patrick Donnelly
4526c74569
qa: use tcmalloc with valgrind in fs:valgrind
Follow-up: dc64ccf063
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-02-25 13:18:37 -08:00
Lucian Petrut
7eae54cb4c rbd: fix mingw 5.0.3 compatibility issue
GetOverlappedResultEx isn't avaialable when using mingw 5.0.3
(default on Ubuntu Bionic).

Since we don't really need the extra parameters, we'll switch
to GetOverlappedResult.

We're doing this mostly for testing purposes as the mingw 5.0.3
runtime is known to be broken.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-25 20:47:23 +00:00
Sage Weil
42d11d46b5 cephadm: provide meta during bootstrap
This is slightly gross, but we need ctx.meta_json for the bootstrap case,
which deploys a mon and mgr.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-25 14:44:26 -05:00
Sage Weil
852ca83933 mgr/cephadm: put service_name in unit.meta and use it when available
Inferring service_name from the daemon name is error-prone.

Fixes: https://tracker.ceph.com/issues/46219
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-25 14:44:26 -05:00
Sage Weil
f58485efb6 cephadm: accept arbitrary dict via --meta-json
e.g., --meta-json '{"foo": "bar"}'

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-25 14:44:26 -05:00
Daniel-Pivonka
674912bfed mgr/cephadm: add ok-to-stop functions for ceph client services
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
Signed-off-by: Adam King <adking@redhat.com>
2021-02-25 14:22:15 -05:00
Nathan Cutler
ffd202a086 rpm: re-disable SUSE lttng build on z390x
This partially reverts 2b1e646f7a which
mistakenly changed a line inside an "%if 0%{?suse_version}" conditional.

Fixes: 2b1e646f7a
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2021-02-25 19:16:59 +01:00
Sebastian Wagner
b092ad3acc
Merge pull request #39649 from Daniel-Pivonka/fixorphaninitialdaemons
cephadm: fix bug in orphan-initial-daemons logic

Reviewed-by: Adam King <adking@redhat.com>
2021-02-25 16:15:37 +01:00
Sebastian Wagner
24ff1c0b6d
Merge pull request #39613 from sebastian-philipp/DaemonDescriptionStatus
mgr/orch: Add DaemonDescriptionStatus

Reviewed-by: Daniel-Pivonka <dpivonka@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-02-25 16:15:03 +01:00
David Galloway
bb854450b9
Merge pull request #39672 from ceph/wip-159-notes
doc/releases/octopus.rst: 15.2.9 Release Notes
2021-02-25 09:54:50 -05:00
Zac Dover
25a914229a doc/dev: rewrite t8y "reporting the issue"
This commit rewrites the "Reporting the Issue"
section, reflecting my better understanding of the
various tools and parties in play. I also added a
TL;DR-style summary (or argument) paragraph at the
top of the section.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-26 00:38:11 +10:00
Sage Weil
0390f1141a qa/suites/rados/monthrash: crank up client side debugging w/ mon thrashing
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-25 08:35:45 -06:00
Duncan Bellamy
387f33e726 src/global/signal_handler.h: fix preprocessor logic for alpine
Fixes: https://tracker.ceph.com/issues/49494
Co-authored-by: Kefu Chai <tchaikov@gmail.com>
Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>
2021-02-25 14:20:55 +00:00
Zac Dover
dc87477a1a doc/dev: add t8y debug section names
This commit adds two section headings to the
Debugging section of the Teuthology Guide:

  1. Viewing Test Results
  2. Identifying Failed Jobs

The text as it stood before seemed to call out for
this explicit sectioning of material, so I've answered
that call.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-26 00:15:56 +10:00
Zac Dover
5c4a1a317d doc/dev: rewrite t8y "triaging" section
This commit simplifes and clarifies the "Triaging
the Cause of Failure" section in the Teuthology
Guide in the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-26 00:05:38 +10:00
Zac Dover
04fbd85e42 doc/dev: rewrite t8y "re-running tests"
This PR rewrites the "re-running tests" section
so that its elegance and readability are improved.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-25 22:45:15 +10:00
Rishabh Dave
c4685f702e qa/ceph_fuse: don't use createfs anymore while mounting
createfs was removed codebase since it didn't created FS regardless of
its value.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-02-25 17:00:54 +05:30
Kefu Chai
126f02008d
Merge pull request #39598 from tchaikov/wip-crimson-op-sequencer
crimson/osd: refactor OperationRepeatSequencer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-02-25 19:21:47 +08:00
Kefu Chai
efc46d96c0
Merge pull request #39659 from tchaikov/wip-pybind-rados-cleanup
pybind/rados: remove rados_nobjects_list_next() from .pxi

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-02-25 19:18:31 +08:00
Kefu Chai
8bf07cd140 pybind/mgr/dashboard: bump up requests to 2.25.1
request 2.20 is not compatible with urllib3 v1.25.2 and up. this causes
trouble of incompatibility with other python modules. for instance, we
now have following error:

ERROR: pip's dependency resolver does not currently take into account
all the packages that are installed. This behaviour is the source of the
following dependency conflicts.
botocore 1.20.14 requires urllib3<1.27,>=1.25.4, but you have urllib3
1.24.3 which is incompatible.

see also https://github.com/psf/requests/pull/5092

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-25 17:58:25 +08:00
Kefu Chai
19abd1e1f4 mgr/{prometheus,telemetry}: appease mypy
update to adapt the type annotation of MgrModule.list_servers()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-25 15:54:12 +08:00
Kefu Chai
b67e4341a3 mgr/ceph_module: add type annotation to BaseMgrModule
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-25 15:54:12 +08:00
Kefu Chai
32bf3c1ac2 mgr/influx: use empty data points to signal shutting down
to appease mypy, which prefer a queue with elements of the same type.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-25 15:54:12 +08:00