Commit Graph

126914 Commits

Author SHA1 Message Date
Patrick Donnelly
dbe5573ed4
qa: add note about where caps are generated
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-05 13:32:15 -04:00
Patrick Donnelly
24bb450d39
qa: use ctx's ceph_manager to run ceph commands by mount
This allows hooks for `cephadm shell` to function so that this code
works with cephadm deployments.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-05 13:32:15 -04:00
Patrick Donnelly
7812cfb674
qa: move CephManager cluster instantiation to subtask
This needs to be available for the cephfs_setup task so administration
mounts can run ceph commands, potentially through `cephadm shell`.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-05 13:32:15 -04:00
Patrick Donnelly
bca21f01ce
pybind/mgr/cephadm: disable allow_standby_replay during CephFS upgrade
Following procedure in [1].

Also: harden checks for active. Ensure "up" and "in" are both [0]. There
should be no standby-replay daemon.

[1] https://docs.ceph.com/en/pacific/cephfs/upgrading/

Fixes: https://tracker.ceph.com/issues/52654
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-05 13:32:14 -04:00
Patrick Donnelly
4affb5c702
pybind/mgr/cephadm: always do mds upgrade sequence
Minor versions also require this sequence.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-05 13:32:14 -04:00
Yuri Weinstein
d5759e030b
Merge pull request #42977 from amathuria/wip-amathuri-scrub-duration
osd/scrub: Add scrub duration to pg dump stats

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2021-10-05 07:49:03 -07:00
Deepika Upadhyay
e4db160015
Merge pull request #43175 from ideepika/wip-supress-jaeger-cmake-logs
cmake,rgw: use jaeger-base target, as a dependency for building dbstore

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-05 14:00:50 +05:30
Josh Durgin
eb9c0f5982
Merge pull request #43410 from zdover23/wip-doc-2021-10-04-min-hardware-typo
docs/start: fix min. hardware rec. typo

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-10-04 11:02:06 -07:00
Zac Dover
14151f2abe docs/start: fix min. hardware rec. typo
The minimum hardware recommendations said that we
recommended a minimum of "24GB+ per daemon".

I changed this to read "2-4GB+ per daemon".

Reddit actually caught this one:
https://www.reddit.com/r/ceph/comments/q14ey2/cephs_mon_minimum_requirements/

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-10-05 02:57:51 +10:00
Deepika Upadhyay
463193bcaf cmake,rgw: use jaeger-base target, as a dependency for building dbstore
* dbstore also has tracing dependencies and missing linking for libraries
* use `using` keyword in common/tracer.cc to silence unused method
  warnings
* log building of external project jaegertracing, opentracing into build
  file, (done to reduce external project build verbosity)

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-04 13:52:49 +05:30
Sage Weil
0bc19b5449
Merge pull request #43402 from liewegas/doc-general-rep
doc/foundation: Martin is new general member rep
2021-10-03 13:42:45 -05:00
Patrick Donnelly
66cc38db1c
Merge PR #43383 into master
* refs/pull/43383/head:
	doc: fixed typo in terminology

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-02 12:27:40 -04:00
Sage Weil
df5be4ba93 doc/foundation: Martin is new general member rep
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-02 10:49:46 -05:00
Ernesto Puerta
ce535dd8e0
Merge pull request #41255 from rhcs-dashboard/angular-11-upgrade
mgr/dashboard: Angular v11 Migration

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-01 18:13:09 +02:00
Ernesto Puerta
c525e74154
Merge pull request #42716 from sshambar/bug-52002
mgr/dashboard,prometheus: fix handling of server_addr

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: sshambar <NOT@FOUND>
2021-10-01 18:12:11 +02:00
zdover23
f5ee2c4bf1
Merge pull request #43277 from anthonyeleven/anthonyeleven/fix-47839
ceph: add example BlueStore migration via the ceph-objectstore-tool dup op

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2021-10-02 00:22:22 +10:00
Jos Collin
d1d4cdcc39
doc: fixed typo in terminology
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-10-01 16:04:44 +05:30
Aishwarya Mathuria
1b4e416f81 osd/scrub: Add scrub duration to pg dump stats
Addition of a new column, SCRUB_DURATION, to the pg stats that stores the time taken for a PG scrub.

Fixes: https://tracker.ceph.com/issues/52605
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2021-10-01 13:27:27 +05:30
Nizamudeen A
fa6fb0087c mgr/dashboard: Remove i18n Warning about duplicate id
Fixes: https://tracker.ceph.com/issues/50268
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-01 12:29:07 +05:30
Nizamudeen A
f7d73d0420 mgr/dashboard: Taking care of deprecation warnings and other errors
1. deprecation  group is deprecated: This API is not typesafe and can result in issues with Closure Compiler renaming. Use the `FormBuilder#group` overload with `AbstractControlOptions` instead.

2. cypress/integration/rgw/users.po.ts(113,23): error TS2345: Argument of type 'void' is not assignable to parameter of type 'string'.

3. Cannot find module 'isomorphic-form-data' from 'node_modules/swagger-ui/dist/swagger-ui.js'

Fixes: https://tracker.ceph.com/issues/50268
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-01 12:29:07 +05:30
Nizamudeen A
de26485018 mgr/dashboard: Angular v11 Migration
Fixes: https://tracker.ceph.com/issues/50268
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-10-01 12:28:40 +05:30
Patrick Donnelly
6939ea034a
Merge PR #43323 into master
* refs/pull/43323/head:
	mgr/volumes/fs: add extra blank line

Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-09-30 12:50:54 -04:00
Patrick Donnelly
8031dee8ea
Merge PR #43322 into master
* refs/pull/43322/head:
	mgr/volumes/fs: add extra blank line

Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-09-30 12:49:37 -04:00
Yuri Weinstein
33bcd2e509
Merge pull request #36733 from majianpeng/rocksdb-remove-redundant-perfcount
kv/RocksdbStore: remove perfcounter l_rocksdb_gets.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-09-30 07:52:30 -07:00
Yuri Weinstein
d14580d44b
Merge pull request #34624 from zhouyh-orcadt/fix_optracker_coredump
common/TrackedOp: fix osd reboot optracker coredump

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-09-30 07:51:32 -07:00
Kefu Chai
96edb523c1
Merge pull request #43355 from rzarzynski/wip-crimson-write_meta
crimson/osd: write the 'osd_key' meta on OSD::mkfs(). 

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-09-30 22:16:51 +08:00
Kefu Chai
130c7261aa
Merge pull request #43364 from rzarzynski/wip-crimson-canceliores-on-stop
crimson/osd: cancel IO reservations on PG::stop().

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-09-30 22:14:11 +08:00
Radoslaw Zarzynski
6b04b96991 crimson/osd: cancel IO reservations on PG::stop().
`PG::request_{local,remote}_recovery_reservation()` dynamically allocates
up to 2 instances of `LambdaContext<T>` and transfers their ownership to
the `AsyncReserver<T, F>`. This is expressed in raw pointers (`new` and
`delete`) notion. Further analysis shows the only place where `delete`
for these objects is called is the `AsyncReserver::cancel_reservation()`.
In contrast to the classical OSD, crimson doesn't invoke the method when
stopping a PG during the shutdown sequence. This would explain the
following ASan issue observed at Sepia:

```
Direct leak of 576 byte(s) in 24 object(s) allocated from:
    #0 0x7fa108fc57b0 in operator new(unsigned long) (/lib64/libasan.so.5+0xf17b0)
    #1 0x55723d8b0b56 in non-virtual thunk to crimson::osd::PG::request_local_background_io_reservation(unsigned int, std::unique_ptr<PGPeeringEvent, std::default_delete<PGPeeringEvent> >, std::unique_ptr<PGPeeringEvent, std::default_delete<PGPeeringEvent> >) (/usr/bin/ceph-osd+0x24d95b56)
    #2 0x55723f1f66ef in PeeringState::WaitDeleteReserved::WaitDeleteReserved(boost::statechart::state<PeeringState::WaitDeleteReserved, PeeringState::ToDelete, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::my_context) (/usr/bin/ceph-osd+0x266db6ef)
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-30 11:28:08 +00:00
Sebastian Wagner
b268cb8a5e
Merge pull request #43202 from karmab/kcli_doc
doc: update kcli information

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-09-30 12:01:54 +02:00
Sebastian Wagner
cab4f9f5f2
Merge pull request #43356 from tchaikov/wip-doc-cephadm
doc/cephadm/services: unindent note section

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-30 10:29:00 +02:00
Kefu Chai
48f5cd5b9b doc/cephadm/services: unindent note section
before this change the note on "apply" command is embedded in the note
on "_no_schedule". and they are not related. so let's move the former
out. also, highlight the yaml file sample in YAML.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-09-30 07:22:11 +08:00
Radoslaw Zarzynski
5b70488bf3 crimson/osd: write the 'ready' meta on OSD::mkfs().
This is for the compliance with the classical OSD. No problem
with lack of this file has been identified yet.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-29 23:18:13 +00:00
Radoslaw Zarzynski
42a1462f30 crimson/osd: write the 'osd_key' meta on OSD::mkfs().
This commit fixes an issue identified during the Rook-crimson effort.

Missing the `write_meta()` on `osd_key` made the CephX inoperational
because of imposibility to load the keyring. Disabling CephX in turn
caused the auth method negotation to fail when reaching out to a monitor.

```
ERROR 2021-09-28 21:19:46,598 [shard 0] none - auth: unable to find a keyring on /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory
ERROR 2021-09-28 21:19:46,598 [shard 0] none - AuthRegistry(0x7fa38c322b68) no keyring found at /var/lib/ceph/osd/ceph-0/keyring, disabling cephx
...
INFO  2021-09-28 21:19:46,601 [shard 0] monc - get_auth_request(con=[client.?(temp_mon_client) 172.17.0.1:0/2910147961@63138 >> mon.? v2:10.108.187.31:3300/0], auth_method=0)
INFO  2021-09-28 21:19:46,601 [shard 0] monc - get_auth_request no methods is supported
...
WARN  2021-09-28 21:20:06,612 [shard 0] monc - cannot establish the active_con with any mon
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-29 20:21:32 +00:00
Samuel Just
e2b05287d5
Merge pull request #43340 from athanatos/sjust/wip-seastore-throttle
crimson/seastore: fix segment_cleaner blocked_io_wake crash

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-09-29 12:29:23 -07:00
Sage Weil
e3bede0008 Merge PR #43287 into master
* refs/pull/43287/head:
	mgr/rook, qa/tasks/rook: change rgw daemon service name
	mgr/rook: fix placement_spec_to_node_selector
	mgr/rook: orch rm no longer uses rook api delete
	qa/tasks/rook: fix cluster deletion hanging due to CephObjectStore CR
	mgr/rook: use default replication size in orch apply rgw
	mgr/rook: add placement specs to apply rgw

Reviewed-by: Sage Weil <sage@redhat.com>
2021-09-29 14:38:47 -04:00
Ernesto Puerta
156defa48e
Merge pull request #43255 from rhcs-dashboard/fix-49344-master
qa/mgr/dashboard: add extra wait to test

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-09-29 20:23:23 +02:00
Kefu Chai
05706e72fe
Merge pull request #43341 from pdvian/wip-doc-mon-bootstrap
doc/dev/mon-bootstrap: fix typo

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-29 23:22:24 +08:00
karmab
064b8de309 doc: update kcli information 2021-09-29 17:07:53 +02:00
Kefu Chai
710fded6e2
Merge pull request #42353 from ifed01/wip-ifed-fix-invalid-offset-repair
os/bluestore: fix writing to invalid offset when repairing

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2021-09-29 22:54:15 +08:00
Kefu Chai
3f52895a28
Merge pull request #39062 from fyzard1991/master
mon/OSDMonitor:Make the pg_num check more accurate

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-09-29 22:52:11 +08:00
Kefu Chai
aca8e1f6ba
Merge pull request #43289 from tchaikov/wip-52705
os/bluestore: list obj which equals to pend

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Xinze Chi <xinze@xsky.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2021-09-29 22:50:02 +08:00
Ernesto Puerta
f947e6ec67
Merge pull request #42821 from rhcs-dashboard/force-maintenance-workflow
mgr/dashboard: Move force maintenance test to the workflow test suite 

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-09-29 13:36:45 +02:00
Sebastian Wagner
32d79d7a55
Merge pull request #43246 from Daniel-Pivonka/cephadm-iscsi-trust-mgr-ips
mgr/cephadm: populate trusted_ip_list in iscsi-gateway.cfg with mgr ips

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-09-29 12:37:44 +02:00
Sebastian Wagner
8a0fc18201
Merge pull request #43309 from guits/iscsi_fixes
cephadm/iscsi: fix a typo

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-29 12:37:02 +02:00
Sebastian Wagner
cabad2ca44
Merge pull request #43319 from guits/workaround_tcmurunner_log
cephadm: modify iscsi containers bindmount

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-29 12:36:42 +02:00
Sebastian Wagner
293e2dcede
Merge pull request #43321 from adk3798/remove-exporter
mgr/cephadm: remove cephadm exporter code

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-29 12:36:06 +02:00
Yuval Lifshitz
c69e45c0e0
Merge pull request #43226 from TRYTOBE8TME/wip-rgw-amqp-minor-nit
src/test: Minor nit from PR #43025
2021-09-29 12:32:55 +03:00
Samuel Just
b344e70e83 crimson/os/seastore/transaction_manager: limit callers to reserve_projected_usage
Adds an exclusive stage for obtaining projected usage as well as an
unordered one for submitting ool writes.  This should allow for a
straightforward wait-list when io is blocked while still allowing
concurrent submission of ool writes otherwise.

Fixes: https://tracker.ceph.com/issues/52698
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-29 00:09:12 -07:00
Samuel Just
fbd30a4b0b crimson/os/seastore/segment_cleaner: track projected usage for in progress operations
We're going to want to permit multiple transactions to be writing
concurrently.  Replace await_hard_limits() with a mechanism that
remembers bytes that will be used by in-progress operations.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-09-29 00:09:09 -07:00
Prashant D
bdb98ad8f9 doc/dev/mon-bootstrap: fix typo
Signed-off-by: Prashant D <pdhange@redhat.com>
2021-09-29 07:13:56 +01:00