Commit Graph

113999 Commits

Author SHA1 Message Date
Kefu Chai
c11594a479 common/WorkQueue,librbd,rgw: use ceph::timespan for representing interval
better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-04 11:20:49 +08:00
Kefu Chai
69c70bd96f
Merge pull request #36442 from rzarzynski/wip-crimson-op_truncate
crimson/osd: add support for CEPH_OSD_OP_TRUNCATE.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-08-04 10:37:12 +08:00
Josh Durgin
2ec09274f8
Merge pull request #36398 from jdurgin/wip-pg-log-doc
osd/PGLog: improve documentation

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-08-03 19:12:41 -07:00
Kiefer Chang
d208d67f9f
mgr/dashboard: fix tasks.mgr.dashboard.test_rbd.RbdTest.test_move_image_to_trash error
The API call is a task and the response status is determined by whether
the call is completed within a pre-defined duration (2 seconds) or not.
We should also allow the status when the call takes longer.

Fixes: https://tracker.ceph.com/issues/46812
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-08-04 10:12:14 +08:00
Josh Durgin
20363b1834 common/options: update pg log bound descriptions to reflect current code
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-08-03 17:38:40 -04:00
Josh Durgin
8877270052 common/options: make pg log options dev-level
Changing these has performance and correctness implications, and is
rarely needed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-08-03 17:38:40 -04:00
Josh Durgin
3fdc4b6e69 common/options: describe osd_pg_log_trim_min
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2020-08-03 17:38:40 -04:00
Daniel-Pivonka
9f7c8f71fc mgr/orchestrator: fix placement of '1' activating dry-run bug
when applying a placement of '1' for mon, mgr, rbd-mirror, crash, alertmanager, grafana, node-exporter, prometheus, or mds a dry-run was activated

Fixes: https://tracker.ceph.com/issues/46819
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2020-08-03 16:37:02 -04:00
Radoslaw Zarzynski
dca8467764 crimson/osd: add support for CEPH_OSD_OP_TRUNCATE.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-03 21:51:05 +02:00
Ernesto Puerta
541ad5a060
github: replace trigger phrase for api tests
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2020-08-03 20:26:54 +02:00
Yuri Weinstein
fac822f131 qa/tests: changes related to #36435 and #36436
Merge only after #36435 and #36436 merged

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2020-08-03 10:22:42 -07:00
Kefu Chai
dbc9689f2c
Merge pull request #36429 from tchaikov/wip-ceph-debug-docker-crimson
ceph-debug-docker: install ceph-crimson-osd for crimson flavor

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-08-03 23:48:52 +08:00
Lenz Grimmer
b77da6cb3b
Merge pull request #35980 from tspmelo/wip-table-status
mgr/dashboard: Display table status in refresh button

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-08-03 17:22:03 +02:00
Jan Fajerski
cb432fe41d ceph-volume: dependency on python-ceph-common
Since e5b585d15d ceph-volume depends on
python-ceph-common. This commit introduces this dependency for the
ceph-osd rpm (which includes ceph-volume) and installs the dependency
for tox runs.

Fixes: https://tracker.ceph.com/issues/46772
Fixes: e5b585d15d

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-08-03 17:14:41 +02:00
Volker Theile
e401946531 cephadm: Deploying alertmanager image is broken
Fixes: https://tracker.ceph.com/issues/46814
Signed-off-by: Volker Theile <vtheile@suse.com>
2020-08-03 16:53:31 +02:00
Kefu Chai
1131f6fb4d
Merge pull request #36401 from tchaikov/wip-pg-stat-ack
messages/MPGStats: print seq instead of tid

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-08-03 22:47:40 +08:00
Sebastian Wagner
40475c6a26
Merge pull request #36375 from ricardoasmarques/force-registry-auth-config-set
cephadm: Fix error bootstraping with '--registry-json'

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-03 16:26:24 +02:00
Sebastian Wagner
2fccac11fc
Merge pull request #36386 from bk201/wip-46045
mgr/cephadm: skip Dashboard configuration if Dashboard is disabled

Reviewed-by: Varsha Rao <varao@redhat.com>
2020-08-03 16:25:44 +02:00
Kefu Chai
0516584d70
Merge pull request #36400 from tchaikov/wip-rip-xio-msgr
cmake: do not check OFED_PREFIX anymore

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-08-03 22:07:45 +08:00
Kefu Chai
cecca8e6a8 ceph-debug-docker: install ceph-crimson-osd for crimson flavor
would be handy if we can have crimson packages when debugging crimson
flavored branch.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-03 21:18:41 +08:00
Kiefer Chang
8f393f8e1b
mgr/dashboard: Fix host attributes like labels are not returned
The problem only happens when a host is reported both from Ceph and the
Orchestrator. There is an error on merging Ceph hosts with Orchestrator
hosts. The unit test is refined to test this case.

Fixes: https://tracker.ceph.com/issues/46761
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-08-03 19:26:33 +08:00
Jason Dillaman
6b727d2cb9
Merge pull request #35060 from yison/rwl-image-writeback-cache-hook-seq1
rbd/cache: Replicated Write Log Hook part 11

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-08-03 06:42:54 -04:00
Kefu Chai
d4e90124f3
Merge pull request #35659 from agayev/track-per-zone-metadata
os/bluestore: track per zone metadata

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-08-03 18:12:29 +08:00
Kefu Chai
4f6443737e
Merge pull request #30838 from ifed01/wip-ifed-single-alloc
os/bluestore: use single allocator for shared bluestore/bluefs device

Reviewed-by: Sage Weil <sage@redhat.com>
2020-08-03 18:00:16 +08:00
Kefu Chai
9fe9dc98bd
Merge pull request #36356 from mgfritch/orch-format-not-supported
mgr/orch: disallow unimplemented format types

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-03 17:58:54 +08:00
Yuan Lu
d1c82d5582 librbd: enable image cache after getting exclusive lock
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-08-03 05:37:13 -04:00
Yuan Lu
99fdc0a3ce librbd: add InitRequest and ShutdownRequest state machine
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-08-03 05:37:13 -04:00
Yuan Lu
7beef299ad librbd: split ReadRequest
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-08-03 05:37:13 -04:00
Yuan Lu
7e0d1a97c6 librbd: add IMAGE_CACHE feature
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-08-03 05:37:13 -04:00
Kefu Chai
1a698a6e19
Merge pull request #36417 from tchaikov/wip-crimson-injectargs
crimson/admin: implement "injectargs" command

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2020-08-03 17:26:30 +08:00
Volker Theile
4b87eeb978 mgr/cephadm: Enhance AlertManagerSpec to allow adding additional webhook receiver URLs
Fixes: https://tracker.ceph.com/issues/46775

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-08-03 10:42:09 +02:00
ghyer
52e5fca550 mon: add "duration" field for monitors' operation log.
Signed-off-by: Haoyu Gao <i@ghyer.com>
2020-08-03 15:36:27 +08:00
Jianpeng Ma
17f375d7e0 kv/RocksdbStore: remove perfcounter l_rocksdb_txns/l_rocksdb_txns_sync
Perfcounter l_rocksdb_submit_latency/l_rocksdb_submit_sync_latency
contain l_rocksdb_txns/l_rocksdb_txns_sync.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2020-08-03 15:32:44 +08:00
Kefu Chai
6c2bd5a327 crimson/osd: ignore SIGHUP
ignore SIGHUP, as crimson does not write log messages using the
facilities offered by ceph::logging. it uses seastar::logger instead.

and seastar::logger writes to stderr, stdout and/or syslog. there is no
need to reopen the log file when the settings change.

and we do send SIGHUP to OSD processes when performing thrash tests. to
avoid killing crimson-osd processes by accident. SIGHUP is ignored.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-03 11:04:40 +08:00
Kefu Chai
9e01fcac91
Merge pull request #36407 from tchaikov/wip-mgr-autopep8
mgr/cephadm: track dependency in tox.ini

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-03 09:33:21 +08:00
Kefu Chai
ebdb392317 common/HeartbeatMap: use std::atomic<time_point> for timeout
since we now depends on GCC-7 and up for C++17 support, we can now use
std::atomic<time_point> for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-03 09:30:47 +08:00
Jason Dillaman
61e1e59c65
Merge pull request #36378 from orozery/librados_test_stub_assert_version
test/librados_test_stub: handle assert_version

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-08-02 16:58:07 -04:00
Harley Gorrell
5cdadf17d6 mgr: Add missing states to PG_STATES in mgr_module.py.
Fixes: https://tracker.ceph.com/issues/46808
Signed-off-by: Harley Gorrell <harley@panix.com>
2020-08-02 11:37:01 -06:00
Kefu Chai
69de458e2f crimson/admin: implement "injectargs" command
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-03 00:54:55 +08:00
Kefu Chai
cdf73b874b crimson/common: add ConfigProxy::inject_args()
prepare for the "injectargs" tell command.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-03 00:54:55 +08:00
Or Ozeri
01045545b6 librbd: support reading multiple extents in the object dispatch interface
This commit extends the object dispatch read function to support multiple object extents (using a single librados ObjectOperation).
In addition, we add an option to return the version number for that read.
This new function will be used by the new crypto object dispatch layer to support read-modify-write operation required for encrypting and writing unaligned data.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2020-08-02 12:27:45 +03:00
Kefu Chai
00fda0bbe4
Merge pull request #36406 from tchaikov/wip-pg-stat-seq
crimson/osd: send pg_stat with correct seq

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-08-02 10:19:47 +08:00
Kefu Chai
a196672b77 mgr/cephadm: track dependency in tox.ini
* track dependency for running tox tests with tox.ini. the fix
  test is not performed by "make check", so `install-deps.sh`
  is not responsible for preparing the autopep8 wheels. hence
  we can just put it in tox.ini
* remove "python_version >= '3'" as we do not support python2

this change addresses a regression introduced by
7d8a3b6032, which fails the rpmbuild like:

RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/share/ceph/mgr/requirements-fix.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 09:39:55 +08:00
Sebastian Wagner
bd0b900325
Merge pull request #36348 from votdev/cephadm_tox_pylint
mgr/cephadm: Introduce tox and autopep8

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-08-01 22:36:16 +02:00
Kefu Chai
6a20c7ffdd crimson/osd: send pg_stat with correct seq
* set the osd_stat in pg_stat message sent to mgr, otherwise the seq id
  in the message would be always 0.
* return the seq id composed of up_epoch and osd_stat_seq when serving
  "flush_pg_stats" tell command.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 00:35:58 +08:00
Kefu Chai
7b258091b4 crimson/osd: remove unused #include
these messages are handled by osd/PeeringState

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 00:35:58 +08:00
Kefu Chai
129658b5f9 common/config: extract check_old_style_section_names()
extract check_old_style_section_names() out from
md_config_t::parse_config_files() into ConfFile class.

for two reasons:

* so it can be reused
* for smaller md_config_t::parse_config_files(), hence better
  readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 00:01:47 +08:00
Kefu Chai
ee70b55e8c common/config: extract get_cluster_name()
for two reasons

* to consolidate the rules for figuring out the cluster name
* to reuse it in another implementation of parse_config_files(),
  crimson will need to restructure the existing parse_config_files(),
  and implement an async version of it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 00:01:47 +08:00
Kefu Chai
502013affb common: let get_my_sections() return the section names
instead of passing the output parameter just let it return the section
names. C++17 enfoces the copy elision in this case, so no need to pass
the output parameter for saving the overhead of creating an temporary
object and copying it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 00:01:47 +08:00
Kefu Chai
cd53ff88b7 common/config: extract get_conffile_paths() out
md_config_t::parse_config_files() performs blocked i/o, and it might be
detected by the stall detector of seastar, and it spills out warning
messages when it sees the hang.

so i plan to implement an asynchronized parse_config_files() for
crimson, and reuse the synchronized part from the blocking
implementation of parse_config_files(). this is the first step.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-08-02 00:01:47 +08:00