Commit Graph

127529 Commits

Author SHA1 Message Date
Deepika Upadhyay
ada9794666
Merge pull request #43127 from majianpeng/librbd-pwl-flush-reorder
librbd/cache/pwl: fix reorder when flush cache-data to osd.

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-11-01 12:31:04 +05:30
Kefu Chai
7f0fa84e43
Merge pull request #43749 from tchaikov/wip-seastore-cleanup
crimson/os/seastore: minor cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-11-01 12:39:06 +08:00
Jianpeng Ma
1fc3be2480 librbd/cache/pwl: fix reorder when flush cache-data to osd.
Consider the following workload:
writeA(0, 4096)
writeB(0, 512).
pwl can makre sure writeA persist to cache before writeB.
But when flush to osd, it use async-read to read data from cache and in
the callback function they issue write to osd.
So although we by order issue aio-read(4096), aio-read(512). But we
can't make sure the return order.
If aio-read(512) firstly return, the write order to next layer is
writeB(0, 512)
writeA(0, 4096).
This is wrong from the user point.

To avoid this occur, we should firstly read all data from cache. And
then send write by order.

Fiexs: https://tracker.ceph.com/issues/52511

Tested-by: Feng Hualong <hualong.feng@intel.com>
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2021-11-01 09:25:52 +08:00
zdover23
9b3fdce0ab
Merge pull request #43676 from zdover23/wip-doc-2021-10-26-rados-config-storage_devices
doc/rados: storage-devices.rst rewrite

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-31 15:17:33 +10:00
Sage Weil
bd093a848d Merge PR #43734 into master
* refs/pull/43734/head:
	cephadm: --config-json overrides --config or --keyring args

Reviewed-by: Adam King <adking@redhat.com>
2021-10-30 15:51:31 -04:00
Sage Weil
3e22bfe8e2 Merge PR #43743 into master
* refs/pull/43743/head:
	mgr/nfs/export: use tool_exec() instead of private _exec()
	pybind/mgr/mgr_module: add tool_exec() command

Reviewed-by: Sage Weil <sage@redhat.com>
2021-10-30 15:51:04 -04:00
Sebastian Wagner
433463b9e6
Merge pull request #43654 from adk3798/limit-info
mgr/cephadm: inform users if limit set for data devices is not met

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-30 20:47:32 +02:00
Sebastian Wagner
e9636ddd24
Merge pull request #43606 from adk3798/suppress-tls-error
mgr/cephadm: suppress TLSV1_ALERT_DECRYPT_ERROR from cherrypy

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-30 20:46:48 +02:00
Kefu Chai
89d9b739e8 crimson/os/seastore: minor cleanups
* pass string by reference. more consistent this way.
* set a bl using list::append(), simpler this way.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-30 15:51:03 +08:00
Samuel Just
8b7542d27f
Merge pull request #43657 from liu-chunmei/osd_meta_to_file
crimson: osd meta read from/write to file. 

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-29 23:46:24 -07:00
Samuel Just
9d19de5439 test/crimson/seastore/test_transaction_manager_test_state: add mock MDStore
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-29 18:04:11 -07:00
Samuel Just
da2b40586a crimson/os/seastore/seastore: factor out MDStore interface
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-29 18:03:59 -07:00
chunmei-liu
c423abaf0f crimson: check mkfs_done before seastore do mkfs
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-10-29 18:03:55 -07:00
chunmei-liu
784aa2f4f2 crimson: write some other osd metas
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-10-29 18:03:49 -07:00
chunmei-liu
0f0e5993a9 crimson: osd meta data read from and write to files
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-10-29 18:03:42 -07:00
Sage Weil
8dcdde7151
mgr/nfs/export: use tool_exec() instead of private _exec()
Fixes: https://tracker.ceph.com/issues/53096
Signed-off-by: Sage Weil <sage@newdream.net>
Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-10-29 14:03:22 -06:00
Sage Weil
e24171f818
pybind/mgr/mgr_module: add tool_exec() command
Run a (ceph) tool with the appropriate keyring and entity name.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-29 14:03:21 -06:00
Casey Bodley
74565072f7
Merge pull request #43625 from alimaredia/wip-marcus-teuthvault-2
qa/rgw: Fix vault token file access.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-10-29 15:14:03 -04:00
Sage Weil
4328c7408c Merge PR #43736 into master
* refs/pull/43736/head:
	Revert "Merge pull request #41325 from rhcs-dashboard/wip-cephadm_box-master"

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-29 14:55:02 -04:00
Ernesto Puerta
f739fc93aa
Merge pull request #43738 from rishabh-d-dave/mgr-dash-pyfakefs
mgr/dashboard: all pyfakefs must be pinned on same version

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: ljflores <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: rishabh-d-dave <NOT@FOUND>
2021-10-29 18:52:44 +02:00
Yuri Weinstein
0f9321552c
Merge pull request #43733 from neha-ojha/wip-fix-octopusx
qa/suites/upgrade/octopus-x: exclude ceph-volume

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2021-10-29 09:45:37 -07:00
Matt Benjamin
852e9e7d7e
Merge pull request #43329 from cfsnyder/wip-48752-generalize-ops-log-socket
rgw: add abstraction for ops log destination and add file logger
2021-10-29 12:19:59 -04:00
Rishabh Dave
24d3a01a30 mgr/dashboard: all pyfakefs must be pinned on same version
Without this patch execution of install-deps.sh fails with an error.

Fixes: https://tracker.ceph.com/issues/53088
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-10-29 20:34:12 +05:30
Sage Weil
b9fcbf2c03 Revert "Merge pull request #41325 from rhcs-dashboard/wip-cephadm_box-master"
This reverts commit 78d6ac0d35, reversing
changes made to 9a8de97f7a.

This broke cephadm (infers config when it should not)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-29 09:53:43 -05:00
Sage Weil
f68f9dd7c3 cephadm: --config-json overrides --config or --keyring args
These args aren't normally combined, but if --fsid and --name are
provided, they may be inferred.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-29 09:49:17 -05:00
Neha Ojha
e297db28a3
Merge pull request #43706 from neha-ojha/wip-doc-corr
doc/releases/pacific.rst: add note about data corruption bug

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Laura Flores <lflores@redhat.com>
2021-10-29 07:34:41 -07:00
Yuri Weinstein
1dc90ce69a
Merge pull request #43714 from ceph/wip-yuriw-crontab-master
qa/tests: added back octopus-x run on master

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-29 07:15:31 -07:00
Sage Weil
b82ab4675c Merge PR #43709 into master
* refs/pull/43709/head:
	qa/tasks/cephfs/test_nfs: wait for fs to come up before exporting

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-29 09:54:01 -04:00
Sebastian Wagner
0b9e7c9567
Merge pull request #43602 from sebastian-philipp/cephadm-rgw-ipv6
mgr/cephadm: Fix RGW ipv6 frontend configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
2021-10-29 12:23:19 +02:00
Sebastian Wagner
34502b1b22
Merge pull request #43260 from cfsnyder/wip-43692-repave-osds
mgr/orchestrator: add --zap flag to 'orch osd rm'

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-29 10:02:11 +02:00
Samuel Just
f94abe64f3
Merge pull request #43539 from orozery/ms-nocrc
msg/async: support disabling data crc for protocol v2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-28 18:37:26 -07:00
Yuri Weinstein
69567f16f3 qa/tests: added pacific-x suite to run on master
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-10-28 16:11:50 -07:00
Kefu Chai
c2c7297360
Merge pull request #31877 from rosinL/wip-fix-dpdk-link
cmake: link against DPDK shared libraries to avoid DPDK EAL double initialization

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-29 06:58:20 +08:00
Neha Ojha
0ec8380990 qa/suites/upgrade/octopus-x: exclude ceph-volume
To fix failure like

Failure Reason:
Command failed on smithi085 with status 1: 'sudo yum -y install ceph-volume'

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-10-28 22:26:34 +00:00
Sebastian Wagner
e13aa78289
Merge pull request #43693 from batrick/i53074
pybind/mgr/cephadm: upgrade MDS if no MDS is "up"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-29 00:25:59 +02:00
Sebastian Wagner
aae2ea3897
Merge pull request #43293 from pcuzner/granular-alerts
mgr/prometheus: expose ceph healthchecks as metrics

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-29 00:23:24 +02:00
Yuri Weinstein
e7cea80450 qa/tests: added back octopus-x run on master
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-10-28 15:13:04 -07:00
Sage Weil
3298e496ca Merge PR #43583 into master
* refs/pull/43583/head:
	changes to logging code requested by Adam
	remove allocation file after rollback to real-fm
	bug-fix close db when umount failed to store allocation file
	Bug-Fix: When restoring allocation from file use a temp allocator and only copy the allocation to the shared-allocator after the file was verified and all extents were cleared

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
2021-10-28 16:13:32 -04:00
Sage Weil
c9e647ab86 Merge PR #43645 into master
* refs/pull/43645/head:
	os/bluestore: Disable compaction then no-column-b is storing allocations to bluefs file

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Gabriel Benhanokh <gbenhano@redhat.com>
2021-10-28 16:12:36 -04:00
Marcus Watts
454cc8a18c Fix vault token file access.
Put the vault token file in a location that ceph can read.
Make it readable only by ceph.

On rhel8 (and indeed, any vanilla rhel machine), $HOME is liable to be
mode 700.  This means the ceph user can't read things in that user's
directory.  This causes radosgw to emit the confusing message "ERROR:
Vault token file ... not found" even though the teuthology log will
plainly show it was created and made readable by ceph.

Fixes: http://tracker.ceph.com/issues/51539
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2021-10-28 14:14:10 -04:00
Sage Weil
d7acc16860 qa/tasks/cephfs/test_nfs: wait for fs to come up before exporting
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-28 13:28:35 -04:00
Neha Ojha
bdca57e08e doc/releases/pacific.rst: add note about data corruption bug
Warn users upgrading their clusters from nautilus/octopus to pacific about
https://tracker.ceph.com/issues/53062

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-10-28 16:56:13 +00:00
Ernesto Puerta
78d6ac0d35
Merge pull request #41325 from rhcs-dashboard/wip-cephadm_box-master
cephadm: add cephadm sandbox (Docker in Docker)

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
2021-10-28 16:59:47 +02:00
Sebastian Wagner
9a8de97f7a
Merge pull request #43692 from liewegas/misc-rook-1
mgr/orchestrator: log exception from wrapper

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-28 16:56:23 +02:00
Sebastian Wagner
558a2ec618
Merge pull request #43330 from sebastian-philipp/CEPHADM_STRAY_DAEMON
doc/cephadm/operations: extend CEPHADM_STRAY_DAEMON

Reviewed-by: Adam King <adking@redhat.com>
2021-10-28 15:05:58 +02:00
Sebastian Wagner
baea97f4fa
Merge pull request #43601 from sebastian-philipp/doc-cephadm-osd-similar-disk-layout
doc/cephadm: improve "Deploying OSDs on multiple similar hosts"

Reviewed-by: Joao Eduardo Luis <joao@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-28 12:14:57 +02:00
Sebastian Wagner
e7fdd7d065
Merge pull request #43622 from Daniel-Pivonka/cephadm-doc-osd-replace-host
doc/cephadm: note that replacment osd needs to be on same host as removed osd

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-28 12:13:23 +02:00
Samuel Just
856b793519
Merge pull request #43681 from cyx1231st/wip-seastore-fix-test-setup-teardown
crimson/os/seastore: initialize tests in seastar reactor

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-10-28 03:11:05 -07:00
Pere Diaz Bou
59dd41027d cephadm/box: add no-tmpfs flag prepare osd bluestore
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-28 09:25:28 +02:00
Patrick Donnelly
d771725664
pybind/mgr/cephadm: upgrade MDS if no MDS is "up"
The upgrade process can get stuck if an MDS crashes. This should be rare
when straddling v16.2.5 where the compatset of the file system inherits
the FSMap "default". The MDS from pre-v16.2.5 do not yet share a
compatset with the mons so the mons will do no promotions, causing
upgrade task to get stuck.

Fixes: https://tracker.ceph.com/issues/53074
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-27 20:36:42 -04:00