Commit Graph

129217 Commits

Author SHA1 Message Date
Sebastian Wagner
24aab16cd0
Merge pull request #44505 from guits/fix-53812
ceph-volume: fix regression introcuded via #43536

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Teoman Onay <tonay@redhat.com>
2022-01-18 16:31:49 +01:00
Sebastian Wagner
7842825a2e
Merge pull request #44489 from adk3798/agent-down-alerts
mgr/cephadm: still check agent deps if it is marked down
2022-01-18 16:23:40 +01:00
Pere Diaz Bou
57c26311de monitoring/grafana: replace filestore osd count
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 14:14:41 +01:00
Pere Diaz Bou
a3cf5c5e9f monitoring/grafana: use Path class instead of split
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
1e4d85d04f monitoring/grafana: remove explicit str casting
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
2b4f3561d2 monitoring/grafana: add generated json files
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
b381a83e9b monitoring/grafana: ValueError instead of RuntimeError
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
4c302234ff monitoring/grafana: Replace missing legendFormat warning with error
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:10 +01:00
Guillaume Abrioux
5c0f0698a5 qa/cephadm: install hwe kernel only for focal
Let's install hwe kernel only on Ubuntu focal, otherwise we only shift the
problem on Ubuntu bionic given that the hwe kernel for bionic is 5.4.

Fixes: https://tracker.ceph.com/issues/53863

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-18 10:36:24 +01:00
myoungwon oh
c93d0b7b1e seastore: rename segment_off_t to seastore_off_t
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-18 18:24:43 +09:00
Samuel Just
51a347456d
Merge pull request #44591 from athanatos/sjust/wip-seastore-flush
crimson/os/seastore: avoid empty Transactions by adding explicit flush() call

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-17 19:36:10 -08:00
Samuel Just
7370d7ded8
Merge pull request #44556 from cyx1231st/wip-crimson-improve-log-journal
crimson/os/seastore: consolidate seastore_journal logs with cleanup and validations

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-01-17 13:19:02 -08:00
Casey Bodley
3b93654d6e rgw: clean up index after full metadata sync
Fixes: https://tracker.ceph.com/issues/40177

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 15:53:26 -05:00
Casey Bodley
dd6bf0b5a8 rgw: clean up index after full data sync
Fixes: https://tracker.ceph.com/issues/40177

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 15:53:26 -05:00
Samuel Just
7c4b3cc7fa crimson/os/seastore: implement FuturizedStore::flush
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-01-17 20:50:58 +00:00
Casey Bodley
a6611a55cc rgw/swift: don't crash on nonexistent bucket in BulkUpload
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 14:59:01 -05:00
Waad AlKhoury
ea55a0b33d mgr/dashboard: Notification banners at the top of the UI have fixed height
Fixes: https://tracker.ceph.com/issues/51575
Signed-off-by: Waad AlKhoury <walkhour@redhat.com>
2022-01-17 20:45:58 +01:00
Casey Bodley
52bfa9a866 qa/rgw: run multisite tests with some async notifications disabled
disable the sending of async datalog notifications on one zone per
cluster. this helps to verify that tests don't rely on notifications to
succeed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 13:54:39 -05:00
Casey Bodley
bf0a4ef1aa rgw: allow rgw_data_notify_interval_msec=0 to disable notifications
the data changes log for multisite will occasionally broadcast recent
changes to other zones, which they can use to prioritize sync of some
of the most recent changes. they'll eventually see all changes as they
replay the data changes log, though, so notifications aren't required
for successful sync. the ability to turn them off is useful for testing

Fixes: https://tracker.ceph.com/issues/49723

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 13:54:36 -05:00
Mykola Golub
078d72e5e6 cls/journal: skip disconnected clients when finding min_commit_position
When a new journal client is registered, all already registered
clients are checked, and a client with min position is selected
as a position for the new client. Thus we may expect that
starting from the registered position all journal entries will be
available (not trimmed) for the new client.

But when looking for a min commit position, the client_register
function did not take into account that a registered client might
be in disconnected state, and in that case the journal entries
might be trimmed for this client.

Fixes: https://tracker.ceph.com/issues/53888
Signed-off-by: Mykola Golub <mgolub@suse.com>
2022-01-17 18:41:34 +00:00
Yuri Weinstein
095f970710 qa/tests: run upgrades on quincy
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2022-01-17 10:18:04 -08:00
Guillaume Abrioux
f8e22fb3da qa/nvme_loop: fix an issue on ubuntu 18.04
The following command:

```
echo /dev/sda | tee /sys/kernel/config/nvmet/subsystems/sda/namespaces/1/device_path
```

makes nvme_loop fail because fascinatingly, it adds an unexpected newline.

See:
```
/dev/sda
/dev/sda

1
tee: /sys/kernel/config/nvmet/subsystems/sda/namespaces/1/enable: No such file or directory
/dev/sda
1
```

Other distros don't have the same behavior:

```
CentOS 8
/dev/sda
/dev/sda
1

Ubuntu 20.04
/dev/sda
/dev/sda
1
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-17 17:10:08 +01:00
Guillaume Abrioux
3c93ffdc92 ceph-volume: fix regression introcuded via #43536
The recent changes from PR #43536 introduced a regeression preventing from
running ceph-volume in a containerized context on Ubuntu 18.04.

Given that the path for the binary `lvs` differs between CentOS 8 and Ubuntu 18.04.
(`/usr/sbin/lvs` and `/sbin/lvs` respictively). It means that ceph-volume running
in the container on CentOS 8 sees the `lvs` binary at `/usr/sbin/lvs` and try to
run it with `nsenter` on the host which is running Ubuntu 18.04.

Fixes: https://tracker.ceph.com/issues/53812

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 95e88cda3df76b59b548ae808df0ef7f19db1f63)
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-17 17:10:03 +01:00
Sebastian Wagner
d05bd30436
Merge pull request #44566 from falcon78921/minor-messaging-nit
mgr/cephadm: fix minor grammar nit in Dry-Runs message
2022-01-17 16:47:53 +01:00
Avan Thakkar
ed2b4e7a56 mgr/dashboard: report ceph tracker bug/feature through GUI
Fixes: https://tracker.ceph.com/issues/44851
Signed-off-by: Shreya Sharma <shreyasharma.ss305@gmail.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-01-17 19:45:31 +05:30
Sebastian Wagner
9e8b2d8282
Merge pull request #44510 from rzarzynski/wip-cephadm-docfix
doc/cephadm: improve the development doc a bit

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2022-01-17 10:21:45 +01:00
Sebastian Wagner
18528ca3f3
Merge pull request #44485 from adk3798/agent-permissions
cephadm: fix permissions on agent files

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2022-01-17 09:40:13 +01:00
Sebastian Wagner
5ff602e5cf
Merge pull request #44506 from sebastian-philipp/orch-suite-add-scsi
qa/suites/orch/cephadm: Also run the rbd/iscsi suite

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Melissa Li <mingkli@redhat.com>
2022-01-17 09:39:50 +01:00
Kotresh HR
7c0d31e52c qa: Add tests snapshot clone failure with quota
Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-01-17 12:14:33 +05:30
Kotresh HR
18b85c53af mgr/volumes: Fix subvoume snapshot clone failure
Problem:
The subvolume snapshot clone fails if the quota on the source
has exceeded. Since the quota is not strictly enforced at the
byte range, this is a possibility.

Cause:
The quota on the clone is set prior to copying the data
from the source. Hence the quota mostly get enforced before
copying the entire data from the source resulting in the
clone failure.

Solution:
Enforce quota on the clone after the data is copied.

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-01-17 12:14:20 +05:30
Avan Thakkar
a6c633d09d mgr/dashboard: include autopep8 for dashboard qa tasks
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-01-17 02:31:51 +05:30
Casey Bodley
7e03511078
Merge pull request #44603 from cbodley/wip-cmake-parquet
rgw: disable parquet by default

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2022-01-14 17:48:07 -05:00
Casey Bodley
fb90d0ebdf build: revert arrow package dependency
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-14 14:54:09 -05:00
Casey Bodley
077e234b74 cmake: disable parquet by default
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-14 14:50:47 -05:00
Neha Ojha
a81f08996a doc/foundation.rst: qct is no longer a member
Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-01-14 19:17:58 +00:00
Ernesto Puerta
581e10f9e8
Merge pull request #44523 from ljflores/wip-telemetry-dashboard
mgr/dashboard/telemetry: reduce telemetry dashboard preview size

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2022-01-14 20:11:15 +01:00
Yuri Weinstein
8b1568832d
Merge pull request #44550 from jdurgin/wip-pool-get-quota
mon/OSDMonitor: avoid null dereference if stats are not available

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-01-14 10:46:49 -08:00
Yuri Weinstein
e419a29be5
Merge pull request #42735 from amathuria/wip-amathuria-scrub-stats
osd/scrub: Add stats to PG dump for number of objects scrubbed

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-14 10:46:28 -08:00
Neha Ojha
82ff85871e
Merge pull request #43667 from ifed01/wip-ifed-fix-ram-gridy-fsck
os/bluestore: make shared blob fsck much less RAM-greedy.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2022-01-14 10:27:31 -08:00
Soumya Koduri
0bd3c05c95
Merge pull request #44440 from soumyakoduri/wip-skoduri-dbstore-fixes
rgw/dbstore: Misc fixes
2022-01-14 23:38:22 +05:30
Neha Ojha
ed7fe35439
Merge pull request #44552 from jdurgin/wip-releases-doc
doc/releases: remove outdated info and versions; mark nautilus eol

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-01-14 09:42:08 -08:00
Casey Bodley
e956537ab8 rgw/dbstore: hide dbstore_log.h from rgw_main.cc
dbstore_log.h sets global dout_subsys/dout_prefix macros, and was
leaking into rgw_main.cc through the common/dbstore.h. this caused all
of rgw_main's log output to start with the wrong prefix "rgw dbstore: "

Fixes: https://tracker.ceph.com/issues/53177

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-14 12:34:08 -05:00
Ernesto Puerta
7225b68e46
qa/dashboard: ensure node 16 is installed
For Ubuntu: https://github.com/nodesource/distributions#manual-installation

Fixes: https://tracker.ceph.com/issues/53843
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2022-01-14 18:08:25 +01:00
Yuri Weinstein
755dc55b8e
Merge pull request #44370 from benhanokh/NCB_expand_device_fix
NCB code doesn't update allocation file when we expand-device

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2022-01-14 09:06:41 -08:00
Yuri Weinstein
f5b4f3f4d9
Merge pull request #44251 from yaarith/telemetry-opt-in
mgr/telemetry: introduce new design for varying report data

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-01-14 09:06:11 -08:00
Yuri Weinstein
31ff4958bf
Merge pull request #43849 from rzarzynski/wip-bs-lucky-buffers
blk, os/bluestore: introduce huge page-based read buffers

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2022-01-14 08:44:06 -08:00
Radoslaw Zarzynski
fba106f172
Merge pull request #42576 from AmnonHanuhov/wip-port_rgw_classes
crimson/osd: Port rgw object classes to run in crimson

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-01-14 16:48:42 +01:00
Yuri Weinstein
c8ea20345d
Merge pull request #44518 from gregsfortytwo/wip-fix-53824
osd: PeeringState: fix selection order in calc_replicated_acting_stretch

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-01-14 07:47:00 -08:00
Yingxin Cheng
5586d07be7 crimson/os/seastore: consolidate seastore_journal logs with structured level and format
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-14 23:06:43 +08:00
Yingxin Cheng
13fb5f676c crimson/os/seastore/journal: validate segments before replay
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-14 23:06:43 +08:00