Commit Graph

129450 Commits

Author SHA1 Message Date
Casey Bodley
e96ae2f363
Merge pull request #44078 from cbodley/wip-rgw-multisite-metadata-retry-error
rgw/multisite: metadata sync only retries on errors

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-01-18 13:18:46 -05:00
Neha Ojha
3e6b8ef30f
Merge pull request #44602 from neha-ojha/wip-qct-remove
doc/foundation.rst: qct is no longer a member

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
2022-01-18 09:17:59 -08:00
myoungwon oh
30d722e861 seastore/seastore_types: fix wrong masking
Fix bit operation due to device_id_t is 8 bit for now

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-19 01:32:42 +09:00
myoungwon oh
fa365d84d1 seastore/random_block_manager: use paddr_t instead of uint64
Reagarding read() and write(), use paddr_t as input parameter
instead of uint64

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-19 01:32:35 +09:00
myoungwon oh
d6f3a29da9 seatore/random_block_manager: remove blocks_per_segment
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-19 01:32:28 +09:00
myoungwon oh
2745fd1ab1 seastore: changes due to adding blk_paddr_t
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-19 01:32:18 +09:00
myoungwon oh
374c60e2a0 seastore: add blk_paddr_t
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-19 01:30:10 +09:00
Nizamudeen A
b6759b75c9 mgr/dashboard: Refactoring dashboard cephadm checks
I isolated all the tests suites into there respective files
so that in future it is easier to add more tests to it.

I also given priority to the host actions.

Create OSD checks are now written in a way that OSDs
are created only on the intended hosts. This will make
the host draining process easier and less time consuming.

Also tried to address the flaky force maintenance checks.

Removed some duplicated codes

Service creation part improved to reduce the time taken
for its completion

Fixes: https://tracker.ceph.com/issues/53905
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-01-18 21:45:26 +05:30
Sebastian Wagner
24aab16cd0
Merge pull request #44505 from guits/fix-53812
ceph-volume: fix regression introcuded via #43536

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Teoman Onay <tonay@redhat.com>
2022-01-18 16:31:49 +01:00
Sebastian Wagner
7842825a2e
Merge pull request #44489 from adk3798/agent-down-alerts
mgr/cephadm: still check agent deps if it is marked down
2022-01-18 16:23:40 +01:00
Pere Diaz Bou
57c26311de monitoring/grafana: replace filestore osd count
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 14:14:41 +01:00
Pere Diaz Bou
a3cf5c5e9f monitoring/grafana: use Path class instead of split
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
1e4d85d04f monitoring/grafana: remove explicit str casting
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
2b4f3561d2 monitoring/grafana: add generated json files
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
b381a83e9b monitoring/grafana: ValueError instead of RuntimeError
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:12 +01:00
Pere Diaz Bou
4c302234ff monitoring/grafana: Replace missing legendFormat warning with error
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-01-18 13:24:10 +01:00
Guillaume Abrioux
5c0f0698a5 qa/cephadm: install hwe kernel only for focal
Let's install hwe kernel only on Ubuntu focal, otherwise we only shift the
problem on Ubuntu bionic given that the hwe kernel for bionic is 5.4.

Fixes: https://tracker.ceph.com/issues/53863

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-18 10:36:24 +01:00
myoungwon oh
c93d0b7b1e seastore: rename segment_off_t to seastore_off_t
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-01-18 18:24:43 +09:00
cao.leilc
362bc473d1 rgw : add error message when rm key
Fixes: https://tracker.ceph.com/issues/53725
Signed-off-by: caolei <halei15848934852@163.com>
2022-01-18 15:03:44 +08:00
Samuel Just
51a347456d
Merge pull request #44591 from athanatos/sjust/wip-seastore-flush
crimson/os/seastore: avoid empty Transactions by adding explicit flush() call

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-01-17 19:36:10 -08:00
Samuel Just
7370d7ded8
Merge pull request #44556 from cyx1231st/wip-crimson-improve-log-journal
crimson/os/seastore: consolidate seastore_journal logs with cleanup and validations

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-01-17 13:19:02 -08:00
Casey Bodley
3b93654d6e rgw: clean up index after full metadata sync
Fixes: https://tracker.ceph.com/issues/40177

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 15:53:26 -05:00
Casey Bodley
dd6bf0b5a8 rgw: clean up index after full data sync
Fixes: https://tracker.ceph.com/issues/40177

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 15:53:26 -05:00
Samuel Just
7c4b3cc7fa crimson/os/seastore: implement FuturizedStore::flush
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-01-17 20:50:58 +00:00
Casey Bodley
a6611a55cc rgw/swift: don't crash on nonexistent bucket in BulkUpload
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 14:59:01 -05:00
Waad AlKhoury
ea55a0b33d mgr/dashboard: Notification banners at the top of the UI have fixed height
Fixes: https://tracker.ceph.com/issues/51575
Signed-off-by: Waad AlKhoury <walkhour@redhat.com>
2022-01-17 20:45:58 +01:00
Casey Bodley
52bfa9a866 qa/rgw: run multisite tests with some async notifications disabled
disable the sending of async datalog notifications on one zone per
cluster. this helps to verify that tests don't rely on notifications to
succeed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 13:54:39 -05:00
Casey Bodley
bf0a4ef1aa rgw: allow rgw_data_notify_interval_msec=0 to disable notifications
the data changes log for multisite will occasionally broadcast recent
changes to other zones, which they can use to prioritize sync of some
of the most recent changes. they'll eventually see all changes as they
replay the data changes log, though, so notifications aren't required
for successful sync. the ability to turn them off is useful for testing

Fixes: https://tracker.ceph.com/issues/49723

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-17 13:54:36 -05:00
Mykola Golub
078d72e5e6 cls/journal: skip disconnected clients when finding min_commit_position
When a new journal client is registered, all already registered
clients are checked, and a client with min position is selected
as a position for the new client. Thus we may expect that
starting from the registered position all journal entries will be
available (not trimmed) for the new client.

But when looking for a min commit position, the client_register
function did not take into account that a registered client might
be in disconnected state, and in that case the journal entries
might be trimmed for this client.

Fixes: https://tracker.ceph.com/issues/53888
Signed-off-by: Mykola Golub <mgolub@suse.com>
2022-01-17 18:41:34 +00:00
Yuri Weinstein
095f970710 qa/tests: run upgrades on quincy
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2022-01-17 10:18:04 -08:00
Guillaume Abrioux
f8e22fb3da qa/nvme_loop: fix an issue on ubuntu 18.04
The following command:

```
echo /dev/sda | tee /sys/kernel/config/nvmet/subsystems/sda/namespaces/1/device_path
```

makes nvme_loop fail because fascinatingly, it adds an unexpected newline.

See:
```
/dev/sda
/dev/sda

1
tee: /sys/kernel/config/nvmet/subsystems/sda/namespaces/1/enable: No such file or directory
/dev/sda
1
```

Other distros don't have the same behavior:

```
CentOS 8
/dev/sda
/dev/sda
1

Ubuntu 20.04
/dev/sda
/dev/sda
1
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-17 17:10:08 +01:00
Guillaume Abrioux
3c93ffdc92 ceph-volume: fix regression introcuded via #43536
The recent changes from PR #43536 introduced a regeression preventing from
running ceph-volume in a containerized context on Ubuntu 18.04.

Given that the path for the binary `lvs` differs between CentOS 8 and Ubuntu 18.04.
(`/usr/sbin/lvs` and `/sbin/lvs` respictively). It means that ceph-volume running
in the container on CentOS 8 sees the `lvs` binary at `/usr/sbin/lvs` and try to
run it with `nsenter` on the host which is running Ubuntu 18.04.

Fixes: https://tracker.ceph.com/issues/53812

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 95e88cda3df76b59b548ae808df0ef7f19db1f63)
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-17 17:10:03 +01:00
Sebastian Wagner
d05bd30436
Merge pull request #44566 from falcon78921/minor-messaging-nit
mgr/cephadm: fix minor grammar nit in Dry-Runs message
2022-01-17 16:47:53 +01:00
Avan Thakkar
ed2b4e7a56 mgr/dashboard: report ceph tracker bug/feature through GUI
Fixes: https://tracker.ceph.com/issues/44851
Signed-off-by: Shreya Sharma <shreyasharma.ss305@gmail.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-01-17 19:45:31 +05:30
Sebastian Wagner
9e8b2d8282
Merge pull request #44510 from rzarzynski/wip-cephadm-docfix
doc/cephadm: improve the development doc a bit

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2022-01-17 10:21:45 +01:00
Sebastian Wagner
18528ca3f3
Merge pull request #44485 from adk3798/agent-permissions
cephadm: fix permissions on agent files

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2022-01-17 09:40:13 +01:00
Sebastian Wagner
5ff602e5cf
Merge pull request #44506 from sebastian-philipp/orch-suite-add-scsi
qa/suites/orch/cephadm: Also run the rbd/iscsi suite

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Melissa Li <mingkli@redhat.com>
2022-01-17 09:39:50 +01:00
Kotresh HR
7c0d31e52c qa: Add tests snapshot clone failure with quota
Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-01-17 12:14:33 +05:30
Kotresh HR
18b85c53af mgr/volumes: Fix subvoume snapshot clone failure
Problem:
The subvolume snapshot clone fails if the quota on the source
has exceeded. Since the quota is not strictly enforced at the
byte range, this is a possibility.

Cause:
The quota on the clone is set prior to copying the data
from the source. Hence the quota mostly get enforced before
copying the entire data from the source resulting in the
clone failure.

Solution:
Enforce quota on the clone after the data is copied.

Fixes: https://tracker.ceph.com/issues/53848
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2022-01-17 12:14:20 +05:30
Xuehan Xu
068c5e7ff1 librgw: move RGWFileHandle::encode/decode to the private sector
To prevent RGWFileHandle::encode/decode methods to be invoked directly by
other modules

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2022-01-17 11:17:28 +08:00
Xuehan Xu
49a35d72e0 librgw: make rgw file handle versioned
The reason that we need this is that there could be the following scenario:

1. rgw_setattr sets the file attr;
2. rgw_write writes some new data, and encodes its attr to store into rados;
3. before the actual persistence of the file's attr bl, rgw_lookup loads the file's
   previous attr and modifies the current file handle's metadata;
4. rgw_write's result persisted to rados;
5. rgw_setattr set the current file handle's metadata which is actually an old one to rados

In this case, the attr in rados would be out of date which means loss of data

Fixes: https://tracker.ceph.com/issues/50194
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2022-01-17 11:17:24 +08:00
Avan Thakkar
a6c633d09d mgr/dashboard: include autopep8 for dashboard qa tasks
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-01-17 02:31:51 +05:30
Casey Bodley
7e03511078
Merge pull request #44603 from cbodley/wip-cmake-parquet
rgw: disable parquet by default

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2022-01-14 17:48:07 -05:00
Casey Bodley
fb90d0ebdf build: revert arrow package dependency
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-14 14:54:09 -05:00
Casey Bodley
077e234b74 cmake: disable parquet by default
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-01-14 14:50:47 -05:00
Neha Ojha
a81f08996a doc/foundation.rst: qct is no longer a member
Signed-off-by: Neha Ojha <nojha@redhat.com>
2022-01-14 19:17:58 +00:00
Ernesto Puerta
581e10f9e8
Merge pull request #44523 from ljflores/wip-telemetry-dashboard
mgr/dashboard/telemetry: reduce telemetry dashboard preview size

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2022-01-14 20:11:15 +01:00
Yuri Weinstein
8b1568832d
Merge pull request #44550 from jdurgin/wip-pool-get-quota
mon/OSDMonitor: avoid null dereference if stats are not available

Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-01-14 10:46:49 -08:00
Yuri Weinstein
e419a29be5
Merge pull request #42735 from amathuria/wip-amathuria-scrub-stats
osd/scrub: Add stats to PG dump for number of objects scrubbed

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2022-01-14 10:46:28 -08:00
Neha Ojha
82ff85871e
Merge pull request #43667 from ifed01/wip-ifed-fix-ram-gridy-fsck
os/bluestore: make shared blob fsck much less RAM-greedy.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2022-01-14 10:27:31 -08:00