Commit Graph

140451 Commits

Author SHA1 Message Date
Venky Shankar
a728d170cc qa: use exisitng ignorelist override list for fs:mirror[-ha]
The custom ignorelist yaml would also require `POOL_APP_NOT_ENABLED`
to be ignored. The custom yaml is unnecessary anyway.

Fixes: http://tracker.ceph.com/issues/62482
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-09-13 12:04:43 +05:30
Venky Shankar
59d1f7655c Merge PR #53153 into main
* refs/pull/53153/head:
	Revert "mds: align quota.max_bytes to 4MB or 4KB"

Reviewed-by: Xiubo Li <xiubli@redhat.com>
2023-09-13 11:10:59 +05:30
Casey Bodley
2cbd795745
Merge pull request #53370 from wowly/rgw-cache-nginx-if-match-header
examples: fixed errors with the if-match header in the examples of rgw-cache nginx configs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-09-12 19:27:00 +01:00
Zac Dover
cc8dce2531 doc/architecture: edit "Cluster Map"
Edit the section "Cluster Map" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-13 03:15:38 +10:00
Anthony D'Atri
c1ddf1d2fb
Merge pull request #53422 from zdover23/wip-doc-2023-09-12-glossary-primary-affinity
doc/glossary: add "primary affinity" to glossary
2023-09-12 12:00:08 -04:00
Ronen Friedman
210dbd4ff1 osd: cleaning stop_for_fast_shutdown()
Removed unsued variables to prevent compiler warnings.
Protected the shard lock.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-09-12 10:55:48 -05:00
Adam Kupczyk
58636b20e9
Merge pull request #52407 from rhcs-dashboard/osd-op-scraper
os/bluestore: osd scrape read/write operations
2023-09-12 17:47:52 +02:00
Pere Diaz Bou
f3cf1bf5bc os/bluestore: osd scrape read/write operations
To run:
```bash
../src/test/objectstore/allocsim/ops_scraper.py --ceph_bin_path bin/ceph
```

Example:
```bash
$ ../src/test/objectstore/allocsim/ops_scraper.py --ceph_bin_path bin/ceph                                                                           [13:06:39]
collecting from osd. 0
op_type | offset/extent | name | pgid | initiated_at | who
-------------------------------------------------------
write 0~4194304 4:c4b97d2c:::be151868021c187609fb97e4c22ae1f9:head 4.3 2023-07-12T13:06:34.959082+0200 client.4831.0:351
write 0~4194304 4:d392075e:::b068dce954dca8ca56d72e02e87f39c8:head 4.b 2023-07-12T13:06:35.093439+0200 client.4831.0:358
write 0~4194304 4:bc1ac643:::7b815bd25a40695c891dc06bdf2b65be:head 4.1d 2023-07-12T13:06:35.128097+0200 client.4831.0:360
write 0~4194304 4:d0d21bd5:::cc531199dd2707370bb461cce82bdef6:head 4.b 2023-07-12T13:06:36.023082+0200 client.4831.0:367
write 0~4194304 4:a9abc92c:::0703d6f43d14bc33d69f94ecb96c522c:head 4.15 2023-07-12T13:06:36.367886+0200 client.4831.0:369
write 0~4194304 4:9b4684ea:::d5856dc0ecad96f811012e0937d5e0ac:head 4.19 2023-07-12T13:06:36.654584+0200 client.4831.0:370
write 0~4194304 4:d15b8c87:::c3344e5de8c8936876965d9fb056ad89:head 4.b 2023-07-12T13:06:36.835690+0200 client.4831.0:371
write 0~4194304 4:ad41de2a:::8a1e2743b577a67d6bc8a1d514391b53:head 4.15 2023-07-12T13:06:37.102741+0200 client.4831.0:373
write 0~4194304 4:fc35d54f:::39cd961e7e33680f4bfd1ca0fe43dda4:head 4.1f 2023-07-12T13:06:38.047612+0200 client.4831.0:377

collecting from osd. 1
op_type | offset/extent | name | pgid | initiated_at | who
-------------------------------------------------------
write 0~4194304 4:4c9ed2e8:::86206ccc35d8aceaf78670a751a6b550:head 4.12 2023-07-12T13:06:34.771423+0200 client.4831.0:340
write 0~4194304 4:106d9524:::fe7534cc604b0eb50498cb23aaca9161:head 4.8 2023-07-12T13:06:35.128105+0200 client.4831.0:359
write 0~4194304 4:a6e3fdb1:::2692491bfc4c584ca5b1efcb896936c2:head 4.5 2023-07-12T13:06:35.567070+0200 client.4831.0:363
write 0~4194304 4:2d9a471a:::ba51255646d00c496c2638bad03e0869:head 4.14 2023-07-12T13:06:35.568290+0200 client.4831.0:364
write 0~4194304 4:2b1381b1:::c58fa31e5d3a4f5550ee143bcb07a49f:head 4.14 2023-07-12T13:06:35.764699+0200 client.4831.0:365
write 0~4194304 4:a22e158f:::b2116e08700da18595b96d58120fe81c:head 4.5 2023-07-12T13:06:36.023101+0200 client.4831.0:366
```

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
2023-09-12 15:51:49 +02:00
Ali Masarwa
0ecc5f1262
Merge pull request #53419 from AliMasarweh/wip-alimasa-persistent-q-test
RGW | bucket notification: adding more sleeping time to prevent test from failing

Reviewed-by: yuvalif <ylifshit@redhat.com>
2023-09-12 16:51:25 +03:00
Jos Collin
0241a82b4d
qa: fix cephfs-top tests for the new fields
Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-09-12 17:35:30 +05:30
Jos Collin
ce3e61eeb7
cephfs-top: include the missing fields in --dump output
Fixes: https://tracker.ceph.com/issues/61397
Signed-off-by: Jos Collin <jcollin@redhat.com>
2023-09-12 17:35:21 +05:30
Zac Dover
f3a3f10e9c doc/glossary: add "primary affinity" to glossary
Add "Primary Affinity" to the glossary and link to the information in
the RADOS documentation that (more or less) explains it.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-12 20:56:03 +10:00
Ali Masarwa
1f8f0a9e66 RGW | bucket notification: adding more sleeping time to prevent test from failing
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-09-12 13:31:31 +03:00
Pedro Gonzalez Gomez
2c04e94071 mgr/dashboard: align charts of landing page
Fixes: https://tracker.ceph.com/issues/62813
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-09-12 11:28:25 +02:00
Milind Changire
7e2546f945
PendingReleaseNotes: add note about 'm' and 'M' periods
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 14:11:03 +05:30
Milind Changire
7431b813a5
doc/cephfs: add note about monthly scheduling and retention period
Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 14:11:03 +05:30
Milind Changire
9c88234791
qa: update minute period multiplier to m
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 14:11:03 +05:30
Milind Changire
e7218e03ae
qa: change minute duration specifier from M to m
Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 13:58:26 +05:30
Milind Changire
2f6d87d181
mgr/snap_schedule: change minute duration specifier from M to m
Problem:
As per the issue tracker, the period spec specifier 'M' is not
consistent with what is used elsewhere, like the period specifiers
displayed in the 'ceph status' command output.
The 'M' period specifier is used as a 'minute' level period specifier by
the cephfs team.
The issue reporter suggests to use 'M' as a 'month' period specifier.

Solution:
Since the 'minute' level period specifer, 'M', is used internally by
the development team, it is failrly easy to swap the 'minute' ('M')
level and 'month' ('m') level period specifers to finally mean that 'm'
implies 'minute' level period and 'M' implies 'month' level period.
Also, since this is the first time that somebody has ever reported that
neither the 'M' nor the 'm' level specifiers work in production, it is a
good idea to fix them once and for all.

Fixes: https://tracker.ceph.com/issues/62494
Signed-off-by: Milind Changire <mchangir@redhat.com>
2023-09-12 13:58:21 +05:30
Rishabh Dave
7d1f6d70f1 qa/cephfs: set omit_sudo for cmd mounting FUSE conn dir
vstart_runner.py removes sudo from the command arguments when "omit_sudo"
is not explicitly set to "False". This leads to 2 error messages in output
of vstart_runner.py which are copied at the end of this commit message.
Setting "omit_sudo=False" for the execution of command should remove both
of these error messages.

This also fixes the potential crash of vstart_runner.py when the FUSE
connections directory is not mounted on the machine and it requires
superuser privileges to mount it.

The 2 error messages copied as it is from the log -

DEBUG:__main__:"sudo" was omitted from the following cmd args before execution and logging using function overriding; check vstart_runner.py for more details.
DEBUG:__main__:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections
mount: /sys/fs/fuse/connections: must be superuser to use mount.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-12 13:26:00 +05:30
Rishabh Dave
c4b2ccd9d6 qa/cephfs: check before mounting fuse connections directory
Check if "/sys/fs/fuse/connections" is already mounted before attempting
to mount it again. Doing so leads to unnecessary error messages in the
vstart_runner.py's output (and most likely in logs for teuthology test
runs too) which confuses the users. See end for the error message.

Also, remove "check_status=False" so that the when the commands crashes
the testing is halted immediately. Initially, this must've been added to
ignore errors generated from re-mounting FUSE connections directory
repeatedly.

The error message -

DEBUG:__main__:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections
mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-12 13:25:45 +05:30
Roy Sahar
b690343128
Merge pull request #53408 from roysahar/nvmeof_fix_omap_state_object_contains_None
nvmeof-gw: omap object name contains None as string due to generated template to ceph-nvmeof.conf
2023-09-12 10:43:24 +03:00
Liu-Chunmei
17df35e9a7
Merge pull request #53232 from myoungwon/wip-enable-rbm-tests
test/crimson/seastore/rbm: add sub-tests regarding RBM to the existing tests

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Liu-Chunmei <chunmei.liu@intel.com>
2023-09-11 23:25:24 -07:00
Myoungwon Oh
73de8937f6 crimson/os/seastore/object_data_handler: consider a RBM case when checking if write can be merged
RBM's paddr always indicates physical address, which means it doesn't have the dealayed.
So, this commit adds a condition that checks if given paddr is used for ongoing write.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-09-12 12:17:47 +09:00
Myoungwon Oh
36982b6657 crimson/os/seastore/cbj: fix a potential overflow bug
CircularBoundedJournal is similar to circular queue, so segment_seq
increases after rolling. However, current implementation always
increases segment_seq_t when rolling occurs, resulting in
the overflow if segment_seq_t hits MAX_SEG_SEQ.

To mitigate this, this commit changes the type of the segment_seq_t
to uint64_t.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-09-12 03:16:04 +00:00
Xiubo Li
5b9df823a2 mds: continue linking if targeti is temporarily located in stray dir
If one inode has more than one hardlink and after the primary dentry
is unlinked it will located the inode in the stray dir temporarily,
which is pending reintegration.

Just before the linkmerge/migrate is triggered a link request comes
it will fail with -EXDEV.

Just skip it and continue the linking.

Fixes: https://tracker.ceph.com/issues/56695
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-12 11:10:18 +08:00
Xiubo Li
8e240303f8 Revert "mds: wait unlink to finish to avoid conflict when creating same dentries"
This reverts commit d4b9431dfe.

Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-12 11:10:18 +08:00
Xiubo Li
4ec2b03f45 Revert "mds: clear the STATE_UNLINKING state when the unlink fails"
This reverts commit 417f24782c.

Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-12 11:10:18 +08:00
Xiubo Li
b0b279aeb7 Revert "mds: wait reintegrate to finish when unlinking"
This reverts commit c3b36723de.

Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-12 11:10:18 +08:00
Xiubo Li
84c7da723b Revert "mds: notify the waiters in replica MDSs"
This reverts commit 478db14a65.

Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-12 11:10:18 +08:00
Xiubo Li
a9dd5df2fe Revert "mds: wait the linkmerge/migrate to finish after unlink"
This reverts commit 48f9a8990b.

Fixes: https://tracker.ceph.com/issues/61818
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2023-09-12 11:10:18 +08:00
Yingxin
eb42e21d79
Merge pull request #53305 from xxhdx1985126/wip-seastore-onode-erase-replay
crimson/os/seastore/onode_manager: populate value recorders of onodes to be erased

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-09-12 09:21:29 +08:00
Roy Sahar
ab27f934a1 nvmeof-gw: omap object name contains None due to generated template contains string None
Signed-off-by: Roy Sahar <royswi@gmail.com>
2023-09-12 01:52:53 +03:00
Laura Flores
c0183c76d7
Merge pull request #53344 from ljflores/wip-tracker-62761
common: add CephContext parameter to tracing::Tracer::init() in !HAVE_JAEGER branch
2023-09-11 18:22:03 -04:00
John Mulligan
d22a9d72d0 cephadm: convert _write_iscsi_unit_poststop_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:05 -04:00
John Mulligan
3ed7a48c7e cephadm: convert _write_osd_unit_poststop_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:05 -04:00
John Mulligan
afd71cb291 cephadm: convert _write_iscsi_unit_run_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
7e2be93a11 cephadm: convert _write_osd_unit_run_commands to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
1b22616aec cephadm: convert _dispatch_deploy to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
962e706359 cephadm: convert get_deployment_type to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
9011b69f31 cephadm: convert get_deployment_container to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
dca0d0a794 cephadm: convert get_container to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
04423ccd2d cephadm: add daemon_name property to DaemonIdentity
Add a convenience property to DaemonIdentity and DaemonSubIdentity that
produces a name in the ceph daemon naming convention: <type>.<id>.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
a7458053a4 cephadm: convert deploy_daemon to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
be96c64bc9 cephadm: convert deploy_daemon_units to to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
9e5ce87614 cephadm: convert get_daemon_args to use DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
c2b877fc25 cephadm: assert that DaemonIdentity has valid values
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:23:04 -04:00
John Mulligan
3b2c2dc1f8 cephadm: convert get_container_mounts to use a DaemonIdentity argument
Convert get_container_mounts to use a DaemonIdentity argument, and in
order to do so: split get_container_mounts_for_type out of
get_container_mounts. get_container_mounts will take a DaemonIdentity
and safely assume that the fsid, daemon_type, daemon_id triple is
available whereas get_container_mounts_for_type has the functionality
of only requiring the fsid and daemon_type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:22:22 -04:00
John Mulligan
b55ff25129 cephadm: convert get_container_binds to use a DaemonIdentity argument
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:09:02 -04:00
John Mulligan
b72d13af6f cephadm: convert _write_custom_conf_files to use a DaemonIdentity arg
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-09-11 15:08:53 -04:00