1
0
mirror of https://github.com/ceph/ceph synced 2025-03-11 02:39:05 +00:00
Commit Graph

142528 Commits

Author SHA1 Message Date
Yuri Weinstein
02805817e8
Merge pull request from kamoltat/wip-ksirivad-fix-tracker-59564
mon/ConnectionTracker.cc: disregard connection scores from mon_rank = -1 

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2024-01-05 07:38:53 -08:00
Yuri Weinstein
556bd56b70
Merge pull request from amathuria/wip-add-test-case-bz-2011756
DaemonServer.cc: fix config show command for RGW daemons

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-05 07:37:54 -08:00
Ronen Friedman
6d6cc71456 osd/scrub: avoid "over clearing" queued_or_active flag
If two StartScrub messages are received in quick succession, the earlier
one might clear the queued_or_active flag as it fails for being from an old
interval.
When that happens - a 3'rd scrub request will actually be allowed to go
through, while the scrubber is still handling the second one.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-05 09:00:10 -06:00
Ronen Friedman
569c07f979 osd/scrub: manage queue registration lifetime in the FSM
As the state of 'being registered in the OSDs scrub queue'
corresponds to the PrimaryActive FSM state.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-05 08:44:21 -06:00
Anthony D'Atri
c6bca26183
Merge pull request from zdover23/wip-doc-2024-01-05-radosgw-admin-add-remove-admin-caps
doc/radosrgw: edit admin.rst
2024-01-05 09:32:54 -05:00
Zac Dover
1f0ff27898 doc/radosrgw: edit admin.rst
Edit "Add / Remove Admin Capabilities" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-05 21:11:59 +10:00
Myoungwon Oh
72c9d6d17f crimson/os/seastore/cache: return false explicitly if delta is not replayed
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:46:43 +09:00
Myoungwon Oh
ad8b3f29ad crimson/os/seastore: check correct crc for inplace-rewritten extents after replay is done
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:44:44 +09:00
Myoungwon Oh
cba051d6e0 test/crimson/seastore: fix overflow issue and make len > 0 when set_contents is called
set_contents causes the overflow at times because
alloc_extent is allowed to use uint32_t.

Specifically, in random_writes case, PADDING_SIZE is 256<<10,
whereas set_contents's len is uint16_t.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:44:37 +09:00
Myoungwon Oh
6d42ff4c7e crimson/os/seastore/epm: detect transaction confict during rbm write using OOL
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:44:31 +09:00
Myoungwon Oh
3ca63d79b8 crimson/os/seastore: introduce modified_region in DATA_BLOCK to keep track of modified region
It has a limitation to keep track of the modified region using the existing
deltas because we can not get the correct region in two cases: 1) a case where replay
is done and 2) duplicate_for_write. This commit introduces modified region
to solve the problem.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:44:25 +09:00
Myoungwon Oh
5307f252d6 crimson/os/seastore: write only modified region when rewriting the extent in an inplace manner
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:44:18 +09:00
myoungwon oh
78dc32bada test/crimson/seastore: add a random overwrite test to test_objec_date_handler
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:44:12 +09:00
Myoungwon Oh
1b29691318 crimson/os/seastore/cache: reset version to 1 if inplace write occurs before in prepare_record
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-01-05 15:43:55 +09:00
Matt Benjamin
2bcc413346
Merge pull request from linuxbox2/wip-lcnotify-cleanup
rgwlc:  additional lc notify cleanups
2024-01-04 21:37:24 -05:00
Adam King
1464da9cf0
Merge pull request from phlogistonjohn/jjm-cephadm-build-with-wheels
cephadm: build with wheels, enabling pyyaml dependency

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-04 20:12:30 -05:00
Adam King
cd3c69782a
Merge pull request from phlogistonjohn/jjm-cephadm-sidecars
cephadm: add support for sidecar containers

Reviewed-by: Adam King <adking@redhat.com>
2024-01-04 20:11:41 -05:00
Matt Benjamin
8a935bb115 rgwlc: even current object versions have a unique instance
Fixes: https://tracker.ceph.com/issues/63546

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-01-04 15:23:36 -05:00
Matt Benjamin
9183de7309 rgwlc: additional lc notify cleanups
Follow the same formula to build up obj_state and version_id
in all call sites.

Resolves: rhbz#2163667

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-01-04 15:23:27 -05:00
Casey Bodley
6704c04a6a
Merge pull request from cbodley/wip-rgw-polymorphic-executor
rgw/beast: switch back to polymorphic executor

Reviewed-by: Mark Kogan <mkogan@ibm.com>
2024-01-04 20:19:54 +00:00
Casey Bodley
fe747f40fa
Merge pull request from cbodley/wip-63829
radosgw-admin: 'zone set' won't overwrite existing default-placement

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2024-01-04 20:19:08 +00:00
Casey Bodley
6787c4dd08
Merge pull request from adamemerson/wip-lazyfifo-no-mutex-over-coro
rgw/multisite: Fix deadlock by not holding mutex over coroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-01-04 20:18:38 +00:00
Samuel Just
d451898573
Merge pull request from Matan-B/wip-matanb-crimson-calc-subsets
crimson: Support Partial Object Recovery

Reviewed-by: Samuel Just <sjust@redhat.com>
2024-01-04 10:13:40 -08:00
Yuval Lifshitz
88a0e91a7a
Merge pull request from yuvalif/wip-yuval-63901
rgw/kafka: make the connection idle timeout configurable

reviwed-by: cbodley
2024-01-04 19:19:43 +02:00
Yuval Lifshitz
a3682b8f5a
Merge pull request from yuvalif/wip-yuval-63859
rgw/notifications/lc: handle publish_commit() failures as warnings

reviewed-by: dang, thotz
2024-01-04 19:19:01 +02:00
Yuval Lifshitz
b37839502e rgw/notifications: dont send notifications if op has failed
Fixes: https://tracker.ceph.com/issues/63855

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-01-04 15:56:12 +00:00
Matan Breizman
b5df21f556 crimson/osd: Keep track of modified_ranges
* `modifies_ranges` interval_set is added to osd_op_params_t
* keep track of modified_ranges while executing relevant ops
* Add `osd_op_params` parameter to `PGBackend::remove()`.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:43:57 +00:00
Aishwarya Mathuria
964e5a7121 src/crimson: Add support for the OSD to receive config changes
1. Adds MMgrReport message in manager client
2. MonClient subscribes to config

Fixes: https://tracker.ceph.com/issues/59241
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2024-01-04 15:39:34 +00:00
Matan Breizman
1367490115 crimson/osd/replicated_recovery_backend: handle_pull_response() - recalc_subsets()
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
0cbe98226f mson/osd/replicated_recovery_backend: Introduce prep_push_to_repli…
…ca()

Intelligently push an object to a replica.  make use of existing
clones/heads and dup data ranges where possible.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
869fc82a74 crimson/osd/replicated_recovery_backend: seperate prep_push_to_replica and prep_push
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
09b638c8db rename prep_push to prep_push_to_replica
dismiss head_ctx

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
35e14f8ff8 crimson/osd/replicated_recovery_backend: prepare_pull to support calc_clone_subsets
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
883f99dd3e crimson/osd/object_metadata_helper: Introduce calc_*_subsets
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Casey Bodley
1e905d2c41 qa/rgw/upgrade: add reef as a starting release
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-01-04 09:37:56 -05:00
Ronen Friedman
d49b2612af
Merge pull request from ronen-fr/wip-rf-crush-dummy
test/crush: removed unused code

Reviewed-by: Laura Flores <lflores@redhat.com>
2024-01-04 14:51:57 +02:00
Leonid S. Usov
22715d4e00
Merge pull request from leonid-s-usov/vstart-localhost-for-real
vstart.sh: make sure that --localhost does only bind to 127.0.0.1 in all cases
2024-01-04 14:15:35 +02:00
Rongqi Sun
bfa87ba85e
Merge pull request from Svelar/memstore
os/memstore: deep copy on partial bufferptrs on setattrs
2024-01-04 14:44:06 +08:00
Rongqi Sun
5f3b7c6bea
Merge pull request from Svelar/memory_is_zero_dev
inline_memory: optimized mem_is_zero for aarch64 by neon intrinsic
2024-01-04 14:43:44 +08:00
zdover23
bcbb68ac36
Merge pull request from zdover23/wip-doc-2024-01-03-radosgw-admin-add-remove-a-key
doc/radosgw: edit "Add/Remove a Key"

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-01-04 12:33:32 +10:00
Samuel Just
066953050c
Merge pull request from athanatos/sjust/wip-crimson-scrub
crimson: add initial scrub support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-01-03 15:26:45 -08:00
Anthony D'Atri
3f5a709263
Merge pull request from zdover23/wip-doc-2023-01-04-release-note-cleanup-following-55049
doc/releases: edit reef.rst
2024-01-03 16:32:37 -05:00
Laura Flores
c047daf730
Merge pull request from ceph/wip-lflores-upgrade-stale-bot
.github/workflows: upgrade stale bot
2024-01-03 11:35:53 -06:00
Laura Flores
159751b680 doc/rados/operations: document ceph balancer status detail
Document change in https://github.com/ceph/ceph/pull/54801

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-03 17:33:32 +00:00
John Mulligan
6e0af1d00d cepahdm: add some specific tests for daemon id systemd name generation
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-03 09:47:47 -05:00
Zac Dover
f62e93cbe7 doc/radosgw: edit "Add/Remove a Key"
Edit the section "Add/Remove a Key" in doc/radosgw/admin.rst. Each
operation (e.g. "Adding an S3 key pair for a user", "Removing an S3 key
pair for a user") now has its own subsection. This increased granularity
should make it easier in the future to link to each of these specific
operations, if needed.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-04 00:29:21 +10:00
Zac Dover
d02178fdbd doc/releases: edit reef.rst
Make minor corrections to doc/releases/reef.rst. These corrections were
suggested by Anthony D'Atri in https://github.com/ceph/ceph/pull/55049.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-04 00:20:51 +10:00
Adam King
e917d66838 mgr/cephadm: make default command timeout field an int
When the fields was "secs" instead, we could hit

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1380, in _run_cephadm_json
    out, err, code = await self._run_cephadm(
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1525, in _run_cephadm
    raise OrchestratorError(
orchestrator._interface.OrchestratorError: cephadm exited with an error code: 2, stderr: usage: cephadm
       [-h] [--image IMAGE] [--docker] [--data-dir DATA_DIR]
       [--log-dir LOG_DIR] [--logrotate-dir LOGROTATE_DIR]
       [--sysctl-dir SYSCTL_DIR] [--unit-dir UNIT_DIR] [--verbose]
       [--timeout TIMEOUT] [--retry RETRY] [--env ENV] [--no-container-init]
       [--no-cgroups-split]
       {version,pull,inspect-image, . . .
       ...
cephadm: error: argument --timeout: invalid int value: '295.0'

where the value ends up as a floating point value
after converting to a string (which is necessary to actually
pass it to the binary). By setting the field to be an
int, we should be able to avoid this.

Signed-off-by: Adam King <adking@redhat.com>
2024-01-03 08:42:38 -05:00
Adam King
efb69ee350 qa/cephadm: add test for cephadm asyncio based timeout
Adds a test that will set the default cephadm command
timeout and then force a timeout to occur by holding
the cephadm lock and triggering a device refresh.
This works because cephadm ceph-volume commands
require the cephadm lock to run, so the command will
timeout waiting for the lock to become available.

Signed-off-by: Adam King <adking@redhat.com>
2024-01-03 08:40:47 -05:00
Adam King
4df348c17b mgr/cephadm: Also catch concurrent.futures.TimeoutError for timeouts
On python 3.6 which Ceph currently uses for its
container builds (which are based on centos 8 stream builds
hence the python version) the exception raised by a timeout
from a concurrent.futures.Future is successfully caught by
looking for asyncio.TimeoutError. However, in builds with
later python versions, e.g. 3.9.16, the timeout is no
longer caught. This results in situations like

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/utils.py", line 79, in do_work
    return f(*arg)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 241, in refresh
    r = self._refresh_host_devices(host)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 352, in _refresh_host_devices
    devices = self.mgr.wait_async(self._run_cephadm_json(
  File "/usr/share/ceph/mgr/cephadm/module.py", line 635, in wait_async
    return self.event_loop.get_result(coro, timeout)
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 63, in get_result
    return future.result(timeout)
  File "/lib64/python3.9/concurrent/futures/_base.py", line 448, in result
    raise TimeoutError()
concurrent.futures._base.TimeoutError

which causes the cephadm module to crash whenever one of these
command timeouts happen. This patch is to also catch the
newer exception type so it works on later python versions as well

Signed-off-by: Adam King <adking@redhat.com>
2024-01-03 08:40:47 -05:00