Commit Graph

142257 Commits

Author SHA1 Message Date
Prashant D
950aa4963d mon: initialize ProgressEvent::add_to_ceph_s
The add_to_ceph_s field was not initialized causing unit
test failures from check-generated.sh:

**** ProgressEvent test 1 dump_json check failed ****
   ceph-dencoder type ProgressEvent select_test 1 dump_json > /tmp/typ-akwRrjNtt
   ceph-dencoder type ProgressEvent select_test 1 encode decode dump_json > /tmp/typ-iDmGaq17A
4c4
<     "add_to_ceph_s": false
---
>     "add_to_ceph_s": true

Fixes: https://tracker.ceph.com/issues/63950

Signed-off-by: Prashant D <pdhange@redhat.com>
2024-01-05 17:31:11 -05:00
Laura Flores
d9686b8f77
Merge pull request #54993 from ljflores/wip-balancer-status-detail-documentation 2024-01-05 12:30:02 -06:00
Yuri Weinstein
b67b33500a
Merge pull request #54890 from kamoltat/wip-ksirivad-fix-63389
src/mon/OSDMonitor.cc: more descriptive loggings for crc mismatch

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-01-05 07:40:50 -08:00
Yuri Weinstein
5e54413c0b
Merge pull request #54708 from amathuria/wip-amathuria-bz-2134786-fix-norecover
mgr/pg_autoscaler: add check for norecover flag

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2024-01-05 07:40:11 -08:00
Yuri Weinstein
02805817e8
Merge pull request #52380 from kamoltat/wip-ksirivad-fix-tracker-59564
mon/ConnectionTracker.cc: disregard connection scores from mon_rank = -1 

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2024-01-05 07:38:53 -08:00
Yuri Weinstein
556bd56b70
Merge pull request #48175 from amathuria/wip-add-test-case-bz-2011756
DaemonServer.cc: fix config show command for RGW daemons

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-05 07:37:54 -08:00
Anthony D'Atri
c6bca26183
Merge pull request #55065 from zdover23/wip-doc-2024-01-05-radosgw-admin-add-remove-admin-caps
doc/radosrgw: edit admin.rst
2024-01-05 09:32:54 -05:00
Zac Dover
1f0ff27898 doc/radosrgw: edit admin.rst
Edit "Add / Remove Admin Capabilities" in doc/radosgw/admin.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-05 21:11:59 +10:00
Matt Benjamin
2bcc413346
Merge pull request #54425 from linuxbox2/wip-lcnotify-cleanup
rgwlc:  additional lc notify cleanups
2024-01-04 21:37:24 -05:00
Adam King
1464da9cf0
Merge pull request #54889 from phlogistonjohn/jjm-cephadm-build-with-wheels
cephadm: build with wheels, enabling pyyaml dependency

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-04 20:12:30 -05:00
Adam King
cd3c69782a
Merge pull request #54722 from phlogistonjohn/jjm-cephadm-sidecars
cephadm: add support for sidecar containers

Reviewed-by: Adam King <adking@redhat.com>
2024-01-04 20:11:41 -05:00
Matt Benjamin
8a935bb115 rgwlc: even current object versions have a unique instance
Fixes: https://tracker.ceph.com/issues/63546

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-01-04 15:23:36 -05:00
Matt Benjamin
9183de7309 rgwlc: additional lc notify cleanups
Follow the same formula to build up obj_state and version_id
in all call sites.

Resolves: rhbz#2163667

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-01-04 15:23:27 -05:00
Casey Bodley
6704c04a6a
Merge pull request #50055 from cbodley/wip-rgw-polymorphic-executor
rgw/beast: switch back to polymorphic executor

Reviewed-by: Mark Kogan <mkogan@ibm.com>
2024-01-04 20:19:54 +00:00
Casey Bodley
fe747f40fa
Merge pull request #54903 from cbodley/wip-63829
radosgw-admin: 'zone set' won't overwrite existing default-placement

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2024-01-04 20:19:08 +00:00
Casey Bodley
6787c4dd08
Merge pull request #54505 from adamemerson/wip-lazyfifo-no-mutex-over-coro
rgw/multisite: Fix deadlock by not holding mutex over coroutine

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-01-04 20:18:38 +00:00
Samuel Just
d451898573
Merge pull request #49594 from Matan-B/wip-matanb-crimson-calc-subsets
crimson: Support Partial Object Recovery

Reviewed-by: Samuel Just <sjust@redhat.com>
2024-01-04 10:13:40 -08:00
Yuval Lifshitz
88a0e91a7a
Merge pull request #55022 from yuvalif/wip-yuval-63901
rgw/kafka: make the connection idle timeout configurable

reviwed-by: cbodley
2024-01-04 19:19:43 +02:00
Yuval Lifshitz
a3682b8f5a
Merge pull request #54967 from yuvalif/wip-yuval-63859
rgw/notifications/lc: handle publish_commit() failures as warnings

reviewed-by: dang, thotz
2024-01-04 19:19:01 +02:00
Matan Breizman
b5df21f556 crimson/osd: Keep track of modified_ranges
* `modifies_ranges` interval_set is added to osd_op_params_t
* keep track of modified_ranges while executing relevant ops
* Add `osd_op_params` parameter to `PGBackend::remove()`.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:43:57 +00:00
Matan Breizman
1367490115 crimson/osd/replicated_recovery_backend: handle_pull_response() - recalc_subsets()
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
0cbe98226f mson/osd/replicated_recovery_backend: Introduce prep_push_to_repli…
…ca()

Intelligently push an object to a replica.  make use of existing
clones/heads and dup data ranges where possible.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
869fc82a74 crimson/osd/replicated_recovery_backend: seperate prep_push_to_replica and prep_push
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
09b638c8db rename prep_push to prep_push_to_replica
dismiss head_ctx

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
35e14f8ff8 crimson/osd/replicated_recovery_backend: prepare_pull to support calc_clone_subsets
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Matan Breizman
883f99dd3e crimson/osd/object_metadata_helper: Introduce calc_*_subsets
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-04 15:18:11 +00:00
Ronen Friedman
d49b2612af
Merge pull request #55037 from ronen-fr/wip-rf-crush-dummy
test/crush: removed unused code

Reviewed-by: Laura Flores <lflores@redhat.com>
2024-01-04 14:51:57 +02:00
Leonid S. Usov
22715d4e00
Merge pull request #53319 from leonid-s-usov/vstart-localhost-for-real
vstart.sh: make sure that --localhost does only bind to 127.0.0.1 in all cases
2024-01-04 14:15:35 +02:00
Rongqi Sun
bfa87ba85e
Merge pull request #54883 from Svelar/memstore
os/memstore: deep copy on partial bufferptrs on setattrs
2024-01-04 14:44:06 +08:00
Rongqi Sun
5f3b7c6bea
Merge pull request #54884 from Svelar/memory_is_zero_dev
inline_memory: optimized mem_is_zero for aarch64 by neon intrinsic
2024-01-04 14:43:44 +08:00
zdover23
bcbb68ac36
Merge pull request #55048 from zdover23/wip-doc-2024-01-03-radosgw-admin-add-remove-a-key
doc/radosgw: edit "Add/Remove a Key"

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-01-04 12:33:32 +10:00
Samuel Just
066953050c
Merge pull request #54214 from athanatos/sjust/wip-crimson-scrub
crimson: add initial scrub support

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-01-03 15:26:45 -08:00
Anthony D'Atri
3f5a709263
Merge pull request #55053 from zdover23/wip-doc-2023-01-04-release-note-cleanup-following-55049
doc/releases: edit reef.rst
2024-01-03 16:32:37 -05:00
Laura Flores
c047daf730
Merge pull request #54994 from ceph/wip-lflores-upgrade-stale-bot
.github/workflows: upgrade stale bot
2024-01-03 11:35:53 -06:00
Laura Flores
159751b680 doc/rados/operations: document ceph balancer status detail
Document change in https://github.com/ceph/ceph/pull/54801

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-03 17:33:32 +00:00
John Mulligan
6e0af1d00d cepahdm: add some specific tests for daemon id systemd name generation
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-03 09:47:47 -05:00
Zac Dover
f62e93cbe7 doc/radosgw: edit "Add/Remove a Key"
Edit the section "Add/Remove a Key" in doc/radosgw/admin.rst. Each
operation (e.g. "Adding an S3 key pair for a user", "Removing an S3 key
pair for a user") now has its own subsection. This increased granularity
should make it easier in the future to link to each of these specific
operations, if needed.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-04 00:29:21 +10:00
Zac Dover
d02178fdbd doc/releases: edit reef.rst
Make minor corrections to doc/releases/reef.rst. These corrections were
suggested by Anthony D'Atri in https://github.com/ceph/ceph/pull/55049.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-04 00:20:51 +10:00
Yuval Lifshitz
dd4589258a rgw/notifications/lc: handle publish_commit() failures as warnings
Fixes: https://tracker.ceph.com/issues/63859

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-01-03 12:29:00 +00:00
Aashish Sharma
16ffaaf819
Merge pull request #53582 from rhcs-dashboard/chartjs-upgrade
mgr/dashboard: chartjs and ng2-charts version upgrade

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-03 12:56:33 +05:30
Venky Shankar
34d5a7fbaa Merge PR #54825 into main
* refs/pull/54825/head:
	mds: change priority of mds rss perf counter to useful

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2024-01-03 09:44:14 +05:30
Samuel Just
4ec04661fe
Merge pull request #54607 from athanatos/sjust/wip-async-recovery-62338-release-note
PendingReleaseNotes: add release note for 62338

Reviewed-by: Neha Ojha <nojha@redhat.com>
2024-01-02 17:32:18 -08:00
Samuel Just
dfb4b26a71 PendingReleaseNotes: add release note for 62338
See https://tracker.ceph.com/issues/62338 and
2fc5486e.

Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-02 09:10:51 -08:00
Yuri Weinstein
ad9b6d7d8b
Merge pull request #54024 from Matan-B/wip-matanb-snap_seq_inc
osd/osd_types: don't increment snap_seq on removal

Reviewed-by: Samuel Just <sjust@redhat.com>
2024-01-02 09:01:34 -08:00
John Mulligan
aaffa66168 cehphadm: disable restarting init containers service
Disable the Restart= line in the init containers systemd template.
It is commented out as there are probably more appropriate workarounds
to use in the future but this should fix testing on centos8 based
systems.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02 09:30:21 -05:00
John Mulligan
e31ef9a379 cephadm: add stop and poststop commands to init container service
Now that there is a more robust init_containers.run script, we can use
the new stop and poststop subcommands from the systemd unit.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02 09:30:21 -05:00
John Mulligan
11ccb59e8e cephadm: convert init containers script to use a template
Convert the init containers run script to be based on a template, like
the sidecar run scripts are. The new script is loosely based on the
sidecars run script but only does actions in batches - logically
iterating over each init container configured.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02 09:30:21 -05:00
John Mulligan
57540a28c5 cephadm: move init container script generation to a function
Move the generation of the init container run script to a small function
fixing a missing `set -e` along the way. This isolates the logic of
generating this run script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02 09:30:21 -05:00
John Mulligan
08e7a56ead cephadm: call functions instead of executing rm in rm_cluster
Convert a bunch of invocations of rm via a subprocess to function calls.
This should make it easier (or possible?) to test the function
in the unit test framework as well as possibly saving a few resources.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02 09:30:21 -05:00
John Mulligan
133b23f13e cephadm: remove a non-functional line in _rm_cluster
Remove a line from _rm_cluster that has no effect. The line uses
call_throws to execute an `rm -rf` command. The argument contains
asterisk chars that indicate that the file(s)/dir(s) to be created are
expected to match the given pattern. However, globs do not work in this
context in contemporary versions of cephadm.

To double check I added the following temporary unit test:
```
def test_does_it_glob(tmp_path):
    from cephadmlib.call_wrappers import call_throws

    d1 = (tmp_path / 'dir1')
    d1.mkdir()
    fns = ['f1.txt', 'f2.txt', 'f3.txt', 'f4']
    for fn in fns:
        with open(d1/fn, "w") as fh:
            fh.write("xxx")
    assert d1.exists()
    for fn in fns:
        assert (d1 / fn).exists()

    ctx = FakeContext()
    call_throws(ctx, ['rm', '-rf', f'{d1}/*.txt'])
    print('files:', os.listdir(d1))
    assert d1.exists()
    for fn in fns:
        if fn.endswith('.txt'):
            assert not (d1 / fn).exists()
        else:
            assert (d1 / fn).exists()
```

If globs worked in this context this test would have passed. It does
not. I confirmed that the current implementation of call/call_wrapper
does not execute the command in a shell context.

I wondered if it was possible that an earlier version of cephadm did
execute this command in a shell context and some changes along the way
changed the behavior. I tracked the origin of the line back to
16ebc62034 the first change to implement
rm-cluster. In this commit the code uses subprocess.check_output
directly. I am familiar with check_output and unless `shell=True` is
passed this function doesn't execute the args in a shell context.  The
`shell=True` argument is not passed to check_output. This means that the
very first implementation of this line suffered from the same issue
- it would have no effect on any files except one named with actual
  asterisk (`*`) characters.

While I'm sure there was a good intention behind this line, the fact
that it persisted in the code so long in a non-functional state and no
one noticed in both production and qa testing makes me feel that it can
be safely removed with no negative effect. Removing the line simplifies
the code and avoids needing to spend effort unit-testing or manually
checking a fix.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2024-01-02 09:30:21 -05:00