Commit Graph

141502 Commits

Author SHA1 Message Date
Zac Dover
d8cd49259d doc/rados: edit "Everything Failed! Now What?"
Edit the section "Everything Failed! Now what?" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthoy D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-27 06:27:28 +01:00
Kefu Chai
e577734645
Merge pull request #53290 from ThomasLamprecht/debian-fix-ceph-mgr-requires
debian: add missing bcrypt to ceph-mgr .requires to fix resulting package dependencies

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2023-11-26 16:28:55 +08:00
zdover23
e989e5729b
Merge pull request #54656 from zdover23/wip-doc-2023-11-25-rados-troubleshooting-mon-mon-store-failures-1-of-x
doc/rados: edit "monitor store failures"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-26 14:04:20 +10:00
Thomas Lamprecht
adc501c2ce debian: add missing bcrypt to manager .requires
The `ceph-mgr` package lost its dependency on `python3-bcrypt` when
the dependencies got moved from d/control to a dh_python3 compatible
requires file. Add it again as the bcrypt module is still used there.

Otherwise one gets errors when, e.g., calling `ceph -s` after a fresh
installation:

> 13 mgr modules have failed dependencies
> Module 'balancer' has failed dependency: No module named 'bcrypt'
> Module 'crash' has failed dependency: No module named 'bcrypt'
> Module 'devicehealth' has failed dependency: No module named 'bcrypt'
> Module 'iostat' has failed dependency: No module named 'bcrypt'
> Module 'nfs' has failed dependency: No module named 'bcrypt'
> Module 'orchestrator' has failed dependency: No module named 'bcrypt'
> Module 'pg_autoscaler' has failed dependency: No module named 'bcrypt'
> Module 'progress' has failed dependency: No module named 'bcrypt'
> Module 'rbd_support' has failed dependency: No module named 'bcrypt'
> Module 'restful' has failed dependency: No module named 'bcrypt'
> Module 'status' has failed dependency: No module named 'bcrypt'
> Module 'telemetry' has failed dependency: No module named 'bcrypt'
> Module 'volumes' has failed dependency: No module named 'bcrypt'

Fixes: https://tracker.ceph.com/issues/63637
Fixes: ef19547e83 ("debian: add .requires for specifying python3 deps")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-25 12:59:55 +01:00
Zac Dover
0a1ce00168 doc/rados: edit "monitor store failures"
Edit the section "Monitor Store Failures" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-25 16:50:32 +10:00
zdover23
68e0d99aa0
Merge pull request #54625 from zdover23/wip-doc-2023-11-23-rados-troubleshooting-mon-client-cant-connect
doc/rados: edit "client can't connect..."

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2023-11-25 07:06:53 +10:00
Yuval Lifshitz
213923165c
Merge pull request #52931 from yuvalif/wip-yuval-lua-teuthology
rgw/test/lua: add lua integration tests suite

reviewed-by: TRYTOBE8TME, cbodley
2023-11-24 11:38:51 +02:00
Yuval Lifshitz
6a42ec5435
Merge pull request #54637 from yuvalif/wip-yuval-62136
rgw/test/kafka: let consumer read events from the beginning

reviewed-by: thotz
2023-11-24 11:36:03 +02:00
Casey Bodley
2c43e1d3aa
Merge pull request #54516 from cbodley/wip-63537
rgw: object lock avoids 32-bit truncation of RetainUntilDate

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-24 02:47:55 +00:00
Casey Bodley
b33e08657e encoding: use concepts for chrono encoding
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 19:16:38 -05:00
Casey Bodley
1fd4309fbb rgw: object lock uses 64-bit encoding for RetainUntilDate
the default encoding of ceph::real_time truncates seconds to uint32_t,
so stores the wrong timestamp for object lock enforcement

Fixes: https://tracker.ceph.com/issues/63537

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 19:16:38 -05:00
Casey Bodley
b7aa16da4b encoding: add round_trip_encode()/decode() for chrono types
the default encodings for chrono types were made to be
backward-compatible with utime_t, so truncated seconds to 32 bits

adds new functions that encode these chrono types using their underlying
representation, which for ceph::real_time and ceph::timespan is
'nanoseconds as uint64_t'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 19:16:38 -05:00
Casey Bodley
54eec5bf8f
Merge pull request #54600 from cbodley/wip-rgw-period-realm-name-encode
rgw: fix RGWPeriod encoding after removing realm_name

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2023-11-24 00:15:53 +00:00
Casey Bodley
424b794bd1
Merge pull request #54297 from linuxbox2/wip-librgw-tcmalloc
rgw: link only radosgw with ALLOC_LIBS

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-24 00:15:02 +00:00
Casey Bodley
55d5116d64
Merge pull request #53504 from davidhall586/Bug62105
rgw: SignatureDoesNotMatch for certain RGW Admin Ops endpoints w/v4 auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-24 00:14:08 +00:00
Casey Bodley
e342927ec9
Merge pull request #50148 from cbodley/wip-58750
rgw: Head/GetObject support partNumber

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-24 00:13:17 +00:00
Casey Bodley
ac7e285fb4
Merge pull request #50790 from jzhu116-bloomberg/wip-51772
rgw/swift: preserve dashes/underscores in swift user metadata names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-24 00:10:01 +00:00
Jane Zhu
a4553f0351 rgw: fix the ret cond check CreateBucket::Execute()
Signed-off-by: Juan Zhu <jzhu4@dev-10-34-20-139.pw1.bcc.bloomberg.com>
2023-11-23 17:40:30 -05:00
Ali Maredia
0540cc65ef rgw: modify ret cond in middle of CreateBucket::Execute()
Return prevents swift metadata from being removed
if bucket already exists.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2023-11-23 17:40:30 -05:00
Jane Zhu
72065c9c71 rgw/swift: convert underscores to hyphens in swift user metadata names
Signed-off-by: Juan Zhu <jzhu4@dev-10-34-20-139.pw1.bcc.bloomberg.com>
2023-11-23 17:40:28 -05:00
Jane Zhu
96ca98a169 rgw/swift: preserve dashes/underscores in swift user metadata names
Signed-off-by: Juan Zhu <jzhu116@bloomberg.net>
2023-11-23 17:40:04 -05:00
Anthony D'Atri
f232219159
Merge pull request #54641 from likid0/ceph_session_tags_fixes
doc/radosgw: Fix JSON typo in Principal Tag example code snippet
2023-11-23 17:38:42 -05:00
Daniel Parkes
68729c951d doc/radosgw: Fix JSON typo in Principal Tag example code snippet
Signed-off-by: Daniel Parkes <dparkes@redhat.com>
2023-11-23 22:15:52 +01:00
Yuval Lifshitz
46500cace6 rgw/test/lua: add lua integration tests suite
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Fixes: https://tracker.ceph.com/issues/63616
2023-11-23 16:36:28 +00:00
Yuval Lifshitz
08e655d8ba rgw/lua: print luarocks config during reload
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-11-23 16:34:21 +00:00
Yuval Lifshitz
4f2ac201ec rgw/test/kafka: let consumer read events from the beginning
in some cases the RGW may publish the notifications before the kafka
consumer started reading the events from the topic

Fixes: https://tracker.ceph.com/issues/62136

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-11-23 14:31:15 +00:00
Anthony D'Atri
14f1647b60
Merge pull request #54632 from zdover23/wip-doc-2023-11-23-rados-troubleshooting-mon-restart-mon-debug-levels-formatting
doc/rados: format sections in tshooting-mon.rst
2023-11-23 07:55:24 -05:00
Zac Dover
910e5efea1 doc/rados: format sections in tshooting-mon.rst
Format definition lists in the section "Do I need to restart a monitor
to adjust debug levels?" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-23 21:54:54 +10:00
Zac Dover
ec298091b5 doc/rados: edit "client can't connect..."
Edit the section "Client can't connect or mount" in the file
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-23 14:59:24 +10:00
Casey Bodley
07614d51c2
Merge pull request #54599 from cbodley/wip-crush-test-warnings
crush: remove unused variables

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-22 13:20:25 +00:00
Ilya Dryomov
0e0b229496
Merge pull request #53291 from petrutlucian94/unicode
common: Windows Unicode support

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-22 12:03:59 +01:00
Lucian Petrut
2cef5b3ce4 test: fix Windows ::_creat
The Windows Universal C Runtime (ucrt) "_creat" function is no
longer POSIX compatible and requires Windows specific mode flags.

We got admin socket test failures after switching from msvcrt to
uscrt.

We'll address the issue with some platform checks.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-11-22 09:15:00 +00:00
Lucian Petrut
750a9483d1 common: Windows Unicode CLI support
Windows CLI arguments use either ANSI (main()) or UTF-16 (wmain()).
Meanwhile, Ceph libraries expect UTF-8 and raise exceptions when
trying to use Unicode CLI arguments or log Unicode output:

  rbd.exe create test_unicode_șțăâ --size=32M
  terminate called after throwing an instance of 'std::runtime_error'
    what():  invalid utf8

We'll use a Windows application manifest, setting the "activeCodePage"
property [1][2]. This enables the Windows UCRT UTF-8 mode so that
functions that receive char* arguments will expect UTF-8 instead of ANSI,
including main(). One exception is CreateProcess, which will need the
UTF-16 form (CreateProcessW).

Despite the locale being set to utf-8, we'll have to explicitly set
the console output to utf-8 using SetConsoleOutputCP(CP_UTF8).

In order to use the UTF-8 locale, we'll have to switch the mingw-llvm
runtime from msvcrt to ucrt.

This also fixes ceph-dokan crashes that currently occur when non-ANSI
paths are logged.

[1] https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests#activecodepage
[2] https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-11-22 09:14:49 +00:00
Brad Hubbard
e877333f07
Merge pull request #54566 from badone/wip-python-version-fedora-39
do_cmake.sh: set python version for Fedora 39

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2023-11-22 09:44:22 +10:00
J. Eric Ivancich
eb4b542976
Merge pull request #54447 from ceph/wip-fix-flight-load-bucket
rgw: fix flight load_bucket call

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 16:36:53 -05:00
J. Eric Ivancich
0af251b856
Merge pull request #47208 from 5cs/fix-lambda-capture-by-ref
rgwlc: lock_lambda overwrites ret val

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-11-21 16:36:27 -05:00
zdover23
0bd8b17bfa
Merge pull request #54598 from zdover23/wip-doc-2023-11-22-rados-troubleshooting-mon-recovering-broken-monmap
doc/rados: edit "recovering broken monmap"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-22 05:56:33 +10:00
Zac Dover
6ccb9f3ca1 doc/rados: edit "recovering broken monmap"
Edit the section "Recovering a monitor's broken monmap" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-22 05:33:37 +10:00
Casey Bodley
c94166de2a rgw: fix RGWPeriod encoding after removing realm_name
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 14:25:14 -05:00
Casey Bodley
9cab99bae8 crush: remove unused variables
[161/715] Building CXX object src/crush/CMakeFiles/crush_objs.dir/CrushTester.cc.o
ceph/src/crush/CrushTester.cc:478:7: warning: variable 'num_devices_active' set but not used [-Wunused-but-set-variable]
  int num_devices_active = 0;
      ^
1 warning generated.
[165/715] Building CXX object src/crush/CMakeFiles/crush_objs.dir/CrushWrapper.cc.o
ceph/src/crush/CrushWrapper.cc:1579:9: warning: variable 'local_changed' set but not used [-Wunused-but-set-variable]
    int local_changed = 0;
        ^

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 13:44:41 -05:00
Casey Bodley
338440ae2c rgw: non-multipart uploads serve entire range on partNumber=1
and omit the x-amz-mp-parts-count response header

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 11:09:23 -05:00
Casey Bodley
c02129eb8c ReleaseNotes: document support for partNumber
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 11:09:23 -05:00
Casey Bodley
6fc57159ef rgw/rados: RadosReadOp::prepare only updates object instance
when called on a versioned object, prepare() may follow olh and look up
a different object instance

but when called on a multipart part, we should not overwrite the
original object name with the part's object name (of the form
mymultipart.2~_XLFNqOW0NuiALg7q4-Hi_7hdtAkZUH.1)

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 11:09:23 -05:00
Casey Bodley
01d8b4c38b rgw/rados: part support for RGWRados::Object::Read
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 11:09:23 -05:00
Casey Bodley
8ae61ca506 rgw/rados: add obj_find_part() to RGWObjManifest
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 11:09:23 -05:00
Casey Bodley
a308e3a1d5 rgw/rados: add get_obj_state() overload for RGWObjStateManifest
add an overload to expose the manifest storage to callers of
get_obj_state(). the existing RGWObjState+RGWObjManifest overload
just calls the RGWObjStateManifest one

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 11:09:21 -05:00
Casey Bodley
60dadf3c8d rgw/rados: remove get_obj_state() overload for follow_olh=true
and just add the follow_olh=true argument to callers

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 10:24:31 -05:00
Casey Bodley
3354b67fbd rgw/s3: add part param and response to GetObj
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-21 10:24:31 -05:00
Casey Bodley
b0b050d31c
Merge pull request #52813 from cbodley/wip-59424
qa/rgw: run s3tests against keystone ec2

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-21 13:33:58 +00:00
Guillaume Abrioux
bc868d9a4f
Merge pull request #53798 from asm0deuz/track_58812
ceph-volume: fixes fallback to stat in is_device and is_partition
2023-11-21 12:25:25 +01:00