Commit Graph

141521 Commits

Author SHA1 Message Date
zdover23
a0312323e6
Merge pull request #54685 from zdover23/wip-doc-2023-11-28-rados-tshooting-log-and-debug-small-repair
doc/rados: improve "Ceph Subsystems"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-29 05:39:10 +01:00
zdover23
ff796bfe73
Merge pull request #54684 from zdover23/wip-doc-2023-11-28-rados-ops-pgs-typo
doc/rados: ops/pgs: s/power of 2/power of two

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-29 05:34:43 +01:00
Zac Dover
7bca5f57cc doc/rados: improve "Ceph Subsystems"
Improve the English in the subsection "Ceph Subsystems" in the section
"Subsystem, Log and Debug Settings" [sic] in
doc/rados/troubleshooting/log-and-debug.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-29 03:01:53 +01:00
Zac Dover
ca81963321 doc/rados: ops/pgs: s/power of 2/power of two
Change "power of 2" to "power of two" in
doc/rados/operations/placement-groups.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-29 02:50:29 +01:00
Daniel Gryniewicz
785c1083fa
Merge pull request #51229 from jsoref/spelling-rgw
rgw: Fix spelling
2023-11-28 09:38:01 -05:00
Venky Shankar
b9b6c42eed
Merge pull request #53456 from petrutlucian94/dokan_case_insensitive
dokan: simple case insensitive emulation

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-11-28 19:28:20 +05:30
Casey Bodley
1f34c2706d
Merge pull request #54621 from cbodley/wip-rgw-posix-warning
rgw/posix: fix unused variable warning

Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-27 17:16:05 +00:00
Josh Soref
ed70d843df rgw: fix spelling errors
* access
* acquire
* across
* algorithm
* alternatively
* ambiguous
* attributes
* authenticated
* automatically
* based
* between
* beware
* bootstrap
* bucket
* cache
* character
* check
* cleanup
* combinations
* committed
* comparison
* compatibility
* concurrent
* conditions
* conjunction
* connections
* considered
* contains
* correct
* creates
* crypto
* dependencies
* dequeued
* dequeuing
* derivative
* described
* determine
* deterministic
* detritus
* discrepancy
* display
* either way
* elements
* entirely
* entries
* error
* everything
* expected
* explicitly
* extend
* false
* fclose
* feature
* following
* foreign
* format
* forward
* functionality
* functions
* generated
* getting
* gracefully
* happened
* hashicorp
* healthcheck
* hierarchy
* however
* http
* identify
* idleness
* ignored
* incremental
* information
* initialization
* length
* locally
* macros
* malformed
* means
* mechanism
* multipart
* multiple
* must
* namespace
* necessary
* nonexistent
* notifications
* occurred
* official
* outputs
* outputted
* overridden
* ownership
* parallelism
* partition
* passthrough
* pause
* penguin
* perform
* permission
* populated
* position
* preferably
* prefixes
* presumably
* previous
* print
* quadruple
* reference
* regardless
* registry
* renewed
* request
* requires
* reshard
* resource
* result to
* satisfy
* secret
* separate
* separator
* setting
* shouldn't
* skipping
* some
* somehow
* something
* specific
* struct
* succeeded
* success
* successful
* successfully
* syntax
* testing
* truncated
* unambiguously
* unittest
* unmodified
* unnecessary
* valid_action
* verifying
* versioning
* visible
* whatever
* whenever

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-11-27 10:27:32 -05:00
Josh Soref
82ec01e208 rgw: rewrite rgw_admin help text
* The line wrapping was fairly random.
* There were cases where words appeared to be doubled.
  - In fact one was an argument
  - the other was the start of the help text.

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-11-27 10:26:58 -05:00
Casey Bodley
f6bc1a8ef8
Merge pull request #54626 from cbodley/wip-cmake-rocksdb-uring
cmake/rocksdb: make sure dependencies build before rocksdb

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2023-11-27 14:20:31 +00:00
Nizamudeen A
68ab0ed81a
Merge pull request #54531 from rhcs-dashboard/port-manipulation-rgw
mgr/dashboard: get rgw port from ssl_endpoint

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2023-11-27 17:48:33 +05:30
Ali Masarwa
c51a3bff0a
Merge pull request #54459 from AliMasarweh/wip-alimasa-2pc-remove-issue
RGW: make new rados support old RGW 2pc remove

Reviewed-by: yuvalif, cbodley
2023-11-27 12:43:46 +02:00
zdover23
83001860de
Merge pull request #54664 from zdover23/wip-doc-2023-11-27-rados-troubleshooting-troubleshooting-mon-everything-failed
doc/rados: edit "Everything Failed! Now What?"

Reviewed-by: Antony D'Atri <anthony.datri@gmail.com>
2023-11-27 15:57:01 +10:00
Zac Dover
d8cd49259d doc/rados: edit "Everything Failed! Now What?"
Edit the section "Everything Failed! Now what?" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthoy D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-27 06:27:28 +01:00
Ali Masarwa
855098f191 RGW: make new rados support old RGW 2pc remove
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-11-26 21:57:51 +02:00
Kefu Chai
e577734645
Merge pull request #53290 from ThomasLamprecht/debian-fix-ceph-mgr-requires
debian: add missing bcrypt to ceph-mgr .requires to fix resulting package dependencies

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2023-11-26 16:28:55 +08:00
zdover23
e989e5729b
Merge pull request #54656 from zdover23/wip-doc-2023-11-25-rados-troubleshooting-mon-mon-store-failures-1-of-x
doc/rados: edit "monitor store failures"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-11-26 14:04:20 +10:00
Thomas Lamprecht
adc501c2ce debian: add missing bcrypt to manager .requires
The `ceph-mgr` package lost its dependency on `python3-bcrypt` when
the dependencies got moved from d/control to a dh_python3 compatible
requires file. Add it again as the bcrypt module is still used there.

Otherwise one gets errors when, e.g., calling `ceph -s` after a fresh
installation:

> 13 mgr modules have failed dependencies
> Module 'balancer' has failed dependency: No module named 'bcrypt'
> Module 'crash' has failed dependency: No module named 'bcrypt'
> Module 'devicehealth' has failed dependency: No module named 'bcrypt'
> Module 'iostat' has failed dependency: No module named 'bcrypt'
> Module 'nfs' has failed dependency: No module named 'bcrypt'
> Module 'orchestrator' has failed dependency: No module named 'bcrypt'
> Module 'pg_autoscaler' has failed dependency: No module named 'bcrypt'
> Module 'progress' has failed dependency: No module named 'bcrypt'
> Module 'rbd_support' has failed dependency: No module named 'bcrypt'
> Module 'restful' has failed dependency: No module named 'bcrypt'
> Module 'status' has failed dependency: No module named 'bcrypt'
> Module 'telemetry' has failed dependency: No module named 'bcrypt'
> Module 'volumes' has failed dependency: No module named 'bcrypt'

Fixes: https://tracker.ceph.com/issues/63637
Fixes: ef19547e83 ("debian: add .requires for specifying python3 deps")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-25 12:59:55 +01:00
Zac Dover
0a1ce00168 doc/rados: edit "monitor store failures"
Edit the section "Monitor Store Failures" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-25 16:50:32 +10:00
zdover23
68e0d99aa0
Merge pull request #54625 from zdover23/wip-doc-2023-11-23-rados-troubleshooting-mon-client-cant-connect
doc/rados: edit "client can't connect..."

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2023-11-25 07:06:53 +10:00
Yuval Lifshitz
213923165c
Merge pull request #52931 from yuvalif/wip-yuval-lua-teuthology
rgw/test/lua: add lua integration tests suite

reviewed-by: TRYTOBE8TME, cbodley
2023-11-24 11:38:51 +02:00
Yuval Lifshitz
6a42ec5435
Merge pull request #54637 from yuvalif/wip-yuval-62136
rgw/test/kafka: let consumer read events from the beginning

reviewed-by: thotz
2023-11-24 11:36:03 +02:00
Casey Bodley
2c43e1d3aa
Merge pull request #54516 from cbodley/wip-63537
rgw: object lock avoids 32-bit truncation of RetainUntilDate

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-24 02:47:55 +00:00
Casey Bodley
b33e08657e encoding: use concepts for chrono encoding
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 19:16:38 -05:00
Casey Bodley
1fd4309fbb rgw: object lock uses 64-bit encoding for RetainUntilDate
the default encoding of ceph::real_time truncates seconds to uint32_t,
so stores the wrong timestamp for object lock enforcement

Fixes: https://tracker.ceph.com/issues/63537

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 19:16:38 -05:00
Casey Bodley
b7aa16da4b encoding: add round_trip_encode()/decode() for chrono types
the default encodings for chrono types were made to be
backward-compatible with utime_t, so truncated seconds to 32 bits

adds new functions that encode these chrono types using their underlying
representation, which for ceph::real_time and ceph::timespan is
'nanoseconds as uint64_t'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 19:16:38 -05:00
Casey Bodley
54eec5bf8f
Merge pull request #54600 from cbodley/wip-rgw-period-realm-name-encode
rgw: fix RGWPeriod encoding after removing realm_name

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2023-11-24 00:15:53 +00:00
Casey Bodley
424b794bd1
Merge pull request #54297 from linuxbox2/wip-librgw-tcmalloc
rgw: link only radosgw with ALLOC_LIBS

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-24 00:15:02 +00:00
Casey Bodley
55d5116d64
Merge pull request #53504 from davidhall586/Bug62105
rgw: SignatureDoesNotMatch for certain RGW Admin Ops endpoints w/v4 auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-24 00:14:08 +00:00
Casey Bodley
e342927ec9
Merge pull request #50148 from cbodley/wip-58750
rgw: Head/GetObject support partNumber

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-11-24 00:13:17 +00:00
Casey Bodley
ac7e285fb4
Merge pull request #50790 from jzhu116-bloomberg/wip-51772
rgw/swift: preserve dashes/underscores in swift user metadata names

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-11-24 00:10:01 +00:00
Jane Zhu
a4553f0351 rgw: fix the ret cond check CreateBucket::Execute()
Signed-off-by: Juan Zhu <jzhu4@dev-10-34-20-139.pw1.bcc.bloomberg.com>
2023-11-23 17:40:30 -05:00
Ali Maredia
0540cc65ef rgw: modify ret cond in middle of CreateBucket::Execute()
Return prevents swift metadata from being removed
if bucket already exists.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2023-11-23 17:40:30 -05:00
Jane Zhu
72065c9c71 rgw/swift: convert underscores to hyphens in swift user metadata names
Signed-off-by: Juan Zhu <jzhu4@dev-10-34-20-139.pw1.bcc.bloomberg.com>
2023-11-23 17:40:28 -05:00
Jane Zhu
96ca98a169 rgw/swift: preserve dashes/underscores in swift user metadata names
Signed-off-by: Juan Zhu <jzhu116@bloomberg.net>
2023-11-23 17:40:04 -05:00
Anthony D'Atri
f232219159
Merge pull request #54641 from likid0/ceph_session_tags_fixes
doc/radosgw: Fix JSON typo in Principal Tag example code snippet
2023-11-23 17:38:42 -05:00
Daniel Parkes
68729c951d doc/radosgw: Fix JSON typo in Principal Tag example code snippet
Signed-off-by: Daniel Parkes <dparkes@redhat.com>
2023-11-23 22:15:52 +01:00
Yuval Lifshitz
46500cace6 rgw/test/lua: add lua integration tests suite
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>

Fixes: https://tracker.ceph.com/issues/63616
2023-11-23 16:36:28 +00:00
Yuval Lifshitz
08e655d8ba rgw/lua: print luarocks config during reload
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-11-23 16:34:21 +00:00
Yuval Lifshitz
4f2ac201ec rgw/test/kafka: let consumer read events from the beginning
in some cases the RGW may publish the notifications before the kafka
consumer started reading the events from the topic

Fixes: https://tracker.ceph.com/issues/62136

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-11-23 14:31:15 +00:00
Anthony D'Atri
14f1647b60
Merge pull request #54632 from zdover23/wip-doc-2023-11-23-rados-troubleshooting-mon-restart-mon-debug-levels-formatting
doc/rados: format sections in tshooting-mon.rst
2023-11-23 07:55:24 -05:00
Zac Dover
910e5efea1 doc/rados: format sections in tshooting-mon.rst
Format definition lists in the section "Do I need to restart a monitor
to adjust debug levels?" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-23 21:54:54 +10:00
Casey Bodley
353360581a cmake/rocksdb: make sure dependencies build before rocksdb
some of rocksdb's dependencies may not have built by the time its
ExternalProject starts, so it can fail with missing headers or
libraries. for example, `uring::uring` may itself be an ExternalProject,
and its include directory won't exist until it starts building:

```
[89/1345] Performing configure step for 'rocksdb_ext'
FAILED: src/rocksdb_ext-prefix/src/rocksdb_ext-stamp/rocksdb_ext-configure build/src/rocksdb_ext-prefix/src/rocksdb_ext-stamp/rocksdb_ext-configure
...
CMake Error in CMakeLists.txt:
  Imported target "uring::uring" includes non-existent path

    "build/src/liburing/src/include"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.
...
[91/1345] Performing download step (git clone) for 'liburing_ext'
Cloning into 'liburing'...
```

use `add_dependencies(rocksdb_ext)` to make sure all of its dependencies
are available before starting the build

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-23 00:19:29 -05:00
Zac Dover
ec298091b5 doc/rados: edit "client can't connect..."
Edit the section "Client can't connect or mount" in the file
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-11-23 14:59:24 +10:00
Casey Bodley
47bae14ad7 rgw/posix: fix unused variable warning
ceph/src/rgw/driver/posix/rgw_sal_posix.cc:37:19: warning: unused variable 'MP_OBJ_PART_FMT' [-Wunused-const-variable]
const std::string MP_OBJ_PART_FMT = "{:0>5}";
                  ^

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-11-22 14:26:52 -05:00
Casey Bodley
07614d51c2
Merge pull request #54599 from cbodley/wip-crush-test-warnings
crush: remove unused variables

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-11-22 13:20:25 +00:00
Ilya Dryomov
0e0b229496
Merge pull request #53291 from petrutlucian94/unicode
common: Windows Unicode support

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-11-22 12:03:59 +01:00
Lucian Petrut
2cef5b3ce4 test: fix Windows ::_creat
The Windows Universal C Runtime (ucrt) "_creat" function is no
longer POSIX compatible and requires Windows specific mode flags.

We got admin socket test failures after switching from msvcrt to
uscrt.

We'll address the issue with some platform checks.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-11-22 09:15:00 +00:00
Lucian Petrut
750a9483d1 common: Windows Unicode CLI support
Windows CLI arguments use either ANSI (main()) or UTF-16 (wmain()).
Meanwhile, Ceph libraries expect UTF-8 and raise exceptions when
trying to use Unicode CLI arguments or log Unicode output:

  rbd.exe create test_unicode_șțăâ --size=32M
  terminate called after throwing an instance of 'std::runtime_error'
    what():  invalid utf8

We'll use a Windows application manifest, setting the "activeCodePage"
property [1][2]. This enables the Windows UCRT UTF-8 mode so that
functions that receive char* arguments will expect UTF-8 instead of ANSI,
including main(). One exception is CreateProcess, which will need the
UTF-16 form (CreateProcessW).

Despite the locale being set to utf-8, we'll have to explicitly set
the console output to utf-8 using SetConsoleOutputCP(CP_UTF8).

In order to use the UTF-8 locale, we'll have to switch the mingw-llvm
runtime from msvcrt to ucrt.

This also fixes ceph-dokan crashes that currently occur when non-ANSI
paths are logged.

[1] https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests#activecodepage
[2] https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-11-22 09:14:49 +00:00
Brad Hubbard
e877333f07
Merge pull request #54566 from badone/wip-python-version-fedora-39
do_cmake.sh: set python version for Fedora 39

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2023-11-22 09:44:22 +10:00