Commit Graph

135619 Commits

Author SHA1 Message Date
Yingxin
d2f2c27acf
Merge pull request #49427 from tchaikov/wip-crimson-fmt
crimson: fmtlib v9 related changes

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-15 10:24:17 +08:00
Yingxin
42db2b00a8
Merge pull request #49408 from tchaikov/wip-crimson-device_spec_t
crimson/os: define device_spec_t as packed

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-15 10:21:22 +08:00
Zac Dover
e1c5bcd73b doc/cephfs: s/all of there are/all of these are/
s/all of there are/all of these are/ in posix.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-15 07:07:44 +10:00
Zac Dover
03bfcb134b doc/cephfs - s/yet to here/yet to hear/ posix.rst
Correct "yet to here" to "yet to hear" in posix.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-15 06:51:29 +10:00
Casey Bodley
76c5d1b972
Merge pull request #48710 from cbodley/wip-rgw-process-env
rgw: pass RGWProcessEnv into process_request()

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-12-14 15:15:46 -05:00
Adam C. Emerson
1a918d16a9
Merge pull request #49395 from adamemerson/wip-policy-useful-error-messages
rgw: Give useful errors when policies fail to parse

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2022-12-14 15:13:51 -05:00
Kamoltat
924e7ec92b mon/Monitor.cc: notify_new_monmap() skips removal of non-exist rank
Problem:
In RHCS the user can choose to manually remove a monitor rank
before shutting the monitor down. Causing inconsistency in monmap.
for example we remove mon.a from the monmap, there is a short period
where mon.a is still operational and will try to remove itself from
monmap but we will run into an assertion in
ConnectionTracker::notify_ranks_removed().

Solution:
In Monitor::notify_new_monmap() we prevent the func
from going into removing our own rank, or
ranks that doesn't exists in monmap.

FYI: this is an RHCS problem only, in ODF,
we never remove a monitor from monmap
before shutting it down.

Fixes: https://tracker.ceph.com/issues/58049

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-12-14 20:13:14 +00:00
Casey Bodley
c53e83f58e
Merge pull request #49141 from cbodley/wip-58111
rgw: rgw_parse_url_bucket() rejects empty bucket names after 'tenant:'

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@cern.ch>
2022-12-14 14:59:58 -05:00
Neha Ojha
a968c6d024
Merge pull request #48226 from sseshasa/wip-fix-recovery-backfill-limits-for-mclock
osd: Reduce backfill/recovery default limits for mClock and other optimizations

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-12-14 10:59:15 -08:00
Mike Perez
1900c02ed6 doc/foundation: Adding Alex Marangone to the board
Signed-off-by: Mike Perez <thingee@gmail.com>
2022-12-14 10:57:50 -08:00
Daniel Gryniewicz
c75995f9f3 RGW - Fix DAOS and MOTR stores to not use rgw_obj_key::to_str()
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-12-14 13:45:31 -05:00
Guillaume Abrioux
770cd09412 ceph-volume: fix a bug in get_all_devices_vgs()
When a physical volume isn't a member of any volume group,
the function `get_all_devices_vgs()` in `ceph_volume.api.lvm`
shouldn't try to return an `VolumeGroup()` object against this
device.

Fixes: https://tracker.ceph.com/issues/57918

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-12-14 17:59:19 +01:00
Laura Flores
960aec07bd .github: automatically add telemetry label to relevant files
Signed-off-by: Laura Flores <ljflores@redhat.com>
2022-12-14 16:20:45 +00:00
Laura Flores
63a77b2c5b
Merge pull request #49178 from JoshSalomon/chg-primary
osd: Add cli upmap command to change the primary OSD of a pg
2022-12-14 09:52:55 -06:00
Pere Diaz Bou
fa43e002e2
Merge pull request #48742 from rhcs-dashboard/add-tooltip
mgr/dashboard: add tooltip mirroring pools table

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-12-14 15:06:09 +01:00
Kefu Chai
161a191a63
Merge pull request #49418 from tchaikov/wip-cls-fifo-fmt
cls/fifo: specialize fmt::formater<> for rados::cls::fifo::info

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-14 21:51:56 +08:00
Pere Diaz Bou
273286d1a6 mgr/dashboard: display real health in rbd mirroring pools
Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
2022-12-14 12:55:08 +01:00
Josh Salomon
e9921ee76c osd, mon: Add 'osd pg-upmap-primary' and 'osd rm-pg-upmap-primary' commands.
Changed OSDMap to keep upmap-primary records
The upmap-primary records are visible in json/xml and in dump commands.

Signed-off-by: Josh Salomon <jsalomon@redhat.com>
2022-12-14 13:37:58 +02:00
Kefu Chai
d56af706d9 crimson/osd: include fmt/ranges.h
so we are able to format ss.clone_snaps, which is a
std::map<snapid_t, std::vector<snapid_t>>. before fmtlib v9,
fmtlib is able to fall back to the operator<<() to format the the types
with the operator<<() defined. but after fmtlib v9, we need to
explicitly define them for accessing the specialized formatter.

in this change, fmt/ranges.h is included so we can access the formatter
for std::map and std::vector.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 19:14:16 +08:00
Kefu Chai
cd553f625f crimson/os: specialize fmt::formater<> for lba_pin_list_t
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<lba_pin_list_t> is defined so
the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 19:14:16 +08:00
Kefu Chai
cb78116a0b crimson/osd: print oid of missing clone
instead of printing the empty optional<>, print the oid of missing
clone. the oid is more interesting in this context.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 19:14:16 +08:00
Kefu Chai
d9c8bb3cf4
Merge pull request #49419 from tchaikov/wip-seastar
seastar: bump up seastar submodule to recent master

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-14 18:23:14 +08:00
cao.leilc
48380f0937 rgw: fix ListOpenIDConnectProviders XML format
Fixes: https://tracker.ceph.com/issues/53171
Signed-off-by: caolei <cao.leilc@inspur.com>
2022-12-14 17:44:52 +08:00
Kefu Chai
3e0d3eb15d crimson/os: specialize fmt::formater<> for types
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<> is specialized for tree_stats_t and
test_item_t so the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 16:56:22 +08:00
Kefu Chai
fe38c3caf1 crimson/os: print pointer using fmt::ptr()
in fmtlib v9, it does not format typed pointers anymore, instead,
we need to print them using fmt::ptr().

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 15:23:08 +08:00
Kefu Chai
c10447b4f0 seastar: bump up seastar submodule to recent master
to address the fmtlib v9 compatibility issues.

please note, the submodule is bump up to a commit which includes

* a revert commit on top of upstream master HEAD. the revert is to
  work around the a recent change which adds a constraint on the parameter
  type of handle_exception(). we are using errorator along with
  handle_exception(), but errorator cannot fulfill the requirement of
  seastar::Future(), so we need to drop the change before we have a
  better solution.
* a commit which defines __NR_pidfd_open if it is not defined.
  this should address the FTBFS on CentOS8 where glibc does not have
  this macro defined.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 15:23:08 +08:00
colemitchell
9b57aff9d4
Merge pull request #49421 from zdover23/wip-doc-2022-12-14-glossary-fqdn
doc/glossary: add "FQDN" entry

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
2022-12-14 02:08:00 -05:00
Zac Dover
e919941036 doc/glossary: add "FQDN" entry
Add "FQDN" definition to doc/glossary.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-12-14 16:42:59 +10:00
Xinyu Huang
13a418b365 crimson/os/seastore: Seastore::omap_list cleanup
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-12-14 06:13:25 +00:00
Kefu Chai
05f5f390d1 test/cls_fio: specialize fmt::formater<> for bpo::options_description
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<bpo::options_description> is defined so
the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 11:01:43 +08:00
Kefu Chai
662163163c cls/fifo: specialize fmt::formater<> for rados::cls::fifo::info
since fmt v9, fmt::formatter<> is not specialized for the types with
operator<<(ostream&, ...) anymore. so we need to specialize it manually.
in this change, fmt::formatter<rados::cls::fifo::info> is defined so
the tree can compile with fmt v9.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-14 11:01:43 +08:00
Yingxin
9c85f735b6
Merge pull request #49242 from myoungwon/wip-enable-rbm-test-seastore
test/crimson/seastore: enable RBM-based unittest-seastore

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-12-14 10:05:57 +08:00
Kefu Chai
7a030c1a81
Merge pull request #49407 from tchaikov/cmake-find-boost
cmake: add 1.79 and 1.80 to known versions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-14 08:42:16 +08:00
Marcus Watts
28864311d3 rgw: added rgw-policy-check
rgw-policy-check - a program to do syntax checking on bucket policy.
This program just reads the policy into memory, so it is not
checking anything except syntax.

Signed-off-by: Marcus Watts <mwatts@redhat.com>

rgw: Fix return value of `rgw-policy-check`

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Use ceph initialization in `rgw-policy-check`

Specifically so we can pull in the options from `ceph.conf` and similar.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-12-13 16:41:52 -05:00
Adam C. Emerson
0b0fd44f42 rgw: Put useful policy error messages in error document
This affects the various create/put operations that take a policy document.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-12-13 16:41:52 -05:00
Adam C. Emerson
0dd2ef4245 rgw: Add rgw_policy_reject_invalid_principals and messages
Reject policies with invalid principals by default and provide more
useful error messages while doing so.

(Log them but do *not* reject the policy if it's set to false.)

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-12-13 16:35:05 -05:00
Adam C. Emerson
613e0108be rgw: Provide useful error messages from policy parser
It would be much nicer to give people an idea why their policies are
failing rather than just telling them where they're failing.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2022-12-13 16:34:34 -05:00
yaarith
3a8f2dcc51
Merge pull request #48214 from ljflores/wip-telemetry-bluestore-compression-mode
mgr/telemetry: add `basic_pool_options_bluestore` collection

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>
2022-12-13 14:34:45 -05:00
Casey Bodley
0ca2f95469 rgw/lua: pass luarocks_path directly, remove from sal
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-12-13 13:52:58 -05:00
Casey Bodley
9b13b83521 rgw/lua: split out a struct RGWLuaProcessEnv
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-12-13 13:52:58 -05:00
Kefu Chai
d1ca83cf47
Merge pull request #49190 from tchaikov/wip-fmt
fmt: pickup v9.1.0

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-12-14 00:40:48 +08:00
Ronen Friedman
aa49dee4e6
Merge pull request #49410 from ronen-fr/wip-rf-cinode-tm
mds: remove unused declaration

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2022-12-13 16:02:24 +02:00
Ronen Friedman
8c46e4c3a3 mds: remove unused declaraion
as some Jenkins builds are set to treat warnings as errors, and
complain.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-12-13 14:24:08 +02:00
Kefu Chai
ae76420ecc crimson/os: define device_spec_t as packed
per C++ standard 20, alignas cannot be applied to scoped enum. despite
that there is a resolution to address this, see
https://cplusplus.github.io/CWG/issues/2354.html, it's not included in
C++20. so we have to use a different way to address

> runtime error: reference binding to misaligned address 0x610000008395
> for type 'device_type_t', which requires 4 byte alignment

otherwise the code would fail to compile with Clang-15, like:

> <source>:3:13: error: 'alignas' attribute cannot be applied to an enumeration

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-13 18:36:17 +08:00
Kefu Chai
06824bc1ec cmake: adapt FindBoost.cmake to our needs
the vanilla FindBoost.cmake pulled from cmake has couple assumptions
which do not hold in our environment. so address them case by case.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-13 18:08:48 +08:00
Kefu Chai
0b69b41f12 cmake: add 1.79 and 1.80 to known versions
sync with
468a995346/Modules/FindBoost.cmake

for v1.79 and v1.80 support

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-12-13 18:08:48 +08:00
Xinyu Huang
b6bb15e1be crimson/os/seastore/omap: adjust omap_list_config_t
Add inclusive behavior for both first and last in omap_list_config_t,
and optimize the usage.
Set first closed last open for range for omap range remove.

Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-12-13 09:49:55 +00:00
Xinyu Huang
e2d544dce8 crimson/os/seastore: introduce omap range remove
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-12-13 09:49:55 +00:00
Xinyu Huang
49b10d72a7 crimson/os/seastore/omap: add tests for range remove
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-12-13 09:49:55 +00:00
Xinyu Huang
a3b5d459ec crimson/os/seastore/omap: introduce omap range remove
Signed-off-by: Xinyu Huang <xinyu.huang@intel.com>
2022-12-13 09:49:55 +00:00