Commit Graph

136673 Commits

Author SHA1 Message Date
Luís Henriques
bf7ddd8e45 qa/tasks/mgr/dashboard/test_health: Add 'max_xattr_size' to the mdsmap schema
Let the mdsmap schema know about the new field 'max_xattr_size'.  This prevents
the following error:

tasks.mgr.dashboard.helper._ValError: \
  In `input['fs_map']['filesystems'][0]['mdsmap']`: unknown keys: {'max_xattr_size'}

URL: https://tracker.ceph.com/issues/55725
Signed-off-by: Luís Henriques <lhenriques@suse.de>
2023-03-01 10:35:28 +00:00
Luís Henriques
ca1a397cca mds: prevent exceeding xattrs limits on initial set
When doing a sync setxattr (MDS_OP_SETXATTR) to set the first xattr in an
inode it is possible for the client to set a huge xattr key/value that would
exceed the limit.  Prevent this from from happening by checking the size
against the maximum allowed size.

URL: https://tracker.ceph.com/issues/55725
Signed-off-by: Luís Henriques <lhenriques@suse.de>
2023-03-01 10:35:28 +00:00
Luís Henriques
13e07ff3e9 mds: prevent clients from exceeding the xattrs key/value limits
Commit eb915d0eec ("cephfs: fix write_buf's _len overflow problem") added
a limit to the total size of xattrs.  This limit is respected by clients
doing a "sync" operation, i.e. MDS_OP_SETXATTR.  However, clients with
CAP_XATTR_EXCL can still buffer these operations and ignore these limits.

This patch prevents clients from crashing the MDSs by also imposing the
xattr limits even when they have the Xx caps.  Replaces the per-MDS knob
"max_xattr_pairs_size" by the new mdsmap setting that the clients can
access.

Unfortunately, clients that misbehave, i.e. old clients that don't respect
this xattrs limit and buffer their xattrs, will see them vanishing.

URL: https://tracker.ceph.com/issues/55725
Signed-off-by: Luís Henriques <lhenriques@suse.de>
2023-03-01 10:35:28 +00:00
Luís Henriques
7b8def5c30 mon: add new configuration to limit filesystem xattrs size
This new configuration option will allow to define the maximum size for a
filesystem xattrs blob.  This is a filesystem-wide knob that will replace
the per-MDS mds_max_xattr_pairs_size option.

Note:

The kernel client patch to handle this new configuration was merged before
the corresponding ceph-side pull-request.  This was unfortunate because in
the meantime PR #43284 was merged and the encoding/decoding of
'bal_rank_mask' got in between.  Hence the 'max_xattr_size' is being
encoding/decoded before 'bal_rank_mask'.

URL: https://tracker.ceph.com/issues/55725
Signed-off-by: Luís Henriques <lhenriques@suse.de>
2023-03-01 10:35:25 +00:00
Radoslaw Zarzynski
0ad603862b
Merge pull request #49286 from rzarzynski/wip-crimson-snaptrimmer
crimson/osd, osd: bring snap trimming to crimson

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-03-01 08:57:27 +01:00
Venky Shankar
cca84e653d Merge PR #48720 into main
* refs/pull/48720/head:
	qa: fix api failure issue caused during a dashboard test
	PendingReleaseNotes: noted new MDSMap field refuse_client_session
	qa: added two testcases
	client: do not initiate session if flag refuse_client_session is set
	mds: do not reconnect when refuse_client_session is set
	mds: add new feature to block clients from establishing sessions

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-01 07:54:18 +05:30
Neha Ojha
707af78f9d
Merge pull request #49006 from yanghonggang/yhg-upgrade-rocksdb-7.7.3
rocksdb: Upgrade rocksdb to latest facebook's v7.9.2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2023-02-28 15:50:21 -08:00
Radoslaw Zarzynski
d90d81585b qa/valgrind.supp: suppress the ColumnFamily ctor called on init
```
    <frame>
      <ip>0x1712F1E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::BytewiseComparator()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/util</dir>
      <file>comparator.cc</file>
      <line>304</line>
    </frame>
    <frame>
      <ip>0x162361C</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::ColumnFamilyOptions::ColumnFamilyOptions()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/options</dir>
      <file>options.cc</file>
      <line>123</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>Options</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/rocksdb/include/rocksdb</dir>
      <file>options.h</file>
      <line>1409</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>RocksDBStore::init(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;)</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/kv</dir>
      <file>RocksDBStore.cc</file>
      <line>400</line>
    </frame>
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 20:58:27 +00:00
Radoslaw Zarzynski
0de4d99c72 qa/valgrind.supp: suppress the get-or-create CacheItemHelper
```
// Get an CacheItemHelper pointer for value type T and role R.
template <typename T, CacheEntryRole R>
Cache::CacheItemHelper* GetCacheItemHelperForRole() {
  static Cache::CacheItemHelper cache_helper(
      BlocklikeTraits<T>::SizeCallback, BlocklikeTraits<T>::SaveToCallback,
      GetCacheEntryDeleterForRole<T, R>());
  return &cache_helper;
}
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 20:43:48 +00:00
Radoslaw Zarzynski
9949151620 qa/valgrind.supp: even broader suppression for rocksdb's init
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 20:43:41 +00:00
Adam King
0aef3d9741
Merge pull request #50308 from adk3798/mock-fqdn-secure-alertmanager
mgr/cephadm: mock get_fqdn in secure alertmanager config test

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2023-02-28 14:57:44 -05:00
Radoslaw Zarzynski
a3aa624db4 qa/valgrind.supp: ingore rocksdb leak in option parsing
This suppression is needed since upgrading to v7.9.2.
See PR https://github.com/ceph/ceph/pull/49006.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 19:13:39 +00:00
Radoslaw Zarzynski
4dbe8f56b6 crimson/osd: fix compilation guard around specialization for libfmt => 9
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
b03ab045e9 crimson/osd: handle snap trim's interruption as any other error
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
38c030e2cb crimson/osd: handle PG_STATE_SNAPTRIM_ERROR when trimming snaps
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
f091a70e3c crimson/osd: errorate SnapTrimEvent
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
e0bb021796 crimson/osd: manage PG_STATE_SNAPTRIM when trimming snaps
After a few tries:

```
$ bin/ceph -s
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2022-12-19T20:18:21.856+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.857+0000 7ff87b46f700  1 build_initial for_mkfs: 0
2022-12-19T20:18:21.860+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.861+0000 7ff87b46f700  1 build_initial for_mkfs: 0
  cluster:
    id:     347aedc9-d72f-4a34-98fc-26cb04b0c2fc
    health: HEALTH_WARN
            11 mgr modules have failed dependencies
            1 pool(s) do not have an application enabled
            1 pool(s) have no replicas configured

  services:
    mon: 1 daemons, quorum a (age 3m)
    mgr: x(active, since 3m)
    osd: 1 osds: 1 up (since 3m), 1 in (since 3m)

  data:
    pools:   1 pools, 1 pgs
    objects: 95 objects, 6 B
    usage:   1.0 GiB used, 100 GiB / 101 GiB avail
    pgs:     1 active+clean+snaptrim

  io:
    client:   1.2 KiB/s wr, 0 op/s rd, 1 op/s wr
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
6923b05d70 crimson/osd: drop C++'s ranges to workaround unbuildability on Clang
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: while checking constraint satisfaction for template 'operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' required here
    for (const auto &i : subops | std::views::keys) {
                                ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: in instantiation of function template specialization 'std::ranges::views::__adaptor::operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' requested here
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/iterator_concepts.h:963:7: note: candidate template ignored: constraints not satisfied [with _Tp = std::ranges::ref_view<const std
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
6dce7df440 crimson/osd: simplify SnapTrimObjSubEvent::start()
Throwing away the NOP-looking-like `maybe_delay` concept became
possible after eradication of the `with_interruption` call nested
within `SnapTrimSubEvent`. Before, on Ubuntu and GCC 11.2.1, lack
of it was the reson behind assertion failures on resetting the
`thread_local` intteruption condition.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
877c272c2f crimson/osd: drop the nested interruptors in snap trimming
`crimson::interruptible` does not support that. See the
`DISABLED_nested_interruptors` unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
c08d06fb0d crimson/test: verify nesting of interruptors
It doesn't work.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
9c719406cc crimson/osd: errorate SnapTrimObjSubEvent::remove_clone()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
912a940f46 crimson/osd: convert SnapTrimObjSubEvent::remove_or_update to ertr
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
30db140344 crimson/osd: dissect snapset update into SnapTrimObjSubEvent::update_head()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
dda6d453d1 crimson/osd: dissect head whiteout removal into SnapTrimObjSubEvent::remove_head_whiteout()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
37f2bda50c crimson/osd: drop data aliases from SnapTrimObjSubEvent::adjust_snaps()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
355b7f7ebd crimson/osd: dissect clone removal into SnapTrimObjSubEvent::remove_clone()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
bb756b5983 crimson/osd: drop data aliases from SnapTrimObjSubEvent::remove_or_update()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
bc5d15e7fc crimson/osd: simplify informing SnapMapper about new mappings
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
b9d81029ae crimson/osd: implement basic osd_snap_trim_sleep support
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
6df8e3965c crimson/osd: introduce WaitSubop stage to SnapTrimEvent
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
9245c7650d crimson/osd: snap trimming informs OpsExecuter about altered objects
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
ad22a9a228 common: add debugs to crimson's green_condition_variable
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
9ce59d7648 crimson/osd: drop unused declaration of PG::fill_op_params_bump_pg_version()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
5d6dd2cba8 osd: SnapMapper::OSDriver uses crimson's interruptor::green_get().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
301b2c6b8a crimson/common: bring green_get() to interruptor
The rationale
=============
Calling `get()` on an unavailable future can result in yielding
the reactor, so -- from the interruptor's POV -- the interrupt
condition must be handled exactly like it is done for `yield()`.

Interruptor takes care about that but, at the moment, only for the
interruptible futures; there is mechanism for plain `seastar::future`.
This patch paves the way for handling them as well.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
dccca6beb9 crimson/common: trace management of interrupt_cond in interruptor::async()
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
b9b4101571 crimson/common: fix method name in interrupt_cond_t::reset()'s debugs
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
ab9bd2430f osd: fix intersecion of green threads and interuptible
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
3e5c64a92d crimson/osd: wire SnapMapper::add_oid() with OpsExecuter
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
f6059ed582 crimson/osd: settle snap_trimq in PG and wire it up
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
cb2f8325e8 crimson/osd: settle SnapMapper within crimson'o PG
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
e57d3bb082 crimson/osd: PerShardState::start_operation() doesn't assume Ret is seastar::future<>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
0a0b5f8ddc crimson/osd: implement SnapMapper::OSDriver for crimson
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
a9b270baa0 crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future
The following corner case was unsupported till now:

```cpp
  return os->omap_get_values(
    ch, hoid, keys
  ).safe_then([] (FuturizedStore::omap_values_t&& vals) {
    return 0;
  }, FuturizedStore::read_errorator::all_same_way([] (auto&) {
    return -1;
  })).get();
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
89b322d387 common/ceph_mutex: add green thread-requiring condition variable for crimson
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
8c01fa651e crimson/osd, osd: share SnapMapper with crimson
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
c77fb5f815 osd: drop the scrub_purged_snaps() declaration from SnapMapper
Nowadays this is a dead code; the actual definition is in `OSD`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
742cb2efb1 crimson/osd: SnapMapper::record_purged_snaps() takes abstratced store and txn
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
3fef67bf36 crimson/osd: convert SnapMapper::record_purged_snaps() to OSTransaction
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-28 16:22:04 +00:00