RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-03-25 11:48:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
136,682 Commits 226 Branches 396 Tags 1.7 GiB
0ce6d03bbc
Commit Graph

136682 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Feng Hualong
0ce6d03bbc Crypto: Add QAT batch mode 
Now, the current code does not allow qat to exert
its acceleration advantage, but leads to poor performance
of qat. The QAT batch mode is implemented here, so that
the QAT performance can be displayed.

When we scale up the number of concurrent requests and worry about
QAT instance will be bottleneck, we can fall back to CPU.

And there add a parameter to set the times of the number of QAT
instance to wait for free instance, which can avoid QAT to be
not busy and make sure QAT full of utilization as much as possible.

max_queue_size is up to max_requests

Add optional_yield in RGWPutObj_BlockEncrypt and
RGWGetObj_BlockDecrypt. Make it with coroutine and non-coroutine mode.

Signed-off-by: Feng Hualong <hualong.feng@intel.com>
 2023-03-10 17:02:42 +08:00
Yingxin
943ccfd0bc
Merge pull request #50317 from xxhdx1985126/wip-rbm-prepare-write 
crimson/os/seastore/epm: also do prepare_write for rbm

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
 2023-03-02 10:07:29 +08:00
zdover23
e83c189e5a
Merge pull request #50335 from adk3798/cephadm-compatability-update 
doc/cephadm: update cephadm compatability and stability page

Reviewed-by: Zac Dover <zac.dover@proton.me>
 2023-03-02 07:36:46 +10:00
Adam King
383f46aa79 doc/cephadm: update cephadm compatability and stability page 
This page is very out of date. This commit probably doesn't
cover everything there is to say about stability and compatability
in cephadm, but it at least gets it noticeably closer to reality

Signed-off-by: Adam King <adking@redhat.com>
 2023-03-01 16:10:41 -05:00
zdover23
0bd1b34742
Merge pull request #50316 from zdover23/wip-doc-2023-03-01-radosgw-multisite-diagram-object-replication-text-revision 
doc/rgw: improve diagram caption

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
 2023-03-02 04:04:25 +10:00
Zac Dover
e8aa50e240 doc/rgw: improve diagram caption 
Improve the syntax and semantics of the caption of the diagram
"Replication of Object Data Between Zones" in doc/radosgw/multisite.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
 2023-03-02 02:27:53 +10:00
Yuval Lifshitz
4478c0941f
Merge pull request #50256 from yuvalif/wip-yuval-zipper-notifications 
rgw/notifications: add bucket notification configuration to zipper

Reviewed-By: dang, cbodley
 2023-03-01 16:13:55 +02:00
Yuval Lifshitz
6850e35555
Merge pull request #48181 from huynp1999/wip-rgw-kafka-add-sasl-mechanism 
src/rgw: SASL mechanism implem

Reviewed-By: ivancich, anthonyeleven, yuvalif
 2023-03-01 16:12:47 +02:00
Radoslaw Zarzynski
0ad603862b
Merge pull request #49286 from rzarzynski/wip-crimson-snaptrimmer 
crimson/osd, osd: bring snap trimming to crimson

Reviewed-by: Samuel Just <sjust@redhat.com>
 2023-03-01 08:57:27 +01:00
Xuehan Xu
1923f8db92 crimson/os/seastore/epm: also do prepare_write for rbm 
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
 2023-03-01 07:42:16 +00:00
Venky Shankar
cca84e653d Merge PR #48720 into main 
* refs/pull/48720/head:
	qa: fix api failure issue caused during a dashboard test
	PendingReleaseNotes: noted new MDSMap field refuse_client_session
	qa: added two testcases
	client: do not initiate session if flag refuse_client_session is set
	mds: do not reconnect when refuse_client_session is set
	mds: add new feature to block clients from establishing sessions

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
 2023-03-01 07:54:18 +05:30
Neha Ojha
707af78f9d
Merge pull request #49006 from yanghonggang/yhg-upgrade-rocksdb-7.7.3 
rocksdb: Upgrade rocksdb to latest facebook's v7.9.2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
 2023-02-28 15:50:21 -08:00
Radoslaw Zarzynski
d90d81585b qa/valgrind.supp: suppress the ColumnFamily ctor called on init 
```
    <frame>
      <ip>0x1712F1E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::BytewiseComparator()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/util</dir>
      <file>comparator.cc</file>
      <line>304</line>
    </frame>
    <frame>
      <ip>0x162361C</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::ColumnFamilyOptions::ColumnFamilyOptions()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/options</dir>
      <file>options.cc</file>
      <line>123</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>Options</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/rocksdb/include/rocksdb</dir>
      <file>options.h</file>
      <line>1409</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>RocksDBStore::init(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;)</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/kv</dir>
      <file>RocksDBStore.cc</file>
      <line>400</line>
    </frame>
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 20:58:27 +00:00
Radoslaw Zarzynski
0de4d99c72 qa/valgrind.supp: suppress the get-or-create CacheItemHelper 
```
// Get an CacheItemHelper pointer for value type T and role R.
template <typename T, CacheEntryRole R>
Cache::CacheItemHelper* GetCacheItemHelperForRole() {
  static Cache::CacheItemHelper cache_helper(
      BlocklikeTraits<T>::SizeCallback, BlocklikeTraits<T>::SaveToCallback,
      GetCacheEntryDeleterForRole<T, R>());
  return &cache_helper;
}
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 20:43:48 +00:00
Radoslaw Zarzynski
9949151620 qa/valgrind.supp: even broader suppression for rocksdb's init 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 20:43:41 +00:00
Adam King
0aef3d9741
Merge pull request #50308 from adk3798/mock-fqdn-secure-alertmanager 
mgr/cephadm: mock get_fqdn in secure alertmanager config test

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
 2023-02-28 14:57:44 -05:00
Radoslaw Zarzynski
a3aa624db4 qa/valgrind.supp: ingore rocksdb leak in option parsing 
This suppression is needed since upgrading to v7.9.2.
See PR https://github.com/ceph/ceph/pull/49006.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 19:13:39 +00:00
Radoslaw Zarzynski
4dbe8f56b6 crimson/osd: fix compilation guard around specialization for libfmt => 9 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
b03ab045e9 crimson/osd: handle snap trim's interruption as any other error 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
38c030e2cb crimson/osd: handle PG_STATE_SNAPTRIM_ERROR when trimming snaps 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
f091a70e3c crimson/osd: errorate SnapTrimEvent 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
e0bb021796 crimson/osd: manage PG_STATE_SNAPTRIM when trimming snaps 
After a few tries:

```
$ bin/ceph -s
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2022-12-19T20:18:21.856+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.857+0000 7ff87b46f700  1 build_initial for_mkfs: 0
2022-12-19T20:18:21.860+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.861+0000 7ff87b46f700  1 build_initial for_mkfs: 0
  cluster:
    id:     347aedc9-d72f-4a34-98fc-26cb04b0c2fc
    health: HEALTH_WARN
            11 mgr modules have failed dependencies
            1 pool(s) do not have an application enabled
            1 pool(s) have no replicas configured

  services:
    mon: 1 daemons, quorum a (age 3m)
    mgr: x(active, since 3m)
    osd: 1 osds: 1 up (since 3m), 1 in (since 3m)

  data:
    pools:   1 pools, 1 pgs
    objects: 95 objects, 6 B
    usage:   1.0 GiB used, 100 GiB / 101 GiB avail
    pgs:     1 active+clean+snaptrim

  io:
    client:   1.2 KiB/s wr, 0 op/s rd, 1 op/s wr
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
6923b05d70 crimson/osd: drop C++'s ranges to workaround unbuildability on Clang 
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: while checking constraint satisfaction for template 'operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' required here
    for (const auto &i : subops | std::views::keys) {
                                ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: in instantiation of function template specialization 'std::ranges::views::__adaptor::operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' requested here
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/iterator_concepts.h:963:7: note: candidate template ignored: constraints not satisfied [with _Tp = std::ranges::ref_view<const std
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
6dce7df440 crimson/osd: simplify SnapTrimObjSubEvent::start() 
Throwing away the NOP-looking-like `maybe_delay` concept became
possible after eradication of the `with_interruption` call nested
within `SnapTrimSubEvent`. Before, on Ubuntu and GCC 11.2.1, lack
of it was the reson behind assertion failures on resetting the
`thread_local` intteruption condition.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
877c272c2f crimson/osd: drop the nested interruptors in snap trimming 
`crimson::interruptible` does not support that. See the
`DISABLED_nested_interruptors` unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
c08d06fb0d crimson/test: verify nesting of interruptors 
It doesn't work.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
9c719406cc crimson/osd: errorate SnapTrimObjSubEvent::remove_clone() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
912a940f46 crimson/osd: convert SnapTrimObjSubEvent::remove_or_update to ertr 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
30db140344 crimson/osd: dissect snapset update into SnapTrimObjSubEvent::update_head() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
dda6d453d1 crimson/osd: dissect head whiteout removal into SnapTrimObjSubEvent::remove_head_whiteout() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
37f2bda50c crimson/osd: drop data aliases from SnapTrimObjSubEvent::adjust_snaps() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
355b7f7ebd crimson/osd: dissect clone removal into SnapTrimObjSubEvent::remove_clone() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
bb756b5983 crimson/osd: drop data aliases from SnapTrimObjSubEvent::remove_or_update() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
bc5d15e7fc crimson/osd: simplify informing SnapMapper about new mappings 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
b9d81029ae crimson/osd: implement basic osd_snap_trim_sleep support 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
6df8e3965c crimson/osd: introduce WaitSubop stage to SnapTrimEvent 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
9245c7650d crimson/osd: snap trimming informs OpsExecuter about altered objects 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
ad22a9a228 common: add debugs to crimson's green_condition_variable 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
9ce59d7648 crimson/osd: drop unused declaration of PG::fill_op_params_bump_pg_version() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:05 +00:00
Radoslaw Zarzynski
5d6dd2cba8 osd: SnapMapper::OSDriver uses crimson's interruptor::green_get(). 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
301b2c6b8a crimson/common: bring green_get() to interruptor 
The rationale
=============
Calling `get()` on an unavailable future can result in yielding
the reactor, so -- from the interruptor's POV -- the interrupt
condition must be handled exactly like it is done for `yield()`.

Interruptor takes care about that but, at the moment, only for the
interruptible futures; there is mechanism for plain `seastar::future`.
This patch paves the way for handling them as well.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
dccca6beb9 crimson/common: trace management of interrupt_cond in interruptor::async() 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
b9b4101571 crimson/common: fix method name in interrupt_cond_t::reset()'s debugs 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
ab9bd2430f osd: fix intersecion of green threads and interuptible 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
3e5c64a92d crimson/osd: wire SnapMapper::add_oid() with OpsExecuter 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
f6059ed582 crimson/osd: settle snap_trimq in PG and wire it up 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
cb2f8325e8 crimson/osd: settle SnapMapper within crimson'o PG 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
e57d3bb082 crimson/osd: PerShardState::start_operation() doesn't assume Ret is seastar::future<> 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
0a0b5f8ddc crimson/osd: implement SnapMapper::OSDriver for crimson 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00
Radoslaw Zarzynski
a9b270baa0 crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future 
The following corner case was unsupported till now:

```cpp
  return os->omap_get_values(
    ch, hoid, keys
  ).safe_then([] (FuturizedStore::omap_values_t&& vals) {
    return 0;
  }, FuturizedStore::read_errorator::all_same_way([] (auto&) {
    return -1;
  })).get();
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2023-02-28 16:22:04 +00:00