Commit Graph

133453 Commits

Author SHA1 Message Date
Ronen Friedman
5e1b230554 osd: modify two ScrubMap data members from fields to full variables
as there is no size advantage in this specific case.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-08-19 19:21:20 +03:00
Ronen Friedman
792e93607c osd: fix ScrubMap::swap() to include all ScrubMap data members
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2022-08-19 19:21:20 +03:00
Casey Bodley
859cab820f
Merge pull request #47632 from linuxbox2/wip-rgw-53455
rgw/s3: add conditional json output for s3:ListBucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-19 11:06:27 -04:00
Casey Bodley
f61b38dc82
Merge pull request #46443 from ybwang0211/head_obj
rgw: The error reported in the "rgw_s3_prepare_decrypt" function is unreasonable

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-19 09:25:12 -04:00
Casey Bodley
16c0ba743b
Merge pull request #46547 from Huber-ming/check_appendable
rgw: check object appendable when check_disk_state

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-19 09:24:35 -04:00
Casey Bodley
22d11aeb19
Merge pull request #47228 from Huber-ming/multi_meta
rgw: 'bucket check' deletes index of multipart meta when its pending_map is nonempty

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-19 09:23:58 -04:00
Casey Bodley
436cbbe0e4
Merge pull request #47547 from mdw-at-linuxbox/wip-master-xamzdate-1
rgw: x-amz-date change breaks certain cases of aws sig v4.

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2022-08-19 09:23:44 -04:00
Rishabh Dave
cef4c8d832
Merge pull request #45036 from rishabh-d-dave/qa-fix-is_mounted
qa/cephfs: upgrade is_mounted() in mount.py

Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2022-08-19 18:43:59 +05:30
Rishabh Dave
775ed5fe64
Merge pull request #45869 from neesingh-rh/fix_55234
mgr/snap_schedule: replace .snap with the client configured snap dir name

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-08-19 18:38:51 +05:30
Rishabh Dave
0a4fe3904f
Merge pull request #46749 from ibm-s390-cloud/fsmap_switch_to_iterator
fsmap: switch to using iterator based loop

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2022-08-19 18:32:38 +05:30
Radoslaw Zarzynski
c04256bab8 msg: reset ProtocolV2's frame assembler in appropriate thread
`set_is_rev1()` actually resets the internal state of `FrameAssembler`,
so -- to avoid racing -- it should be performed by the same thread that
takes care about executing `ProtocolV2::write_message()`.

```cppp
void set_is_rev1(bool is_rev1) {
  m_descs.clear();
  m_flags = 0;
  m_is_rev1 = is_rev1;
}
```

See comments in the tracker for analysis.

Fixes: https://tracker.ceph.com/issues/55851
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-08-19 12:15:41 +00:00
Yingxin Cheng
046477f52e crimson/os/seastore/transaction_manager: change read_extent_iertr to use get_pin_iertr
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-08-19 17:21:41 +08:00
Yingxin
d925198451
Merge pull request #47691 from myoungwon/wip-cbjournal-integration
crimson/os/seastore: CBJournal integration works

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-08-19 17:03:34 +08:00
Radoslaw Zarzynski
6db862f15c
Merge pull request #47597 from NitzanMordhai/wip-nitzan-crimson-watch-list-empty-return-code
crimson/osd: watch list empty return incorrect

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Sam Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-08-19 10:59:01 +02:00
Kefu Chai
f143841394
Merge pull request #47670 from rzarzynski/wip-common-ftbfs-systemd
common: fix a FTBFS in Journald.cc when building without systemd

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-08-19 15:34:47 +08:00
Kefu Chai
132de8c844
Merge pull request #47680 from tchaikov/wip-crimson-cleanups
crimson: cleanups to use `IsFullKey` concept

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-08-19 12:40:06 +08:00
myoungwon oh
02a38a00bd crimson/os/seastore: introduce finish_commit to update tails after trim transaction
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-08-19 12:42:25 +09:00
myoungwon oh
1c33a2aaf5 crimson/os/seastore/cbjournal: rename journal_tail to dirty_tail
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-08-19 12:42:06 +09:00
myoungwon oh
41469d628e crimson/os/seastore/cbjournal: make cbjournal to store alloc_tail
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-08-19 12:41:23 +09:00
myoungwon oh
8aab9256b1 crimson/os/seastore/cbjournal: make journal_tail and written_to to use journal_seq_t instead of rbm_abs_addr
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-08-19 12:40:43 +09:00
myoungwon oh
4ba6ade57c crimson/os/seastore: divide the role of open_device_read_header into device.mount()
and open_for_mount()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2022-08-19 12:39:42 +09:00
Matthew Taylor
a0d82e2704 doc: Remove inactive Ceph mirrors.
hk.ceph.com - notified mirror maintainer list of mirror decommission on 2020-08-31
us-east.ceph.com - mirror has been offline for years.

Signed-off-by: Matthew Taylor <matthew.taylor@digitalpacific.com.au>
2022-08-19 09:47:26 +08:00
Casey Bodley
74272753b8
Merge pull request #47615 from adamemerson/wip-objv-doc
rgw: Document `RGWOvjVersionTracker`

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-08-18 18:22:15 -04:00
Ilya Dryomov
ad504b10f6 librbd/cache/pwl: generate image cache state json under m_lock
The previous commit moved the entirety of write_image_cache_state()
from under m_lock.  This was a step too far because the generated image
cache state json is no longer guaranteed to be consistent.

Arrange for m_lock to still be held during image cache json generation
but released before owner_lock is grabbed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-08-18 20:11:12 +02:00
Patrick Donnelly
e9730461a2
qa/workunites/fs: add first damage test script
Not part of the QA test suites because we don't want to install the
associated script.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-08-18 13:57:38 -04:00
Patrick Donnelly
5fa0825bf3
tools/cephfs: add basic detection/cleanup tool for dentry first damage
This tool iterates through the metadata pool looking for dentries with
non-sensical "first" (snapshot) metadata. It only specifically looks for
damage of the kind where the "first" value is greater than the latest
generated snapshot for the file system. It does not detect other kinds
of genuine damage to the "first" metadata.

Fixes: https://tracker.ceph.com/issues/56140
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-08-18 13:57:28 -04:00
Kefu Chai
82dff11ef0 crimson/os: make _node_fields_013_t::insert_at() use IsFullKey
less repeatings this way

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-19 00:55:26 +08:00
Daniel Gryniewicz
00bc8579a4 RGW - Admin - Pass user to created bucket
The user was not being passed to the created bucket.  This means that,
in some operations, radosgw-admin was crashing trying to de-reference
the user.

Fixes: https://tracker.ceph.com/issues/56810

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-08-18 12:30:04 -04:00
Josh Durgin
857f2bdb60
Merge pull request #47288 from batrick/i56274
debian: add libsqlite3-mod-ceph dependency for ceph-mgr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-08-18 09:29:28 -07:00
Kefu Chai
72f91fad9e crimson/os: define *::from_key() with IsFullKey
more compact this way.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-19 00:28:45 +08:00
Kefu Chai
b374301ece crimson: move helper classes down
so we can use the concept of `IsFullKey` for defining accessors in the
incoming commit.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-18 23:45:23 +08:00
Kefu Chai
d143757d24 crimson: define encode_key() with IsFullKey constrait
also, let the compiler to deduce the type of the template parameter,
less repeating and less error-prone this way.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-08-18 23:10:23 +08:00
Matan Breizman
f3d7c7e8f6 include/buffer: clear_and_dispose() formatting
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 12:34:34 +00:00
Radosław Zarzyński
dc0d636af7 common: fix a FTBFS in Journald.cc when building without systemd
This came from Josh Salomon:

```
/home/jsalomon/src/prim_score/src/common/Journald.h:93:24: error: ‘SubsystemMap’ does not name a type
   93 |   JournaldLogger(const SubsystemMap *) {}
      |                        ^~~~~~~~~~~~
/home/jsalomon/src/prim_score/src/common/Journald.h:94:23: error: ‘Entry’ does not name a type
   94 |   int log_entry(const Entry &) {
      |
```

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-08-18 13:45:24 +02:00
Arthur Outhenin-Chalandre
50f4580686
Merge pull request #47633 from bosc0/alerts_multicluster
prometheus: add multicluster support to alerts
2022-08-18 13:35:24 +02:00
Matan Breizman
dcca182d9f qa/suites/crimson-rados: Enable test_rados snap test in Crimson's suite
Re-enanling snap tests in test_rados.py.
Currently disabling rollback tests.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 11:00:11 +00:00
Matan Breizman
7171ca64c3 crimson/osd: hold head object state before executing ops
In case that the head object will be removed, we should hold
its ObjectState before the removal.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
0f89eb672a crimson/osd: whiteout removed head object with existing clones
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
405f822518 crimson/osd/pg: with_clone_obc: avoid coid capture
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
13225cdeee crimson/osd: add try-catch block when decoding ObjectState
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
c769d2a80c crimson/osd: apply stats after make_writeable
make_writeable keeps track of few stats counters (e.g num_object_clones).
Therfore, we should apply the stats after it.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
b0c165a7b3 crimson/osd: pg_backend formatting changes
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
80103ca585 crimson/osd: Snapshots: copy-on-write clones
Creating copy-on-write (COW) clones in the backing store when a write arrives
after a snapshot is taken of an object.
Both for Pool or Self-managed snapshotting mechanisms.

Trello: https://trello.com/c/yTDCGNCf

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan Breizman
6cbbb501f2 crimson/osd/pg: with_clone_obc() return object_corrupted when not found
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2022-08-18 10:59:26 +00:00
Matan
521f83aa21
Merge pull request #47554 from Matan-B/wip-matanb-c-rados-python
qa/suite/crimson-rados: Fix rados_python test

Reviewed by: Samuel Just <sjust@redhat.com>
Reviewed by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed by: Nitzan Mordechai <nmordech@redhat.com>
2022-08-18 13:52:45 +03:00
Neeraj Pratap Singh
d3552e8107 doc: drop perf stats o/p string.
Dropping the output of this command entirely, as it is
difficult to maintain it in the docs.A minor change in
the command output invalidates the docs.

Fixes: https://tracker.ceph.com/issues/56162
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
2022-08-18 14:31:38 +05:30
Matan
bccd2dbb20
Merge pull request #47653 from Matan-B/wip-matanb-snapmapper-pending-typo
PendingReleaseNotes: fix typo in 15.2.17

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
2022-08-18 08:56:38 +03:00
Matan
da4e28a53f
Merge pull request #47392 from Matan-B/wip-matanb-memstore-subsys
common: MemStore debug subsystem

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-08-18 08:48:59 +03:00
Yingxin
9d6d507792
Merge pull request #47643 from aisakaki/wip-omap-merge
crimson/os/seastore/omap: optimize merge policy

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-08-18 13:20:55 +08:00
Yingxin
8b15e5f976
Merge pull request #47558 from rzarzynski/wip-crimson-extradbgs
crimson/{net,osd}: add debugs after bughunt's investigations

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2022-08-18 09:23:53 +08:00