Commit Graph

130087 Commits

Author SHA1 Message Date
chunmei-liu
260b15ab05 crimson/seastore: change seastore_device_size
to be half of the original value.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-03-10 13:46:59 -08:00
Daniel Gryniewicz
8966edd2fb RGW - Get quota on OPs with a bucket
Bucket head OPs should have quota in the output.  However, we were only
fetching quota on OPs that also had an object.  The object itself is not
necessary for quota (although a bucket is).  Change it so that we get
quota on bucket OPs as well.

Fixes https://tracker.ceph.com/issues/54488

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-03-10 15:01:38 -05:00
Kamoltat Sirivadhna
5bfc71e85d
Merge pull request #45200 from kamoltat/wip-ksirivad-pg-num-max-reorder
osd/osd_types: pg_num_max reordering
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-03-10 12:01:27 -05:00
Nizamudeen A
79f8545deb mgr/dashboard: BDD approach for the dashboard cephadm e2e
Files under the directory cypress/integration/common/* will contain
common specs which can be used on all the .feature files. We can change
the common directory to cypress/integration/* from the package.json, but
if we do that now then we'll need to take care of all the absolute
import in that path. So for now at least that's not a good choice.

The bug in the cypress-browserify-preprocessor which doesn't allow to
take our tsconfig.json file forced me to go with relative imports rather
than the absolute import. We'll need to wait for this to be fixed before
 changing all our tests to BDD.

Fixes: https://tracker.ceph.com/issues/54224
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-03-10 15:33:16 +05:30
Anthony D'Atri
49c4be1174
Merge pull request #45330 from wxypro/fix-typos
doc/cephfs: fix typos
2022-03-10 00:55:19 -08:00
wangxinyu
b132d6d69b doc/cephfs: fix typos
fix typos

Signed-off-by: wangxinyu <wangxinyu@inspur.com>
2022-03-10 15:44:18 +08:00
Aashish Sharma
4690238130 mgr/cephadm: fix cephadm errors
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2022-03-10 10:35:57 +05:30
Samuel Just
40610e8fb7
Merge pull request #43474 from xxhdx1985126/wip-epm-strategy
crimson/os/seastore: implement Sprite LFS' gc and extent placement strategy to lower gc cost

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-03-09 18:34:00 -08:00
Neha Ojha
f436973332
Merge pull request #45244 from Matan-B/wip-matanb-revert-keyring
Revert "doc/dev: Running workunits locally"

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-09 15:54:21 -08:00
Neha Ojha
1187405ead
Merge pull request #45305 from Thingee/update-foundation-mem-202203
docs: Updating Foundation member list for 202203

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
Reviewed-by:Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-09 15:19:35 -08:00
Yuri Weinstein
c6da0c55a2
Merge pull request #44913 from benhanokh/safe_shutdown_v2
OSD::Modify OSD Fast-Shutdown to work safely i.e. quiesce all activit…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2022-03-09 13:53:43 -08:00
Kamoltat Sirivadhna
36b0dc0954
Merge pull request #45318 from kamoltat/wip-fix-autoscale-doc
doc/rados/operations/placement-groups:  ...fix typo...
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by:  Neha Ojha <nojha@redhat.com>
2022-03-09 12:41:01 -05:00
David Galloway
6917edc743
Merge pull request #45272 from petrutlucian94/build_fix
Fix Windows build
2022-03-09 10:42:36 -05:00
Kamoltat
7f1c7637e2 doc/rados/operations/placement-groups: typo two 'the'
typo, should not have two 'the' next to each
other.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-03-09 15:36:20 +00:00
Kamoltat Sirivadhna
791ea7e7e8
Merge pull request #45279 from kamoltat/wip-ksirivad-fix-bulk-doc
doc/rados/operations/placement-groups: fix --bulk commands
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-09 10:13:21 -05:00
Kefu Chai
b04655645b cls/rbd: define SnapshotNamespace's ctor using its parent
simpler this way. also this prevent the compiler from trying to
convert a random value into SnapshotNamespace just because it
has a templated constructor when it tries to lookup a candidate
of operator<<(ostream&, Random value).

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-09 21:53:17 +08:00
Kefu Chai
4382062ba3
Merge pull request #45302 from rzarzynski/wip-crimson-fix-noenoent-no-remove
crimson/osd: fix no ENOENT when removing already removed object

Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-09 21:34:47 +08:00
Kefu Chai
9dfaa10791
Merge pull request #45310 from inspur-wyq/wip-doc-cephadm
doc/cephadm/adoption.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-09 21:23:06 +08:00
Kefu Chai
4bc772aed3
Merge pull request #45266 from tchaikov/wip-sphinx
admin/doc-requirements: bump sphinx to 4.4.0

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2022-03-09 21:20:24 +08:00
Ilya Dryomov
249e5e352c
Merge pull request #45297 from idryomov/wip-test-notify-rebuild-object-map
test/librbd/test_notify.py: effect post object map rebuild assert

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-03-09 10:15:56 +01:00
wangyunqing
e4db28f6b2 doc/cephadm/adoption.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2022-03-09 16:57:20 +08:00
Samuel Just
1af515da6d
Merge pull request #45278 from rzarzynski/wip-crimson-fix-logs-in-cls
crimson, cls: fix the inability to print logs from plugins.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-08 23:07:42 -08:00
Nikhilkumar Shelke
cc64c57c5c qa: verify idempotent subvolume creation for 'mode' argument
Fixes: https://tracker.ceph.com/issues/54375
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-03-09 11:55:14 +05:30
Nikhilkumar Shelke
d164e49bde mgr/volumes: the 'mode' should honor idempotent subvolume creation
currently the 'mode' argument is not honored idempotent
subvolume creation for existing subvolume.

Fixes: https://tracker.ceph.com/issues/54375
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
2022-03-09 11:25:27 +05:30
chunmei-liu
3cca3ed552 qa/suites/crimson-rados: add fixed-1 for seastore
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2022-03-08 21:29:04 -08:00
Kamoltat
4a01fc7798 doc/rados/operations/placement-groups: fix --bulk commands
Some parts of the documents regarding
the bulk flag  have typos.

Command for creating a pool

was: `ceph osd create test_pool --bulk`

should be: `ceph osd pool create test_pool --bulk`

Command for setting bulk value in a pool

was: `ceph osd pool set test_pool bulk=<true/false/1/0>`

should be: `ceph osd pool set test_pool bulk <true/false/1/0>`

Also removed a bit of trailing white spaces.

Changed `complements` to `complement`.

https://tracker.ceph.com/issues/54485

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-03-09 04:27:33 +00:00
Mike Perez
5e105778a7 docs: Update Foundation member list for March #45305
Signed-off-by: Mike Perez <thingee@gmail.com>
2022-03-08 14:06:04 -08:00
Adam King
e338e1f7d2
Merge pull request #45211 from phlogistonjohn/jjm-nfs-cleanups3
mgr/nfs: Additional fixes for NFS export management without orchestration

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-03-08 13:42:07 -05:00
Adam King
43cc29466a
Merge pull request #45281 from rkachach/fix_issue_54487
mgr/cephadm: fixing MDSSpec ctr

Reviewed-by: Adam King <adking@redhat.com>
2022-03-08 13:33:54 -05:00
Adam King
03a0c3b384
Merge pull request #45039 from Matan-B/wip-matanb-nfs-timeout
qa/tasks/cephfs: increase timeout in test_nfs.py

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-03-08 13:31:40 -05:00
Radoslaw Zarzynski
66fa95f318 crimson/osd: fix no ENOENT when removing already removed object
This patch deals with the following problem:

```
[rzarzynski@o06 build]$ RBD_FEATURES="21" ./bin/ceph_test_cls_rbd --gtest_filter=TestClsRbd.create
Running main() from gmock_main.cc
Note: Google Test filter = TestClsRbd.create
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from TestClsRbd
[ RUN      ] TestClsRbd.create
../src/test/cls_rbd/test_cls_rbd.cc:467: Failure
Expected equality of these values:
  -2
  ioctx.remove(oid)
    Which is: 0
[  FAILED  ] TestClsRbd.create (10 ms)
[----------] 1 test from TestClsRbd (10 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (2805 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TestClsRbd.create

 1 FAILED TEST
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-08 18:27:27 +00:00
Soumya Koduri
87c4cf0539 rgw: Update "CEPH_RGW_DIR_SUGGEST_LOG_OP" for remove entries
dir_suggest() expects "CEPH_RGW_DIR_SUGGEST_LOG_OP" flag to be set to log
completion of any pending Bucket Index transactions. This flag was not updated
for CEPH_RGW_REMOVE op entries in check_disk_state().
This change fixes the same.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-03-08 22:32:55 +05:30
Casey Bodley
dad598dc3e
Merge pull request #45214 from cbodley/wip-rgw-ancient-curl-workaround
cmake/rgw: remove workaround for ancient libcurl

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
2022-03-08 10:32:39 -05:00
Casey Bodley
7eaed47aa6
Merge pull request #44488 from cfsnyder/wip-53788-reopen-ops-log-sighup
rgw: reopen ops log file on sighup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 10:30:36 -05:00
Casey Bodley
e11e62e5e5
Merge pull request #45212 from cbodley/wip-54435
rgw: prevent spurious/lost notifications in the index completion thread

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-03-08 10:29:48 -05:00
Casey Bodley
e98697fdcb
Merge pull request #44859 from cfsnyder/wip-54112-rgw-segfault-init-fetch
rgw: fix segfault in UserAsyncRefreshHandler::init_fetch

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 08:31:54 -05:00
Casey Bodley
eb25bf2557
Merge pull request #44946 from adamemerson/wip-54208
rgw: Fix data race in ChangeStatus

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 08:31:15 -05:00
Casey Bodley
112c260875 rgw: initialize RGWIndexCompletionManager thread after related members
resolves valgrind issues about RGWIndexCompletionManager::process()
using uninitialized memory

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 08:28:35 -05:00
Ilya Dryomov
5b94f080a3 test/librbd/test_notify.py: effect post object map rebuild assert
Instead of just optionally skipping update_features test, commit
9c0b239d70 ("qa/upgrade: conditionally disable update_features
tests") moved it after rebuild_object_map test.  This isn't right
because update_features test invalidates the object map as a side
effect and rebuild_object_map test is what makes it valid again:

  assert((image.flags() & RBD_FLAG_OBJECT_MAP_INVALID) == 0)

Let's make this assert effective, at least when update_features
test isn't skipped.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-08 13:59:58 +01:00
Samuel Just
78af510044
Merge pull request #45274 from Matan-B/wip-matanb-c-ops-debug
crimson/osd: do_pgnls_common debug message format

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-07 19:15:36 -08:00
Radoslaw Zarzynski
45c7b4b51a crimson/osd, osd: don't use VLA in cls_log() anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-08 01:51:09 +00:00
Radoslaw Zarzynski
f62329f9a4 crimson, cls: fix the inability to print logs from plugins.
`cls_log()` of the interface between an OSD and a plugin
(a Ceph Class) is implemented on top of the `dout` macros
and shared between crimson and the classical OSD.

Unfortunately, when a plugin is hosted inside crimson,
this causes the inability to send to any in-plugin generated
message to the `seastar::logger` for the `objclass` subsystem.

This patch differtiates the implementation of `cls_log()`,
and thus allow the `seastar::logger`-based implementation
of `dout` to be used.

After the fix:

```
DEBUG 2022-03-07 14:12:14,124 [shard 0] osd - handling op call on object 1:424638ea:::image1:head
DEBUG 2022-03-07 14:12:14,124 [shard 0] osd - calling method rbd.create, num_read=0, num_write=0
DEBUG 2022-03-07 14:12:14,125 [shard 0] objclass - <cls> ../src/cls/rbd/cls_rbd.cc:787: create object_prefix=image1 size=0 order=22 features=0
DEBUG 2022-03-07 14:12:14,125 [shard 0] osd - handling op omap-get-vals-by-keys on object 1:424638ea:::image1:head
DEBUG 2022-03-07 14:12:14,125 [shard 0] osd - omap_get_vals_by_keys: object does not exist: 1:424638ea:::image1:head
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-08 01:31:10 +00:00
Adam C. Emerson
ec81f225e3
Merge pull request #44831 from adamemerson/wip-prepend
build: PREPEND -Og, not APPEND

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2022-03-07 15:51:24 -05:00
Ilya Dryomov
25cf302049
Merge pull request #43858 from rzarzynski/wip-msgr-no-cpp_error-on-read
msg/async: remove the misleading error description on read failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-07 18:30:03 +01:00
Kefu Chai
f6b9693b43
Merge pull request #45277 from rzarzynski/wip-crimson-fix-bo-in-pgbackend
crimson/osd: fix buffer overflow due to missed debug's param

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-08 00:28:22 +08:00
Kefu Chai
ded0b4d25f submodule: bump up rapidjson submodule
to include the C++20 fixes introduced recently.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-08 00:24:48 +08:00
Avan Thakkar
41ca50c81c mgr/dashboard: fix columns in host table with NaN Undefined
Fixes: https://tracker.ceph.com/issues/54068
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2022-03-07 21:44:38 +05:30
Redouane Kachach
eb4e3f2494
mgr/cephadm: fixing MDSSpec ctr
Fixes issue: https://tracker.ceph.com/issues/54487

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-03-07 17:03:08 +01:00
Gabriel BenHanokh
8d052558be os/bluestore: Fix problem with allocation desync
Close window for possibility to capture allocator state and bluefs state
that are not in sync.

Signed-off-by: Gabriel Benhanokh <gbenhano@redhat.com>
2022-03-07 17:42:40 +02:00
Adam Kupczyk
4fd98ce035 os/bluestore/bluefs: Fix sync compaction
Fixes problem with sync compaction (_rewrite_log_and_layout_sync).
There was a problem with not updating log_seq after compacting log.

It cause to stop _replay log right after first transaction.

... 20 bluefs _replay 0x0:  op_dir_create sharding
... 20 bluefs _replay 0x0:  op_dir_link  sharding/def to 21
... 20 bluefs _replay 0x0:  op_jump_seq 1025
... 10 bluefs _read h 0x555557c46400 0x1000~1000 from file(ino 1 size 0x1000 mtime 0.000000 allocated 410000 alloc_commit 410000 extents [1:0x1540000~410000])
... 20 bluefs _read left 0xff000 len 0x1000
... 20 bluefs _read got 4096
... 10 bluefs _replay 0x1000: stop: seq 1025 != expected 1026

This is a product of bluefs fine grain locks refactor.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 2f8e37064ca079c960929d7bb91e84fbf7f5cd47)

Conflicts:
	src/test/objectstore/test_bluefs.cc
2022-03-07 17:42:40 +02:00