Commit Graph

129967 Commits

Author SHA1 Message Date
Casey
433782dbd5 cmake: use arrow's find_package modules
Signed-off-by: Casey <cbodley@redhat.com>
2022-03-14 10:19:02 -04:00
Casey
ed60aeed0b cmake: add WITH_SYSTEM_ARROW to skip submodule build
relies on a hack to find the installed ParquetConfig.cmake

Signed-off-by: Casey <cbodley@redhat.com>
2022-03-14 10:19:02 -04:00
Casey Bodley
2d80f0cd25 ceph.spec.in: seastar drops _FORTIFY_SOURCE from CFLAGS also
the arrow submodule builds some C sources that trip up on _FORTIFY_SOURCE in debug builds

[ 79%] Building C object src/arrow/CMakeFiles/arrow_objlib.dir/vendored/musl/strptime.c.o
In file included from /usr/include/time.h:25,
                 from /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-10531-gc73e1fda/rpm/el8/BUILD/ceph-17.0.0-10531-gc73e1fda/src/arrow/cpp/src/arrow/vendored/strptime.h:20,
                 from /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-10531-gc73e1fda/rpm/el8/BUILD/ceph-17.0.0-10531-gc73e1fda/src/arrow/cpp/src/arrow/vendored/musl/strptime.c:4:
/usr/include/features.h:381:4: error: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Werror=cpp]
  381 | #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
      |    ^~~~~~~
cc1: all warnings being treated as errors
make[5]: *** [src/arrow/CMakeFiles/arrow_objlib.dir/build.make:2543: src/arrow/CMakeFiles/arrow_objlib.dir/vendored/musl/strptime.c.o] Error 1

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-03-14 10:19:02 -04:00
Casey Bodley
b10364dc21 cmake: add submodule for utf8proc at v2.2.0
adds utf8proc submodule, needed by the arrow submodule in centos. add a
WITH_SYSTEM_UTF8PROC option that controls whether or not utf8proc is
built from submodule

non-system utf8proc is built as a static library to avoid conflicts with
system-provided libraries

ceph.spec.in sets WITH_SYSTEM_UTF8PROC=OFF until it's available in
centos

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-03-14 10:19:02 -04:00
Casey Bodley
2ca6d75521 cmake: add submodule for Apache Arrow at v6.0.1
adds an arrow submodule. when WITH_RADOSGW_SELECT_PARQUET is enabled,
the submodule is built as an external project and rgw links against its
imported Arrow::Parquet target

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-03-14 10:19:02 -04:00
Ilya Dryomov
0b44878cb2
Merge pull request #45235 from tchaikov/rbd-std-variant
librbd: s/boost::variant/std::variant/

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-14 11:36:39 +01:00
Yuri Weinstein
ad7a527c34
Merge pull request #44827 from ceph/wip-yuriw-crontab-master2
qa/tests: added quincy and removed master runs

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Adam King adking@redhat.com
2022-03-13 09:10:27 -07:00
Kefu Chai
974811e819
Merge pull request #45276 from tchaikov/wip-rapidjson
submodule: bump up RapidJSON

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-13 20:38:35 +08:00
Kefu Chai
3a93b40721 librbd: s/boost::variant/std::variant/
boost::variant explicitly prevent it from being compared other types
by marking the return type of, for instance operator==(const T&) as
"void", if T is not identical to the variant type. so address this,
let's use std::variant<> instead.

more standard compliant and simpler this way.

because of https://cplusplus.github.io/LWG/issue3052, libstdc++
only specializes for std::variant<> for std::visit(), while libc++
allows the derived types of std::variant<> to be "visited", hence
we add an adaptor for SnapshotNamespace.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-11 20:46:39 +08:00
Zack Cerza
e39304dcd6
Merge pull request #45034 from ceph/cephadm-conditional-hostname
qa/tasks/cephadm: Make hostname change idempotent
2022-03-10 19:07:17 -07:00
Yuri Weinstein
21928dd489
Merge pull request #44925 from JoshSalomon/balancer-perf
balancer: Improve performance in rare cases

Reviewed-by: Laura Flores <lflores@redhat.com>
2022-03-10 15:24:47 -08:00
Kamoltat Sirivadhna
5bfc71e85d
Merge pull request #45200 from kamoltat/wip-ksirivad-pg-num-max-reorder
osd/osd_types: pg_num_max reordering
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2022-03-10 12:01:27 -05:00
Anthony D'Atri
49c4be1174
Merge pull request #45330 from wxypro/fix-typos
doc/cephfs: fix typos
2022-03-10 00:55:19 -08:00
wangxinyu
b132d6d69b doc/cephfs: fix typos
fix typos

Signed-off-by: wangxinyu <wangxinyu@inspur.com>
2022-03-10 15:44:18 +08:00
Samuel Just
40610e8fb7
Merge pull request #43474 from xxhdx1985126/wip-epm-strategy
crimson/os/seastore: implement Sprite LFS' gc and extent placement strategy to lower gc cost

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-03-09 18:34:00 -08:00
Neha Ojha
f436973332
Merge pull request #45244 from Matan-B/wip-matanb-revert-keyring
Revert "doc/dev: Running workunits locally"

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-09 15:54:21 -08:00
Neha Ojha
1187405ead
Merge pull request #45305 from Thingee/update-foundation-mem-202203
docs: Updating Foundation member list for 202203

Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>
Reviewed-by:Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-09 15:19:35 -08:00
Yuri Weinstein
c6da0c55a2
Merge pull request #44913 from benhanokh/safe_shutdown_v2
OSD::Modify OSD Fast-Shutdown to work safely i.e. quiesce all activit…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2022-03-09 13:53:43 -08:00
Kamoltat Sirivadhna
36b0dc0954
Merge pull request #45318 from kamoltat/wip-fix-autoscale-doc
doc/rados/operations/placement-groups:  ...fix typo...
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by:  Neha Ojha <nojha@redhat.com>
2022-03-09 12:41:01 -05:00
David Galloway
6917edc743
Merge pull request #45272 from petrutlucian94/build_fix
Fix Windows build
2022-03-09 10:42:36 -05:00
Kamoltat
7f1c7637e2 doc/rados/operations/placement-groups: typo two 'the'
typo, should not have two 'the' next to each
other.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-03-09 15:36:20 +00:00
Kamoltat Sirivadhna
791ea7e7e8
Merge pull request #45279 from kamoltat/wip-ksirivad-fix-bulk-doc
doc/rados/operations/placement-groups: fix --bulk commands
Reviewed-by: Neha Ojha <nojha@redhat.com>
2022-03-09 10:13:21 -05:00
Kefu Chai
b04655645b cls/rbd: define SnapshotNamespace's ctor using its parent
simpler this way. also this prevent the compiler from trying to
convert a random value into SnapshotNamespace just because it
has a templated constructor when it tries to lookup a candidate
of operator<<(ostream&, Random value).

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-09 21:53:17 +08:00
Kefu Chai
4382062ba3
Merge pull request #45302 from rzarzynski/wip-crimson-fix-noenoent-no-remove
crimson/osd: fix no ENOENT when removing already removed object

Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2022-03-09 21:34:47 +08:00
Kefu Chai
9dfaa10791
Merge pull request #45310 from inspur-wyq/wip-doc-cephadm
doc/cephadm/adoption.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-09 21:23:06 +08:00
Kefu Chai
4bc772aed3
Merge pull request #45266 from tchaikov/wip-sphinx
admin/doc-requirements: bump sphinx to 4.4.0

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2022-03-09 21:20:24 +08:00
Ilya Dryomov
249e5e352c
Merge pull request #45297 from idryomov/wip-test-notify-rebuild-object-map
test/librbd/test_notify.py: effect post object map rebuild assert

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2022-03-09 10:15:56 +01:00
wangyunqing
e4db28f6b2 doc/cephadm/adoption.rst: fix typos
Signed-off-by: wangyunqing <wangyunqing@inspur.com>
2022-03-09 16:57:20 +08:00
Samuel Just
1af515da6d
Merge pull request #45278 from rzarzynski/wip-crimson-fix-logs-in-cls
crimson, cls: fix the inability to print logs from plugins.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2022-03-08 23:07:42 -08:00
Kamoltat
4a01fc7798 doc/rados/operations/placement-groups: fix --bulk commands
Some parts of the documents regarding
the bulk flag  have typos.

Command for creating a pool

was: `ceph osd create test_pool --bulk`

should be: `ceph osd pool create test_pool --bulk`

Command for setting bulk value in a pool

was: `ceph osd pool set test_pool bulk=<true/false/1/0>`

should be: `ceph osd pool set test_pool bulk <true/false/1/0>`

Also removed a bit of trailing white spaces.

Changed `complements` to `complement`.

https://tracker.ceph.com/issues/54485

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2022-03-09 04:27:33 +00:00
Mike Perez
5e105778a7 docs: Update Foundation member list for March #45305
Signed-off-by: Mike Perez <thingee@gmail.com>
2022-03-08 14:06:04 -08:00
Adam King
e338e1f7d2
Merge pull request #45211 from phlogistonjohn/jjm-nfs-cleanups3
mgr/nfs: Additional fixes for NFS export management without orchestration

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-03-08 13:42:07 -05:00
Adam King
43cc29466a
Merge pull request #45281 from rkachach/fix_issue_54487
mgr/cephadm: fixing MDSSpec ctr

Reviewed-by: Adam King <adking@redhat.com>
2022-03-08 13:33:54 -05:00
Adam King
03a0c3b384
Merge pull request #45039 from Matan-B/wip-matanb-nfs-timeout
qa/tasks/cephfs: increase timeout in test_nfs.py

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-03-08 13:31:40 -05:00
Radoslaw Zarzynski
66fa95f318 crimson/osd: fix no ENOENT when removing already removed object
This patch deals with the following problem:

```
[rzarzynski@o06 build]$ RBD_FEATURES="21" ./bin/ceph_test_cls_rbd --gtest_filter=TestClsRbd.create
Running main() from gmock_main.cc
Note: Google Test filter = TestClsRbd.create
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from TestClsRbd
[ RUN      ] TestClsRbd.create
../src/test/cls_rbd/test_cls_rbd.cc:467: Failure
Expected equality of these values:
  -2
  ioctx.remove(oid)
    Which is: 0
[  FAILED  ] TestClsRbd.create (10 ms)
[----------] 1 test from TestClsRbd (10 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (2805 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TestClsRbd.create

 1 FAILED TEST
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-08 18:27:27 +00:00
Casey Bodley
dad598dc3e
Merge pull request #45214 from cbodley/wip-rgw-ancient-curl-workaround
cmake/rgw: remove workaround for ancient libcurl

Reviewed-by: Yehuda Sadeh <ysadehwe@redhat.com>
2022-03-08 10:32:39 -05:00
Casey Bodley
7eaed47aa6
Merge pull request #44488 from cfsnyder/wip-53788-reopen-ops-log-sighup
rgw: reopen ops log file on sighup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 10:30:36 -05:00
Casey Bodley
e11e62e5e5
Merge pull request #45212 from cbodley/wip-54435
rgw: prevent spurious/lost notifications in the index completion thread

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2022-03-08 10:29:48 -05:00
Casey Bodley
e98697fdcb
Merge pull request #44859 from cfsnyder/wip-54112-rgw-segfault-init-fetch
rgw: fix segfault in UserAsyncRefreshHandler::init_fetch

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 08:31:54 -05:00
Casey Bodley
eb25bf2557
Merge pull request #44946 from adamemerson/wip-54208
rgw: Fix data race in ChangeStatus

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 08:31:15 -05:00
Casey Bodley
112c260875 rgw: initialize RGWIndexCompletionManager thread after related members
resolves valgrind issues about RGWIndexCompletionManager::process()
using uninitialized memory

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-03-08 08:28:35 -05:00
Ilya Dryomov
5b94f080a3 test/librbd/test_notify.py: effect post object map rebuild assert
Instead of just optionally skipping update_features test, commit
9c0b239d70 ("qa/upgrade: conditionally disable update_features
tests") moved it after rebuild_object_map test.  This isn't right
because update_features test invalidates the object map as a side
effect and rebuild_object_map test is what makes it valid again:

  assert((image.flags() & RBD_FLAG_OBJECT_MAP_INVALID) == 0)

Let's make this assert effective, at least when update_features
test isn't skipped.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-08 13:59:58 +01:00
Samuel Just
78af510044
Merge pull request #45274 from Matan-B/wip-matanb-c-ops-debug
crimson/osd: do_pgnls_common debug message format

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-07 19:15:36 -08:00
Radoslaw Zarzynski
45c7b4b51a crimson/osd, osd: don't use VLA in cls_log() anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-08 01:51:09 +00:00
Radoslaw Zarzynski
f62329f9a4 crimson, cls: fix the inability to print logs from plugins.
`cls_log()` of the interface between an OSD and a plugin
(a Ceph Class) is implemented on top of the `dout` macros
and shared between crimson and the classical OSD.

Unfortunately, when a plugin is hosted inside crimson,
this causes the inability to send to any in-plugin generated
message to the `seastar::logger` for the `objclass` subsystem.

This patch differtiates the implementation of `cls_log()`,
and thus allow the `seastar::logger`-based implementation
of `dout` to be used.

After the fix:

```
DEBUG 2022-03-07 14:12:14,124 [shard 0] osd - handling op call on object 1:424638ea:::image1:head
DEBUG 2022-03-07 14:12:14,124 [shard 0] osd - calling method rbd.create, num_read=0, num_write=0
DEBUG 2022-03-07 14:12:14,125 [shard 0] objclass - <cls> ../src/cls/rbd/cls_rbd.cc:787: create object_prefix=image1 size=0 order=22 features=0
DEBUG 2022-03-07 14:12:14,125 [shard 0] osd - handling op omap-get-vals-by-keys on object 1:424638ea:::image1:head
DEBUG 2022-03-07 14:12:14,125 [shard 0] osd - omap_get_vals_by_keys: object does not exist: 1:424638ea:::image1:head
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-03-08 01:31:10 +00:00
Adam C. Emerson
ec81f225e3
Merge pull request #44831 from adamemerson/wip-prepend
build: PREPEND -Og, not APPEND

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
2022-03-07 15:51:24 -05:00
Ilya Dryomov
25cf302049
Merge pull request #43858 from rzarzynski/wip-msgr-no-cpp_error-on-read
msg/async: remove the misleading error description on read failures

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2022-03-07 18:30:03 +01:00
Kefu Chai
f6b9693b43
Merge pull request #45277 from rzarzynski/wip-crimson-fix-bo-in-pgbackend
crimson/osd: fix buffer overflow due to missed debug's param

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-03-08 00:28:22 +08:00
Kefu Chai
ded0b4d25f submodule: bump up rapidjson submodule
to include the C++20 fixes introduced recently.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2022-03-08 00:24:48 +08:00
Redouane Kachach
eb4e3f2494
mgr/cephadm: fixing MDSSpec ctr
Fixes issue: https://tracker.ceph.com/issues/54487

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-03-07 17:03:08 +01:00