Commit Graph

125896 Commits

Author SHA1 Message Date
Kefu Chai
05e4145856 mgr/dashboard/api: set a UTF-8 locale when running pip
ansible-core started to include files whose filenames are encoded in
non-ascii characters, so we have to use a more capable encoding for the
locale in order to install this package. otherwise we'd have following
error:

Collecting ansible-core<2.12,>=2.11.3
  Using cached ansible-core-2.11.4.tar.gz (6.8 MB)
ERROR: Exception:

Traceback (most recent call last):
  File "/tmp/tmp.fX76ASIrch/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
...
  File "/tmp/tmp.fX76ASIrch/venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 226, in untar_file
    with open(path, "wb") as destfp:
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 137-140: ordinal not in range(256)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-17 15:57:38 +08:00
Liu-Chunmei
a4558a2d26
Merge pull request #42731 from liu-chunmei/seastore-alloc-extent-hint
crimson/seastore alloc extent hint

Replace L_ADDR_MIN by obj hash in onode, omap, object_data when alloc_extent.

 Reviewed-by: Sam Just <sjust@redhat.com> and Cheng, Yingxin <yingxin.cheng@intel.com>
2021-08-16 17:15:22 -07:00
Chunmei Liu
1c0e6e7c8d crimson/seastore: change unit test using hint
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2021-08-16 13:56:08 -07:00
chunmei-liu
6633e30a32 crimson/seastore: replace L_ADDR_MIN by hint in omap
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-08-16 13:54:48 -07:00
Chunmei Liu
21adad11d7 crimson/seastore: use get_hint in object_data_handler
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2021-08-16 13:54:42 -07:00
Chunmei Liu
23693c1010 crimson/seastore: replace L_ADDR_MIN by obj hash in Onode
Signed-off-by: Chunmei Liu <chunmei.liu@intel.com>
2021-08-16 13:54:07 -07:00
Casey Bodley
f74b748a42
Merge pull request #42340 from mkogan1/wip-d3n-docs
doc/radosgw/d3n: add user docs

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2021-08-16 16:11:33 -04:00
Mark Kogan
542a304453 doc/radosgw/d3n: add user docs
(high-level feature description, setup instructions, config options)
Fixes: https://tracker.ceph.com/issues/51409

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2021-08-16 18:18:08 +03:00
Kefu Chai
dbdc677f11
Merge pull request #42790 from ronen-fr/wip-ronenf-std-fix
key_value_store: fix missing std

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-16 23:05:41 +08:00
Kefu Chai
ce5975486c
Merge pull request #42795 from tchaikov/wip-cmake-quiet
cmake: silence build output when building external deps

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-08-16 22:38:58 +08:00
Kefu Chai
89d8c79f50 cmake: do not quote string
otherwise we have

CMake Error at /home/jenkins-build/build/workspace/ceph-pull-requests/build/spdk-ext-prefix/src/spdk-ext-stamp/spdk-ext-build-.cmake:37 (message):
  Command failed: 2

   'env' '-i' 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' 'CC=/usr/bin/clang-10' '/usr/bin/make' 'EXTRA_CFLAGS=' '-fPIC' '-Wno-address-of-packed-member""'

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 21:27:57 +08:00
Kefu Chai
1fdd632d0c cmake: silence build output when building external deps
when download/building grafonnet-lib, dpdk, spdk, liburing and fio,
they dump lots of output during configuration and building phrases,
all of which is irrelevant to us. so let's just silence it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 21:27:57 +08:00
Kefu Chai
ab5aebf306
Merge pull request #42784 from tchaikov/wip-rgw_dencoder
rgw/rgw_dencoder: set identity_type when creating samples

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-08-16 21:12:05 +08:00
Ronen Friedman
eb0eef8d86 key_value_store: fix missing std
following PR #42742

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-08-16 13:09:35 +00:00
Kefu Chai
ca2d04f935
Merge pull request #42785 from tchaikov/wip-no-warnings
run-make-check: pass -Werror when compiling C++ code and a couple cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-08-16 21:08:33 +08:00
Kefu Chai
dac200dade
Merge pull request #42714 from ronen-fr/wip-ronenf-last-update
osd/scrub: only telling the scrubber of 'updates' events if these eve…

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-08-16 20:20:12 +08:00
Kefu Chai
2247e85395
Merge pull request #42754 from aclamk/wip-aclamk-bluefs-improve-allocation-check
os/bluestore/bluefs: Cleanup allocation consistency check code

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-08-16 20:18:02 +08:00
Kefu Chai
ba5d5488cf run-make-check: pass -Werror when compiling C++ code
to raise the bar so we can identify potential issues earlier.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 17:37:01 +08:00
Kefu Chai
33545a7b1c cmake: use -Wno-unused-but-set-variable when compiling SPDK if supported
to silence warnings like:

In file included from nvme_tcp.c:52:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/spdk/include/spdk_internal/nvme_tcp.h:369:17: warning: variable 'plen' set but not used [-Wunused-but-set-variable]
        uint32_t hlen, plen;
                       ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 17:32:34 +08:00
Ilya Dryomov
076a39ce2b
Merge pull request #42580 from hualongfeng/solve_competition_between_read_and_retire
librbd/cache/pwl/ssd: solve competition between read and retire

Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-16 11:30:01 +02:00
Kefu Chai
706997b7e6 cmake: use string(APPEND ..) to append to string
more readable and less repeating this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 17:29:37 +08:00
Kefu Chai
164fb13fc9 rgw/rgw_rest: do not set unused variable
`len` is used to track the total length of received data, but we already
collect the data in `bl`, so we can always check `bl` for the total
length. this change also silences following warning from Clang:

../src/rgw/rgw_rest.cc:1477:21: warning: variable 'len' set but not used [-Wunused-but-set-variable]
  int read_len = 0, len = 0;
                    ^
1 warning generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 17:26:10 +08:00
Kefu Chai
436e9309ed os/bluestore/BlueStore: make maybe unused variable [[maybe_unused]]
cct is defined so it can be used by dout(), but when the source code is
compiled for crimson, dout() does not use cct at all, so in that case,
compiler complains like:

../src/os/bluestore/BlueStore.cc:17413:8: warning: unused variable 'cct' [-Wunused-variable]
  auto cct  = onode->c->store->cct;
       ^
1 warning generated.

in this change, cct is marked with [[maybe_unused]] to silence this
warning.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 17:17:50 +08:00
Kefu Chai
7c5848cad9
Merge pull request #42777 from tchaikov/wip-clang-build
run-make-check,cmake: use clang in run-make-check.sh

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-08-16 16:59:09 +08:00
Kefu Chai
9aafa0248e rgw/rgw_json_enc: dump rgw_log_entry::identity_type
should dump it for better debugging experience

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 16:33:18 +08:00
Kefu Chai
cd30a7e5b4 rgw/rgw_json_enc: dump obj using dump_stream()
use dump_stream() instead of dump_string() for dumping stringify object.

more readably this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 16:33:18 +08:00
Kefu Chai
22095f4bd8 rgw/rgw_dencoder: set identity_type when creating samples
otherwise identity_type is a random number on stack. and there is good
chance that it does not match with the one decoded from the encoded
blob.

this change should address the failures like:

**** rgw_log_entry test 2 binary reencode check failed ****
   ceph-dencoder type rgw_log_entry select_test 2 encode export /tmp/typ-O8hVYK2Gb
   ceph-dencoder type rgw_log_entry select_test 2 encode decode encode export /tmp/typ-G09itHHIC
   cmp /tmp/typ-O8hVYK2Gb /tmp/typ-G09itHHIC

The following tests FAILED:
	132 - check-generated.sh (Failed)
..

Fixes: https://tracker.ceph.com/issues/52278
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 16:33:10 +08:00
Kefu Chai
57877075c0 test/crimson: disable logging for unittest-*fltree*
these two tests are very chatty, just silence it. we can always re-enable the
logging, if something goes wrong with any of them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 14:23:17 +08:00
Kefu Chai
d3c315703a cmake: pass -Wunused-but-set-variable when building dpdk
otherwise we have:

src/spdk/dpdk/lib/librte_eal/linux/eal.c:591:11: error: variable 'total_mem' set but not used [-Werror,-Wunused-but-set-variable]
        uint64_t total_mem = 0;
                 ^
1 error generated.

when building dpdk with Clang. dpdk passes -Werror to compiler, so
we have to disable this option.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 14:23:17 +08:00
Kefu Chai
10baab3fc8 cmake/modules/BuildFIO: use ceph fork
to pick up the clang build fix. we should use the upstream repo, once the
clang build fix gets merged.

bumping up the fio helps to address following error, as this part was rewritten:

src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/fio_ceph_objectstore.cc.o.d -o src/test/fio/CMakeFiles/fio_ceph_objectstore.dir/fio_ceph_objectstore.cc.o -c ../src/test/fio/fio_ceph_objectstore.cc
In file included from ../src/test/fio/fio_ceph_objectstore.cc:26:
In file included from src/fio/fio.h:18:
In file included from src/fio/thread_options.h:6:
In file included from src/fio/options.h:8:
src/fio/parse.h:128:13: error: arithmetic on a pointer to void
        return ret + offset;
               ~~~ ^
1 error generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-16 14:23:17 +08:00
Radoslaw Zarzynski
0a4034c10f build: switch the make check bot to Clang.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-08-16 14:23:17 +08:00
Kefu Chai
8e064be4b2
Merge pull request #42781 from cyx1231st/wip-seastore-disable-test-intr-future
crimson/onode-staged-tree: force test to work with invalidated transactions

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-16 14:21:58 +08:00
Yingxin Cheng
8b6428c4ac crimson/onode-staged-tree: force test work with invalidated transactions
is_then() is failed to interrupt the continuation chain and cause
unit test failure. So add a workaround to make the test pass again.

Fixes: https://tracker.ceph.com/issues/52259

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-16 11:27:34 +08:00
Kefu Chai
c6329042c9
Merge pull request #42778 from tchaikov/wip-crimson-cleanup
crimson/os/seastore: do not capture unused variable

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-16 11:10:00 +08:00
Kefu Chai
f435859ec4
Merge pull request #42776 from tchaikov/wip-freebsd
msg/async: build without "using namespace std"

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2021-08-14 18:28:37 +08:00
Kefu Chai
a5f690fe28 test: s/INSTANTIATE_TEST_CASE_P/INSTANTIATE_TEST_SUITE_P/
to silence warnings like:

../src/test/osd/TestOSDMap.cc:2076:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P
[-Wdeprecated-declarations]
INSTANTIATE_TEST_CASE_P(
^
../src/googletest/googletest/include/gtest/gtest-param-test.h:505:38: note: expanded from macro 'INSTANTIATE_TEST_CASE_P'
  static_assert(::testing::internal::InstantiateTestCase_P_IsDeprecated(), \
                                     ^
../src/googletest/googletest/include/gtest/internal/gtest-internal.h:1298:1: note: 'InstantiateTestCase_P_IsDeprecated' has been explicitly marked deprecated here
GTEST_INTERNAL_DEPRECATED(
^
../src/googletest/googletest/include/gtest/internal/gtest-port.h:2249:59: note: expanded from macro 'GTEST_INTERNAL_DEPRECATED'
 #define GTEST_INTERNAL_DEPRECATED(message) __attribute__((deprecated(message)))
                                                          ^
1 warning generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-14 10:29:41 +08:00
Kefu Chai
59b29381ac crimson/os/seastore: do not capture unused variable
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-14 10:29:41 +08:00
Kefu Chai
499fb72745
Merge pull request #42744 from wjwithagen/wjw-fix-varlength-init
rgw/store: Do not init var-length arrays

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-14 10:28:52 +08:00
Kefu Chai
4d0e11176f msg/async: build without "using namespace std"
* add "std::" prefix in headers

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-14 09:51:38 +08:00
Kefu Chai
6e44813168
Merge pull request #42730 from tchaikov/wip-cmake-build-type
cmake: set CMAKE_BUILD_TYPE only if it is not specified

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2021-08-14 00:14:12 +08:00
Kefu Chai
8713c241a1
Merge pull request #42742 from tchaikov/wip-no-using-namespace-std
src: build without "using namespace std"

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-08-14 00:12:06 +08:00
Kefu Chai
0bb2e68a77 include/object.h: do not "using namespace std"
so the identifiers in "std" namespace do not conflict the ones in C code
or the idenfitiers in other namespaces.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 16:56:55 +08:00
Kefu Chai
20552c5e8a dokan: : build without "using namespace std"
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 16:56:55 +08:00
Kefu Chai
b11c205950 java: build without "using namespace std"
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 13:26:17 +08:00
Kefu Chai
8a3a888041 tools/cephfs: build without "using namespace std"
* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 12:23:39 +08:00
Kefu Chai
c2aa30306c tools/rbd_nbd: build without "using namespace std"
* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 12:23:39 +08:00
Kefu Chai
0b35ce68e4 mount.ceph: build without "using namespace std"
* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 12:23:39 +08:00
Kefu Chai
5fbac6d853 crimson/tools: build without "using namespace std"
* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 12:23:39 +08:00
Kefu Chai
f6b157e946 tools/erasure-code: build without "using namespace std"
* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 12:23:39 +08:00
Kefu Chai
ef9682e9e8 cephfs-mirror: build without "using namespace std"
* add "std::" prefix in headers
* add "using" declarations in .cc files.

so we don't rely on "using namespace std" in one or more included
headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-13 12:23:38 +08:00