Commit Graph

123070 Commits

Author SHA1 Message Date
Kefu Chai
1f6668bfc8
Merge pull request #41165 from tchaikov/wip-cephfs-shell-manpage
doc/man: build and install cephfs-shell manpage

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-05 17:43:26 +08:00
Kefu Chai
d415257831
Merge pull request #40977 from xxhdx1985126/wip-seastore-xattr
crimson/os/seastore: formal extended attributes support

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-05 15:23:00 +08:00
Kefu Chai
1f6cf5e403 cmake: remove cflags from CC
this change make it more explicit how we override CC, CFLAGS and
CPPFLAGS:

* CC only for the compiler
* CFLAGS only for the flags consumed by C compiler
* CPPFLAGS only used for the preprocessor, it is used by both C and C++
  compilers.

this change does not address any specific test failure. it just
improves the correctness and readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-05 15:19:27 +08:00
Kefu Chai
9f49c4955b
Merge pull request #41109 from markhpc/wip-bluefs_buffered_io-desc
src/common: Update bluefs_buffered_io descrption.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-05-05 14:45:40 +08:00
Kefu Chai
1c04ce6f3a doc/man: build and install cephfs-shell manpage
otherwise the manpage of cephfs-shell is not built and installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-05 14:41:50 +08:00
Kefu Chai
8431e9c27f
Merge pull request #41142 from tchaikov/wip-cmake-global
cmake: let libglobal_obj depend on legacy-option-headers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-05 12:52:53 +08:00
Kefu Chai
6a48787c4d doc/man/8/cephfs-shell: reformat options
* format global options using option directive
* fix the header, so man/conf.py is able to parse
  the description
* define "Synopsis" section to be consistent with other manpages.
* drop reference to glossary using "term" as manapge does not have
  reference to glossary entries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-05 09:27:53 +08:00
Patrick Donnelly
2c569e7f30
Merge PR #41160 into master
* refs/pull/41160/head:
	Revert "Merge PR #41134 into master"

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-05-04 18:00:27 -07:00
David Galloway
20301d8599 doc: 16.2.2 Release Notes
Signed-off-by: David Galloway <dgallowa@redhat.com>
2021-05-04 19:39:39 -04:00
Patrick Donnelly
52faaf848e
Revert "Merge PR #41134 into master"
This reverts commit 93bca2d0e9, reversing
changes made to 7e70b15c43.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-04 14:46:47 -07:00
Ali Maredia
6cfc2ce2ce rgw: add DPP's to logging for most ops
This commit contains the following major
changes:

- Most log output lines for major ops now have
DoutPrefixProviders.

- Create new macro ldpp_subdout. This macro is meant
as a replacement for lsubdout for headerfiles that
do not define the rgw subsys.

- Changes to RGWObjManifest begin and end iterators.

- Make it so that rgw_dencoder.cc should only contain the
logic related to encoding.

Also add dpp to ldouts and lderr's already using
req_state and replace sync_env/env->dpp's with dpp's

Signed-off-by: Ali Maredia <amaredia@redhat.com>
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-05-04 16:28:11 -04:00
Ilya Dryomov
67bb6cf524 msg/async/ProtocolV2: catch correct bad_alloc exception
We want buffer::bad_alloc, not std::bad_alloc.  Otheriwise, we end
up with a confusing error

  failed decoding of frame header: Bad allocation

from ProtocolV2::run_continuation(), printed after frame header is
successfully decoded.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-05-04 22:17:07 +02:00
Ilya Dryomov
aa31ddf0e7 common/buffer: adjust align before calling posix_memalign()
posix_memalign() requires alignment argument to be a multiple of
sizeof(void *).  Since it is an implementation detail of buffer,
it needs to be adjusted there -- buffer consumers have no way of
knowing that passing e.g. align == 4 is incorrect.

One place already does the adjustment, but only for align == 0.
The other just asserts.  Fix both and remove the "power of two"
assertion.  Let posix_memalign() return EINVAL and handle that
by throwing buffer::bad_alloc, as expected by the consumers.

Fixes: https://tracker.ceph.com/issues/50646
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-05-04 22:17:07 +02:00
Radoslaw Zarzynski
14706f9a56 log: fix the formatting when dumping thread IDs.
When switching to `fmt::format()` we accidentaly changed
the format of how we dump threads ID in `Log::dump_recent()`.
This commit rectifies that.

Fixes: https://tracker.ceph.com/issues/50653
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-04 19:23:50 +00:00
Casey Bodley
4854791236
Merge pull request #41152 from linuxbox2/wip-restore-rgw-lua
cmake: restore Lua scripting support for RGW

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-05-04 15:14:29 -04:00
Ernesto Puerta
b6e4d6fd25
Merge pull request #41024 from rhcs-dashboard/generate-manifest
mgr/dashboard: Generate NPM dependencies manifest

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-05-04 20:52:30 +02:00
Ernesto Puerta
9f5acc392d
Merge pull request #41062 from rhcs-dashboard/50519-fix-ssl-cmd
mgr/dashboard: fix set-ssl-certificate{,-key} commands

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-05-04 20:46:05 +02:00
Sage Weil
3ad3f06d2c mgr/orchestrator: fix help formatting
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 13:36:00 -04:00
Sage Weil
729ebaabd5 mgr/cephadm: remove haproxy+keepalived container image from service spec
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 12:13:21 -04:00
Matt Benjamin
bbe461bb74 cmake: restore Lua scripting support for RGW
RGW linkage with Lua was broken by the following commit:

commit 7471fbfa4b
Author: Kefu Chai <kchai@redhat.com>
Date:   Mon Apr 26 22:44:16 2021 +0800

    cmake: require lua if WITH_CEPHFS is enabled

    in src/mds/Mantle.cc, the lua C binding APIs are used, so we should only
    require lua if WITH_CEPHFS is enabled.

    Signed-off-by: Kefu Chai <kchai@redhat.com>

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-05-04 12:09:45 -04:00
Sage Weil
ee5a0c96a0 ceph_test_rados_api_service: stop threads before asserting
Otherwise, if we assert, we'll hang here:

Thread 1 (Thread 0x7f74eba79580 (LWP 1688617)):
#0  0x00007f74eb2aa529 in futex_wait (private=<optimized out>, expected=132, futex_word=0x7ffd642b4b54) at ../sysdeps/unix/sysv/linux/futex-internal.h:61
#1  futex_wait_simple (private=<optimized out>, expected=132, futex_word=0x7ffd642b4b54) at ../sysdeps/nptl/futex-internal.h:135
#2  __pthread_cond_destroy (cond=0x7ffd642b4b30) at pthread_cond_destroy.c:54

#3  0x0000563ff2e5a891 in LibRadosService_StatusFormat_Test::TestBody (this=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:78
#4  0x0000563ff2e9dc3a in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (location=0x563ff2ea72e4 "the test body", method=<optimized out>, object=0x563ff422a6d0)
    at ./src/googletest/googletest/src/gtest.cc:2605
#5  testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=object@entry=0x563ff422a6d0, method=<optimized out>, location=location@entry=0x563ff2ea72e4 "the test body")
    at ./src/googletest/googletest/src/gtest.cc:2641
#6  0x0000563ff2e908c3 in testing::Test::Run (this=0x563ff422a6d0) at ./src/googletest/googletest/src/gtest.cc:2680
#7  0x0000563ff2e90a25 in testing::TestInfo::Run (this=0x563ff41a3b70) at ./src/googletest/googletest/src/gtest.cc:2858
#8  0x0000563ff2e90ec1 in testing::TestSuite::Run (this=0x563ff41b6230) at ./src/googletest/googletest/src/gtest.cc:3012
#9  0x0000563ff2e92bdc in testing::internal::UnitTestImpl::RunAllTests (this=<optimized out>) at ./src/googletest/googletest/src/gtest.cc:5723
#10 0x0000563ff2e9e14a in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (location=0x563ff2ea8728 "auxiliary test code (environments or event listeners)",
    method=<optimized out>, object=0x563ff41a2d10) at ./src/googletest/googletest/src/gtest.cc:2605
#11 testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x563ff41a2d10, method=<optimized out>,
    location=location@entry=0x563ff2ea8728 "auxiliary test code (environments or event listeners)") at ./src/googletest/googletest/src/gtest.cc:2641
#12 0x0000563ff2e90ae8 in testing::UnitTest::Run (this=0x563ff30c0660 <testing::UnitTest::GetInstance()::instance>) at ./src/googletest/googletest/src/gtest.cc:5306

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 11:09:08 -05:00
Sage Weil
f130f15a45 ceph_test_rados_api_service: more retries for servicemkap
Otherwise, this fails intermittently with

/build/ceph-16.2.1-257-g717ce59b/src/test/librados/service.cc:166: Failure
Expected: (0) != (retry), actual: 0 vs 0

Fixes: https://tracker.ceph.com/issues/50595
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 11:01:38 -05:00
Dennis Körner
188e6b3fc5
Add Rocky Linux to supported DISTRO_NAMES
Rocky Linux is a RHEL clone. I did a test-installation of ceph pacific on Rocky Linux RC1 with cephadm. As far as I can see, everything works as expected.

Signed-off-by: Dennis Körner <koerner@netzwerge.de>
2021-05-04 17:59:59 +02:00
Patrick Donnelly
93bca2d0e9
Merge PR #41134 into master
* refs/pull/41134/head:
	doc/man: build and install cephfs-shell manpage
	doc/man/8/cephfs-shell: reformat options

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-04 07:32:51 -07:00
Navin Barnwal
f2aa58a5fa mgr/dashboard: fix rename inventory to disks
Inventory -> Physical Disks changed in total 3 locations

Fixes: https://tracker.ceph.com/issues/50314
Signed-off-by: Navin Barnwal <knbarnwal@gmail.com>
2021-05-04 19:59:36 +05:30
Kefu Chai
6007448501 cmake: let libglobal_obj depend on legacy-option-headers
to address following build failure:

FAILED: src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o ...
src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o -MF src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o.d -o src/global/CMakeFiles/libglobal_objs.dir/global_init.cc.o -c
../src/global/global_init.cc
In file included from ../src/global/global_init.cc:26:
In file included from ../src/common/config.h:26:
In file included from ../src/common/config_values.h:59:
../src/common/options/legacy_config_opts.h:1:10: fatal error: 'global_legacy_options.h' file not found
         ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-04 21:09:25 +08:00
J. Eric Ivancich
7e70b15c43
Merge pull request #40926 from ivancich/wip-orphan-list-pools
rgw: allow rgw-orphan-list to process multiple data pools

Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
2021-05-04 09:08:48 -04:00
J. Eric Ivancich
220ef4b22d rgw: fix bucket object listing when marker matches prefix
When an iniitial marker that ends with a delimiter is provided, it
prevents listing of that "subdirectory" due to new logic at the cls
level to make listing more efficient. The fix catches that situation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-05-04 08:42:32 -04:00
J. Eric Ivancich
459d59f505 rgw: radosgw_admin remove bucket not purging past 1,000 objects
The zipper refactoring combines use of old is_truncated variable and
newer structure that manages is_truncated. This prevented advancing
past the first page of 1,000 entries.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-05-04 08:41:12 -04:00
Sage Weil
b5d39be188 mgr/cephadm: apply hostname/addr checks to 'orch host set-addr' too
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 08:21:19 -04:00
Sage Weil
a8acc354dc mgr/cephadm: make 'host add' idempotent
Do not let prime_empty_host blow away inventory.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 08:21:19 -04:00
Sage Weil
2f2c997652 mgr/cephadm: set host crush location based on HostSpec
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 08:21:19 -04:00
Sage Weil
d2a9a35993 python-common: add location property to HostSpec, + tests
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-04 08:21:19 -04:00
Casey Bodley
b7718366c4
Merge pull request #40631 from cbodley/wip-49780
rgw/sts: read_obj_policy() consults iam_user_policies on ENOENT

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
2021-05-04 08:08:51 -04:00
Arthur Outhenin-Chalandre
c6e2953fdb
librbd/mirror/snapshot: avoid UnlinkPeerRequest with a unlinked peer
CreatePrimaryRequest could create some UnlinkPeerRequest with an already
unlinked peer in a scenario where you have multiple peers. This request
will not remove the peer (as it's already not linked to the requested
peer) and will skip deletion of the mirror snapshot if another peer
remains. Eventually the code will go through an infinite recursive loop
between CreatePrimaryRequest and UnlinkPeerRequest and segfault.

This commit adds an extra condition to make sure to not submit a
UnlinkPeerRequest if the peer is not linked to the current snapshot. If
there is already no peer in the list it will submit a UnlinkPeerRequest
to remove the snapshot.

Fixes: https://tracker.ceph.com/issues/50439
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2021-05-04 13:37:40 +02:00
Kefu Chai
e870f7a428
Merge pull request #41033 from tchaikov/wip-osx-build
cmake,common: various cleanup for building on MacOS

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-04 19:11:52 +08:00
Xuehan Xu
1c798dc6c7 test/crimson: add tests for enhanced get/set attrs apis
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-05-04 18:39:12 +08:00
Xuehan Xu
c0f6c28e87 crimson/os/seastore: allow upper layer to set attrs other than OI_ATTR and SS_ATTR
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-05-04 18:39:09 +08:00
Xuehan Xu
2dcb836e2d crimson/os: change get/set attrs value to the bufferlist type
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-05-04 18:26:30 +08:00
Sebastian Wagner
becbe5fc4a
Merge pull request #41045 from jmolmo/fix_cv_lvm_lock
mgr/cephadm: ceph-volume verbose only when fails

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-05-04 11:08:39 +02:00
Sebastian Wagner
2e195e977f
Merge pull request #40313 from jmolmo/purge_iscsi_config
mgr/cephadm: Remove gateway.conf from iscsi pool when service is removed

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-05-04 10:43:57 +02:00
Samuel Just
a47f9bf6b2
Merge pull request #40799 from myoungwon/wip-fix-null-promote-obc
osd: fix not set promote_obc when manifest object is rollbacked

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-05-04 00:55:33 -07:00
Kefu Chai
5191008aa4 doc/man: build and install cephfs-shell manpage
otherwise the manpage of cephfs-shell is not built and installed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-04 15:11:22 +08:00
Kefu Chai
0fa568b54a doc/man/8/cephfs-shell: reformat options
* format global options using option directive
* fix the header, so man/conf.py is able to parse
  the description
* define "Synopsis" section to be consistent with other manpages.
* drop reference to glossary using "term" as manapge does not have
  reference to glossary entries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-04 15:11:22 +08:00
Misono Tomohiro
5a6c6267a1 os/FileStore: fix to handle readdir error correctly
Currently filestore code does not handle readdir error.
As man readdir(3) says, we need to check errno after readdir
returns NULL to determine if error happens or not.

This patch fixes the all readdir() calls to check errono and
handle it appropriately:
 - FileStore.cc ... abort if EIO error happens
 - BtrfsFileStoreBAckend.cc/LFNindex.cc
                ... return error to upper layer

Without this fixes, primary PG could fail to correctly perform
backfill operation and could lead data loss propagation described
in #50558.

Fixes: https://tracker.ceph.com/issues/50558
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
2021-05-04 10:31:59 +09:00
Patrick Donnelly
fc74911784
Merge PR #41066 into master
* refs/pull/41066/head:
	mds: session dumps num_completed_flushes
	mds: completed_requests -> num_completed_requests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-03 17:51:37 -07:00
Patrick Donnelly
5c1f350c8b
Merge PR #40963 into master
* refs/pull/40963/head:
	mds: do not trim the inodes from the lru list in standby_replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-03 17:46:16 -07:00
Patrick Donnelly
a52cc67249
Merge PR #40962 into master
* refs/pull/40962/head:
	test: add test to validate snap synchronization with parent directory snapshots
	cephfs-mirror: ignore parent directory snapshots when building snap map

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-03 17:43:47 -07:00
Patrick Donnelly
5fe5ec095c
Merge PR #40927 into master
* refs/pull/40927/head:
	mds: reset the return value for heap command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-03 17:39:41 -07:00
Patrick Donnelly
d65632289c
Merge PR #40903 into master
* refs/pull/40903/head:
	test: add test case for MDS privated inos accessing
	mds: make the lost+found dir accessible from clients
	mds: move the inos 1,2 and 3 macros to ceph_fs.h

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-05-03 17:37:34 -07:00