Commit Graph

125322 Commits

Author SHA1 Message Date
Sage Weil
7e22d2a31d rgw: default ms_mon_client_mode = secure
Require a secure connection to the monitor.  This is important because
we may store SSL certs on the mon.

Note that secure mode was introduce before Nautilus, so this can be
backported at least to Pacific.

Fixes: https://tracker.ceph.com/issues/52000
Signed-off-by: Sage Weil <sage@newdream.net>
2021-08-02 11:23:35 -05:00
Kefu Chai
ff3161f8a5
Merge pull request #42502 from tchaikov/wip-bloomfilter-cleanups
include/intarith, common/bloom_filter: add popcount() and cleanups

Reviewed-by: Sage Weil <sage@redhat.com>
2021-07-28 22:56:52 +08:00
Yuval Lifshitz
aa14f3dd17
Merge pull request #42169 from Matan-B/lua-nats-example
rgw: Add an example for lua scripting
2021-07-28 10:16:30 +03:00
Yuval Lifshitz
e527b8ce55
Merge pull request #42350 from TRYTOBE8TME/wip-rgw-notification
rgw/notifications: Change in multipart upload notification behavior
2021-07-28 10:13:10 +03:00
Kefu Chai
4f9fa25ce0 common/bloom_filter: move bloom_filter::density() to .cc
for faster compilation, also, bloom_filter::density()
is not sitting in the critical paths, so no need to make it an
inline function.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-28 12:57:13 +08:00
Kefu Chai
146c796a7f
Merge pull request #42389 from asannes/ingress-fix
mgr/cephadm: ingress: fix typo in spec.virtual_interface_networks reference

Reviewed-by: Sage Weil <sage@redhat.com>
2021-07-28 12:48:04 +08:00
Kefu Chai
781f31e386
Merge pull request #42510 from rzarzynski/wip-crimson-use-random-nonce
crimson/osd: fix misdirecting msgs when an OSD flips at Sepia.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-28 11:06:11 +08:00
Josh Durgin
624068d244
Merge pull request #42322 from ljflores/wip-lflores-telemetry-docs
doc/mgr/telemetry: update Telemetry Module docs to include perf channel

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
2021-07-27 17:31:33 -07:00
Josh Durgin
60cb985cc7
Merge pull request #42492 from zdover23/wip-doc-rados-top-level-section-2021-07-26
doc/rados: rewrite index.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-27 17:28:58 -07:00
Casey Bodley
7b61667711
Merge pull request #42473 from tchaikov/wip-cmake-rgw-cleanup
cmake: make rgw_common a static library

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-27 14:31:40 -04:00
Sage Weil
ecc0b4597b Merge PR #42406 into master
* refs/pull/42406/head:
	mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'
	doc/mgr/rook: update title
	doc/mgr/nfs: reference customizing ingress
	doc/mgr/nfs: add section for manual ganesha config; reframe
	doc/mgr/nfs: document ingress in more detail
	doc/mgr/nfs: typo
	doc/mgr/nfs: add note about incomplete ingress
	qa/suites/orch/cephadm: add rgw nfs export test
	mgr/cephadm: ingress: tolerate no daemons
	mgr/nfs: add --squash option to 'nfs export create rgw ...'
	mgr/nfs: use bucket owner creds for rgw bucket export
	mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image
	qa/tasks/python: simple task to run python code
	doc/mgr/nfs: revisions
	mgr/nfs/export: nicer exceptions on cap update

Reviewed-by: Varsha Rao <varao@redhat.com>
2021-07-27 14:11:56 -04:00
Radoslaw Zarzynski
5293d1ad1e crimson/osd: make the get_nonce() static.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-27 16:37:53 +00:00
Radoslaw Zarzynski
c79f2aaba9 crimson/osd: print OSD::nonce on start-up.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-27 16:37:53 +00:00
Radoslaw Zarzynski
d0a5dd7e36 crimson/osd: fix misdirecting msgs when an OSD flips at Sepia.
For description please refer to:
  * https://gist.github.com/rzarzynski/6cbc20fb28d7ac096ecc4b68500454bf#gistcomment-3830762,
  * https://gist.github.com/rzarzynski/6cbc20fb28d7ac096ecc4b68500454bf#gistcomment-3830939.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-27 16:29:45 +00:00
Kefu Chai
a8dac22376
Merge pull request #42476 from tchaikov/wip-crimson-more-aliens
doc, vstart.sh: adapt to the generalize AlienStore support

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-27 23:58:55 +08:00
Casey Bodley
b3cffa193d
Merge pull request #42222 from flipkart-incubator/aws_sse_s3
rgw: Introduce BucketEncryption APIs to support SSE-S3 feature

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-27 10:52:23 -04:00
Kefu Chai
587a8c9c22 doc/dev/crimson: document cyanstore and alienstore
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 22:50:15 +08:00
Sage Weil
2db3cfabc3 mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'
This argument is not used.  The rgw exports currently only work on the
default realm.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-27 10:47:55 -04:00
Sage Weil
9aeefbc666 doc/mgr/rook: update title
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-27 10:47:55 -04:00
Sage Weil
7d3443412c doc/mgr/nfs: reference customizing ingress
Link to the cephadm docs on modifying the service directly.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-27 10:47:55 -04:00
Sage Weil
b5a32d632d doc/mgr/nfs: add section for manual ganesha config; reframe
This documentation is incomplete because this mode of operation is not
tested/validated.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-27 10:47:55 -04:00
Kefu Chai
26e7034752 common/bloom_filter: use popcount() to implement density()
instead of counting the bits using a loop, it'd be more efficient to do
this using the SIMD instruction.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
f8b3d9dcb3 common/bloom_filter: s/bits_per_char/CHAR_BIT/
use macro defined by standard libary instead of inventing our own.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
197368cfb0 include/intarith: add popcount()
for counting "1" in an integer.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
943b91a11b test/test_intarith: adapt to the change in intarith.h
back in 2623fec1cd, the vaiants of, for
instance, ctz() are consolidated to a single template. so the
ctz<>() dispatches by the size of argument after that change.
but the tests were not updated accordingly.

in this change:

* the tests are updated to use the template.
* instead of using integer literal postfix, use the macros like
  UINT64_C to define integer constants for better portability on
  different architectures where the lengths of integer *might* be
  different from amd64. also, it's more readable than postfixes
  like ULL in this context, as we really care about the exact
  length of an integer in this context when counting the leading
  zeros.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
49bd620f75 cmake: add unittest_intarith back
this unit test was missed when we migrated from autotools to CMake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
5d68abe194 common/bloom_filter: use vector for holding bits
to be more consistent with the upstream at https://github.com/ArashPartow/bloom/blob/master/bloom_filter.hpp
and for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
5079e56b0c common/bloom_filter: use "using" for better readability
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
89770fe691 common/bloom_filter: return by return values not by input params
it'd be easier for the static analyzer (like GCC), to reason about if
a variable is initialized before being used.

this change also helps to improve the readability, and to silence the
false alarm like:

In file included from ../src/os/bluestore/BlueStore.h:42,
                 from ../src/os/bluestore/BlueStore.cc:26:
../src/common/bloom_filter.hpp: In member function 'void std::vector<_Tp, _Alloc>::_M_fill_insert(std::vector<_Tp, _Alloc>::iterator, std::vector<_Tp, _Alloc>::size_type, const value_type&) [with _Tp = bloom_filter; _Alloc = mempool::pool_allocator<mempool::mempool_bluestore_fsck, bloom_filter>]':
../src/common/bloom_filter.hpp:118:46: warning: '*((void*)(& __tmp)+8).bloom_filter::table_size_' may be used uninitialized in this function [-Wmaybe-uninitialized]
  118 |  mempool::bloom_filter::alloc_byte.deallocate(bit_table_, table_size_);
      |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 20:03:32 +08:00
Kefu Chai
f0027c05c6
Merge pull request #42457 from cyx1231st/wip-seastore-segment-metrics
crimson/seastore: implement metrics in BlockSegmentManager

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-27 14:37:33 +08:00
Yingxin Cheng
3758671ee9 crimson/os/seastore/segment_manager: implement metrics in BlockSegmentManager
* data read/write effort;
* metadata write effort;
* opened/closed/released segments;
* unused bytes from closed segments;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-07-27 12:43:26 +08:00
Kefu Chai
0edc573fca include/denc: always initialize local variable
this change silences following false positive warning:

In file included from ../src/include/encoding.h:41,
                  from ../src/kv/KeyValueDB.h:12,
                  from ../src/os/bluestore/bluestore_common.h:20,
                  from ../src/os/bluestore/BlueFS.cc:5:
 ../src/include/denc.h: In function ‘std::enable_if_t<(is_same_v<T, bluefs_extent_t> || is_same_v<T, const bluefs_extent_t>)> _denc_friend(T&, P&) [with T = bluefs_extent_t; P = ceph::buffer::v15_2_0::p$
 ../src/include/denc.h:639:11: warning: ‘shift’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   639 |     shift += 7;
       |     ~~~~~~^~~~
 ../src/include/denc.h:613:7: note: ‘shift’ was declared here
   613 |   int shift;
       |       ^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-27 11:43:30 +08:00
Kefu Chai
d7862cb607
Merge pull request #42475 from tchaikov/wip-crimson-cleanup
crimson/osd: cleanups

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-07-27 10:37:07 +08:00
Kefu Chai
03ac43a4b4
Merge pull request #42483 from Astroakanksha24/master
mgr/volumes/fs/operations/group.py: add extra blank lines

Reviewed-by: Varsha Rao <varao@redhat.com>
2021-07-27 10:32:29 +08:00
Kefu Chai
d637dcda22
Merge pull request #42485 from tchaikov/wip-strtoll
common/strtol: replace `const char*` with `std::string_view`

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-07-27 10:31:41 +08:00
Sage Weil
f8f7b86571 Merge PR #42292 into master
* refs/pull/42292/head:
	qa/suites/upgrade: log_to_journald=false

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-26 19:23:26 -04:00
Sage Weil
ac63ab6125 Merge PR #42489 into master
* refs/pull/42489/head:
	qa/suites/upgrade/pacific-x/stress-split: do not avoid_pacific_features

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 19:23:03 -04:00
Sage Weil
8d9db910f7 doc/mgr/nfs: document ingress in more detail
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
0448a8d39d doc/mgr/nfs: typo
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
684c6a2dba doc/mgr/nfs: add note about incomplete ingress
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
cd089ee74e qa/suites/orch/cephadm: add rgw nfs export test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
bae406f746 mgr/cephadm: ingress: tolerate no daemons
This doesn't normally happen, but did before the daemon inventory breakage
(see previous patches) was fixed.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
ad5fe4452d mgr/nfs: add --squash option to 'nfs export create rgw ...'
The cephfs variant already has it.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Sage Weil
7d2f7efe3f mgr/nfs: use bucket owner creds for rgw bucket export
The bucket owner can always read/write to the bucket, so use those creds
for the export.  This is less complicated than setting up a dedicated
user anyway.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
deaff0c42c mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image
We were using CEPH_TYPES + GATEWAY_TYPES, but that isn't really accurate.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
45737fe95a qa/tasks/python: simple task to run python code
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
8b96d9c22b doc/mgr/nfs: revisions
- clean up language
- move config hierarchy to the bottom (this is an implementation detail
that is only useful if managing ganesha externally)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:06 -04:00
Sage Weil
b73f730749 mgr/nfs/export: nicer exceptions on cap update
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:21:24 -04:00
Guillaume Abrioux
4bfa87ee55
Merge pull request #42181 from lyind/51526-fix-ceph-volume-zero-division-error
ceph-volume: lvm batch: fast_allocations(): avoid ZeroDivisionError
2021-07-26 21:58:07 +02:00
Zac Dover
d27bb21cc1 doc/rados: rewrite index.rst
This PR makes minor alterations to the
text at the beginning of the RADOS Guide.

Most notably, the monitor daemon has been
added to the list of types of daemons that
constitute a Ceph cluster.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-07-27 05:16:41 +10:00