Commit Graph

124567 Commits

Author SHA1 Message Date
Volker Theile
8292281161 mgr/dashboard: User database migration has been cut out
This PR will revert changes done by https://tracker.ceph.com/issues/49645 to auto-migrate user database v1 to v2.

Fixes: https://tracker.ceph.com/issues/51443

Signed-off-by: Volker Theile <vtheile@suse.com>
2021-06-30 16:35:23 +02:00
Kefu Chai
eb20648daf
Merge pull request #42089 from tchaikov/wip-xattr-cleanup
osd, crimson/osd: do_cmp_xattr() related cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-06-30 19:30:09 +08:00
Kefu Chai
6772c71114
Merge pull request #42054 from tchaikov/wip-os-xattrs
os: use transparent comparator in ObjectStore::getattrs()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2021-06-30 19:29:17 +08:00
Kefu Chai
85b7b1d91e
Merge pull request #42043 from tchaikov/wip-ceph-argparse-ann
pybind/ceph_argparse: do not set self.typeargs in ctor of CephArgtype

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-30 19:28:07 +08:00
Kefu Chai
a2876c7319
Merge pull request #42087 from tchaikov/wip-common-buffers
common/buffers: check _num directly in list::c_str()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-30 19:27:34 +08:00
Kefu Chai
e5c9315b11
Merge pull request #42084 from tchaikov/wip-49638
qa: run e2e test on centos only

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2021-06-30 19:26:42 +08:00
Kefu Chai
21f9046914
Merge pull request #42047 from tchaikov/wip-autoscaler-ann
pybind/mgr/pg_autoscaler: add typing annotations and cleanups

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-30 19:26:01 +08:00
Kefu Chai
c2a91c0d40
Merge pull request #42082 from fengchunsong/master
common: Use double instead of long double to improve performance

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-30 19:25:21 +08:00
Matt Benjamin
4bd071ea1a
Merge pull request #37866 from pritha-srivastava/wip-sts-47809
rgw/sts: fix for copy object operation using sts
2021-06-30 07:17:22 -04:00
Kefu Chai
1b0b32e44c pybind/mgr/pg_autoscaler: add typing annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
83f841295d pybind/mgr/pg_autoscaler: do not index pool_root with pool_name
pool_root is indexed by pool_id, and we never index it with pool_name.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
f725e09ae6 pybind/mgr/pg_autoscaler: do not create a tuple long_desc
long_desc is supposed to be a str, not a tuple of str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
9bb55e8480 pybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out
as it also serves as part of interface of get_subtree_resource_status(),
not only its internals. to ease adding the type annotations, this class
is promoted out of the class.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
4efd740080 pybind/mgr/pg_autoscaler: define commands using CLICommand
simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
b40de19c23 pybind/mgr/pg_autoscaler: define options using Option
more consistent and less error-prune this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
c42712dd18
Merge pull request #41919 from agayev/zoned-more-cleaning-support
os/bluestore: More support for cleaning zones.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-06-30 11:07:24 +08:00
Kefu Chai
0796dd1b5e
Merge pull request #42100 from rzarzynski/wip-crimson-assert_moveable
crimson: introduce assert_moveable().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-30 10:22:24 +08:00
Kefu Chai
abce58d203 pybind/ceph_argparse: add more type annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 10:18:47 +08:00
Kefu Chai
8b8b943561
Merge pull request #41976 from tchaikov/wip-crimson-alienstore-lockless
crimson/os: use lockfree queue for sharded queue

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-30 08:38:41 +08:00
Patrick Donnelly
1cbcecdb5a
Merge PR #42101 into master
* refs/pull/42101/head:
	cephsqlite: add comment on atexit

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 11:20:36 -07:00
Kefu Chai
d846120186 crimson/os/alienstore: use semaphore to manage tasks in thread pool
* implement std::counting_semaphore in C++17
* use the homebrew counting_semaphore as the synchronization primitive
  to access the pending tasks, for better performance than the
  implementation using mutex and condition_variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 00:01:12 +08:00
Patrick Donnelly
be86f191e7
cephsqlite: add comment on atexit
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-29 08:01:56 -07:00
Radoslaw Zarzynski
fad97b3543 crimson: introduce assert_moveable().
In C++ `std::moving` a `const`-qualified value yields a constant
r-value reference (`const T&&`) which won't be matched with a callable
taking non-constant r-value reference (like move constructors) but
can play with one taking a constant l-value reference (like copy
constructors do). This behaviour is surprising especially in lambas
where adding or removing the `mutable` specifier may lead to different
behaviour. The problem isn't obvious and it's easy to wrongly drop
the `mutable` druing a clean-up. Therefore introducing a tool for
developers to fail at compile-time if that happens seems desired.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-29 14:55:51 +00:00
Kefu Chai
05dae35e53
Merge pull request #42002 from tchaikov/wip-common-armor
common/armor: mark dst_end a const pointer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-29 22:54:36 +08:00
Kefu Chai
7fb626ce3e
Merge pull request #42035 from batrick/i51372
libcephsqlite: shutdown RADOS in destructor

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 22:33:10 +08:00
zdover23
08e032c828
Merge pull request #42094 from zdover23/wip-doc-upgrading-ceph-monitoring-the-upgrade-2021-06-29
doc/cephadm: improving "Monitoring the Upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-30 00:08:11 +10:00
zdover23
04216d37d2
Merge pull request #42079 from zdover23/wip-doc-upgrading-ceph-first-section-2021-06-29
doc/cephadm: improve "Upgrading Ceph" (main)

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-30 00:07:28 +10:00
Zac Dover
2dcdef0d43 doc/cephadm: improving "Monitoring the Upgrade"
This PR improves the section "Monitoring the Upgrade"
in the "Upgrading Ceph" chapter of the cephadm documentation.

This PR introduces a couple of section breaks with signposting
information in their titles, and rewrites some sentences in order
to reduce the cognitive load of the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-06-29 22:42:29 +10:00
Kefu Chai
f019a91098
Merge pull request #42042 from tchaikov/wip-51375
common/options: convert a millisecs opt to a chrono::milliseconds and cleanups

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-29 19:28:19 +08:00
Kefu Chai
aa3f2028c1 crimson/os/alienstore: return attrs retrieved from bluestore directly
since ObjectStore returns a map<...,less<>> already, there is not point to
convert the returned map to the type expected by AlienStore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 11:27:25 +00:00
Kefu Chai
a456e031ca os: use transparent comparator in ObjectStore::getattrs()
for two reasons:

- better performance when looking on in the return map if the key
  is not a string, as we don't need to create a temporary string
  as the key
- improve the performance of crimson::AlienStore, as the latter
  uses the transparent comparator. as, without this change, we'd
  have to perform a deep copy to fill up the returned map with
  its non-transparent-comparator version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 11:27:25 +00:00
Kefu Chai
6ca774dbc7 crimson/os: use lockfree queue for sharded queue
each sharded queue has multiple producers and a single consumer queue,
but the producer side is in seastar world, so ideally, we should not
guard the queue using a POSIX lock.

in this change, the lockfree multiple-writer/multiple-reader queue is
used to replace the std::queue to drop the lock guarding it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 18:58:09 +08:00
Kefu Chai
a41b62bd85 mgr,mon: s/boost::optional/std::optional/
since the encoding schemes of boost::optional and std::optional are
identical, they can be used interchangeably.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:59:33 +08:00
Kefu Chai
32e62f9a64 common/config: s/boost::variant/std::variant/
we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:59:33 +08:00
Kefu Chai
863d8385d0 common/options: s/boost::variant/std::variant/
we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:56:58 +08:00
Kefu Chai
812e58c597
Merge pull request #42013 from ronen-fr/wip-ronenf-scrubs-config
qa/suites/rados: add simultaneous scrubs to the thrasher

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-06-29 16:21:52 +08:00
Kefu Chai
135b6dde5e
Merge pull request #42031 from sebastian-philipp/normalize_image_digest_local-registry
cephadm: Fix normalize_image_digest for local registries

Reviewed-by: Adam King <adking@redhat.com>
2021-06-29 16:20:28 +08:00
Kefu Chai
7e9d5fa02d
Merge pull request #41933 from liewegas/fix-48311
mgr: set debug_mgr=2/5 (so INFO goes to mgr log by default)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:18:17 +08:00
Kefu Chai
2026a7f0d3
Merge pull request #41999 from tchaikov/wip-rpm-deb
rpm,debian: cleanups related to python3-setuptools dependencies

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-06-29 16:16:32 +08:00
Kefu Chai
f91183d986
Merge pull request #42076 from liu-chunmei/crimson-pgnls
crimson: fix pgnls exception

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 15:10:48 +08:00
Kefu Chai
34642418c8 osd/PrimaryLogPG: reorder #includes
so the oder of includes complies to https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 15:00:20 +08:00
Kefu Chai
8fe0305992 osd/PrimaryLogPG: use std::from_chars() to convert str to integer
* no need to create a temporary string for using strtoull()
* use std::from_chars() so it's consistent with its counterpart in
  in crimson.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
f39d71ecaa crimson/osd: check invalid input in do_xattr_cmp_u64()
we should return -EINVAL if the string does not represent a decimal
integer.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
4561fb36f8 crimson/osd: include used header
for using std::from_chars()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
f02a20120c crimson/osd: extract do_cmp_xattr()
to consolidate the logic to dispatch by op.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
9fa07cb89e osd: extract do_cmp_xattr()
to consolidate the logic to dispatch by op.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
6e8fbc1373
Merge pull request #42039 from liu-chunmei/crimson-cmpxattr
crimson: add cmp_xatt support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:09:15 +08:00
Kefu Chai
d6334e5c43 common/buffers: check _num directly in list::c_str()
no need to create temporary iterator for comparing it with cend().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 13:56:02 +08:00
chunmei-liu
f5fa339036 crimson: fix pgnls exception
has_pg_op is always false, since m->ops is empty at that time.
so pgnls operation will go to process_op and report unknown operations.
move m->finish_decode ahead to fill m->ops.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-06-28 22:20:55 -07:00
Kefu Chai
02b8b0f490 qa: run e2e test on centos only
it's a regression introduced by the restrcuture of the test suites,
let's pin the test to CentOS8.

See-also: https://tracker.ceph.com/issues/49638
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 13:09:53 +08:00