Commit Graph

124680 Commits

Author SHA1 Message Date
Kefu Chai
f725e09ae6 pybind/mgr/pg_autoscaler: do not create a tuple long_desc
long_desc is supposed to be a str, not a tuple of str.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
9bb55e8480 pybind/mgr/pg_autoscaler: extract CrushSubtreeResourceStatus out
as it also serves as part of interface of get_subtree_resource_status(),
not only its internals. to ease adding the type annotations, this class
is promoted out of the class.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
4efd740080 pybind/mgr/pg_autoscaler: define commands using CLICommand
simpler this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Kefu Chai
b40de19c23 pybind/mgr/pg_autoscaler: define options using Option
more consistent and less error-prune this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 14:03:33 +08:00
Zac Dover
ee455f00ab doc/cephadm: improve "Canceling an Upgrade"
This PR improves the section "Canceling an Upgrade"
in the "Upgrading Ceph" chapter of the cephadm
documentation.

I removed an extraneous prompt and rewrote a sentence
so that it was congruent with other sentences in similar
places elsewhere in the documentation.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-06-30 15:20:13 +10:00
Patrick Donnelly
891c2773e9
qa: refactor reading debug file code
No need to invoke Python to read a file!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-29 20:13:41 -07:00
Patrick Donnelly
97a86453f8
qa: get mount id before failing fs
Otherwise getxattr will hang.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-29 20:13:29 -07:00
Kefu Chai
c42712dd18
Merge pull request #41919 from agayev/zoned-more-cleaning-support
os/bluestore: More support for cleaning zones.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-06-30 11:07:24 +08:00
Kefu Chai
0796dd1b5e
Merge pull request #42100 from rzarzynski/wip-crimson-assert_moveable
crimson: introduce assert_moveable().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-30 10:22:24 +08:00
Kefu Chai
abce58d203 pybind/ceph_argparse: add more type annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 10:18:47 +08:00
Kefu Chai
8b8b943561
Merge pull request #41976 from tchaikov/wip-crimson-alienstore-lockless
crimson/os: use lockfree queue for sharded queue

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-30 08:38:41 +08:00
Dan van der Ster
3f15749de0 mgr/DaemonServer: skip redundant update of pgp_num_actual
During PG merge the MGR was observed repeatedly sending identical
set pgp_num_actual values, leading to osdmap churn at 2000/hr.

Skip the redundant osd set pgp_num_actual command if the
pgp_num is already our computed next.

Fixes: https://tracker.ceph.com/issues/51433
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2021-06-29 23:00:33 +02:00
Patrick Donnelly
1cbcecdb5a
Merge PR #42101 into master
* refs/pull/42101/head:
	cephsqlite: add comment on atexit

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 11:20:36 -07:00
Samuel Just
95ace7e700 crimson/os/seastore: add collection ordering
Adds a mutex to SeastoreCollection which ensures that transactions are
ordered on a per-collection basis.  Future optimizations could enable
the transaction construction phase to be pipelined for a single collection,
but would require correctly handling the case where a more recently
submitted transaction encounters a conflict.

Fixes: https://tracker.ceph.com/issues/51358
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-06-29 10:41:15 -07:00
Samuel Just
c96faa8e8b crimson/os/seastore/seastore: preserve Transaction::handle on reset for writes
We're going to need to preserve the pipeline phase locking on conflict.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-06-29 10:14:45 -07:00
Samuel Just
e7ac30f36a crimson/os/seastore/ordering_handle: fix phase descriptions
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-06-29 10:08:55 -07:00
Samuel Just
a2b89dc01a crimson/os/seastore/transaction: reorder members, make handle private
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-06-29 10:08:55 -07:00
Samuel Just
8bc427aa75 crimson/os/seastore/transaction: remove unused initiated_after
retired_gate_token has this information, Transaction::initiated_after
was left by accident.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-06-29 10:08:55 -07:00
Kefu Chai
d846120186 crimson/os/alienstore: use semaphore to manage tasks in thread pool
* implement std::counting_semaphore in C++17
* use the homebrew counting_semaphore as the synchronization primitive
  to access the pending tasks, for better performance than the
  implementation using mutex and condition_variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-30 00:01:12 +08:00
Patrick Donnelly
be86f191e7
cephsqlite: add comment on atexit
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-29 08:01:56 -07:00
Radoslaw Zarzynski
fad97b3543 crimson: introduce assert_moveable().
In C++ `std::moving` a `const`-qualified value yields a constant
r-value reference (`const T&&`) which won't be matched with a callable
taking non-constant r-value reference (like move constructors) but
can play with one taking a constant l-value reference (like copy
constructors do). This behaviour is surprising especially in lambas
where adding or removing the `mutable` specifier may lead to different
behaviour. The problem isn't obvious and it's easy to wrongly drop
the `mutable` druing a clean-up. Therefore introducing a tool for
developers to fail at compile-time if that happens seems desired.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-29 14:55:51 +00:00
Kefu Chai
05dae35e53
Merge pull request #42002 from tchaikov/wip-common-armor
common/armor: mark dst_end a const pointer

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-29 22:54:36 +08:00
Kefu Chai
7fb626ce3e
Merge pull request #42035 from batrick/i51372
libcephsqlite: shutdown RADOS in destructor

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 22:33:10 +08:00
zdover23
08e032c828
Merge pull request #42094 from zdover23/wip-doc-upgrading-ceph-monitoring-the-upgrade-2021-06-29
doc/cephadm: improving "Monitoring the Upgrade"

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-30 00:08:11 +10:00
zdover23
04216d37d2
Merge pull request #42079 from zdover23/wip-doc-upgrading-ceph-first-section-2021-06-29
doc/cephadm: improve "Upgrading Ceph" (main)

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-30 00:07:28 +10:00
Zac Dover
2dcdef0d43 doc/cephadm: improving "Monitoring the Upgrade"
This PR improves the section "Monitoring the Upgrade"
in the "Upgrading Ceph" chapter of the cephadm documentation.

This PR introduces a couple of section breaks with signposting
information in their titles, and rewrites some sentences in order
to reduce the cognitive load of the reader.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-06-29 22:42:29 +10:00
Kefu Chai
f019a91098
Merge pull request #42042 from tchaikov/wip-51375
common/options: convert a millisecs opt to a chrono::milliseconds and cleanups

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-29 19:28:19 +08:00
Kefu Chai
aa3f2028c1 crimson/os/alienstore: return attrs retrieved from bluestore directly
since ObjectStore returns a map<...,less<>> already, there is not point to
convert the returned map to the type expected by AlienStore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 11:27:25 +00:00
Kefu Chai
a456e031ca os: use transparent comparator in ObjectStore::getattrs()
for two reasons:

- better performance when looking on in the return map if the key
  is not a string, as we don't need to create a temporary string
  as the key
- improve the performance of crimson::AlienStore, as the latter
  uses the transparent comparator. as, without this change, we'd
  have to perform a deep copy to fill up the returned map with
  its non-transparent-comparator version.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 11:27:25 +00:00
Kefu Chai
6ca774dbc7 crimson/os: use lockfree queue for sharded queue
each sharded queue has multiple producers and a single consumer queue,
but the producer side is in seastar world, so ideally, we should not
guard the queue using a POSIX lock.

in this change, the lockfree multiple-writer/multiple-reader queue is
used to replace the std::queue to drop the lock guarding it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 18:58:09 +08:00
Kefu Chai
a41b62bd85 mgr,mon: s/boost::optional/std::optional/
since the encoding schemes of boost::optional and std::optional are
identical, they can be used interchangeably.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:59:33 +08:00
Kefu Chai
32e62f9a64 common/config: s/boost::variant/std::variant/
we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:59:33 +08:00
Kefu Chai
863d8385d0 common/options: s/boost::variant/std::variant/
we should use standard library for more well defined behavior, and
less dependencies on 3rd party libraries.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:56:58 +08:00
Kefu Chai
812e58c597
Merge pull request #42013 from ronen-fr/wip-ronenf-scrubs-config
qa/suites/rados: add simultaneous scrubs to the thrasher

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-06-29 16:21:52 +08:00
Kefu Chai
135b6dde5e
Merge pull request #42031 from sebastian-philipp/normalize_image_digest_local-registry
cephadm: Fix normalize_image_digest for local registries

Reviewed-by: Adam King <adking@redhat.com>
2021-06-29 16:20:28 +08:00
Kefu Chai
7e9d5fa02d
Merge pull request #41933 from liewegas/fix-48311
mgr: set debug_mgr=2/5 (so INFO goes to mgr log by default)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 16:18:17 +08:00
Kefu Chai
2026a7f0d3
Merge pull request #41999 from tchaikov/wip-rpm-deb
rpm,debian: cleanups related to python3-setuptools dependencies

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-06-29 16:16:32 +08:00
Kefu Chai
f91183d986
Merge pull request #42076 from liu-chunmei/crimson-pgnls
crimson: fix pgnls exception

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 15:10:48 +08:00
Kefu Chai
34642418c8 osd/PrimaryLogPG: reorder #includes
so the oder of includes complies to https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 15:00:20 +08:00
Kefu Chai
8fe0305992 osd/PrimaryLogPG: use std::from_chars() to convert str to integer
* no need to create a temporary string for using strtoull()
* use std::from_chars() so it's consistent with its counterpart in
  in crimson.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
f39d71ecaa crimson/osd: check invalid input in do_xattr_cmp_u64()
we should return -EINVAL if the string does not represent a decimal
integer.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
4561fb36f8 crimson/osd: include used header
for using std::from_chars()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
f02a20120c crimson/osd: extract do_cmp_xattr()
to consolidate the logic to dispatch by op.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
9fa07cb89e osd: extract do_cmp_xattr()
to consolidate the logic to dispatch by op.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:54:08 +08:00
Kefu Chai
6e8fbc1373
Merge pull request #42039 from liu-chunmei/crimson-cmpxattr
crimson: add cmp_xatt support

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-29 14:09:15 +08:00
Kefu Chai
d6334e5c43 common/buffers: check _num directly in list::c_str()
no need to create temporary iterator for comparing it with cend().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 13:56:02 +08:00
chunmei-liu
f5fa339036 crimson: fix pgnls exception
has_pg_op is always false, since m->ops is empty at that time.
so pgnls operation will go to process_op and report unknown operations.
move m->finish_decode ahead to fill m->ops.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-06-28 22:20:55 -07:00
Kefu Chai
02b8b0f490 qa: run e2e test on centos only
it's a regression introduced by the restrcuture of the test suites,
let's pin the test to CentOS8.

See-also: https://tracker.ceph.com/issues/49638
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 13:09:53 +08:00
chunmei-liu
5313440847 crimson: add cmp_xatt support
fix Exceptional future ignored: std::runtime_error (op 'cmpxattr' not supported)

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-06-28 21:57:47 -07:00
Chunsong Feng
58753cf50e common: Use double instead of long double to improve performance
To convert namoseconds to seconds, the precision needs to be 10,
and the precision of double is 15, which is enough to use.
On aarch64, double division uses the div instruction, while long
double uses the gcc buildin _divtf3, which has poor performance.
Therefore, use double instead of long double for better performance.

Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
2021-06-29 03:26:07 +00:00