Commit Graph

119591 Commits

Author SHA1 Message Date
Sage Weil
38e14f9235 qa/suites/rados/cephadm/upgrade: change starting version by distro
centos/rhel have podman 2, which does not like conflicting --cap-add and
--privileged arguments.  cephadm versions prior to 15.2.5 use both args,
however, which means the rhel/centos upgrade test has to start at 15.2.5
to work at all on those distros (with the updated podman).

Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-11 14:08:19 -05:00
Samuel Just
498124f2bf
Merge pull request #39037 from liu-chunmei/seastore_collection
crimson/seastore: add collection implementation

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-02-10 17:51:37 -08:00
Ken Dreyer
a153b1f2ed
Merge pull request #39398 from ktdreyer/cephadm-sys-fs-selinux
cephadm: make /sys/fs/selinux empty
2021-02-10 18:34:12 -07:00
Neha Ojha
2c26f2837e
Merge pull request #39384 from neha-ojha/wip-cephadm-compress
Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-02-10 15:17:27 -08:00
Mykola Golub
b4d9cc45d6
Merge pull request #39155 from dillaman/wip-49037
librbd: correct incremental deep-copy object-map inconsistencies

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-02-10 18:37:34 +02:00
Ken Dreyer
f0f96445b2 cephadm: make /sys/fs/selinux empty
When the following conditions are true:

  1) A host has selinux-policy-targeted,
  2) We mount the host's /sys into a privileged container,
  3) The container has SELINUXTYPE=targeted in /etc/selinux/config,
  4) The container does not have an selinux-policy-targeted package,

then SELinux-enabled applications like restorecon or DNF do not work inside
the container.

Resolve this by making /sys/fs/selinux an empty directory.

Fixes: https://tracker.ceph.com/issues/49239

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2021-02-10 08:25:10 -07:00
Ernesto Puerta
6c5ccc10b0
Merge pull request #38963 from rhcs-dashboard/refresh-crushmap
mgr/dashboard: Automatically refresh crush map metadata table

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-02-10 15:03:09 +01:00
Sebastian Wagner
d8fd08eebb
Merge pull request #39355 from sebastian-philipp/doc-install-rm-deepsea
doc/install: cephadm is the successor of DeepSea

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-02-10 13:29:30 +01:00
Sebastian Wagner
ca41e2a559
Merge pull request #38505 from sebastian-philipp/doc-dev-cephadm-scalability-notes
doc/dev/cephadm: scalability notes

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-02-10 13:17:00 +01:00
Venky Shankar
e40a9f4caf
Merge pull request #39287 from david-z/wip-fix-join-fscid
mon/MDSMonitor.cc: fix join fscid not applied with pending fsmap at boot
2021-02-10 17:30:14 +05:30
Jan Fajerski
40d5c37930 Merge PR #38318 into master
* refs/pull/38318/head:
	common: Add Windows ETW support

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-10 11:50:09 +01:00
chunmei-liu
4fdd451edd crimson/seastore: add collection manager implementation
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-02-09 22:48:37 -08:00
Kefu Chai
fe69a2abf4
Merge pull request #39387 from xxhdx1985126/wip-client-request-pg-ref
crimson/osd: capture pgref by value if it is to be referenced asynchronously

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-10 13:00:13 +08:00
Kefu Chai
2045b8322c
Merge pull request #39388 from ivzhh/master
cmake: don't pass ccache argument to RocksDB build

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-10 12:56:46 +08:00
Sheng Mao
59a5f333fa cmake: don't pass ccache argument to RocksDB build
RocksDB's cmake has a similar logic to turn on ccache as the main cmake.
Currently BuildRocksDB.cmake explicitly passes ccache to RocksDB cmake
arguments as CMAKE_CXX_COMPILER_LAUNCHER. This causes a compilation error
on ccache 4.0, especially in make -j mode.

ccache: error: Recursive invocation (the name of the ccache binary must
be "ccache")

This commit lets cmake from RocksDB decide RULE_LAUNCH_COMPILE by itself.

Signed-off-by: Sheng Mao <shngmao@gmail.com>
2021-02-09 20:17:53 -07:00
Xuehan Xu
affa3a6cce crimson/osd: capture pgref by value if it is to be referenced asynchronously
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-02-10 10:51:08 +08:00
zdover23
718af4faaf
Merge pull request #39351 from zdover23/wip-doc-dev-getting-binaries-rewrite-08-Feb-2021
doc/dev: rewrite "getting binaries" section

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-02-10 12:00:15 +10:00
Brad Hubbard
6aa9528c17
Merge pull request #39383 from badone/wip-unmask-deep-scrub-stdout-lazy_omap_stats_test
test/lazy-omap-stats: Immediate deep-scrub output

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-02-10 11:37:17 +10:00
Samuel Just
3ccf7cb8ca
Merge pull request #39333 from cyx1231st/wip-seastore-onode-tree-empty-oid-ns
crimson/onode-staged-tree: support empty ns and oid

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-02-09 15:39:13 -08:00
Neha Ojha
3ced450cdd Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"
This reverts commit 6d7ba1180e.

Fixes: https://tracker.ceph.com/issues/49228
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-02-09 22:15:37 +00:00
Brad Hubbard
3e79a035ee test/lazy-omap-stats: Immediate deep-scrub output
Show stdout from the child process as it happens.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-02-10 08:02:00 +10:00
Casey Bodley
7f72ff359e
Merge pull request #38871 from cbodley/wip-qa-rgw-valgrind-on
qa/rgw: reenable valgrind in rgw/verify suite

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-02-09 16:22:42 -05:00
Zac Dover
cce1fc800d doc/dev: rewrite "getting binaries" section
This rewrites the "Getting Binaries" section
of the "Workflow" section of the "Tests:
Integration Tests" section of the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-10 06:46:48 +10:00
Casey Bodley
a635e523be qa/rgw: test_rgw_reshard.py just tries common ports
the 'sudo netstat | grep radosgw' command is causing failures:

  File "qa/workunits/rgw/test_rgw_reshard.py", line 53, in get_radosgw_port
    x = out.decode('utf8').split(" ")
AttributeError: 'bool' object has no attribute 'decode'

because when valgrind is enabled, the process name is 'valgrind' instead
of 'radosgw'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-09 14:26:26 -05:00
Casey Bodley
c706740dbe test/rgw: re-enable valgrind in rgw verify suite
this only enables it for radosgw itself, not the other daemons

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-09 14:26:19 -05:00
Casey Bodley
58e0da8fc9 qa/rgw: move valgrind config into valgrind.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-09 14:26:19 -05:00
Nizamudeen A
bc8562ef2a mgr/dashboard: Automatically refresh the crush map metadata table
If we make any change to the osd crush map like do an osd crush reweight from cli, for that change to be reflected on metadata table we need to reload the entire page. Instead this PR takes care of auto refreshing the tree view.

Fixes: https://tracker.ceph.com/issues/48922
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2021-02-09 23:51:07 +05:30
Casey Bodley
8d7cf601a2
Merge pull request #39211 from cbodley/wip-rgw-admin-formatter
radosgw-admin: fix leaks with make_unique()

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-02-09 07:31:05 -05:00
Casey Bodley
aefacf1c98
Merge pull request #39301 from cbodley/wip-49169
rgw: fix snprintf() truncate warning in ETagVerifier

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-02-09 07:30:40 -05:00
Sebastian Wagner
f82e374341 doc/dev/cephadm: scalability notes
Add a few notes about possible future scalability improvements

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-09 12:14:26 +01:00
Sebastian Wagner
8069dea427 doc/install: cephadm is the successor of DeepSea
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-09 12:05:02 +01:00
Alin Gabriel Serdean
8e7042f4ca common: Add Windows ETW support
This patch adds generic ETW logging on top of syslog calls.

The events are described in the message compiler file `event_logging.mc`.

Using the cross platform utility `windmc`, we will generate the header file,
`event_logging.h`, needed by the ETW implementation, and the resource file
`event_logging.rc`.

Over the generated resource file we will run another utility called `windres`.
this tool is used to compile the binary objects needed by the OS utilities
(i.e. `Event Viewer`) to view the logged events.

On usage, a registry key needs to be added/removed on the target computer.
The registry looks like the following:
[HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\<get_process_name>]
"EventMessageFile"="<Folder_location_to>\\event_logging.dll"

I.E.:
[HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\rbd-wnbd]
"EventMessageFile"="C:\\Program Files\\Ceph\\bin\\event_logging.dll"

Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-authored-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-09 10:46:24 +02:00
Kefu Chai
d5290aad46
Merge pull request #39345 from tchaikov/wip-crimson-move
crimson/osd: do not pass oop as a const ref and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-02-09 13:51:01 +08:00
Venky Shankar
9f039ceeda
Merge pull request #39353 from lxbsz/minor
client: use scoped_lock instead of unique_lock
2021-02-09 10:43:10 +05:30
Yingxin Cheng
54f8710664 crimson/onode-staged-tree: be explicit about string_key markers
To be explicit that string_kew_view_t::MIN and MAX are not valid
string lengths. They are reserved as markers.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-02-09 12:48:59 +08:00
Kefu Chai
8c49c91fc9
Merge pull request #39367 from runsisi/wip-fix-spec
ceph.spec.in: fix missing backslash for line continuation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-09 12:23:50 +08:00
Yingxin Cheng
42ae838af5 crimson/onode-staged-tree: support empty ns and oid
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-02-09 11:42:43 +08:00
Kefu Chai
5fcddda37b crimson/os: disable code block only executed by Debug build
silences warning like:

warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
  ).safe_then([this, target_size] {
               ^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
3177df3142 crimson/osd: capture labels using new variables
structured binding does not define variables, let's capture the bound
"variables" using variables defined in the capture list.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
da7ea289e5 crimson/common: mark ExitBarrier final
to silence -Wfinal-dtor-non-final-class warnings like

operation.h:461:20: warning: class with destructor marked 'final' cannot
be inherited from [-Wfinal-dtor-non-final-class]
    ~ExitBarrier() final {
                   ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
e85d1f759c crimson/osd: refactor ClientRequest::do_process()
instead of using a local lambda which tries to have a single point
for handling both the happy path and failure path, it'd be simpler
to just early return if we fail to handle a request,

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
68af4a9f12 crimson/osd: do not pass oop as a const ref
before this change, std::move(oop) will create a copy of oop.
after this change, std::move(oop) actually moves oop.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
3c17ea9010 crimson/osd: do not pass ops to submit_transaction() separately
the order of evaluation of function arguments is unspecified with some
exceptions. but in this case, the reason why ops is still valid is that
the parameter type of `osd_op_p` is `const osd_op_params_t&`. so a copy
is passed to it.

to enable us to actually move `osd_op_p`, in this change, the
osd_op_p.req->ops is used instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
e730737e8b crimson/osd: extract methods out of ClientRequest::process_op()
* do_recover_missing()
* do_process()

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
c6dd356e4c crimson/osd: do not move op_params_p in fill_op_params_bump_pg_version()
it's confusing to move the parameter for updating it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
9eca9243f4 crimson/osd: do not add parentheses unless necessary
do not use the notation of

[]() -> return_type {

unless it helps with readability or we have to do so.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
f37e915e9f crimson/osd: remove no-op func passed to OpsExecuter::flush_changes()
no need to create an empty func for an request which creates no
transactions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
28712f215d crimson/osd: rename PG::rep_repair_primary_object() to repair_object()
as we should expose the internals using the name of interface, and there
is no need to differentiate rep_repair_primary_object() from another
repair method.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-09 11:38:22 +08:00
Kefu Chai
1f6f9b24e4
Merge pull request #39361 from sebastian-philipp/orch-fix-flake8
mgr/orch: Fix flake8 error

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-09 10:15:36 +08:00
luo.runbing
893924c20d
ceph.spec.in: fix missing backslash for line continuation
it errors while building without lua_packages:

+ -DBOOST_J=102 -DWITH_GRAFANA=ON
/var/tmp/rpm-tmp.aq2X3J: line 91: -DBOOST_J=102: command not found

Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
2021-02-09 09:39:41 +08:00