Commit Graph

126791 Commits

Author SHA1 Message Date
Brad Hubbard
e7a8d094bc
Merge pull request #43291 from badone/wip-missing-hostname-dependency
rpm, debian: We depend on 'hostname'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-27 14:21:14 +10:00
Brad Hubbard
cf932436c8 rpm, debian: Add hostname as a "make check" dependency
test_hostname.cc depends on the hostname binary.

Fixes: https://tracker.ceph.com/issues/52695

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-09-27 11:03:55 +10:00
Ronen Friedman
1a2329c46b
Merge pull request #43191 from ronen-fr/wip-rf-scrub-locations
osd/scrub: collecting scrub-related files into a separate directory

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-09-26 20:33:49 +03:00
Kefu Chai
48c044b618
Merge pull request #43304 from inspur-wyq/wip-doc-3
doc/rbd/rbd-windows.rst: fix typos

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-26 21:55:57 +08:00
Ronen Friedman
7248f12768 osd/scrub: collecting scrub-related files into a separate directory
Cleaning src/osd from scrub implementation files. Triggered by:
- the matching Crimson scrub structure;
- the proliferation of scrub related code files (inc. in coming PRs);

scrubber_common.h, which defines the scrubber's interface, remains
in src/osd.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-09-26 13:42:54 +00:00
Ronen Friedman
c98b0bced3
Merge pull request #40984 from ronen-fr/wip-ronenf-scrub-sched
osd/scrub: extracting scrub scheduling code from OSD.cc

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-26 14:59:49 +03:00
Ilya Dryomov
6de678890b
Merge pull request #43220 from idryomov/wip-keyring-resolve-error
auth,mon: don't log "unable to find a keyring" error when key is given

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-26 12:09:43 +02:00
Ilya Dryomov
60e5539976
Merge pull request #43207 from idryomov/wip-full-try-regressions
librados,neorados,librbd: fix FULL_TRY handling regressions

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-09-26 12:07:06 +02:00
Ilya Dryomov
75824a113b
Merge pull request #43228 from idryomov/wip-rbd-mirror-snapshot-rx-only
rbd-mirror: unbreak one-way snapshot-based mirroring

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-09-25 10:58:17 +02:00
Ilya Dryomov
803e03d629
Merge pull request #43282 from Madhu-1/csi-ceph-conf
doc/rbd/rbd-kubernetes: add ceph.conf configmap

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-24 18:04:20 +02:00
Sebastian Wagner
4adcde2a5b
Merge pull request #43294 from badone/wip-run-tox-python-common-timedelta-fix
mgr/dashboard: More robust handling of timedelta() representation

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-24 17:36:57 +02:00
Casey Bodley
80026d3af3
Merge pull request #41931 from pritha-srivastava/wip-rgw-sts-51219
rgw/sts: code for returning an error when an IAM policy

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2021-09-24 11:04:36 -04:00
Madhu Rajanna
4f48612c2c doc/rbd/rbd-kubernetes: add ceph.conf configmap
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-24 16:44:46 +02:00
Sage Weil
0d0a25aa92 Merge PR #42384 into master
* refs/pull/42384/head:
	mgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date
	mgr/cephadm: fix upgrade to version with agent
	mgr/cephadm: delay actions on agent daemons if root cert not created
	mgr/cephadm: handle making certs when we have hostname but no address
	mgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks
	mgr/cephadm: offline host handling improvements for agent
	mgr/cephadm: handle use_agent being turned on and off
	mgr/cephadm: better handling of offline hosts with agent
	mgr/cephadm: convert networks from set to list + don't reset con on down agent hosts
	mgr/cephadm: add ceph volume to metadata agent reports
	mgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line
	cephadm: allow mgr listener to handle variable length json strings
	mgr/cephadm: cephadm agent 2.0

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-24 10:36:41 -04:00
Sebastian Wagner
fcd9e4e4fe
Merge pull request #43298 from sebastian-philipp/cephadm-check_time_sync-mock
cephadm: TestCheckHost: also mock check_time_sync 

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2021-09-24 15:48:40 +02:00
Adam King
d4b070e033 mgr/cephadm: kick serve loop if new metadata makes all hosts metadata up to date
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
e2643798a9 mgr/cephadm: fix upgrade to version with agent
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
64c2a0326c mgr/cephadm: delay actions on agent daemons if root cert not created
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
90ccfdff24 mgr/cephadm: handle making certs when we have hostname but no address
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
d0900bf0c0 mgr/cephadm: make agent certs compatible with OpenSSL hostname and ip checks
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
f0139dd983 mgr/cephadm: offline host handling improvements for agent
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
c8b41564cd mgr/cephadm: handle use_agent being turned on and off
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
f15e1ec255 mgr/cephadm: better handling of offline hosts with agent
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
c35375948e mgr/cephadm: convert networks from set to list + don't reset con on down agent hosts
networks needs to be list so they can be encoded in  a json string
resetting con on the hosts where agent isn't reporting (possibly offline hosts)

Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
eeb18384c3 mgr/cephadm: add ceph volume to metadata agent reports
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
d7072d87ec mgr/cephadm: implement 2-way ssl in mgr -> MgrListener comm line
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:51 -04:00
Adam King
be72463d03 cephadm: allow mgr listener to handle variable length json strings
Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:50 -04:00
Adam King
78983ad0d0 mgr/cephadm: cephadm agent 2.0
Creates an http endpoint in mgr/cephadm to receive
http requests and an agent that can be deployed on
each host that will gather metadata on the host and
send it to the mgr/cephadm http endpoint. Should save the
cephadm mgr module a lot of time it would have to spend
repeatedly ssh-ing into each host to gather the metadata
and help performance on larger clusters.

Fixes: https://tracker.ceph.com/issues/51004

Signed-off-by: Adam King <adking@redhat.com>
2021-09-24 07:23:50 -04:00
Sebastian Wagner
767564864f
cephadm: Fix TestMaintenance trying to list /var/lib/ceph
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-24 12:55:03 +02:00
Sebastian Wagner
fc9d9b3914
cephadm: TestCheckHost: also mock check_time_sync
Fixes: https://tracker.ceph.com/issues/52722

```
TestCheckHost.test_container_engine fails at cephadm:5834: Error cephadm.Error: No time synchronization is active
```

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-24 12:47:04 +02:00
Ilya Dryomov
366e9c51a8 qa/suites/rbd: test case for one-way snapshot-based mirroring
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-09-24 12:30:14 +02:00
Ernesto Puerta
17f3685d40
Merge pull request #43037 from ljflores/perf-histogram-formatting
mgr/dashboard: improve formatting of histograms in Telemetry preview form

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2021-09-24 11:55:59 +02:00
Samuel Just
080b0e12fd
Merge pull request #43295 from liu-chunmei/fix_store_nbd
crimson/store-nbd: fix store_nbd build error for futurized store mkfs

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-09-23 23:29:12 -07:00
chunmei-liu
75d95dcda3 crimson/store-nbd: fix store_nbd build error for futurized store mkfs
Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-09-23 21:20:39 -07:00
Samuel Just
4a9f1d7909
Merge pull request #43261 from rzarzynski/wip-crimson-mkfs_ertr
crimson/{common,os,osd}: errorate the FuturizedStore::mkfs() paths

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-23 19:49:08 -07:00
Samuel Just
d8ee8f03b9
Merge pull request #43288 from rzarzynski/wip-crimson-skip-our-frames-in-bt
crimson/common: skip first 4 frames when dumping a backtrace.

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2021-09-23 19:48:04 -07:00
Brad Hubbard
ad1a4f2410 mgr/dashboard: More robust handling of timedelta() representation
The representation of timedelta() appears to have changed recently so
we should try and be more accommodating in that regard.

Fixes: https://tracker.ceph.com/issues/52696

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2021-09-24 12:22:42 +10:00
Sebastian Wagner
166cd9976c
Merge pull request #43275 from adk3798/maint-target
mgr/cephadm: base maintenance mode enter/exit success/failure on returned message

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-23 22:15:28 +02:00
Sebastian Wagner
d5d1f154bc
Merge pull request #43121 from mgfritch/cephadm-pull-error
cephadm: raise error during `pull` failure

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
2021-09-23 22:12:01 +02:00
Sebastian Wagner
44eb16dd13
Merge pull request #43062 from strenuous-life/wip-cephadm-zap-device
mgr/cephadm: osd should not be zap when it is running

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-09-23 22:11:30 +02:00
Kamoltat Sirivadhna
e3bbfcfd2e
Merge pull request #37544 from kamoltat/wip-mgr-progress-global-efficiency
mgr/progress: optimize global recovery module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-09-23 14:12:16 -04:00
Laura Flores
94fbcefaf1 mgr/dashboard: improve unittest
Signed-off-by: Laura Flores <lflores@redhat.com>
2021-09-23 17:34:05 +00:00
Radoslaw Zarzynski
295268a11a tests/crimson: make the virtual methods of SeaStoreTestState final.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 17:27:39 +00:00
Radoslaw Zarzynski
c04f5a2acf crimson/os: workaround the segfaulting GCC 11 issue.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 17:27:39 +00:00
Radoslaw Zarzynski
e5f7ff249d crimson/osd, crimson/os: errorate the FuturizedStore::mkfs() paths.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 17:27:27 +00:00
Radoslaw Zarzynski
56208ed326 crimson/common: skip first 4 frames when dumping a backtrace.
It's all about these items:

```
 0# print_backtrace(std::basic_string_view<char, std::char_traits<char> >) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:80
 1# FatalSignal::signaled(int, siginfo_t const&) at /opt/rh/gcc-toolset-9/root/usr/include/c++/9/ostream:570
 2# FatalSignal::install_oneshot_signal_handler<11>()::{lambda(int, siginfo_t*, void*)#1}::_FUN(int, siginfo_t*, void*) at /home/rzarzynski/ceph1/build/../src/crimson/common/fatal_signal.cc:
62
 3# 0x00007F16BBA13B30 in /lib64/libpthread.so.0
```

They are part of our backtrace handling and typically developers
are not interested in them. Let's be consistent with the classical
OSD and hide them.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-09-23 16:44:50 +00:00
Samuel Just
2612268e0d
Merge pull request #43262 from rzarzynski/wip-crimson-alienstore-fix-oncommit
crimson/os/alienstore: fix nullptr deref in OnCommit::finish().

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-09-22 22:59:57 -07:00
Pritha Srivastava
a79fa82939 rgw/sts: code to check IAM policy and return an
appropriate error incase Resource specified in the
IAM policy is incorrect and is discarded. The IAM
policy can be a resource policy or an identity policy.
This is for policies that have already been set.

Fixes: https://tracker.ceph.com/issues/51219

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2021-09-23 10:38:26 +05:30
Pritha Srivastava
be42b2cf04 rgw/sts: code for returning an error when an IAM policy
resource belongs to someone else's tenant.

While parsing the policy it discards the resource element,
but then when an operation is evaluated, since the resource element
is empty, it doesnt evaluate the resource at all and the policy
ends up erroneously allowing actions on resources in other tenants.

Fixes: https://tracker.ceph.com/issues/51219

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2021-09-23 10:31:47 +05:30
Adam King
393330bae6 mgr/cephadm: base maintenance enter/exit success off of return message
rather than on whether there was any stdout from the command

Signed-off-by: Adam King <adking@redhat.com>
2021-09-22 20:21:10 -04:00