Commit Graph

127920 Commits

Author SHA1 Message Date
Casey Bodley
874b2b304a
Merge pull request #42880 from Kalkulus1/patch-1
doc: adds a way to allow users set_contents_from_filename for rgw

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-11-16 11:01:00 -05:00
Ronen Friedman
2766946460
Merge pull request #43912 from ronen-fr/wip-rf-test-dump
qa/standalone: fix osd-scrub-dump following changes to 'pg dump pgs' output

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-11-16 17:58:07 +02:00
Casey Bodley
696968a898
Merge pull request #43854 from cbodley/wip-53199
cls/rgw: index cancelation still cleans up remove_objs

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2021-11-16 10:28:44 -05:00
Guillaume Abrioux
76560c15f0
Merge pull request #43300 from cfsnyder/wip-52730-fix-dbwal-slot-size-calc
ceph-volume: fix bug with miscalculation of required db/wal slot size for VGs with multiple PVs
2021-11-16 15:15:28 +01:00
Sébastien Han
1654f68aea
Merge pull request #43927 from guits/fix_raw_activate
ceph-volume: follow-up on PR #42727
2021-11-16 14:11:10 +01:00
Sebastian Wagner
343708fb6e
Merge pull request #43780 from mgfritch/nfs-rgw-v4
doc/radosgw/nfs: add note about NFSv3 deprecation

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Sage Weil <sage@newdream.net>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-11-16 11:38:03 +01:00
Sebastian Wagner
4ec317af08
Merge pull request #43933 from ceph/dependabot/pip/src/ceph-volume/plugin/zfs/pip-21.1
build(deps-dev): bump pip from 19.2 to 21.1 in /src/ceph-volume/plugin/zfs

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-16 10:39:37 +01:00
Neha Ojha
d31130d451
Merge pull request #43938 from neha-ojha/wip-quick-fix
qa/suites/upgrade/octopus-x: set bluestore fsck quick fix on mount to true

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2021-11-15 16:09:30 -08:00
Neha Ojha
94e3a240ad qa/suites/upgrade/octopus-x: set bluestore fsck quick fix on mount to true
First step towards catching bugs like https://tracker.ceph.com/issues/53062.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-11-15 21:39:15 +00:00
Samuel Just
16e152e161
Merge pull request #43928 from josephsawaya/futurize-futurized-store
crimson: futurize FuturizedStore::create()

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-11-15 12:17:33 -08:00
Matan
02348afc73
Merge pull request #43344 from Matan-B/wip-matanb-crimson-clog
crimson/osd: Implementing cluster logging
2021-11-15 21:37:49 +02:00
Casey Bodley
879dd9543c
Merge pull request #43761 from cbodley/wip-beast-timeout-alloc
rgw/beast: optimizations for request timeout

Reviewed-by: Mark Kogan <mkogan@redhat.com>
2021-11-15 13:31:18 -05:00
Casey Bodley
b848ccae96 rgw/rados: index transactions pass remove_objs to cancel() too
whenever an index transaction uses remove_objs for complete(), it also
needs to pass them for cancel() to avoid leaking index entries

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-15 13:16:07 -05:00
Casey Bodley
8b27472bbd cls/rgw: index cancelation still cleans up remove_objs
when multipart uploads complete their final bucket index transaction,
they pass the list of part objects in 'remove_objs' for bulk removal -
the part objects, along with their bucket stats, get replaced by the
head object

but if CompleteMultipart races with another upload, the head object
write will fail with ECANCELED and the bucket index transaction gets
canceled with CLS_RGW_OP_CANCEL. these canceled uploads still need to
clean up their 'remove_objs', but cancelation was returning too early.
as a result, these bucket index entries get orphaned and leave the
bucket stats inconsistent

this commit reworks rgw_bucket_complete_op() so that CLS_RGW_OP_CANCEL
is handled the same way as OP_ADD and OP_DEL, so always runs the loop to
clean up 'remove_objs'

Fixes: https://tracker.ceph.com/issues/53199

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-15 13:16:05 -05:00
Casey Bodley
f3325fc76f cls/rgw: add complete_remove_obj() helper for remove_objs
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-15 13:06:46 -05:00
dependabot[bot]
3306425d38
build(deps-dev): bump pip in /src/ceph-volume/plugin/zfs
Bumps [pip](https://github.com/pypa/pip) from 19.2 to 21.1.
- [Release notes](https://github.com/pypa/pip/releases)
- [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst)
- [Commits](https://github.com/pypa/pip/compare/19.2...21.1)

---
updated-dependencies:
- dependency-name: pip
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-15 17:48:13 +00:00
Casey Bodley
d7ec0b281c cls/rgw: helpers take const input params
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-15 12:34:07 -05:00
Matan Breizman
7b5cb61bd9 crimosn/osd: Implementing cluster logging
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2021-11-15 16:29:15 +00:00
Ernesto Puerta
689c213d1a
Merge pull request #43903 from rhcs-dashboard/edit-service-feature
mgr/dashboard: Edit a service feature 

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-11-15 17:25:33 +01:00
Sage Weil
22871aa2e0 Merge PR #43894 into master
* refs/pull/43894/head:
	qa/suites/orch/cephadm: verify that 'orch ls' reports OSDs properly
	mgr/cephadm: show unmanaged OSDs under 'osd' service

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-15 11:13:11 -05:00
Mark Kogan
c0c1dc389c rgwi/beast: stream timer with duration 0 disables timeout
fixes all S3 operations failing with:
`2021-11-15T15:46:05.992+0000 7ffee17fa700 20 failed to read header: Bad file descriptor`
when `--rgw_frontends="beast port=8000 request_timeout_ms=0"`

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2021-11-15 15:57:42 +00:00
Joseph Sawaya
6fe9dea75b crimson: futurize FuturizedStore::create()
This commit changes the FuturizedStore::create() function to return a
seastar::future containing its original return value.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-11-15 15:17:13 +00:00
Guillaume Abrioux
2c228a9a40 ceph-volume: follow-up on PR #42727
PR #42727 introduced a regression in `ceph-volume raw activate`

since it dropped `nargs='+'` from the argument `--device`, the variable
is no longer a list but a string.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-11-15 16:02:46 +01:00
Venky Shankar
785edd08a0
Merge pull request #42110 from joscollin/wip-doc-cephfs-top-misc-fixes
cephfs-top: fix Refresh interval

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-11-15 11:50:22 +05:30
Deepika Upadhyay
44cd7c7650
Merge pull request #42046 from CongMinYin/align-entry-bit
librbd/cache/pwl/ssd: make log entry 64 bit and add ssd version control

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-11-13 17:27:27 +05:30
Deepika Upadhyay
b287de4219
Merge pull request #43837 from majianpeng/librbd-fix-reorder-problem-between-process_writeback_dirty_entries
librbd/cache/pwl: fix reorder issue between func process_writeback_dirty_entries

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-11-13 17:26:24 +05:30
Casey Bodley
2a6fce4edc
Merge pull request #43773 from linuxbox2/wip-rgwnfs-crash
librgw: treat empty root path as "/" on mount

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-11-12 15:57:37 -05:00
Casey Bodley
a50dcf6dbd
Merge pull request #43563 from Zhiwei-Dai/wip-fix-mutex-unlock-twice
common: avoid pthread_mutex_unlock twice

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2021-11-12 15:56:18 -05:00
Casey Bodley
51eafb38ac
Merge pull request #43829 from cbodley/wip-rgw-auth-uninitialized-optional
rgw/auth: perm_mask uses std::optional

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-11-12 15:53:10 -05:00
Casey Bodley
f1bbf70272
Merge pull request #43103 from ivancich/wip-none-stats-fix
rgw: fix rgw.none statistics

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-11-12 15:52:46 -05:00
Ronen Friedman
9dda986bd5 qa/standalone: fix scrub/osd-scrub-dump following changes to 'pg dump pgs' output
Make osd-scrub-dump test ignore the 'scrubbing' that might be late to disappear
from the modified (PR #43403) 'pg dump' output.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-11-12 18:43:41 +00:00
Ernesto Puerta
6fe330c800
Merge pull request #43877 from rhcs-dashboard/fix-spelling-network-address
mgr/dashboard: Spelling mistake in host-form Network address field

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-11-12 19:10:23 +01:00
Patrick Donnelly
6588d8ff77
Merge PR #43896 into master
* refs/pull/43896/head:
	doc: document subvolume (group) pins

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2021-11-12 10:20:05 -05:00
Casey Bodley
a68fbb18a9 rgw/beast: reference count Connections for timeout_handler
resolves a use-after-free in the timeout_handler, where a timeout fires
and schedules the timeout_handler for execution, but the coroutine exits
and destroys the socket before asio executes the timeout_handler

timeout_handler now holds a reference on the Connection to extend its
lifetime

now that the Connection is allocated on the heap, we can include the
parse_buffer in this memory instead of allocating it separately

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-12 09:53:01 -05:00
Casey Bodley
3f853538b1 rgw/beast: replace beast::tcp_stream with manual timeouts
remove the beast::tcp_stream wrapper from the socket, and track timeouts
manually with a timeout_timer. this timer uses ceph's coarse_mono_clock
which is cheaper to sample than std::chrono::steady_clock

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-12 09:37:09 -05:00
Casey Bodley
0bee64d875 rgw/beast: use explicit executor type for tcp socket and stream
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-12 09:37:09 -05:00
Casey Bodley
9d9258e06b spawn: use explicit strand executor
the default spawn::yield_context uses the polymorphic boost::asio::executor
to support any executor type

rgw's beast frontend always uses the same executor type for these
coroutines, so we can use that type directly to avoid the overhead of
type erasure and virtual function calls

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-12 09:37:09 -05:00
Casey Bodley
44f4b083de rgw: clean up WITH_RADOSGW_BEAST_OPENSSL
the #ifdef was covering more includes than it should have

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-12 09:37:09 -05:00
Yin Congmin
3da4a9401c qa/suites/rbd/persistent-writeback-cache: add test case
Add the test case which size is 8GB, So that some problems that occur
only in test scenarios above 4GB may be found in this test. For example,
the variables of 32-bit may be unexpected value when it operates with
a 64 bit value.

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-11-12 17:31:00 +08:00
Nizamudeen A
3c3d8822d4 mgr/dashboard: Edit a service feature
Fixes: https://tracker.ceph.com/issues/53077
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-11-12 13:44:51 +05:30
Neha Ojha
9a4b3c1926
Merge pull request #43570 from ljflores/wip-pg-stats
mgr/telemetry: modify stats_per_pool and add stats_per_pg

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>
2021-11-11 11:51:59 -08:00
Sage Weil
e1cd91098f qa/suites/orch/cephadm: verify that 'orch ls' reports OSDs properly
Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-11 13:53:33 -05:00
Sage Weil
f13a3e5b3c mgr/cephadm: show unmanaged OSDs under 'osd' service
1- If the unit.meta file service_name = osd.NNN (which is true for lots of
OSDs deployed on older version of cephadm) then ignore the field entirely.

2- If an OSD has not service_name (see above) then show it under the 'osd'
service (instead of 'osd.unmanaged').

Sample 'ceph orch ls' output with a drivegroup + unmanaged OSD:

NAME      PORTS  RUNNING  REFRESHED  AGE  PLACEMENT
...
osd                    1  85s ago    -    <unmanaged>
osd.hdds               5  85s ago    2s   *
...

Fixes: https://tracker.ceph.com/issues/53235
Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-11 13:53:33 -05:00
Yuval Lifshitz
d69f196c0f
Merge pull request #43587 from zenomri/wip-omri-tracer-opentelemetry
common/tracer: Tracer implementation using opentelemetry sdk
2021-11-11 19:40:12 +02:00
Patrick Donnelly
6e569bfd14
doc: document subvolume (group) pins
Fixes: https://tracker.ceph.com/issues/53236
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-11 12:30:02 -05:00
Ernesto Puerta
45eb9dd328
Merge pull request #43464 from rsommer/wip-prometheus-standby-behaviour
mgr/prometheus: Make prometheus standby behaviour configurable

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-11-11 17:36:30 +01:00
Sebastian Wagner
0eb651ca7f
Merge pull request #43874 from liewegas/qa-podman-add-stream
qa/suites/orch/cephadm: add 8.stream + container_tools

Reviewed-by: Adam King <adking@redhat.com>
2021-11-11 16:43:34 +01:00
Sage Weil
25c65fddd1 Merge PR #43046 into master
* refs/pull/43046/head:
	mgr/rook: get running pods, auth rm, better error checking for orch nfs
	qa/tasks/rook: add apply nfs to rook qa task
	mgr/rook: prevent creation of NFS clusters not in .nfs rados pool
	mgr/rook, mgr/nfs: update rook orchestrator to create and use .nfs pool

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-11-11 10:31:22 -05:00
Patrick Donnelly
e40142fa8d
Merge PR #43851 into master
* refs/pull/43851/head:
	mds/FSMap: allow upgrade when no MDS is "in"

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-11-11 10:17:56 -05:00
Patrick Donnelly
5902d10c3c
Merge PR #43800 into master
* refs/pull/43800/head:
	pybind/mgr/cephadm: disable FSMap sanity checks during MDS upgrade
	mds/FSMap: assign v16.2.4 compat to pre-v16.2.5 standby daemons

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-11-11 10:14:50 -05:00