Commit Graph

101267 Commits

Author SHA1 Message Date
Matt Benjamin
0d3338c2d2 rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-08 18:57:15 -04:00
Sage Weil
6c690ae285 os/bluestore: fsck: warning (not error) by default on no per-pool omap
Unconditionally error if the global per_pool_omap was set, though!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:37 -05:00
Sage Weil
dee8f8cb7e os/bluestore: fsck: int64_t for error count
Just to be on the safe side.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
dbdd1d9b6e os/bluestore: default size of 1 TB for testing
Less ENOSPC from vstart.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
52a2d4b24d os/bluestore: behave if we *do* set PGMETA and PERPOOL flags
pgmeta trumps perpool.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
9bbe8d04c7 os/bluestore: do not set both PGMETA_OMAP and PERPOOL_OMAP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
1eb10f3233 os/bluestore: fsck: only generate 1 error per omap_head
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
b1e44c3298 os/bluestore: make fsck repair convert to per-pool omap
- error if objects not per-pool
- error if per_pool_omap not set
- convert as we go, and set the flag at the end

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Yuri Weinstein
690385362c
Merge pull request #29541 from yuriw/wip-yuriw-upgrade-rbd-master
qa/tests: added rbd task on ec
2019-08-08 15:22:48 -07:00
Sage Weil
0d9410036a Merge PR #27542 into master
* refs/pull/27542/head:
	qa/tasks: Fix ambiguous store_thrash, thrash_store

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-08 15:52:58 -05:00
Sage Weil
8827bc1022 Merge PR #29493 into master
* refs/pull/29493/head:
	qa/tasks/mgr/mgr_test_case: get mgrmap from 'mgr dump', not status
	qa/tasks/ceph_manager: no newlines in 'ceph -s' output
	mon: make mon summary more concise in 'ceph -s'
	mon/MgrStatMonitor: set initial service_map 'modified' to cluster mkfs
	mon: remove double-nesting of "osdmap" for ceph status
	mon/MgrMap: make print_summary (used by 'ceph -s') more concise

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-08 15:52:45 -05:00
Sage Weil
a0fca4eaf6 Merge PR #29511 into master
* refs/pull/29511/head:
	common/config: respect POD_MEMORY_REQUEST *and* POD_MEMORY_LIMIT env vars
	common/config: let diff show non-build defaults
	common/config: do no include multiple 'default' values

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-08 15:49:29 -05:00
Sage Weil
01fcf0e401 Merge PR #29517 into master
* refs/pull/29517/head:
	osd: include PastInterals in pg_notify_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-08-08 15:49:17 -05:00
Adam C. Emerson
ad88d4dced Merge pull request #29559 from adamemerson/wip-putobj-crash
rgw: Move upload_info declaration out of conditional

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2019-08-08 15:14:03 -04:00
Adam C. Emerson
a63881eaf7 rgw: Move upload_info declaration out of conditional
That way we don't deconstruct it right after assigning a reference to
part of it.

Fixes: https://tracker.ceph.com/issues/41172
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2019-08-08 13:12:46 -04:00
Sage Weil
99c56559d1 os/bluestore: whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 11:23:27 -05:00
Sage Weil
c69c9539f6 os/bluestore/BlueFS: add bluefs_shared_alloc_size
Add a separate config option that controls the alloc_size for the shared
device (BDEV_SLOW).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 11:23:27 -05:00
Neha
27aa0db8e5 os/bluestore/BlueStore.cc: start should be >= _get_ondisk_reserved()
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-08-08 11:23:27 -05:00
Casey Bodley
24a3056171
Merge pull request #29449 from cbodley/wip-vstart-client-rgw
vstart: move common rgw config to [client.rgw]

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-08 11:33:52 -04:00
Yuri Weinstein
644ed83e24 qa/tests: added rbd task on ec
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-08-08 08:10:37 -07:00
Sage Weil
72d6426025 osd: include PastInterals in pg_notify_t
We use a pair<pg_notify_t,PastIntervals> everywhere a pg_notify_t is used.
This is silly; just make it a member instead.

Include some minor compat cruft so we can speak to pre-octopus OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 09:14:03 -05:00
Stephan Müller
f07c1939d4 mgr/dashboard: CephFS client tab switch
The problem was that it was not possible to click on the clients tab
inside the CephFS details and switch to it. This commit fixes this.

Fixes: https://tracker.ceph.com/issues/41165
Signed-off-by: Stephan Müller <smueller@suse.com>
2019-08-08 16:11:36 +02:00
Casey Bodley
6e976c5aa7
Merge pull request #29538 from cbodley/wip-rgw-data-changes-shared-unlock
rgw: fix unlock of shared lock in RGWDataChangesLog

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-08 10:11:29 -04:00
Casey Bodley
57887f6a36 rgw: http client drops mutex before suspending coroutine
the lock can't be held over async_wait(), because the call to finish()
won't be able to reacquire the lock in order to fire the completion.
coroutines may also resume on a different thread, and you can't unlock a
mutex that was locked on another thread

Reported-by: Yuval Lifshitz <yuvalif@yahoo.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-08 10:06:40 -04:00
Casey Bodley
fef63ad340 rgw: http client doesn't need lock to access atomic 'done'
Reported-by: Yuval Lifshitz <yuvalif@yahoo.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-08 10:04:14 -04:00
Varsha Rao
c57f6bdf15 cephfs-shell: Fix onecmd TypeError
Fixes: https://tracker.ceph.com/issues/41164
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-08-08 19:05:15 +05:30
Mykola Golub
82a45573d5
Merge pull request #29468 from dillaman/wip-41032
pybind/mgr/rbd_support: use image ids to detect duplicate tasks

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-08-08 16:32:33 +03:00
Sage Weil
0b719c4382 Merge PR #29514 into master
* refs/pull/29514/head:
	mgr/hello: typos
	mgr/hello: a better command example
	mgr/hello: boilerplate for handling config options
	mgr/hello: fully document our options

Reviewed-by: Tim Serong <tserong@suse.com>
2019-08-08 08:24:33 -05:00
Sage Weil
390f76d525 qa/tasks/mgr/mgr_test_case: get mgrmap from 'mgr dump', not status
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 08:17:04 -05:00
Varsha Rao
4c968b1f30 cephfs-shell: Convert paths type from string to bytes
Almost for all paths type is changed to bytes, by specifying custom byte
conversion action class and passing it to argparser add_argument() method.
While printing of paths and concating with colorama module arguments, paths
type is string.

Fixes: https://tracker.ceph.com/issues/41163
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-08-08 18:42:57 +05:30
Sage Weil
0527d9c8cd Merge PR #29546 into master
* refs/pull/29546/head:
	mgr/telemetry: clear the event after being awaken by it

Reviewed-by: Sage Weil <sage@redhat.com>
2019-08-08 08:12:53 -05:00
Lenz Grimmer
6d5939f8d1
mgr/dashboard: Prevent clone when layering not enabled on pare… (#29317)
mgr/dashboard: Prevent clone when layering not enabled on parent image

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-08-08 14:38:58 +02:00
Lenz Grimmer
ee66b6be91
mgr/dashboard: fix RBD snapshot protected label (#29534)
mgr/dashboard: fix RBD snapshot protected label

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-08 14:34:17 +02:00
Lenz Grimmer
95ec11a080
mgr/dashboard: Verify fields on Crush map page (#29452)
mgr/dashboard: Verify fields on Crush map page

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-08 14:01:49 +02:00
Lenz Grimmer
f22fc76453
mgr/dashboard: Verify fields on Monitors page (#29450)
mgr/dashboard: Verify fields on Monitors page

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-08 14:00:52 +02:00
Lenz Grimmer
e56e3d78fb
mgr/dashboard: E2E Dashboard Page Test Suite (#29432)
mgr/dashboard: E2E Dashboard Page Test Suite

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-08 13:59:42 +02:00
Lenz Grimmer
c2caec2ebe
mgr/dashboard: Verify manager modules have editing functionali… (#29407)
mgr/dashboard: Verify manager modules have editing functionality

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-08-08 13:58:51 +02:00
Soumya Koduri
eb6eddbe8d Validate bucket names as per revised s3 spec
As per amazon s3 spec -
https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

* The s3 bucket names should not contain upper case letters or underscore.
* Name cannot end with dash or have consecutive periods, or dashes adjacent
  to periods.
* Each label in the bucket name must start and end with a lowercase
  letter or a number.
* Name cannot exceed 63 characters.

This change is to enforce these rules if rgw_relaxed_s3_bucket_names is set to
'false' which is by default.

Fixes: https://tracker.ceph.com/issues/36293

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2019-08-08 16:54:12 +05:30
Kefu Chai
0dddb20685 mgr/telemetry: clear the event after being awaken by it
otherwise telemetry will have a busy-loop once it's signaled.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-08 12:24:56 +08:00
David Zafman
b1c14b7f6e
Merge pull request #29494 from dzafman/wip-scrub-test
test: Bump sleep time for slower machines

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-07 18:30:31 -07:00
Jianpeng Ma
ff71ad472e os/bluestore: add kv_drain_preceding_waiters indicate drain_preceding.
Condition: (txc->state == TransContext::STATE_PREPARE &&  deferred_aggressive) isn't
correctly for drain_preceding. So represent this event by adding this
parameter.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2019-08-08 08:50:25 +08:00
Jianpeng Ma
3f64cc71f5 os/bluestore: reduce unecessary notify.
In fact, only drain()/drain_preceding() need wakeup.
Other case it depend on kv_submitted_waiters.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2019-08-08 08:50:14 +08:00
Casey Bodley
d02c1cab5c rgw: add rename_swift_keys for user rename
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
8c4338cb63 rgw: consolidate RGWUser::execute_rename
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
cfabb2f05f rgw: user rename calls update() to write final user
this calls remove_old_indexes() to clean up the uid and buckets index
objects, and also sets 'old_info = user_info' so that radosgw-admin
dumps the json output of the renamed user instead of the old user

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
c7ea6dc6d6 rgw: user rename overwrites new user with --yes-i-really-mean-it
if a previous rename attempt fails to complete, it can be restarted with
--yes-i-really-mean-it to overwrite the 'stub' user created previously

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
ff707aa303 rgw: user rename operates on 'stub' user
user rename creates a 'stub' user and links all buckets to that before
modifying the existing user. once everything succeeds, the stub user is
updated to match the existing user and all index objects are updated to
point at it

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
b8a60eb046 rgw: add rgw_remove_user_buckets_index
helper function to remove the user:buckets object.
rgw_remove_uid_index() now omits the object version tracker argument to
avoid reading the user info

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
4375204a4a rgw: simplify bucket chown
removes unused 'attrs' argument, and takes uid + display name instead
of RGWUserInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-08-07 16:25:12 -04:00
Casey Bodley
17fc695047
Merge pull request #28813 from smanjara/wip-user-rename-working
rgw : Bucket mv, bucket chown and user rename utilities

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-07 16:05:07 -04:00