Commit Graph

101064 Commits

Author SHA1 Message Date
Yuval Lifshitz
929c062ae9 rgw: don't throw when accept errors are happening on frontend
Signed-off-by: Yuval Lifshitz <yuvalif@yahoo.com>
2019-08-11 10:06:05 +03:00
Josh Durgin
3f18ed55aa
Merge pull request #28227 from sseshasa/monCachePriority
mon/OSDMonitor: Use generic priority cache tuner for mon caches

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-09 14:23:39 -07:00
Casey Bodley
bc45261470
Merge pull request #29540 from cbodley/wip-rgw-user-rename
rgw: followup for 'user rename'

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-09 16:57:25 -04:00
Sage Weil
9346d3c3bc os/bluestore: do not set osd_memory_target default from cgroup limit
On the aarch64 box I'm testing, this gives us a value of
7378697629483768832, which is not what we want.

I think we are better off relying on this limit being explicitly set via
environment variables (POD_* by kuberentes/rook) or via the command line.

This partially reverts 5c6b533697, but not
all of it, since we wan to keep the option itself, as it is now used by
common/config.cc when dealing with the POD_MEMORY_LIMIT env var.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 12:25:59 -05:00
Casey Bodley
13f1595335
Merge pull request #29558 from theanalyst/rgw-cache-lock
rgw: fix unlock of shared lock in RGWCache

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-09 13:03:35 -04:00
Sage Weil
377fdb1484 os/bluestore: refuse to mkfs or mount if osd_max_object_size >= MAX_OBJECT_SIZE
BlueStore has its own object size limit (2^32-1).  Make sure the cluster
limit is below that or refuse to mkfs or mount.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 10:57:14 -05:00
Abhishek Lekshmanan
2b6dbe31c8 rgw: fix unlock of shared lock in RGWCache
similar to https://github.com/ceph/ceph/pull/29538/ we unlock a shared_lock with
unlock causing a crash. Also scope the single line if statements to make the
code more concise

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-09 17:54:22 +02:00
Sage Weil
f011c13547 Merge PR #29292 into master
* refs/pull/29292/head:
	os/bluestore: warn on no per-pool omap
	os/bluestore: fsck: warning (not error) by default on no per-pool omap
	os/bluestore: fsck: int64_t for error count
	os/bluestore: default size of 1 TB for testing
	os/bluestore: behave if we *do* set PGMETA and PERPOOL flags
	os/bluestore: do not set both PGMETA_OMAP and PERPOOL_OMAP
	os/bluestore: fsck: only generate 1 error per omap_head
	os/bluestore: make fsck repair convert to per-pool omap
	os/bluestore: teach fsck to tolerate per-pool omap
	os/bluestore: ondisk format change to 3 for per-pool omap
	mon/PGMap: add data/omap breakouts for 'df detail' view
	osd/osd_types: separate get_{user,allocated}_bytes() into data and omap variants
	mon/PGMap: fix stored_raw calculation
	mon/PGMap: add in actual omap usage into per-pool stats
	osd: report per-pool omap support via store_statfs_t
	os/bluestore: set per_pool_omap key on mkfs
	osd/osd_types: count per-pool omap capable OSDs
	os/bluestore: report omap_allocated per-pool
	os/bluestore: add pool prefix to omap keys
	kv/KeyValueDB: take key_prefix for estimate_prefix_size()
	os/bluestore: fix manual omap key manipulation to use Onode::get_omap_key()
	os/bluestore: make omap key helpers Onode methods
	os/bluestore: add Onode::get_omap_prefix() helper
	os/bluestore: change _do_omap_clear() args

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-09 10:40:45 -05:00
Alfredo Deza
8363d89a4d
Merge pull request #29528 from tchaikov/wip-build-doc-with-python3
admin/build-doc: use python3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-08-09 11:17:19 -04:00
Sage Weil
9426974195 os/bluestore/BlueFS: fix device_migrate_to_* to handle varying alloc sizes
The previous implementation moved extents individually.  This caused
problems when moving an extent with a small alloc_size that wasn't
a multiple of the target device's alloc_size.

Instead, identify files with extents that need to be moved, and then read
the file in its entirety and rewrite it in its entirety.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 10:10:12 -05:00
Sage Weil
e8b5a458c3 os/bluestore/BlueFS: apply shared_alloc_size to shared device
Keep an alloc_size vector so that we have this value handy at all times.
Allow bluestore to fetch this value directly instead of looking at the
bluefs_* config options since this encapsulates things a bit better, and
also isn't vulnerable to the config setting changing at runtime.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 10:10:12 -05:00
Abhishek Lekshmanan
fac4ab71fb rgw: url decode PutUserPolicy params
Since these are sent as a part of a POST request which is usually urlencoded,
the json parser would later return invalid json for jsons containing whitespace

Fixes: https://tracker.ceph.com/issues/41189
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-09 16:57:25 +02:00
Sage Weil
39db4d7c4b os/bluestore/KernelDevice: print aio error extent in hex
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 09:23:25 -05:00
Matt Benjamin
a7b29647fd
Merge pull request #29560 from linuxbox2/wip-rgwf-advance
rgw_file: dont deadlock in advance_mtime()
2019-08-09 10:01:03 -04:00
Sage Weil
b8501164ef os/bluestore: warn on no per-pool omap
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-09 08:21:18 -05:00
Jan Fajerski
b8d6dcbe9f ceph-volume: never log to stdout, use stderr instead
We should never print log messages to stdout, as this should be reserved
for output of ceph-volume.

Fixes: https://tracker.ceph.com/issues/41158

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-08-09 14:26:16 +02:00
Sebastian Krah
9edf36f464 mgr/dashboard: Update language files
Signed-off-by: Sebastian Krah <skrah@suse.com>
2019-08-09 10:50:29 +02:00
Sebastian Krah
d33b5d33b5 mgr/dashboard: Add transifex-i18ntool
Adds the npm package transifex-i18ntool which manages the translation
files of the ceph dashboard

Signed-off-by: Sebastian Krah <skrah@suse.com>
2019-08-09 10:50:29 +02:00
alfonsomthd
bc8e811b08 mgr/dashboard: adapt bucket tenant tests to new behaviour
Fixes: https://tracker.ceph.com/issues/41175
Signed-off-by: alfonsomthd <almartin@redhat.com>
2019-08-09 10:05:46 +02:00
Kefu Chai
2db496017a
Merge pull request #29400 from wjwithagen/wjw-fix-do_freeBSD.sh
do_freebsd.sh: update build scripts to resemble Jenkins scripts

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-09 14:09:42 +08:00
Kefu Chai
ed8ff905fc
Merge pull request #29495 from ifed01/wip-ifed-finisher-improve
common/Finisher: remove some lock acquisitions.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-09 12:24:28 +08:00
Kefu Chai
e5165db985
Merge pull request #29477 from tchaikov/wip-osd-pg-as-mutex
osd: pg as a mutex

Reviewed-by: Greg Farnum gfarnum@redhat.com
Reviewed-by: Samuel Just <sjust@redhat.com>
2019-08-09 12:22:26 +08:00
Kefu Chai
abce85bbd2
Merge pull request #29486 from runsisi/wip-fix-verbose
ceph.in: fix verbose print

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-08-09 12:21:29 +08:00
Kefu Chai
32b33d3258
Merge pull request #29465 from penglaiyxy/wip_bluestore_caculated_revert
os/bluestore: no need to add tail length (revert PR#29185)

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-08-09 12:20:46 +08:00
Kefu Chai
88c6c039d1
Merge pull request #29488 from majianpeng/bluestore-remove-lock
os/bluestore: no need protected by OpSequencer::qlock.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-08-09 12:19:27 +08:00
Kefu Chai
3efc51fa1d
Merge pull request #29385 from kamoltat/wip-qa-tasks-mgr-test-progress-bug-fix
qa/tasks/mgr/test_progress.py: fix bug in 9b4dbf0

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-09 12:18:40 +08:00
Dan Mick
8096d69f9a
Merge pull request #29365 from liewegas/wip-telemetry-post
telemetry/server: misc fixes
2019-08-08 16:32:15 -07:00
Song Shun
612db362ef osd: add device_id to list_devices to help get smart info easily
Signed-off-by: Song Shun <song.shun3@zte.com.cn>
2019-08-08 18:57:38 -04:00
Matt Benjamin
0d3338c2d2 rgw_file: dont deadlock in advance_mtime()
Fixes: https://tracker.ceph.com/issues/41173

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-08 18:57:15 -04:00
Sage Weil
6c690ae285 os/bluestore: fsck: warning (not error) by default on no per-pool omap
Unconditionally error if the global per_pool_omap was set, though!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:37 -05:00
Sage Weil
dee8f8cb7e os/bluestore: fsck: int64_t for error count
Just to be on the safe side.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
dbdd1d9b6e os/bluestore: default size of 1 TB for testing
Less ENOSPC from vstart.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
52a2d4b24d os/bluestore: behave if we *do* set PGMETA and PERPOOL flags
pgmeta trumps perpool.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
9bbe8d04c7 os/bluestore: do not set both PGMETA_OMAP and PERPOOL_OMAP
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
1eb10f3233 os/bluestore: fsck: only generate 1 error per omap_head
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Sage Weil
b1e44c3298 os/bluestore: make fsck repair convert to per-pool omap
- error if objects not per-pool
- error if per_pool_omap not set
- convert as we go, and set the flag at the end

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 17:26:24 -05:00
Yuri Weinstein
690385362c
Merge pull request #29541 from yuriw/wip-yuriw-upgrade-rbd-master
qa/tests: added rbd task on ec
2019-08-08 15:22:48 -07:00
Sage Weil
0d9410036a Merge PR #27542 into master
* refs/pull/27542/head:
	qa/tasks: Fix ambiguous store_thrash, thrash_store

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-08 15:52:58 -05:00
Sage Weil
8827bc1022 Merge PR #29493 into master
* refs/pull/29493/head:
	qa/tasks/mgr/mgr_test_case: get mgrmap from 'mgr dump', not status
	qa/tasks/ceph_manager: no newlines in 'ceph -s' output
	mon: make mon summary more concise in 'ceph -s'
	mon/MgrStatMonitor: set initial service_map 'modified' to cluster mkfs
	mon: remove double-nesting of "osdmap" for ceph status
	mon/MgrMap: make print_summary (used by 'ceph -s') more concise

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-08 15:52:45 -05:00
Sage Weil
a0fca4eaf6 Merge PR #29511 into master
* refs/pull/29511/head:
	common/config: respect POD_MEMORY_REQUEST *and* POD_MEMORY_LIMIT env vars
	common/config: let diff show non-build defaults
	common/config: do no include multiple 'default' values

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-08-08 15:49:29 -05:00
Sage Weil
01fcf0e401 Merge PR #29517 into master
* refs/pull/29517/head:
	osd: include PastInterals in pg_notify_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-08-08 15:49:17 -05:00
Adam C. Emerson
ad88d4dced Merge pull request #29559 from adamemerson/wip-putobj-crash
rgw: Move upload_info declaration out of conditional

Reviewed-By: Casey Bodley <cbodley@redhat.com>
2019-08-08 15:14:03 -04:00
Adam C. Emerson
a63881eaf7 rgw: Move upload_info declaration out of conditional
That way we don't deconstruct it right after assigning a reference to
part of it.

Fixes: https://tracker.ceph.com/issues/41172
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2019-08-08 13:12:46 -04:00
Sage Weil
99c56559d1 os/bluestore: whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 11:23:27 -05:00
Sage Weil
c69c9539f6 os/bluestore/BlueFS: add bluefs_shared_alloc_size
Add a separate config option that controls the alloc_size for the shared
device (BDEV_SLOW).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 11:23:27 -05:00
Neha
27aa0db8e5 os/bluestore/BlueStore.cc: start should be >= _get_ondisk_reserved()
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-08-08 11:23:27 -05:00
Casey Bodley
24a3056171
Merge pull request #29449 from cbodley/wip-vstart-client-rgw
vstart: move common rgw config to [client.rgw]

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>
2019-08-08 11:33:52 -04:00
Yuri Weinstein
644ed83e24 qa/tests: added rbd task on ec
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-08-08 08:10:37 -07:00
Sage Weil
72d6426025 osd: include PastInterals in pg_notify_t
We use a pair<pg_notify_t,PastIntervals> everywhere a pg_notify_t is used.
This is silly; just make it a member instead.

Include some minor compat cruft so we can speak to pre-octopus OSDs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-08-08 09:14:03 -05:00
Stephan Müller
f07c1939d4 mgr/dashboard: CephFS client tab switch
The problem was that it was not possible to click on the clients tab
inside the CephFS details and switch to it. This commit fixes this.

Fixes: https://tracker.ceph.com/issues/41165
Signed-off-by: Stephan Müller <smueller@suse.com>
2019-08-08 16:11:36 +02:00