Commit Graph

99124 Commits

Author SHA1 Message Date
Sage Weil
35c0d75888 osd: add hdd and ssd variants for osd_recovery_max_active
Semi-arbitrarily set the SSD max to 10 (instead of 3).  This should be
tuned based on some real data.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-06-20 16:24:51 -05:00
Kefu Chai
790472df84
Merge pull request #28676 from tchaikov/wip-stop.sh-with-crimson
stop.sh: add --crimson option

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-06-21 03:55:05 +08:00
Neha Ojha
bd15824567
Merge pull request #28204 from dzafman/wip-39555
mon: Improve health status for backfill_toofull and recovery_toofull

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-06-20 11:12:10 -07:00
Patrick Donnelly
50fec41094
Merge PR #28663 into master
* refs/pull/28663/head:
	cephfs-shell: use consistent naming for paths
	cephfs-shell: remove unnecessary instances of to_bytes
	cephfs-shell: refactor list_items
	pybind/cephfs: improve user-friendliness of DirResult
	pybind/cephfs: avoid unicode check

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2019-06-20 11:09:37 -07:00
Patrick Donnelly
475f2deaf3
Merge PR #28561 into master
* refs/pull/28561/head:
	vstart_runner: upgrade the check for commands to be run as another user
	vstart_runner: split unicode arguments into lists

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 11:09:01 -07:00
Casey Bodley
51a8f37b5f
Merge pull request #28049 from theanalyst/rgw-lc-bl-fix
rgw_lc: use a new bl while encoding RGW_ATTR_LC

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-20 13:30:51 -04:00
Casey Bodley
bc065cae78
Merge pull request #28276 from liuchang0812/multi-site-request-forward
rgw: forwarded some requests to master zone

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-20 13:23:47 -04:00
Casey Bodley
01104c4d3e
Merge pull request #26167 from yaozongyou/rgw-lc-add-more-check
rgw: check lc objs not empty after fetching

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-20 13:15:57 -04:00
Casey Bodley
e796d7ecda
Merge pull request #26538 from zhangsw/feature-rgw-object-lock
rgw: add S3 object lock feature to support object worm

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-20 13:14:51 -04:00
Casey Bodley
d28197e64a
Merge pull request #26351 from linuxbox2/wip-rgw-gcretire
rgw: perfcounters: add gc retire counter

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-06-20 13:13:57 -04:00
Casey Bodley
d4e1a08a2c
Merge pull request #28527 from tianshan/fix_ret
rgw: fix miss get ret in STSService::storeARN

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
2019-06-20 13:12:33 -04:00
Casey Bodley
2f7133d0a9
Merge pull request #28327 from linuxbox2/wip-rgw-email
rgw: conditionally allow non-unique email addresses

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-20 13:12:10 -04:00
Casey Bodley
96b0777379
Merge pull request #28308 from joke-lee/usage_dump
rgw: usage dump_unsigned instead dump_int

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-20 13:11:28 -04:00
Casey Bodley
891fc11670
Merge pull request #28172 from joke-lee/rgw-sts-bad-token
rgw: fix rgw crash and set correct error code

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
2019-06-20 13:10:31 -04:00
Rishabh Dave
21a1b30a3e vstart_runner: upgrade the check for commands to be run as another user
Rectify the condition that checks if command to be issued as another
user using sudo is issued as a single argument after "-c".

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-06-20 21:35:38 +05:30
Mykola Golub
961df6356a
Merge pull request #28539 from dillaman/wip-40072
librbd: improve journal performance to match expected degradation

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-06-20 19:02:14 +03:00
Kefu Chai
c1ef3ff3de stop.sh: add --crimson option
so we can pkill crimson-osd if this option is specified

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-20 22:16:19 +08:00
Casey Bodley
e4114eb9a9
Merge pull request #28655 from dengchj/multisite_doc
doc: fixed --read-only argument value in multisite doc

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-06-20 09:36:34 -04:00
Patrick Donnelly
f26493cc25
Merge PR #28194 into master
* refs/pull/28194/head:
	test_volume_client: declare only one default for python version
	test_volume_client: don't shadow class variable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 06:27:04 -07:00
Patrick Donnelly
b2e8a02f59
cephfs-shell: use consistent naming for paths
And some small changes to make some commands take variable arguments and fix
defaults.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 06:20:57 -07:00
Patrick Donnelly
ba9f0eefa5
cephfs-shell: remove unnecessary instances of to_bytes
cephfs binding already automatically converts paths and some arguments to
bytes.

Fixes: http://tracker.ceph.com/issues/40455

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 06:20:57 -07:00
Patrick Donnelly
2a32752981
cephfs-shell: refactor list_items
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 06:20:57 -07:00
Patrick Donnelly
dff5b1a5d0
pybind/cephfs: improve user-friendliness of DirResult
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 06:20:56 -07:00
Patrick Donnelly
9bc49fefcd
pybind/cephfs: avoid unicode check
py3 does not have the unicode built-in. Instead, simply try to do the encoding
and catch failures appropriately. If it quacks like a duck...

(Note that cython apparently cheats and allows the unicode check but this is
simpler.)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 06:20:56 -07:00
Patrick Donnelly
753da4ec7b
Merge PR #28638 into master
* refs/pull/28638/head:
	qa: elide python version config
	cephfs-shell: assert python3

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2019-06-20 06:08:47 -07:00
Kefu Chai
f8102f5c43
Merge pull request #28498 from kamoltat/wip-mgr-progress-add-event-osd-marked-in
mgr/progress: Add recovery event when OSD marked in

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-20 17:30:17 +08:00
Kefu Chai
585d3fe31f
Merge pull request #28564 from ceph/store_statfs-log
log: fix store_statfs log line

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-20 17:26:30 +08:00
Kefu Chai
1f5e3e8da6
Merge pull request #28621 from Aran85/fix-ec-rf
osd: accident of rollforward may need to mark pglog dirty

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2019-06-20 17:21:38 +08:00
Kefu Chai
e10116657a
Merge pull request #28656 from dvanders/zstd-upgrade
zstd: upgrade to v1.4.0

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-20 17:19:09 +08:00
Nathan Cutler
20966a3d72 mgr/zabbix: encode string for Python 3 compatibility
Credits to Jochem Kuijpers for the fix.

Fixes: https://tracker.ceph.com/issues/36318
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-06-20 10:32:36 +02:00
Rishabh Dave
d78e035432 test_volume_client: declare only one default for python version
Fixes: http://tracker.ceph.com/issues/40460
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-06-20 12:18:49 +05:30
Rishabh Dave
da441cb381 vstart_runner: split unicode arguments into lists
Split not just string arguments but also unicode arguments into lists.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-06-20 11:04:59 +05:30
Xie Xingguo
dba845bac6
Merge pull request #28570 from xiexingguo/wip-objecter-direct-spgid
osdc/Objecter: _calc_target - inline spgid

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-20 11:15:44 +08:00
Chenjiong Deng
c65832c1ff doc: fixed --read-only argument value in multisite doc
Fixes: https://tracker.ceph.com/issues/40458

Signed-off-by: Chenjiong Deng <dengchenjiong@umcloud.com>
2019-06-20 10:59:23 +08:00
Xie Xingguo
14cee82049
Merge pull request #28568 from xiexingguo/wip-off-by-one-max-osd
mon/OSDMonitor: fix off-by-one when updating new_last_in_change

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-20 10:41:21 +08:00
Xie Xingguo
b294dc6749
Merge pull request #28629 from xiexingguo/wip-osd-df-by-pool-name
mgr, osd: 'osd df' by pool

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-20 10:40:40 +08:00
David Zafman
fa698e18e1 mon: Improve health status for backfill_toofull and recovery_toofull
Treat backfull_toofull as a warning condition because it can resolve itself.
Includes test case for PG_BACKFILL_FULL
Includes test case for recovery_toofull / PG_RECOVERY_FULL

Fixes: https://tracker.ceph.com/issues/39555

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-06-20 02:22:01 +00:00
Patrick Donnelly
ff1f04f4d5
qa: elide python version config
This test doesn't actually use the config and cephfs-shell is py3 only.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-19 18:22:51 -07:00
Patrick Donnelly
de429c48c4
cephfs-shell: assert python3
There's little reason to support this new tool with python2.

Fixes: http://tracker.ceph.com/issues/40418
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-19 18:22:50 -07:00
Jason Dillaman
9497655da5
Merge pull request #28553 from trociny/wip-40071
librbd: mirror image enable/disable should enable/disable journaling

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 20:21:37 -04:00
Jason Dillaman
57c4669ff4
Merge pull request #28627 from trociny/wip-40409
journal: properly advance read offset after skipping invalid range

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 16:58:44 -04:00
Sage Weil
43bcf94bd8 osd: share curmap in handle_osd_ping
We just took the curmap ref above; do not call get_osdmap() again.

I think it may explain a weird segv I saw here in ~shared_ptr, although
I'm not quite certain.  Regardless, this change is correct and better.

(gdb) bt
#0  raise (sig=sig@entry=11) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00005596e5a98261 in reraise_fatal (signum=11) at ./src/global/signal_handler.cc:326
#2  handle_fatal_signal(int) () at ./src/global/signal_handler.cc:326
#3  <signal handler called>
#4  0x00005596f4fe80e0 in ?? ()
#5  0x00005596e5464068 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x5596f4b7cf60) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#6  std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x5596f4b7cf60) at /usr/include/c++/9/bits/shared_ptr_base.h:148
#7  0x00005596e543377f in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=0x7f2b25044e28, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#8  std::__shared_ptr<OSDMap const, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x7f2b25044e20, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr_base.h:1169
#9  std::shared_ptr<OSDMap const>::~shared_ptr (this=0x7f2b25044e20, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/shared_ptr.h:103
#10 OSD::handle_osd_ping(MOSDPing*) () at ./src/osd/OSD.cc:4662

Signed-off-by: Sage Weil <sage@redhat.com>
2019-06-19 14:38:53 -05:00
Kefu Chai
b2f75ba2aa
Merge pull request #28460 from tchaikov/wip-cmake-detect-libnl
cmake: rewrite Findgenl to support components argument

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-20 00:36:53 +08:00
Sage Weil
afd0b508c2 qa/suites/rados/thrash: force normal pg log length with cache tiering
When we are doing cache tiering, we are more sensitive to short PG logs
because the dup op entries are not perfectly promoted from the base to
the cache.

See:
 http://tracker.ceph.com/issues/38358
 http://tracker.ceph.com/issues/24320

This works around the problem by not testing short pg logs in combination
with cache tiering.  This works because the short_pg_log.yaml fragment
sets the short log in the [global] section but the cache workloads overload
it (back to a large/default value) in the [osd] section.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-06-19 11:15:25 -05:00
Jason Dillaman
751bf69570 rbd: use the ordered throttle for the export action
This allows exports to STDOUT to use multiple concurrent operations
and also fixes a potential race condition with concurrent callbacks
and file seeking.

Fixes: http://tracker.ceph.com/issues/40435
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 11:09:54 -04:00
Dan van der Ster
fa9cd3faad zstd: compat with v1.4.0
In zstd d8e215cbee03b038fffe74aebad63b625c42f23c
ZSTD_compress_generic() is renamed to ZSTD_compressStream2().

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2019-06-19 16:57:13 +02:00
Jason Dillaman
5f3b043896 librbd: tweaks to improve throughput for journaled IO
Increase the per-object append flush bytes to 1MiB to mimic the 16MiB
in-memory writeback cache. Finally, default enable the new journal
writethrough until flush option.

Performance results comparing no journal, pre-change, and post-change:

	no journal	original	modified
4K	40310.84	7223.73		27427.27
32K	21530.37	3256.39		10284.05
256K	3159.67		564.73		1627.16
4M	223.03		48.03		91.91

Fixes: http://tracker.ceph.com/issues/40072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 10:38:53 -04:00
Jason Dillaman
172ad649fb librbd: new rbd_journal_object_writethrough_until_flush option
When set to true, the journal will not attempt to batch appends until
after it receives the the first flush request from the user.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 10:38:53 -04:00
Jason Dillaman
c0322a13c8 journal: support dynamically updating recorder flush options
Default to disabling writeback-style append flushes unless overridden
by a call to 'set_append_batch_options'.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 10:38:51 -04:00
Jason Dillaman
a4e160e410 journal: fix broken append batching implementation
The original flush_interval/bytes/age batching options no longer
actually do anything. Integrate these better with the max in-flight
AIO handling.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-06-19 10:37:22 -04:00