Commit Graph

4237 Commits

Author SHA1 Message Date
David Zafman
02964703de
Merge pull request #24749 from dzafman/wip-36474
Add support for osd_delete_sleep configuration value

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-11-07 14:05:55 -08:00
Sage Weil
5b9be42bf5 Merge PR #15047 into master
* refs/pull/15047/head:
	tool/ceph_objectstore_tool: add new op that reset last_complete to last_update

Reviewed-by: Sage Weil <sage@redhat.com>
2018-11-06 10:47:18 -06:00
Jason Dillaman
cc592823fa
Merge pull request #24717 from trociny/wip-rbd-mirror-status-instance
rbd: show info about mirror daemon instance in image mirror status output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-11-06 11:11:49 -05:00
Mykola Golub
7d2ffd981b rbd: show info about mirror instance in image mirror status output
It is particularly useful when running multiple rbd-mirror instances
in Active-Passive or Active-Active mode.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-11-05 22:34:39 +02:00
Mykola Golub
472abc2ed9 test/librbd: migration supporting namespace tests
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-11-03 19:36:44 +02:00
Casey Bodley
cf9a0a28ea
Merge pull request #24895 from votdev/feature_36681
rgw: Return tenant field in bucket_stats function

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-11-02 16:28:26 -04:00
Yuri Weinstein
a24793ed6c
Merge pull request #22923 from ceph/wip-fix-latest-distro
qa/tests: update links for centos latest to point to 7.5

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2018-11-02 12:30:41 -07:00
Volker Theile
9440776f06 rgw: Return tenant field in bucket_stats function
Fixes: https://tracker.ceph.com/issues/36681

Signed-off-by: Volker Theile <vtheile@suse.com>
2018-11-02 13:26:10 +01:00
Vasu Kulkarni
67bfc94f17 qa/tests: update links for centos latest to point to 7.5
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-11-01 11:09:12 -07:00
Kefu Chai
b4a5f76243 qa: add librados3 to exclude list of pre-nautilus install tasks
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
083b846198 qa: s/librados2/librados3/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
0475dfdea3 qa: update test_envlibrados_for_rocksdb.sh for libradospp split
include a patch so rocksdb can use libradospp instead of librados. will
upstream the patch and make it work for both pre-nautilus librados and
nautilus libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
3730d10623 librados: move C++ APIs into libradospp
the goal is to decouple C++ API from C API, and to version them
differently, as they are targeting different consumers.

this allows us to change the C++ API and bumping up its soversion
without requiring consumer to recompile the librados client for
using the new librados. in this way, C++ API can move faster than
C API. for example, if bufferlist interface is changed for better
performance, and this breaks existing API/ABI, we can bump up
the C++ library's soversion, and and the C library's version unchanged
but ship the new librados's C binding. so the librados client linked
against librados's C library will be able to take advantage of
the improvement in C++ library. while the librados client
linked against C++ library won't break at runtime due to unresolved
symbol or changed structure layout.

this is massive change, the genereal idea is to

* split librados.cc into two source files: librados_c.cc and
  librados_cxx.cc, the former for implementing C APIs, the later
  for C++ APIs.
* extract the C++ API in librados into librados-cxx, the library
  name will be libradospp. but we can change it before nautilus
  is released.
* link these librados libraries with static libraries which it
  depends on, so "-Wl,--exclude-libs,ALL" link flags can help
  hide the non-public symbols.
* extract the tests exercising librados' C++ API into a different
  source file named *_cxx.cc. for instance, to move the C++ tests
  in aio.cc into aio_cxx.cc
* extract the shared helper functions which do not use any librados
  or librados-cxx APIs into test_shared{.cc,h}. the "shared" here
  means, *shared* by C++ and C tests.
* extract the test fixtures, i.e., the subclasses of testing::Test,
  for testing C++ APIs into testcase_cxx.cc.
* update qa/workunits/rados/test.sh accordingly to add the splitted
  tests
* update the consumers of librados to link against librados-cxx
  instead, if they are using the C++ API.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:30 +08:00
Sage Weil
9ab9dcfc0d Merge PR #24809 into master
* refs/pull/24809/head:
	os/bluestore: omit redundant '/' in OSD path for ceph-bluestore-tool if
	os/bluestore: improve error handling for migrate ops in
	qa/standtalone/osd-bluefs-volume-ops: remove redundant code.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-30 15:09:45 -05:00
David Zafman
3f621a1190 test: Set any value for osd_delete_sleep to guarantee we are testing even SSD
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-10-30 11:42:05 -07:00
Lenz Grimmer
88719afc73
Merge pull request #22833 from rjfd/wip-dashboard-jwt
mgr/dashboard: JWT authentication

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-10-30 10:50:20 +01:00
Ricardo Dias
2f5e7c3392
mgr/dashboard: backend: JWT based authentication
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-10-29 15:47:14 +00:00
Sage Weil
c40685ebdd Merge PR #24787 into master
* refs/pull/24787/head:
	Merge PR #24796 into nautilus
	osd: fix heartbeat_reset unlock
	Merge PR #24780 into nautilus
	Merge PR #24761 into nautilus
	Merge PR #24651 into nautilus
	osd: fix race between op_wq and context_queue
	test: Make sure kill_daemons failure will be easy to find
	test: Add flush_pg_stats to make test more deterministic
2018-10-29 08:36:34 -05:00
Igor Fedotov
5d38f8b49b qa/standtalone/osd-bluefs-volume-ops: remove redundant code.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-10-29 16:30:36 +03:00
Sage Weil
143f601155 Merge PR #24651 into nautilus
* refs/pull/24651/head:
	test: Make sure kill_daemons failure will be easy to find
	test: Add flush_pg_stats to make test more deterministic

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-10-26 21:07:09 -05:00
Xie Xingguo
e6f9241aeb
Merge pull request #24657 from xiexingguo/wip-rm-device-class-fix
mon/OSDMonitor: two "ceph osd crush class rm" fixes

Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-27 09:49:57 +08:00
Sage Weil
80bb0664d6 Merge PR #24713 into master
* refs/pull/24713/head:
	mon: drop repeated 'goodchars' and add osd crush ls testcase

Reviewed-by: João Eduardo Luís <joao@suse.de>
2018-10-26 09:18:13 -05:00
John Spray
c5fd31dfcc
Merge pull request #24767 from votdev/issue_36581
mgr/dashboard/qa: Fix various vstart_runner.py issues

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-10-26 15:09:59 +01:00
Lenz Grimmer
0c84be2306
Merge pull request #24727 from zmc/wip-dashboard-gzip
mgr/dashboard: Enable gzip compression

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-10-26 11:05:27 +02:00
Volker Theile
00e3022710 mgr/dashboard/qa: CephfsTest - admin_socket() got an unexpected keyword argument 'timeout'
Adapt method arguments of LocalRemote::run() according to teuthology.orchestra.run.run() (see https://github.com/ceph/teuthology/blob/master/teuthology/orchestra/run.py#L364) to be able to run QA tests locally in a vstart cluster.

Fixes: http://tracker.ceph.com/issues/36581
Signed-off-by: Volker Theile <vtheile@suse.com>
2018-10-26 09:59:11 +02:00
Zack Cerza
03de8f9557 mgr/dashboard: Enable gzip compression
This is related to http://tracker.ceph.com/issues/36453. It is far from
a complete solution, but seems like a positive move.

I tested this change by first disabling my browser cache, and then used
the /docs endpoint to query /api/dashboard/health. Before compression:
  Content-Length: 60748
  Time: 615ms
After:
  Content-Length: 7505
  Time: 92ms

Then, I logged into the dashboard as normal and reloaded the page once I
was in. Some values for the reload operation before compression:
  Total page load time: 58.48s
  vendor.js Content-Length: 6486025
  vendor.js time: 48.09s
After:
  Total page load time: 14.55s
  vendor.js Content-Length: 1143178
  vendor.js time: 4.50s

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-10-24 16:04:37 -06:00
Zack Cerza
bd09bc0462 DashboardTestCase: add assertHeaders()
Signed-off-by: Zack Cerza <zack@redhat.com>
2018-10-24 16:04:37 -06:00
Zack Cerza
b134972035 LocalCephManager.admin_socket: add timeout kwarg
This fixes "TypeError: admin_socket() got an unexpected keyword argument
'timeout'". The value is never used.

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-10-24 16:04:37 -06:00
Jason Dillaman
484dc12089 qa/tasks/qemu: use unique clone directory to avoid race with workunit
If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 5d56014c61)
2018-10-24 10:30:43 -04:00
Patrick Donnelly
60c63f71f6
Merge PR #24533 into master
* refs/pull/24533/head:
	qa: add timeouts for remote ops for client mounts

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-10-23 14:46:34 -07:00
xie xingguo
5bcac35213 mon/OSDMonitor: do not remove device class still referenced by ec-profiles
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 21:17:56 +08:00
xie xingguo
4bc54587a1 mon/OSDMonitor: make "ceph osd crush class rm" idempotent
Removing a non-existent device class should be generally okay.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-23 21:17:56 +08:00
Yan Jun
1e98c72dfc mon: drop repeated 'goodchars' and add osd crush ls testcase
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2018-10-23 16:32:45 +08:00
Mykola Golub
5dd0599bdf
Merge pull request #24696 from dillaman/wip-36542
qa/tasks/qemu: use unique clone directory to avoid race with workunit

Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-10-23 09:41:43 +03:00
Sage Weil
a350131d16 Merge PR #24698 into master
* refs/pull/24698/head:
	Merge PR #24697 into nautilus
	ceph_test_msgr: fix authorizer behavior
	Merge pull request #24667 from liewegas/wip-ec-thrash-full
	Merge PR #24689 into nautilus
	qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 22:46:23 -05:00
Jason Dillaman
5d56014c61 qa/tasks/qemu: use unique clone directory to avoid race with workunit
If there is a workunit task associated with the same client, the two
tasks will attempt to clone the suite repo to the same directory.
Worse, if it's parallel tasks, the two clones will clobber each
other.

Fixes: http://tracker.ceph.com/issues/36542
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-10-22 10:44:40 -04:00
Josh Durgin
36ca230776
Merge pull request #24667 from liewegas/wip-ec-thrash-full
qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-10-22 07:39:26 -07:00
Kefu Chai
4af71e7c00
Merge pull request #23103 from ifed01/wip-ifed-bluefs-migrate
os/bluestore: allow ceph-bluestore-tool to coalesce, add and migrate BlueFS backing volumes

Reviewed-by:  Sage Weil <sage@redhat.com>
2018-10-22 22:33:08 +08:00
Sage Weil
ae583f5dde Merge PR #24689 into master
* refs/pull/24689/head:
	qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
	Merge PR #24625 into nautilus
	qa/suites/rados/mgr/tasks/module_selftest: whitelist 'foo bar security'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-22 09:19:46 -05:00
liuchang0812
7c008d279e tool/ceph_objectstore_tool: add new op that reset last_complete to last_update
Fixes: http://tracker.ceph.com/issues/19382

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2018-10-22 11:03:06 +08:00
Sage Weil
b678356594 qa/tasks/ceph_manager: fix get_stuck_pgs from pg dump change
Fixes 95b7d2340c

Fixes: http://tracker.ceph.com/issues/36485
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-21 10:52:38 -05:00
Sage Weil
98fc7ebc99 Merge PR #24184 into master
* refs/pull/24184/head:
	mgr/DaemonServer: remove any upmaps on merging PGs
	mgr/DaemonServer: prevent merge if either pg is remapped|upmap
	mgr/DaemonServer: move pending merge check for more consistent code
	qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
	mgr/DaemonServer: add option to bypass careful throttling for thrasher
	PendingReleaseNotes: note about mgr/balancer/max_misplaced change
	mgr/DaemonServer: remove stale/misleading check
	mgr/DaemonServer: throttle pgp_num changes based on misplaced %
	mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
	mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
	mon/PGMap: add get_recovery_stats()
	mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
	pybind/mgr/mgr_module: add get_option()
	mgr/DaemonServer: allow pg_num increases that abort pending merges
	mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
	mon/OSDMonitor: make pgp_num track pg_num more consistently

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-10-20 16:40:22 -05:00
Sage Weil
86ae8fb6b8 qa/suites/rados/thrash*/thrashers/careful.yaml: thrash with mgr controller
Thrash such that we still exercise the careful throttling in the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
70ec5bda23 mgr/DaemonServer: add option to bypass careful throttling for thrasher
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-20 15:21:58 -05:00
Sage Weil
c22d5a0fce qa/suites/rados/thrash-erasure-code*/thrashers/*: less likely resv rejection injection
For EC pools we have a lot of shards, and 30% probability on each one
means we are very like to repeatedly fail backfill reservations.. long
enough that teuthology gives up waiting.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-18 17:17:41 -05:00
David Zafman
da3c556aa2 test: Make sure kill_daemons failure will be easy to find
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-10-17 16:54:45 -07:00
David Zafman
b33edbc4f6 test: Add flush_pg_stats to make test more deterministic
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-10-17 16:54:45 -07:00
Igor Fedotov
02b5768a4f tests: add qa test case for bluefs volume coalescence
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-10-17 22:39:27 +03:00
Patrick Donnelly
092801ae34
qa: add timeouts for remote ops for client mounts
Fixes: https://tracker.ceph.com/issues/36390

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-17 10:39:13 -07:00
Patrick Donnelly
b1dca00f90
Merge PR #24503 into master
* refs/pull/24503/head:
	qa: increase timeout for cleanup

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2018-10-17 10:18:50 -07:00