Commit Graph

119 Commits

Author SHA1 Message Date
Kefu Chai
f8d47223e2 qa/workunits/rados/test_envlibrados_for_rocksdb: use cmake not make
* so we just rely on a single build system instead of two of them, the
  other place we use cmake is cmake/modules/BuildRockDB.cmake.
* disable gflags when building rocksdb, it's optional and does not help
  in the sense of testing librados support.
* disable prompts when installing on debian, to silence warnings like:
  debconf: unable to initialize frontend: Dialog
* drop --force-yes option, as it is deprecated, and is replaced with
  --allow-downgrades, --allow-remove-essential,
  --allow-change-held-packages,  but none of them apply in our case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-21 13:28:23 +08:00
Kefu Chai
63a35625a3 qa/workunits/rados/test_envlibrados_for_rocksdb: install g++ not g++-4.7
since fog now deploys ubuntu 18.04 where g++-4.7 is not available
anymore, so g++ is good enough.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-20 16:35:27 +08:00
Sage Weil
c894d528c2 qa/workunits/rados/test.sh: ensure tee output is valid filename
In the future we'll add --gtest_filter bits to the items, which confuse
tee.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-18 14:38:26 -05:00
Kefu Chai
1a95710441 qa: disable -Werror when compiling env_librados_test
to silence warnings like

utilities/env_librados.cc:175:33: warning: unused parameter ‘offset’ [-Wunused-parameter]
   Status InvalidateCache(size_t offset, size_t length) {
                                 ^~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-15 22:38:09 +08:00
Brad Hubbard
aba1e74e03 qa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-04-13 18:41:40 +10:00
Nathan Cutler
3a058a2e42 tests: rados tool: test -f option
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-28 12:52:34 +01:00
Sage Weil
ff9fa835da qa/workunits/rados/test_rados_tool: remove stray |, fix race
Fixes: http://tracker.ceph.com/issues/22676
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:35:14 -06:00
Sage Weil
ea2c99ec87 qa/workunits/rados/test_rados_tool.sh: -x
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-13 10:26:24 -06:00
Sage Weil
c1ec2752ee
Merge pull request #19180 from theanalyst/tools/rados-omap-clear
tools: rados add a cli option to clear omap keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-11 15:50:12 -06:00
Casey Bodley
f473d57013 librados: add async interfaces for use with Networking TS
Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-09 09:58:18 -05:00
Abhishek Lekshmanan
83631d9bd8 qa: add test for clearomap cli option
Fixes:http://tracker.ceph.com/issues/22255
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-12-05 11:06:22 +01:00
Brad Hubbard
71bf04775b osd: Warn about objects with too many omap entries
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-24 17:27:57 +10:00
xie xingguo
b60d3498e2 osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC
We now set full flag if a pool is currently running out of space and
set both full and full_no_quota flags if it is running out of quota.
Therefore the full_no_quota flag should be instead used to uniquely
identify whether we are running out of quota or not.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 08:49:42 +08:00
xie xingguo
5dd3458eec qa: fix pool-quota related tests
https://github.com/ceph/ceph/pull/17371 introduces support of
per-pool space-full flag, which turns out to set both
full and full_no_quota flags now if a pool is currently running out
of quota.

Actually this test is fragile as long as we keep appending new flags
at pool granularity, but let's not bother with that complexity now.

Fixes: http://tracker.ceph.com/issues/21409
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-16 13:50:18 +08:00
Kefu Chai
30b5b4627c Merge pull request #16494 from asomers/bin_bash
misc: Fix bash path in shebangs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-27 10:14:14 +08:00
Alan Somers
3aae5ca6fd scripts: fix bash path in shebangs
/bin/bash is a Linuxism.  Other operating systems install bash to
different paths.  Use /usr/bin/env in shebangs to find bash.

Signed-off-by: Alan Somers <asomers@gmail.com>
2017-07-27 13:24:26 -06:00
xie xingguo
a27fd9d25c crush: kill "class create" command
The device class is now self and automatically managed.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-07-26 22:40:17 +08:00
Jason Dillaman
fa90be842e test: enable pool applications for new pools
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-07-19 13:13:01 -04:00
Sage Weil
12c6cfa5f1 qa/workunits/rados/test_health_warnings: test down msgs vs crush classes
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-13 12:15:26 -04:00
John Spray
44bce9e02c Merge pull request #15858 from liewegas/wip-mgr-servicemap
mgr,librados: service map

Reviewed-by: John Spray <john.spray@redhat.com>
2017-07-10 15:03:07 +01:00
Sage Weil
aabe0e35a0 librados: allow service registrations
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-09 14:09:04 -04:00
Sage Weil
dc7a2aaf7a erasure-code: ruleset-* -> crush-*
1) ruleset is an obsolete term, and
2) crush-{rule,failure-domain,...} is more descriptive.

Note that we are changing the names of the erasure code profile keys
from ruleset-* to crush-*.  We will update this on upgrade when the
luminous flag is set, but that means that during mon upgrade you cannot
create EC pools that use these fields.

When the upgrade completes (users sets require_osd_release = luminous)
existing ec profiles are updated automatically.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-06 15:01:03 -04:00
David Zafman
6e5f3cdc69 test: Use correct argument --read-perecent
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-06-23 08:02:50 -07:00
Kefu Chai
b86cda7040 Merge pull request #15764 from tchaikov/wip-20342
qa/suites/upgrade/hammer-jewel-x: add luminous.yaml

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-06-23 22:26:03 +08:00
Kefu Chai
5d252a27a1 a/workunits/rados/test_envlibrados_for_rocksdb.sh: git clone --depth 1
no need to clone the whole history of rocksdb, we just need the HEAD of
master. so "--depth 1" is better and faster in this case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-22 12:19:01 +08:00
Brad Hubbard
a921882e7c osd: Reverse order of op_has_sufficient_caps and do_pg_op
Fixes: http://tracker.ceph.com/issues/19790

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-06-19 15:23:17 +10:00
Kefu Chai
69bb53b06f qa/workunits/rados/test_health_warning: misc fixes
* do not let osd shutdown itself by enlarge osd_max_markdown_count and
  shorten osd_max_markdown_period
* do not shutdown all osds in the last test. if all osds are shutdown at
  the same time. none of them will get updated osdmap after noup is
  unset. we should leave at least one of them, so the gossip protocol
  can kick in, and populate the news to all osds.

Fixes: http://tracker.ceph.com/issues/20174
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-06-04 14:35:15 -04:00
Sage Weil
aecaebe087 qa/workunits/rados/test_health_warning: misc fixes
- use 'noup' flag
- wait for healthy between each test
- check counts for each type

Fixes: http://tracker.ceph.com/issues/19990
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-22 10:09:19 -04:00
Neha Ojha
6ecb03299a mon: add test for crush type down health warnings
Signed-off-by: Neha Ojha <nojha@redhat.com>
2017-05-16 14:16:19 -07:00
Josh Durgin
4a7c058183 qa: use 4k stripe_width again for test_rados_tool.sh
The stripe_unit change altered the default stripe_width to depend on k.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-04-19 17:45:44 -07:00
Kefu Chai
787f6ac699 qa/workunits/rados/test.sh: print test name when it fails
we have

2017-02-04T16:15:46.090 INFO:tasks.workunit.client.0.mira032.stdout:error in 22088
2017-02-04T16:15:46.092 INFO:tasks.workunit.client.0.mira032.stderr:bash: line 1: 22092 Alarm clock             ceph_test_rados_api_aio 2>&1
2017-02-04T16:15:46.096 INFO:tasks.workunit.client.0.mira032.stderr:     22093 Done                    | tee ceph_test_rados_api_aio.log
2017-02-04T16:15:46.099 INFO:tasks.workunit.client.0.mira032.stderr:     22094 Done                    | sed "s/^/                  api_aio: /"
2017-02-04T16:15:46.102 INFO:tasks.workunit.client.0.mira032.stderr:+

if a unittest in rados/test.sh fails in teuthology.log, but it would
be desirable to have the failed test name in the line of "error in
22088".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-02-10 23:54:29 +08:00
Kefu Chai
8b07aad0f0 Merge pull request #12706 from Liuchang0812/wip-qa-rados-1228
test: update test_rados_tool.sh, use POOL and OBJ var

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-01-02 14:21:03 +08:00
liuchang0812
96f5c92dbf test: update test_rados_tool.sh, use POOL and OBJ var
* replace hard-code pool name with $POOL
* replace hard-code object name with $OBJ
* introduce a new variable called $POOL_EC
* clean up pool
* simplify test case

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-12-29 16:05:37 +00:00
Willem Jan Withagen
4b9b562465 qa/test_rados_tool.sh: POSIX dd only accepts 'k' as multiplier
FreeBSD does not want KB as 1k multiplier.
see also: http://www.unix.com/man-page/posix/1posix/dd/

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-12-28 12:13:29 +01:00
Sage Weil
2e1a096a82 Merge pull request #12674 from Liuchang0812/tools-rados-offset-support
tools/rados: add a parameter "--offset" to rados put command

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-12-27 16:35:50 -06:00
Sage Weil
c467d7fb23 Merge pull request #12675 from Liuchang0812/fix-test-case-in-rados
test: replace hard-code binary names with varibles

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-12-27 16:35:29 -06:00
liuchang0812
fe3d3821cb tools/rados: add parameter offset for put cmd and test case for it
add a parameter "--offset" for put command
add a test case for rados put command

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-12-27 16:06:41 +08:00
liuchang0812
14de3a1ee1 test: replace hard-code binary names with varibles
This test case couldn't run under customize env CEPH_BIN.

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
2016-12-27 14:18:29 +08:00
Sage Weil
5666fd61d6 qa/workunits: include extension for nose tests
When you have a relative path you have to include the extension.
Weird.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-19 14:08:11 -05:00
Sage Weil
cf294777ea qa/workunits: use relative path instead of wget from git
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-15 15:10:28 -05:00
Jason Dillaman
286ceb1e03 rados: optionally support reading omap key from file
Fixes: http://tracker.ceph.com/issues/18123
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-07 09:57:16 -05:00
Sage Weil
f9882f4abe qa/workunits/rados/test_envlibrados_for_rocksdb: force librados-dev install
On trusty we see

 WARNING: The following packages cannot be authenticated!
   librados-dev
 E: There are problems and -y was used without --force-yes

Signed-off-by: Sage Weil <sage@redhat.com>
2016-11-12 08:30:59 -05:00
David Disseldorp
87cd4c7350 qa/workunits/rados/test_rados_tool.sh: test for segv without --pool
Ensure that the rados client binary doesn't segfault when specifying a
number of parameters without a corresponding --pool parameter.

Signed-off-by: David Disseldorp <ddiss@suse.de>
2016-10-25 13:19:58 +02:00
Yuri Weinstein
fc70cd9ac3 Merge pull request #11036 from tomy-ac/wip-tc-rados_append
Added append functionality to rados tool.

Reviewed-by: Samuel Just <sjust@redhat.com>
2016-10-24 14:45:42 -07:00
Tomy Cheru
cb919cae34 tools/rados: Added append functionality to rados tool.
Signed-off-by: Tomy Cheru <tomy.cheru@sandisk.com>
2016-09-28 09:23:37 +05:30
Ryne Li
958fa7dcf2 add qa test of envlibrados for rocksdb
Signed-off-by: Ryne Li <lizhenqiangsnake@gmail.com>
2016-09-20 20:12:47 -07:00
Oleh Prypin
20c5bcf592 qa/workunits/rados/test_python.sh: Allow specifying Python executable
(through an environment variable)

Signed-off-by: Oleh Prypin <oleh@pryp.in>
2016-08-18 23:36:32 +03:00
Kefu Chai
56df5dcda4 Merge pull request #10166 from wjwithagen/wip-wjw-cmake-test_rados_tool.sh
test_rados_tool.sh: Make script work under ctest

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-15 17:03:59 +08:00
Kefu Chai
53af28db6e test: remove ceph_test_rados_api_tmap_migrate
see also: http://tracker.ceph.com/issues/16144

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-12 16:08:09 +08:00
Willem Jan Withagen
a939e99341 rados/test_rados_tool.sh: Improve on testing conditions
- have test aka [] use numeric comparisons
 - suppress the filename on 'wc -l' by pipping the file

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-07 14:57:16 +02:00