Commit Graph

145 Commits

Author SHA1 Message Date
Kefu Chai
b76a6c729f qa/workunits/rados/test_envlibrados_for_rocksdb: support SUSE distros
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 17:46:46 +08:00
Kefu Chai
be89e66f74 qa/workunits/rados/test_librados_build.sh: install build deps
to build helloworld of librados, we need to install the build
dependencies.

Fixes: https://tracker.ceph.com/issues/40155
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 17:46:46 +08:00
Kefu Chai
0072d487aa qa/workunits/rados/test_envlibrados_for_rocksdb: use helper script
instead of having its own copy of `install()`, try to leverage the
helper in ceph-helpers-root.sh.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-12 17:46:46 +08:00
Nathan Cutler
f9f824448a qa: add version number sanity singleton to rados suite
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-05-08 11:31:34 +02:00
myoungwon oh
8a8d5e89bb src/test: remove unused objects to calculate accurate dedup ratio.
Signed-off-by: Myoungwon Oh <ommw@sk.com>
2019-04-24 18:22:37 +09:00
myoungwon oh
d72cc5a0ee src/tools: fix dedup tool's name
Signed-off-by: Myoungwon Oh <ommw@sk.com>
2019-04-08 19:19:09 +09:00
myoungwon oh
a095765f2f qa/workunit: add rabin chunk test
Signed-off-by: Myoungwon Oh <ommw@sk.com>
2019-04-04 23:58:57 +09:00
Jason Dillaman
9683869442 Revert "qa: update test_envlibrados_for_rocksdb.sh for libradospp split"
This reverts commit 0475dfdea3.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-21 08:28:43 -05:00
myoungwon oh
e11fb7ce4d src/tools: add stats (fixed objects,total objects)
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2019-01-26 12:11:51 +09:00
Vikhyat Umrao
8a694fc2f9 qa: specify filestore for misc tests
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 13:09:19 -06:00
Sage Weil
d8d16ba6aa qa/workunits/rados/test_health_warnings: prevent out osds
Fixes: http://tracker.ceph.com/issues/37776
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-31 11:18:05 -06:00
Igor Fedotov
25664c4ccf test: add test coverage for rados df reports
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-12-06 18:54:21 +03:00
Nathan Cutler
6c6ae9e30d qa/workunits/rados/test_librados_build.sh: grab files from explicit git branch
Fixes: https://tracker.ceph.com/issues/37392
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-11-27 00:34:17 +01:00
Kefu Chai
60dc1c2a32 qa: patch rocksdb the right way
quote from patch(1)

>       but usually just
>
>       patch -pnum <patchfile

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 13:03:00 +08:00
Kefu Chai
2dc0f86227 qa: use FOUND_VAR to be backward compatible with cmake 2.8.12
before this change, we assume that the variable set if rados::radospp is
found will be radospp_FOUND, but this is a feature cmake 3, see
https://cmake.org/cmake/help/v3.3/module/FindPackageHandleStandardArgs.html

while the cmake shipped by centos is cmake 2.8.12, where the variable
name will be <UPPERCASED_NAME>_FOUND, see
https://cmake.org/cmake/help/v2.8.12/cmake.html#module:FindPackageHandleStandardArgs

in the test of test_envlibrados_for_rocksdb.sh, we are using cmake not
the cmake3 offered by EPEL7, so RADOSPP_FOUND will be set instead. that's why
executable env_librados_test will fail to link against rados::radospp.
as rados::radospp won't be defined if radospp_FOUND is not defined/set.

after this change, the 2nd mode of FIND_PACKAGE_HANDLE_STANDARD_ARGS()
is used instead to ensure that radospp_FOUND is defined even if cmake
2.8.12 is used.

also, the message() commands for debugging purpose are removed.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-15 11:57:52 +08:00
Kefu Chai
0e5bdd9d56 librados: remove rados_tmap_*
we have switched from tmap to omap long ago.

but keep the server side implementation around, in case ancient
client is still using these tmap APIs.

also, tmap_update() is kept, because librbd is using it for v1 image
backward compatibility.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-07 14:05:25 -08:00
Kefu Chai
0475dfdea3 qa: update test_envlibrados_for_rocksdb.sh for libradospp split
include a patch so rocksdb can use libradospp instead of librados. will
upstream the patch and make it work for both pre-nautilus librados and
nautilus libradospp

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:31 +08:00
Kefu Chai
3730d10623 librados: move C++ APIs into libradospp
the goal is to decouple C++ API from C API, and to version them
differently, as they are targeting different consumers.

this allows us to change the C++ API and bumping up its soversion
without requiring consumer to recompile the librados client for
using the new librados. in this way, C++ API can move faster than
C API. for example, if bufferlist interface is changed for better
performance, and this breaks existing API/ABI, we can bump up
the C++ library's soversion, and and the C library's version unchanged
but ship the new librados's C binding. so the librados client linked
against librados's C library will be able to take advantage of
the improvement in C++ library. while the librados client
linked against C++ library won't break at runtime due to unresolved
symbol or changed structure layout.

this is massive change, the genereal idea is to

* split librados.cc into two source files: librados_c.cc and
  librados_cxx.cc, the former for implementing C APIs, the later
  for C++ APIs.
* extract the C++ API in librados into librados-cxx, the library
  name will be libradospp. but we can change it before nautilus
  is released.
* link these librados libraries with static libraries which it
  depends on, so "-Wl,--exclude-libs,ALL" link flags can help
  hide the non-public symbols.
* extract the tests exercising librados' C++ API into a different
  source file named *_cxx.cc. for instance, to move the C++ tests
  in aio.cc into aio_cxx.cc
* extract the shared helper functions which do not use any librados
  or librados-cxx APIs into test_shared{.cc,h}. the "shared" here
  means, *shared* by C++ and C tests.
* extract the test fixtures, i.e., the subclasses of testing::Test,
  for testing C++ APIs into testcase_cxx.cc.
* update qa/workunits/rados/test.sh accordingly to add the splitted
  tests
* update the consumers of librados to link against librados-cxx
  instead, if they are using the C++ API.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-11-02 00:15:30 +08:00
John Spray
67d147c00d
Merge pull request #23622 from renhwztetecs/renhw-wip-25103
mgr: fixup pgs show in unknown state

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-10-10 13:28:33 +01:00
huanwen ren
e5096b41ca common: adapt to the new pg dump format.
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2018-09-27 23:22:50 +08:00
Nathan Cutler
c46c890d02 qa: add test that builds example librados programs
Fixes: http://tracker.ceph.com/issues/15100
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-09-25 13:18:04 +02:00
Kefu Chai
4b0e2c8ed4 qa: fix typos
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-21 12:41:42 +08:00
Sage Weil
2c26fb0fe1 rados: drop mkpool, rmpool commands
- mkpool and rmpool users should use the normal cli/mon commands

Signed-off-by: Sage Weil <sage@redhat.com>
2018-08-31 09:27:36 -05:00
Sage Weil
d213b2531f rados: remove 'chown' command
Signed-off-by: Sage Weil <sage@redhat.com>
2018-08-31 09:27:36 -05:00
Dan Mick
298a1d92d2 qa/suites/rados, qa/workunits/rados: Add suite/workunit for ceph-crash
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2018-08-13 13:53:26 -07:00
Jason Dillaman
97e3f0aa01 qa/workunits/rados: test pool op permissions
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-06-26 11:41:19 -04:00
Kefu Chai
f8d47223e2 qa/workunits/rados/test_envlibrados_for_rocksdb: use cmake not make
* so we just rely on a single build system instead of two of them, the
  other place we use cmake is cmake/modules/BuildRockDB.cmake.
* disable gflags when building rocksdb, it's optional and does not help
  in the sense of testing librados support.
* disable prompts when installing on debian, to silence warnings like:
  debconf: unable to initialize frontend: Dialog
* drop --force-yes option, as it is deprecated, and is replaced with
  --allow-downgrades, --allow-remove-essential,
  --allow-change-held-packages,  but none of them apply in our case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-21 13:28:23 +08:00
Kefu Chai
63a35625a3 qa/workunits/rados/test_envlibrados_for_rocksdb: install g++ not g++-4.7
since fog now deploys ubuntu 18.04 where g++-4.7 is not available
anymore, so g++ is good enough.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-05-20 16:35:27 +08:00
Sage Weil
c894d528c2 qa/workunits/rados/test.sh: ensure tee output is valid filename
In the future we'll add --gtest_filter bits to the items, which confuse
tee.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-18 14:38:26 -05:00
Kefu Chai
1a95710441 qa: disable -Werror when compiling env_librados_test
to silence warnings like

utilities/env_librados.cc:175:33: warning: unused parameter ‘offset’ [-Wunused-parameter]
   Status InvalidateCache(size_t offset, size_t length) {
                                 ^~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-15 22:38:09 +08:00
Brad Hubbard
aba1e74e03 qa/workunits/rados/test_large_omap_detection: Scrub pgs instead of OSDs
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-04-13 18:41:40 +10:00
Nathan Cutler
3a058a2e42 tests: rados tool: test -f option
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-01-28 12:52:34 +01:00
Sage Weil
ff9fa835da qa/workunits/rados/test_rados_tool: remove stray |, fix race
Fixes: http://tracker.ceph.com/issues/22676
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-14 10:35:14 -06:00
Sage Weil
ea2c99ec87 qa/workunits/rados/test_rados_tool.sh: -x
Signed-off-by: Sage Weil <sage@redhat.com>
2018-01-13 10:26:24 -06:00
Sage Weil
c1ec2752ee
Merge pull request #19180 from theanalyst/tools/rados-omap-clear
tools: rados add a cli option to clear omap keys

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-01-11 15:50:12 -06:00
Casey Bodley
f473d57013 librados: add async interfaces for use with Networking TS
Defines asynchronous librados operations that satisfy all of the
"Requirements on asynchronous operations" imposed by the C++ Networking
TS [1] in section 13.2.7. These operations are implemented in terms of
boost::asio, but the interfaces themselves are free of boost types -
this makes the transition to std::net trivial when it's available.

These interfaces conform to the Extensible Asynchronous Model [2] that
originated in boost::asio. This model allows the last 'handler' argument
to either be a callback that gets the result, a coroutine yield_context
that will suspend until completion, or a 'use_future' tag to request the
result in a std::future (see the unit tests for examples of each). The
'Extensible' part also enables further integration with new frameworks.

For now, only async_read(), async_write(), and the read/write variants
of async_operate() are provided.

[1] Working Draft, C++ Extensions for Networking
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4711.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-01-09 09:58:18 -05:00
Abhishek Lekshmanan
83631d9bd8 qa: add test for clearomap cli option
Fixes:http://tracker.ceph.com/issues/22255
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2017-12-05 11:06:22 +01:00
Brad Hubbard
71bf04775b osd: Warn about objects with too many omap entries
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-10-24 17:27:57 +10:00
xie xingguo
b60d3498e2 osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC
We now set full flag if a pool is currently running out of space and
set both full and full_no_quota flags if it is running out of quota.
Therefore the full_no_quota flag should be instead used to uniquely
identify whether we are running out of quota or not.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-20 08:49:42 +08:00
xie xingguo
5dd3458eec qa: fix pool-quota related tests
https://github.com/ceph/ceph/pull/17371 introduces support of
per-pool space-full flag, which turns out to set both
full and full_no_quota flags now if a pool is currently running out
of quota.

Actually this test is fragile as long as we keep appending new flags
at pool granularity, but let's not bother with that complexity now.

Fixes: http://tracker.ceph.com/issues/21409
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-16 13:50:18 +08:00
Kefu Chai
30b5b4627c Merge pull request #16494 from asomers/bin_bash
misc: Fix bash path in shebangs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-08-27 10:14:14 +08:00
Alan Somers
3aae5ca6fd scripts: fix bash path in shebangs
/bin/bash is a Linuxism.  Other operating systems install bash to
different paths.  Use /usr/bin/env in shebangs to find bash.

Signed-off-by: Alan Somers <asomers@gmail.com>
2017-07-27 13:24:26 -06:00
xie xingguo
a27fd9d25c crush: kill "class create" command
The device class is now self and automatically managed.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-07-26 22:40:17 +08:00
Jason Dillaman
fa90be842e test: enable pool applications for new pools
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-07-19 13:13:01 -04:00
Sage Weil
12c6cfa5f1 qa/workunits/rados/test_health_warnings: test down msgs vs crush classes
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-13 12:15:26 -04:00
John Spray
44bce9e02c Merge pull request #15858 from liewegas/wip-mgr-servicemap
mgr,librados: service map

Reviewed-by: John Spray <john.spray@redhat.com>
2017-07-10 15:03:07 +01:00
Sage Weil
aabe0e35a0 librados: allow service registrations
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-09 14:09:04 -04:00
Sage Weil
dc7a2aaf7a erasure-code: ruleset-* -> crush-*
1) ruleset is an obsolete term, and
2) crush-{rule,failure-domain,...} is more descriptive.

Note that we are changing the names of the erasure code profile keys
from ruleset-* to crush-*.  We will update this on upgrade when the
luminous flag is set, but that means that during mon upgrade you cannot
create EC pools that use these fields.

When the upgrade completes (users sets require_osd_release = luminous)
existing ec profiles are updated automatically.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-06 15:01:03 -04:00
David Zafman
6e5f3cdc69 test: Use correct argument --read-perecent
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-06-23 08:02:50 -07:00
Kefu Chai
b86cda7040 Merge pull request #15764 from tchaikov/wip-20342
qa/suites/upgrade/hammer-jewel-x: add luminous.yaml

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-06-23 22:26:03 +08:00