Commit Graph

41722 Commits

Author SHA1 Message Date
Jason Dillaman
f0c14e0ecd test/cli-integration/rbd: verify formatting for 'rbd disk-usage'
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-13 11:20:08 -04:00
Kefu Chai
53514ad97f ceph.in: do not throw on unknown errno
some of the errnos are not listed in errno.errorcode. if we happen
to run into them, print 'Unknown' instead.

Fixes: #11354
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-13 22:42:32 +08:00
Jason Dillaman
6306beda38 qa/workunits/rados/test_rados_tool: added test for omapsetval via stdin
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-13 10:39:33 -04:00
Jason Dillaman
7ffafc56b1 rbd: add disk usage tool
The new disk usage tool uses the new fast diff object map feature
(when enabled) to quickly calculate the provisioned vs actual usage
of images and associated snapshots within a pool.

Fixes: #7746
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-13 10:25:45 -04:00
Ilya Dryomov
92ed6e584d tests: disable deep-flatten feature for the krbd fsx test
krbd doesn't support fast-diff or deep-flatten yet.  Had to do this
with two commits because fast-diff is in next and deep-flatten isn't.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-13 17:10:39 +03:00
Jason Dillaman
7dea175741 rados cli: setomapval can now read from stdin
This will allow a user to store binary values within omap.  Since
RBD uses binary omap values, this will assist with certain recovery
operations.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-13 10:10:02 -04:00
Ilya Dryomov
a2ad46408d Merge remote-tracking branch 'origin/next' 2015-05-13 17:00:44 +03:00
Ilya Dryomov
3ab13f161c Merge pull request #4665 from ceph/wip-11551-next
tests: disable new RBD features for the krbd fsx test

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-13 16:57:05 +03:00
Josh Durgin
b16a6ea788 Merge pull request #4644 from ceph/wip-11405
librbd: new QA client upgrade tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-13 06:53:07 -07:00
Jason Dillaman
faa210ee36 tests: disable fast-diff feature for the krbd fsx test
The kernel does not yet support fast-diff.

Fixes: #11551
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-13 09:52:19 -04:00
Kefu Chai
3c13c69a20 Merge pull request #4627 from dachary/wip-install-deps-clean
install-deps.sh: only pip install after make clean

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-13 21:44:04 +08:00
Nathan Cutler
61303656fa ceph.spec.in: include SUSE in _with_systemd
The master specfile newly defines a _with_systemd variable that should be true
for the set of distros that are using systemd. Since this set of distros
includes SUSE/openSUSE (at least for the more recent versions where ceph is
supported), this commit sets _with_systemdto true on SUSE/openSUSE.

http://tracker.ceph.com/issues/11610 Fixes: #11610

Signed-off-by: Nathan Cutler <ncutler@suse.cz>
2015-05-13 15:32:38 +02:00
John Spray
c324dc1b9e Merge pull request #4536 from ceph/wip-mds-snaps
Wip mds snaps

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-05-13 12:56:09 +02:00
Loic Dachary
283de972f2 ceph-detect-init: plain setup.py install on make install
If no DESTDIR environment variable is set, just

  python setup.py install

instead of setting --root, --prefix or --install-layout. These options
are tested to do the right thing in the context of RPM and deb packaging
but they are presumably not what someone using make install manually
would want.

http://tracker.ceph.com/issues/11598 Fixes: #11598

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 12:45:58 +02:00
Loic Dachary
f88275dd04 tests: tiering health report reworked
Instead of

* setting limit
* populate the cache
* check the health warnings

do the following

* populate the cache
* set limits below the content of the cache
* check the health warnings

The problem with the former approach is that the limits stored by the
OSD internally do not exactly match the one set by the user: they are
converted in ratios and there may be rounding errors.

Also replace the busy loop waiting for pg stats to flush with

  ceph tell osd.* flush_pg_stats || true

for simplicity.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 10:39:37 +02:00
Loic Dachary
4fa0559eaa Merge pull request #4646 from theanalyst/doc/release/v0.94.2
doc: more entries for release notes in v0.94.2

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 09:32:45 +02:00
Loic Dachary
5f252d6135 tests: no agent when testing tiering agent border case
On a machine slow enough, the tiering agent can be activated while
testing border cases where the cache is almost full. Prevent that
by deactivating the tiering agent.

http://tracker.ceph.com/issues/11359 Fixes: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 09:28:06 +02:00
Loic Dachary
b2c40d5e76 tests: uncomment tiering agent tests
http://tracker.ceph.com/issues/11359 Refs: #11359

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 09:25:57 +02:00
Abhishek Lekshmanan
8af9c051da doc: more entries for release notes in v0.94.2
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
2015-05-13 12:50:30 +05:30
Loic Dachary
78c73c512a install-deps.sh: only pip install after make clean
Do not re-install all python dependencies if there already exists a
wheelhouse directory. It makes it so running install-deps.sh twice will
only access the network the first time. The directories where the python
dependencies are installed are removed by make clean. Refreshing the
python dependencies cache can be done via make clean ; install-deps.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 08:35:53 +02:00
Kefu Chai
b241510fc9 Merge pull request #4521 from tchaikov/wip-fix-malformed_input-macros
common: fix the macros for malformed_input::what()

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-13 11:29:57 +08:00
Yan, Zheng
61030e62cf client: pin request->target when target dentry is not pinned
In most cases, target dentry is pinned by request, so no one will
free the target inode. But for LOOKUP and LOOKUPPARENT requests,
target dentry is not pinned. So we should pin the target inode

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-05-13 11:24:50 +08:00
Yan, Zheng
9591df3581 client: invalidate kernel dcache when cache size exceeds limits
make client try invalidating kernel dcache when its cache size
exceeds "client_cache_size" config option.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-05-13 11:24:49 +08:00
Kefu Chai
54fd4e054f common: fix the macros for malformed_input::what()
the thrown exception of malformed_input should carry the function name in
which it was thrown. but what we have now is something like:
"buffer::malformed_input: __PRETTY_FUNCTION__ unknown encoding version >
100"
because __PRETTY_FUNCTION__ is not a macro any more. see
- https://gcc.gnu.org/onlinedocs/gcc-3.1/gcc/Function-Names.html
- https://gcc.gnu.org/onlinedocs/gcc/Function-Names.html
and it is not a string literal, so neither can we can concat it with
the literal err message.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-13 10:08:10 +08:00
Xiaoxi Chen
801799ddf8 test/KeyValueDBMemory : update init()
make it build.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:26:01 +08:00
Xiaoxi Chen
9754764ca2 os/RocksDBStore: interpret some configurations.
compact_on_mount,
compaction_threads,
flusher_threads,
disableWAL.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:25:53 +08:00
Xiaoxi Chen
fcea61045d common/str_map: trim start/tailing writesapce in key and value.
Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
9d6f9c9b54 /s/omap_rocksdb_option/filestore_rocksdb_option
Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
7f6eed9a52 os/RocksDBStore: compact_on_mount not support now
will fix it.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
7c16916d25 keyvaluestore:use keyvaluestore_rocksdb_options if backend is rocksdb
use keyvaluestore_rocksdb_options if using Rocksdb as backend.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
dac287806d os/FileStore:use omap_rocksdb_options if backend is Rocksdb
use omap_rocksdb_options if using RocksDB as omap backend.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
c0311cc385 mon/MonitorDBStore: use mon_rocksdb_options if backend is Rocksdb
use mon_rocksdb_options if using RocksDB as backend for monstore.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
39dc866773 common/config_opts.h: use string for rocksdb options.
Now we can drop the ugly rocksdb interpret options, and
have
  // rocksdb options that will be used for keyvaluestore(if backend is rocksdb)
  OPTION(keyvaluestore_rocksdb_options, OPT_STR, "")
  // rocksdb options that will be used for omap(if omap_backend is rocksdb)
  OPTION(omap_rocksdb_options, OPT_STR, "")
  // rocksdb options that will be used in monstore
  OPTION(mon_rocksdb_options, OPT_STR, "")

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:45 +08:00
Xiaoxi Chen
9c64bae3c8 Add an test checking if frequent used options works.
Now most frequent used options are tested passed.

Known issue:
  BackgroundThreads(we may need to interpret by ourselves).
  DisableWAL(need to interpret since it's in wopitions).

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:44 +08:00
Xiaoxi Chen
8915dba9d6 os/RocksDB: use GetOptionsFromString
use GetOptionsFromString API, instead of interpret them.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2015-05-13 08:22:44 +08:00
Josh Durgin
bf0bdbc09f Merge pull request #4656 from ceph/wip-rbd-xfstests-201505
RBD: expunge test broken by xfs commit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-12 09:16:21 -07:00
Loic Dachary
9127e4008b Merge pull request #4655 from ceph/wip-add-ceph-create-keys.rst
doc: add ceph-create-keys.rst to dist tarball

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-12 18:09:57 +02:00
Douglas Fuller
944eb28a36 RBD: expunge test broken by xfs commit
expunge generic/231, broken for disk and rbd by kernel commit 04162bb

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2015-05-12 09:00:28 -07:00
Kefu Chai
a97566ad66 doc: add ceph-create-keys.rst to dist tarball
Fixes: #10725
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-12 07:34:49 -07:00
Loic Dachary
44d6429a5d Merge pull request #4412 from tchaikov/wip-add-man-page
doc: add man page for ceph-create-keys

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-12 16:03:07 +02:00
Loic Dachary
28f0230c12 erasure-code: bench.sh compares isa & jerasure, vandermonde & cauchy
ISA and Jerasure can be compared for the default stripe width (4KB) and
the two most commonly used Reed Solomon matrices. Comparing the
bandwidth for large chunks (1MB) is not relevant because it is not
commonly used.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-12 14:58:23 +02:00
John Spray
745a565cd8 Merge pull request #4522 from ceph/wip-cephfs-reset-fixes
Fixes to resets of tables and journal with damaged metadata

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-05-12 13:45:34 +02:00
Loic Dachary
753122de54 erasure-code: add --erased option to ceph_erasure_code_benchmark
The --erased option specifies which chunks are erased when running the
decode workload. It is useful to precisely measure the code path
involved by a given recovery strategy. If only one chunk is missing
the code path won't be the same as if two chunks are missing.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-12 12:07:52 +02:00
Loic Dachary
613e200243 erasure-code: add display_chunks helper to ceph_erasure_code_benchmark
To display the erased chunk in a way that is human readable. Extract the
helper by refactoring code from --erasures-generation exhaustive

./ceph_erasure_code_benchmark --plugin isa --parameter directory=.libs \
   --workload decode --verbose --parameter technique=reed_sol_van \
   --parameter k=4 --parameter m=2 --erasures-generation exhaustive

chunks (0)  1   2   3   4   5  (X) is an erased chunk
chunks  0  (1)  2   3   4   5  (X) is an erased chunk
chunks  0   1  (2)  3   4   5  (X) is an erased chunk
chunks  0   1   2  (3)  4   5  (X) is an erased chunk
chunks  0   1   2   3  (4)  5  (X) is an erased chunk
chunks  0   1   2   3   4  (5) (X) is an erased chunk

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-12 12:05:35 +02:00
John Spray
676014580d tools: fix tabletool reset of nonexistent sessionmap
If the object didn't exist, the omap clear was failing
and preventing the subsueent omap set header from
executing.  Set the FAILOK flag on the omap clear
sub-operation.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-12 10:23:27 +01:00
John Spray
a4925933d3 tools: improve journaltool reset
Handle the case where no journal objects exist at all,
rather than just resetting a journal that already exists.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-12 10:23:27 +01:00
John Spray
237f76b6b4 mds: expose static default file layout methods
So that tools can invoke these to get the defaults
without a fully baked MDCache.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-12 10:23:27 +01:00
John Spray
95becebac4 tools: fix tabletool reset snap
SnapServer has an encode method defined that
is different to encode_state, whereas in InoTable
the two were synonmous.  This code was working
previously for inotable but not for snapserver.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-12 10:23:27 +01:00
John Spray
008344542e client: subscribe to map on pool full
Similar to what the objecter does for us
when the cluster-wide full flag is set.  This
allows us to pick up when the pool isn't full
any more.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-12 10:20:43 +01:00
John Spray
482615296f client: clarify some comments
...related to the ENOSPC/pool quota handling.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-12 10:20:43 +01:00