Commit Graph

31396 Commits

Author SHA1 Message Date
Loic Dachary
fd13307f63 mon: minor improvement to test-helpers.sh
* kill -9 daemons to better cope with stuck daemons
* reduce paxos propose interval to speedup tests
* look for erasure code plugins from the sources directory

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
4b687ba673 mon: if crush rule create-simple exists return
And it is consistent with the above. Also use a slightly different
string to allow the caller to differentiate between the two cases.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
a7e9a7b648 crush: display item name in CrushWrapper::dump_rule()
When an item name is available used it instead of displaying the number.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
e5b6eb45ba crush: cosmetic changes to CrushWrapper::dump_rule()
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
dcfeee66cd crush: add the dump_rule(int ruleset) method
And implement dump_rules() using dump_rule(). The indentiation and
variable names are intentionaly left as is to not confuse code being
moved around and the code changes.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
f77e8ea268 crush: add CrushWrapper::dump_rules() unit test
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
462ad76a47 crush: fix leaks in TestCrushWrapper.cc
To get a meaningfull valgrind output.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 12:14:05 +01:00
Loic Dachary
ee46908b8e Merge pull request #1222 from dachary/wip-pending-osdmap
mon: control pending OSDMap for testing

Reviewed-by: Joao Eduardo Luis <joao@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-13 10:16:40 +01:00
Loic Dachary
6ad6b092df mon: remove racy tests from osd-pool-create.sh
Use the admin socket to create the conditions by which a pool creation
is made to wait for the next paxos proposal because the required crush
ruleset is pending.

It replaces a fragile time sensitive workaround that could fail because
of race conditions. It also has the benefit of increase the speed of the
test because there is no need to wait for a long time just to accomodate
the slowest machines.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 10:14:11 +01:00
Loic Dachary
8dea58843c mon: add osdmonitor_prepare_command to the admin socket
It provides a developer path allowing functional tests to modify the
pending OSDMap without triggering a PaxosProposal.

It can be used as follows:

   echo '{"prefix":"osdmonitor_prepare_command","prepare":"osd crush tunables","profile":"bobtail"}' | nc -U out/mon.a.asok

It will transform the command into:

  {"prefix":"osd crush tunables","profile":"bobtail"}

and feed it to OSDMonitor::prepare_command_impl(). The pending OSDMap won't
be proposed because it short circuit PaxosService::dispatch. It will,
however, be proposed next time PaxosService::dispatch() gets a chance.

It cannot be used via the ceph command line.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 09:52:23 +01:00
Loic Dachary
07c494bd7c mon: add --mon-advanced-debug-mode
When this flag is true, the mon is expected to provide functionalities
that are for developer oriented debug purposes only. It is meant to
be used by the developer and not the system administrator, because it
would allow a non-developer to break things in ways that would be very
difficult to diagnose. It should probably not be documented.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 09:50:46 +01:00
Loic Dachary
5ea156a09a mon: Monitor::send_reply gracefully handles no connection
If req->get_connection() returns a null pointer, drop the reply instead
of crashing the monitor.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 09:45:55 +01:00
Loic Dachary
53e92f62a0 mon: split prepare_command in two
So that it is possible to call prepare_command without a session
established and a cmdmap that has already been parsed.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-13 09:45:55 +01:00
Sage Weil
9035227e9d doc/release-notes: do not downgrade from v0.67.6
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-12 18:02:58 -08:00
Sage Weil
7533b3b7b5 doc/release-notes: note about dumpling xattr backport
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-12 17:55:50 -08:00
Sage Weil
cc1e84490d PendingReleaseNotes: note about cephfs backtrace updates
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-12 17:42:16 -08:00
Sage Weil
9482850750 Merge pull request #1219 from ceph/wip-update-backtrace
mds: force update backtraces for previously created FS

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-12 17:37:07 -08:00
Sage Weil
af4c142d95 mon/OSDMonitor: fix legacy tunables warning
Warn on legacy tunables, not on non-optimal tunables.  Optimal is a moving
target, but it is really the legacy defaults that we want to push people
off of.

Fixes: #7399
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-12 13:18:04 -08:00
Sage Weil
dce5196973 Merge pull request #1224 from kdreyer-inktank/packaging-libdir
packaging: do not package libdir/ceph recursively
2014-02-12 10:35:07 -08:00
Alexandre Oliva
5a6c9508e0 packaging: do not package libdir/ceph recursively
Package libdir/ceph non-recursively, to avoid duplicates, and
package libdir/ceph/ceph_common.sh explicitly.

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
2014-02-12 11:17:07 -07:00
Sage Weil
f7cf25f293 Merge pull request #1215 from ceph/wip-7385
Remove the max cached objects restriction for librbd

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-12 09:54:30 -08:00
Loic Dachary
f6afe8cc6e Merge pull request #1199 from ceph/wip-doc-librados-intro
Wip doc librados intro

Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-12 17:25:42 +01:00
Loic Dachary
48bcbc75ea Merge pull request #1221 from dachary/wip-filestore
tests: fix packaging for s/filestore/objectstore/

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
2014-02-12 12:52:02 +01:00
Loic Dachary
840e918fd4 tests: fix packaging for s/filestore/objectstore/
The binaries file name have changed and need to be updated in the
packaging files for deb and rpm. Fix a few leftovers as well.

Fixing 1a588f18ba

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-12 12:46:14 +01:00
Kri5
b88b257241 Merge pull request #1220 from dachary/wip-filestore
tests: fix objectstore tests
2014-02-12 12:26:28 +01:00
Loic Dachary
b64f1e39a8 tests: fix objectstore tests
The objectstore test from 1a588f18ba was
missing a few changes.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-12 11:52:37 +01:00
Loic Dachary
a065e2c0d0 Merge pull request #1212 from ywang19/master
correct one command line at building packages section

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-12 09:06:31 +01:00
Yan, Zheng
f34eb1b85b mds: force update backtraces for previously created FS
Update inode format version to 10, treat any previously created
inode as no backtrace. When inode with no backtrace is modified,
force update its backtrace.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2014-02-12 14:24:09 +08:00
Sage Weil
ed0980c925 Merge pull request #1168 from yuyuyu101/wip-refactor-objectstore-test
Rename test/filestore to test/objectstore

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-11 21:17:05 -08:00
Sage Weil
7f76e78bf2 Merge pull request #1218 from yuyuyu101/wip-misc-fix
Fix bad dealloctor

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-11 21:12:09 -08:00
Haomai Wang
b5c10bf059 Fix bad dealloctor
Memory allocated by malloc() should be deallocated by free(), not 'delete'

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-02-12 12:04:30 +08:00
ywang19
a4b3b786ff correct one command line at building packages section
Signed-off-by: Wang, Yaguang <yaguang.wang@intel.com>
2014-02-12 10:38:16 +08:00
Sage Weil
33692a2c02 osdmaptool: fix cli test
Encoding the extra tunable byte threw off the output here.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 18:38:16 -08:00
Sage Weil
fed83969b2 tset_bufferlist: fix signed/unsigned comparison
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 18:16:58 -08:00
Loic Dachary
8533b6ac2e Merge pull request #1185 from ceph/wip-crush
crush: "vary_r" tunable

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-11 23:42:53 +01:00
John Wilkins
2682b64c51 doc: Incorporated feed back from Loic and Dan.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-02-11 13:28:33 -08:00
John Wilkins
0da9621a89 doc: Adds additional terms for use with librados.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-02-11 13:28:04 -08:00
Josh Durgin
e1a49e5386 ObjectCacher: remove unused target/max setters
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-11 12:15:35 -08:00
Josh Durgin
0559d31db2 librbd: remove limit on number of objects in the cache
The number of objects is not a significant indicated of when data
should be written out for rbd. Use the highest possible value for
number of objects and just rely on the dirty data limits to trigger
flushing. When the number of objects is low, and many start being
flushed before they accumulate many requests, it hurts average request
size and performance for many concurrent sequential writes.

Fixes: #7385
Backport: emperor, dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-11 12:14:13 -08:00
Josh Durgin
db034acf54 ObjectCacher: use uint64_t for target and max values
All the options are uint64_t, but the ObjectCacher was converting them
to int64_t. There's never any reason for these to be negative, so
change the type.

Adjust a few conditionals so that they only convert known-positive
signed values to uint64_t before comparing with the target and max
values. Leave the actual stats accounting as loff_t for now, since
bugs in accounting will have bad effects if negative values wrap
around.

Backport: emperor, dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-11 12:06:20 -08:00
Josh Durgin
bf8cf2d6d2 ObjectCacher: remove max_bytes and max_ob arguments to trim()
These are never passed, so replace them with the defaults.

Backport: emperor, dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-11 12:06:10 -08:00
Sage Weil
d136eb4cbd mon: allow firefly crush tunables to be selected
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 11:12:56 -08:00
Sage Weil
e3309bce03 doc/rados/operations/crush: describe new vary_r tunable
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 11:12:56 -08:00
Sage Weil
525b2d2663 crush: add firefly tunables baseline test
This is a user's map that gives different results when the vary_r tunable
is adjusted.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 11:12:56 -08:00
Sage Weil
37f840b499 crushtool: new cli tests for the vary-r tunable
These illustrate the variation in mapping results as the vary_r tunable
is adjusted.  Note:

1- For the vary_r=0 case, we have several inputs that map to only a single
output:

      rule 3 (delltestrule) num_rep 4 result size == 1:\t27/1024 (esc)
      rule 3 (delltestrule) num_rep 4 result size == 2:\t997/1024 (esc)

This is the behavior we are fixing.  For all of the other values of
vary_r, we get 2 outputs for all inputs.

2- If we use vary_r 1, which is likely the most efficient computation,
we get lots of inputs that change.  By setting larger values of vary_r,
we can trade a bit of extra computation to get a mapping that is more
similar to the legacy behavior. This is useful for legacy clusters:

    $ for f in `seq 1 4` ; do diff -u test-map-vary-r-0.t test-map-vary-r-$f.t | grep -c -- +  ; done
    3030
    1629
    645
    228

The crushmap here comes from a user who was seeing a bad mapping for certain
pgs after some OSDs were reweighted by utilization.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 11:11:25 -08:00
Sage Weil
e88f843c99 crush: add infrastructure around SET_CHOOSELEAF_VARY_R rule step/command
This will let you vary the vary_r tunable on a per-rule basis.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 08:48:14 -08:00
Sage Weil
f944ccc20a crush: add SET_CHOOSELEAF_VARY_R step
This lets you adjust the vary_r tunable on a per-rule basis.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 08:48:14 -08:00
Sage Weil
e20a55d906 crush: add infrastructure around new chooseleaf_vary_r tunable
- encoding
- feature bit
- decompile/compile

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 08:48:14 -08:00
Sage Weil
dd200c924e Merge pull request #1207 from dachary/wip-7378
common: admin socket fallback to json-pretty format

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-11 08:30:31 -08:00
Loic Dachary
c36a6ed4c3 Merge pull request #1198 from dachary/wip-mailmap
mailmap updates

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-02-11 16:14:49 +01:00