Commit Graph

301 Commits

Author SHA1 Message Date
Samuel Just
035d6cc2a3 osd/: refer to pg by spg_t and replica by pg_shard_t
We may have multiple pg shards on the same osd for an
ec pool.  Thus, replicas must be referred to by <osd, shard>
and pgs by <pgid, shard>.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:12:10 -08:00
Samuel Just
1c93e03f04 osd/: Add a shard parameter to SnapMapper to handle multiple pg shards
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-17 20:11:05 -08:00
Sage Weil
af848d4a4a Merge pull request #1176 from ceph/wip-primary-affinity
osd: primary affinity

Added primary-affinity thrashing to thrashosd.py.

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-15 16:59:35 -08:00
Sage Weil
c360c604aa ceph_psim: some futzing to test primary_affinity
- map to acting
- count first position, primary

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-15 10:50:09 -08:00
Loic Dachary
d8964b2f33 Merge pull request #1235 from ceph/wip-osdmaptool-pool-fix
wip-osdmaptool-pool-fix

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-14 13:33:26 +01:00
Ilya Dryomov
f98435a45f osdmaptool: add --pool option for --test-map-pgs mode to usage()
--test-map-pgs mode allows to map all pgs from either all pools or just
one pool.  Mention it in usage output.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-02-14 12:24:32 +02:00
Ilya Dryomov
eedbf501e6 osdmaptool: fix --pool option for --test-map-object mode
Commit 7f1b12f2ef ("osdmaptool: add --test-map-pgs mode") broke
--pool for --test-map-object mode.  Fix it, and improve --pool option
handling for both modes while at it (report strict_strtol() errors,
check if specified pool exists).

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-02-14 12:24:25 +02:00
Loic Dachary
8533b6ac2e Merge pull request #1185 from ceph/wip-crush
crush: "vary_r" tunable

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-11 23:42:53 +01:00
Sage Weil
e20a55d906 crush: add infrastructure around new chooseleaf_vary_r tunable
- encoding
- feature bit
- decompile/compile

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-11 08:48:14 -08:00
Josh Durgin
32aa9fdf66 Merge branch wip-librados-timeout
Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-10 14:12:35 -08:00
Josh Durgin
79c1874346 rados: check return values for commands that can now fail
A few places were not checking the return values of commands, since
they could not fail before timeouts were added.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-10 12:53:12 -08:00
Sage Weil
9caaef716f osdmaptool: allow a completely random placement
This useful for comparison purposes and sanity-checking the results.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-07 07:10:13 -08:00
Sage Weil
7f1b12f2ef osdmaptool: add --test-map-pgs mode
This command will map all pgs from all pools (or just one pool) to osds
and summarize the placement and calculate the actual standard deviation and
the expected value.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-07 07:10:13 -08:00
Noah Watkins
4c4e1d0d47 libc++: use ceph:: namespaced data types
Switches the implemetnation of smart pointers and unordered map/set to
use the ceph:: versions.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-18 14:03:20 -08:00
Greg Farnum
93d481a5d2 mon, osdmaptool: switch to primary-specifying pg_to_up_acting_osds
Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-15 16:33:05 -08:00
Greg Farnum
5b699782d1 OSDMap: switch pg_to_osds to have an explicit primary param
Use pointers instead of references for the out params, too!

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-15 16:33:05 -08:00
Sage Weil
de8e8b5d09 osd/ReplicatedPG: make cache-[try-]flush CACHE instead of WR ops
This will allow us to send a flush op on a snap.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-13 16:19:43 -08:00
Sage Weil
4e8259db4f osd/ReplicatedPG: allow cache-evict on snaps
We do three things here:

 - make cache-evict a CACHE instead of WR op, allowing us to submit it
   on snaps (not just head)
 - allow eviction of a snap
 - verify that all snaps are missing before evicting a head

Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-13 16:19:42 -08:00
Loic Dachary
26f7fa96b0 crush: crushtool copyright notice update
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:33 +01:00
Loic Dachary
b705e52304 crush: crushtool emacs compile helper
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:33 +01:00
Loic Dachary
d3d75a2165 crush: crushtool --build informative messages
* dump the crush tree created by --build at debug level 1.

* display a warning at debug level 1 if there is more than one root. In
  most cases it is not what the user wants and it may be confusing
  because the ruleset will only apply to the first of root and have less
  devices under it as expected.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:33 +01:00
Loic Dachary
5b28405a85 crush: crushtool --build uses OSDMap helpers for rulesets
Instead of creating a ruleset from scratch, use the
OSDMap::build_simple_crush_rulesets helper. It is more likely to match
the user expecations.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:33 +01:00
Loic Dachary
1368229e04 crush: print --build debug information when verbose 2
instead of verbose 0

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:32 +01:00
Loic Dachary
5b95d183c8 crush: display args on crushtool failure
When the number of args provided to --build is not a multiple of 3,
display the arguments which do not comply.

For instance the --debug_crush 0 option is not consumed by global_init
in crushtool because, unlike most ceph tools, the arguments are not
passed to global_init. As a result --debug_crush 0 become part of the
arguments and triggers the failure.

   crushtool --debug_crush 0 --build --num_osds 320 node straw 4
   remaining args: [--debug_crush,0,node,straw,4]
   layers must be specified with 3-tuples of (name, buckettype, size)

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:32 +01:00
Loic Dachary
2765f81aff crush: parse CEPH_ARGS in crushtool
The arguments are not given to global_init because the -c option would
conflict. Reading arguments from CEPH_ARGS the way other ceph tools do
is the only way to control verbosity ( via --debug_crush 0 for instance ).

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-12 17:48:32 +01:00
Loic Dachary
229740f422 ceph-conf: display arguments when an option cannot be parsed
unable to parse option: ''

Is difficult to figure out. It is much better if shown in the context in
which it was found:

   ceph-conf --help ''
   unable to parse option: ''
   args: '--help' ''

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-01 21:23:55 +01:00
Bjørnar Ness
be5afa2e66 rados: allow listomapvals to list all k/v pairs
Current code only lists first 512 k/v pairs.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-30 13:36:31 -08:00
Loic Dachary
6e92ed1ea2 osd: factorize build_simple and build_simple_from_conf
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-26 10:01:19 +01:00
Sage Weil
1e529972f3 Merge remote-tracking branch 'gh/master' into wip-cache
Conflicts:
	src/osdc/Objecter.h
	src/vstart.sh

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-22 15:33:59 -08:00
Sage Weil
e6f9fb5b32 Merge pull request #980 from ceph/port/misc
Misc portability patches

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-22 09:34:12 -08:00
Loic Dachary
a0d1521ddb replace pool type REP with REPLICATED
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-22 08:15:24 +01:00
Noah Watkins
79c5a2e8c8 make: add libcommon for missing symbols
On OSX without linking in libcommon at the end of these make targets
there is a missing reference to pipe_cloexec, even though the dependency
is present indirectly through libglobal.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-21 13:13:09 -06:00
Sage Weil
71cd4a2278 rados: add cache-flush, cache-evict, cache-flush-evict-all commands
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-19 16:39:58 -08:00
Sage Weil
d650474059 Merge pull request #923 from dachary/wip-crush-test
CrushTester patches and documentation

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-10 09:06:31 -08:00
Loic Dachary
fbc4f99080 crush: remove scary message string
The string is no longer used and can be removed.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 21:10:29 +01:00
Sage Weil
7d000e3411 Merge pull request #918 from ceph/port/misc
Misc portability patches

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-09 11:16:49 -08:00
Loic Dachary
41152a6317 crush: --show-utilization* implies --show-statistics
--show-utilization* outputs only if --show-statistics is set, which is
confusing. Instead of failing, set --show-statistics to avoid the
confusion.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 10:57:17 +01:00
Sage Weil
94da2153d1 Merge pull request #869 from ceph/wip-crush
crush changes for erasure coding

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-07 20:59:22 -08:00
Sage Weil
096f9b3268 Merge pull request #917 from ceph/port/compat
compat: define replacement TEMP_FAILURE_RETRY

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-07 14:01:14 -08:00
Noah Watkins
60a25093a4 rados_sync: remove unused private variable
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:46 -08:00
Noah Watkins
e2be099118 compat: define replacement TEMP_FAILURE_RETRY
Not all platforms have it.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:18:51 -08:00
Sage Weil
8535ceda03 crushtool: remove scary tunables messages
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:15 -08:00
Sage Weil
caa0e22e15 crush: CHOOSE_LEAF -> CHOOSELEAF throughout
This aligns the internal identifier names with the user-visible names in
the decompiled crush map language.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Yehuda Sadeh
516788d15b Merge remote-tracking branch 'origin/next' 2013-12-06 11:24:06 -08:00
Joao Eduardo Luis
47ee79704f mon: ceph-kvstore-tool: get size of value for prefix/key
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-06 01:06:17 +00:00
Joao Eduardo Luis
c98c1043e3 tools: ceph-kvstore-tool: output value contents to file on 'get'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-06 01:06:17 +00:00
Sage Weil
51cd75080d Merge pull request #838 from xinglin/boost-program-option-check
autoconf: add check for the boost_program_options library

Reviewed-by: Noah Watkins <noah.watkins@inktank.com>
2013-11-29 22:25:28 -08:00
Samuel Just
878f3540d1 ceph-filestore-tool: add tool for fixing lost objects
Used to repair: #6761
Backport: emperor
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-11-14 21:38:38 -08:00
Xing Lin
dcef9fb9bd automake: replaced hardcoded '-lboost_program_options' with a macro
replaced '-lboost_program_options' with the BOOST_PROGRAM_OPTIONS_LIBS
macro in related Makefile.am files

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-11-10 17:13:07 -07:00
Gary Lowell
aef3378bd7 Merge branch 'next' 2013-11-08 05:19:11 +00:00