Commit Graph

29907 Commits

Author SHA1 Message Date
Sage Weil
caa0e22e15 crush: CHOOSE_LEAF -> CHOOSELEAF throughout
This aligns the internal identifier names with the user-visible names in
the decompiled crush map language.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
431a13eb37 osd/OSDMap: fix feature calculation for CACHEPOOL
We need to include the faeture in the mask.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
03911b07e0 crush/CrushCompiler: [de]compile set_choose[leaf]_tries rule step
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
09ce7a2bd3 crush/CrushWrapper: set chooseleaf_tries to 5 for 'simple' indep rules
When making a generic indep rule, set the recursive retry to 5.  This gives
better overall results.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
d1b97462cf crush/mapper: add SET_CHOOSE_TRIES rule step
Since we can specify the recursive retries in a rule, we may as well also
specify the non-recursive tries too for completeness.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
64aeded50d crush/mapper: apply chooseleaf_tries to firstn mode too
Parameterize the attempts for the _firstn choose method, and apply the
rule-specified tries count to firstn mode as well.  Note that we have
slightly different behavior here than with indep:

 If the firstn value is not specified for firstn, we pass through the
 normal attempt count.  This maintains compatibility with legacy behavior.
 Note that this is usually *not* actually N^2 work, though, because of the
 descend_once tunable.  However, descend_once is unfortunately *not* the
 same thing as 1 chooseleaf try because it is only checked on a reject but
 not on a collision.  Sigh.

 In contrast, for indep, if tries is not specified we default to 1
 recursive attempt, because that is simply more sane, and we have the
 option to do so.  The descend_once tunable has no effect for indep.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
cb88763ccb crush/mapper: fix up the indep tests
Fix indentation.
Simplify+fix the changed vs moved calculation.
Use the new SET_CHOOSE_LEAF_TRIES command.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
c43c893be8 crush/mapper: new SET_CHOOSE_LEAF_TRIES command
Explicitly control the number of sample attempts, and allow the number of
tries in the recursive call to be explicitly controlled via the rule. This
is important because the amount of time we want to spend looking for a
solution may be rule dependent (e.g., higher for the wide indep pool than
the rep pools).

(We should do the same for the other tunables, by the way!)

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
2731d3030d crush/mapper: pass parent r value for indep call
Pass down the parent's 'r' value so that we will sample different values in
the recursive call when the parent tries multiple times.  This avoids doing
useless work (calling multiple times and trying the same values).

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
1b567ee089 crush/mapper: clarify numrep vs endpos
Pass numrep (the width of the result) separately from the number of results
we want *this* iteration.  This makes things less awkward when we do a
recursive call (for chooseleaf) and want only one item.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
d51a219a10 osd/osd_types: pg_pool_t: fix /// -> ///< comments
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
14138372a0 mon, crush: add mode to "osd crush rule create-simple ..."
Add a mode (firstn or indep) to the create-simple command.  Make it
optional and default to firstn (for compatiblity and simplicity).

Note: a "default=..." option for mon commands would be easier.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
6dc68f4f55 osd/OSDMap: do not shift result when removing nonexistent osds
If it is a replicated pool, remove and shift to the left.  For erasure
pools, replace nonexistent items with CRUSH_ITEM_NONE.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
9085286f88 osd, crush: add 'erasure' pool/pg type
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
94350996cb crush/mapper: strip firstn conditionals out of crush_choose, rename
Now that indep is handled by crush_choose_indep, rename crush_choose to
crush_choose_firstn and remove all the conditionals.  This ends up
stripping out *lots* of code.

Note that it *also* makes it obvious that the shenanigans we were playing
with r' for uniform buckets were broken for firstn mode.  This appears to
have happened waaaay back in commit dae8bec9 (or earlier)... 2007.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
0b9bb54448 crush: add a few unit tests for INDEP mode
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:25 -08:00
Sage Weil
4551fee9ad crush/mapper: add note about r in recursive choose
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:24 -08:00
Sage Weil
7967ae0e0f crush/CrushWrapper: add dump_tree()
This is based on the OSDMap::print_tree(), but is a bit simpler because
we do not have up/down information at this level.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:24 -08:00
Sage Weil
86e978036a crush: use breadth-first search for indep mode
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:24 -08:00
Sage Weil
b1d4dd4eb0 crush: return CRUSH_ITEM_UNDEF for failed placements with indep
For firstn mode, if we fail to make a valid placement choice, we just
continue and return a short result to the caller.  For indep mode, however,
we need to make the position stable, and return an undefined value on
failed placements to avoid shifting later results to the left.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:24 -08:00
Sage Weil
1cfe140bf2 crush: eliminate CRUSH_MAX_SET result size limitation
This is only present to size the temporary scratch arrays that we put on
the stack.  Let the caller allocate them as they wish and remove the
limitation.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 14:41:24 -08:00
Gregory Farnum
e94ca0bd9c Merge pull request #891 from ceph/wip-filestore-remount
allow various threaded services to be restarted to allow mount->umount->mount to work

Reviewed-by: Greg Farnum <greg@inktank.com>
2013-12-03 12:56:25 -08:00
Sage Weil
8f377cc1f7 Merge pull request #879 from xinglin/coverity-fixes
Coverity fixes

The rgw patch is
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>

The rest is
Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-03 12:47:25 -08:00
Sage Weil
858a21b90f common/WorkQueue: allow start() after stop()
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 12:27:14 -08:00
Sage Weil
36505e85b4 os/FileStore: allow mount after umount
Clear the stop flag on successful shutdown so that we can start again
later.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 12:27:14 -08:00
Sage Weil
cd6be292b1 common/Finisher: allow finisher to be restarted
Allow a start() after stop() by clearing the stop flag on a successful
stop.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-03 12:27:14 -08:00
Sage Weil
f60f5fe943 Merge pull request #843 from ceph/wip-osd-metadata
feed osd info about os, kernel, memory, arch to the mons

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
2013-12-03 10:48:57 -08:00
Josh Durgin
9770d5d84e Merge pull request #860 from dachary/wip-6827
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-03 10:39:17 -08:00
Xing Lin
dc06e23e23 test/mon/workloadgen: added check of return value for statfs()
check the return value of statfs() in update_osd_stat()

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

test/mon/workloadgen: fixed typo in error message

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-12-03 11:19:13 -07:00
Xing Lin
0fd3cbf631 rgw/rgw_op: added check of return value for rgw_unlink_bucket()
checked the return value of rgw_unlink_bucket() in RGWCreateBucket::execute()

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-12-03 11:19:13 -07:00
Xing Lin
9c8df34919 os/WBThrottle: added check of return value for posix_fadvise()
check the return value of posix_fadvise() in WBThrottle::entry()

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-12-03 11:19:13 -07:00
Xing Lin
de09778561 test/filestore: add check of return values in StoreTest::SetUp
add return value check for function calls, including mkdir, mkfs
and mount

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

test/filestore: fix return value check for mkdir

allow EEXIST from mkdir()

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-12-03 11:18:55 -07:00
Xing Lin
2884c81766 test/filestore/workloadgen: added check of return value for mkdir
check return value of mkdir

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>

test/filestore/workloadgen: fix return value check for mkdir

allow EEXIST from mkdir()

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-12-03 11:18:19 -07:00
Sage Weil
6ac9269d45 Merge pull request #889 from dachary/wip-osd-metadata
depends on #843 : CPU info + check osd metadata
2013-12-03 08:13:03 -08:00
Loic Dachary
602d5d53c0 osd: include CPU info
It will be handy when trying to figure out CPU bound problems / benchmarks.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-03 15:39:36 +01:00
Loic Dachary
f5025307d9 pybind: syntax check osd metadata
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-03 15:12:23 +01:00
Loic Dachary
6570197871 ceph-disk: blacklist /dev/fd0
blkid -s TYPE /dev/fd0 has been verified to hang forever on a
H8DMR-82 supermicro motherboard running

3.8.0-33-generic #48~precise1-Ubuntu SMP Thu Oct 24 16:28:06 UTC 2013
x86_64

It is unlikely that ceph will ever be used on floppy disks, they
can be blacklisted.

http://tracker.ceph.com/issues/6827 fixes: #6827

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-03 09:31:05 +01:00
Xing Lin
bcb6bfd29d test/ceph_decoder: add return value check for read_file()
check the return value of read_file function call

Signed-off-by: Xing Lin <xinglin@cs.utah.edu>
2013-12-02 23:36:20 -07:00
Sage Weil
3f1352ff38 doc/rados/operations/crush-map: describe crush locations, hook
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-02 21:48:54 -08:00
Sage Weil
396ee0d423 add script/run-coverity
Keep this in-tree.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-02 21:46:25 -08:00
Sage Weil
283de99c74 Merge pull request #887 from jdurgin/wip-peek-meta
osd: read into correct variable for magic string

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-02 17:36:32 -08:00
John Wilkins
a5eda4fcc3 doc: Updated scrub defaults and added note about deep scrubbing.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-12-02 15:26:19 -08:00
Josh Durgin
f11b380e9a osd: read into correct variable for magic string
4d140a71a1 refactored this and
introduced a bug. peek_meta() was accidentally reading into magic,
then replacing magic with val, which was always the empty string,
resulting in the osd always failing to start due to 'mismatched'
magic values.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-02 15:16:46 -08:00
athanatos
cc71fae70b Merge pull request #881 from ceph/wip-objectstore
osd: clean up the osd / objectstore separation

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-02 13:07:19 -08:00
John Wilkins
c77ce90f5b doc: Fixed hyperlink to the manual installation section.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-12-02 10:49:04 -08:00
John Wilkins
648f3bcd39 doc: Added a link to get packages.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-12-02 10:48:27 -08:00
Sage Weil
16b75768b8 osd/OSDMap: fix typo and crush types helper
Bah!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-02 10:21:28 -08:00
Sage Weil
1242642dab Merge pull request #882 from ceph/wip-6699
sysvinit, upstart: use df -P when weighting new OSDs

Reviewed-by: Noah Watkins <noah.watkins@inktank.com>
2013-12-02 10:19:30 -08:00
Sage Weil
a4254f7f0a Merge pull request #876 from javacruft/wip-missed-module
Mark libcls_kvs as a module

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-02 10:10:45 -08:00
Sage Weil
5274b3ef02 Merge pull request #841 from ceph/wip-crush-chassis
osd/OSDMap: add 'chassis' to default type hierarchy

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-12-02 09:59:39 -08:00