Commit Graph

30042 Commits

Author SHA1 Message Date
Noah Watkins
3b39a8a9f1 librbd: rename howmany to avoid conflict
A howmany macro exists on some platforms in standard headers, but there
really isn't any sort of standard that I've found. We just avoid the
conflict entirely this way.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 18:07:03 -08:00
Sage Weil
096f9b3268 Merge pull request #917 from ceph/port/compat
compat: define replacement TEMP_FAILURE_RETRY

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-07 14:01:14 -08:00
Sage Weil
96068bfad6 Merge pull request #919 from ceph/port/fdatasync
wbthrottle: use feature check for fdatasync

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-07 14:00:40 -08:00
Noah Watkins
539fe26109 wbthrottle: use feature check for fdatasync
Checking for fdatasync uses the same approach as the qemu configure
script. The relevant commit is d1722a27f552a22561104210e0afad4577878e53.
Here is a copy of the commit message which explains the check:

Under Darwin, a symbol exists for the fdatasync() function, so that our
link test succeeds. However _POSIX_SYNCHRONIZED_IO is set to '-1'.

According to POSIX:2008, a value of -1 means the feature is not
supported.
A value of 0 means supported at compilation time, and a value greater 0
means supported at both compilation and run time.

Enable fdatasync() only if _POSIX_SYNCHRONIZED_IO is '>0'.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:37:00 -08:00
Noah Watkins
663da61c02 rados_sync: fix mismatched tag warning
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:46 -08:00
Noah Watkins
60a25093a4 rados_sync: remove unused private variable
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:46 -08:00
Noah Watkins
43c1676778 mon: check for sys/vfs.h existence
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:20 -08:00
Noah Watkins
c99cf265fd make: increase maximum template recursion depth
With clang on OSX spirit blows up without this.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:22:54 -08:00
Noah Watkins
e2be099118 compat: define replacement TEMP_FAILURE_RETRY
Not all platforms have it.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:18:51 -08:00
Sage Weil
a52ef1df49 Merge remote-tracking branch 'gh/wip-fix-3x'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-06 16:56:10 -08:00
Sage Weil
0386095ea0 Merge remote-tracking branch 'gh/wip-fix-tunables'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-06 16:55:54 -08:00
Sage Weil
3b3cbf52fb crush/CrushCompiler: make current set of tunables 'safe'
We can reenable this error the next time we add new tunables.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:16 -08:00
Sage Weil
8535ceda03 crushtool: remove scary tunables messages
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:15 -08:00
Sage Weil
4eb8891d8d crush/CrushCompiler: start with legacy tunables when compiling
Ensure that a crush file always compiled deterministically, even though
the default values for *new* maps has changed.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:15 -08:00
Sage Weil
e8fdef217f crush: add indep data set to cli tests
This will help us catch things if we break the mapping.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:22:59 -08:00
Sage Weil
564de6ea05 osdmaptool: fix cli tests for 3x
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:22:26 -08:00
Sage Weil
6704be68d4 osd: default to 3x replication
3x is the recommendation; it should be the default too.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:21:37 -08:00
Sage Weil
308e4f9def Merge pull request #913 from dachary/wip-crush-unittest
CrushWrapper::move_bucket unittest and minor fixes

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-06 16:10:00 -08:00
Sage Weil
38647f7627 Revert "osd: default to 3x replication"
This reverts commit cb26fbde52f31b449af60acce3ced34e593d6e1e.

Fix unit tests and do integration tests first; this may have unexpected
consequences.
2013-12-06 15:48:39 -08:00
Loic Dachary
cbeb1f4510 crush: detach_bucket must test item >= 0 not > 0
Since detach_bucket is a private helper solely used by move_bucket which
contains another ( correct ) safeguard, the code cannot be reached and
the problem can never happen. If another function uses detach_bucket,
it may happen.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:31:54 +01:00
Loic Dachary
2cd73f9d3e crush: remove obsolete comments from link_bucket
Probably copy/pasted from move_bucket.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:27:09 +01:00
Loic Dachary
e00324b2bc crush: remove redundant code from move_bucket
The following was introduced in 2012 by a2d0cff1b071bed84ac439e4fcf9ddfb936f89c8

  // un-set the device name so we can use add_item later
  build_rmap(name_map, name_rmap);
  name_map.erase(id);
  name_rmap.erase(id_name);

when insert_item refused to move a bucket for which a name already
exists. It was changed in 2013 by
4e2557a038dc1e8c68993ad8571d74e2eb8ea90a and now supports it. The
TestCrushWrapper unittest for move_bucket pass.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:21:16 +01:00
Loic Dachary
8ef80a4c67 crush: unittest CrushWrapper::move_bucket
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:20:31 +01:00
Sage Weil
865880b5b1 Merge pull request #888 from ceph/wip-crush-tunables
default to bobtail-era crush tunables.

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-06 14:45:57 -08:00
Sage Weil
650f896c4d Merge pull request #903 from ceph/wip-memstore
memstore: reference ObjectStore backend

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-06 14:38:15 -08:00
Sage Weil
1d5427a790 Merge pull request #907 from ceph/wip-3x
osd: default to 3x replication
2013-12-06 14:25:38 -08:00
Sage Weil
384f01dfd3 crush/mapper: dump indep partial progression for debugging
...if DEBUG_INDEP is #defined.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
e632a79b3c PendingReleaseNotes: note change of CRUSH indep mode in release notes
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
c853019475 crush: add feature CRUSH_V2 for new indep mode and SET_*_TRIES rule steps
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
caa0e22e15 crush: CHOOSE_LEAF -> CHOOSELEAF throughout
This aligns the internal identifier names with the user-visible names in
the decompiled crush map language.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
431a13eb37 osd/OSDMap: fix feature calculation for CACHEPOOL
We need to include the faeture in the mask.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
03911b07e0 crush/CrushCompiler: [de]compile set_choose[leaf]_tries rule step
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
09ce7a2bd3 crush/CrushWrapper: set chooseleaf_tries to 5 for 'simple' indep rules
When making a generic indep rule, set the recursive retry to 5.  This gives
better overall results.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
d1b97462cf crush/mapper: add SET_CHOOSE_TRIES rule step
Since we can specify the recursive retries in a rule, we may as well also
specify the non-recursive tries too for completeness.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
64aeded50d crush/mapper: apply chooseleaf_tries to firstn mode too
Parameterize the attempts for the _firstn choose method, and apply the
rule-specified tries count to firstn mode as well.  Note that we have
slightly different behavior here than with indep:

 If the firstn value is not specified for firstn, we pass through the
 normal attempt count.  This maintains compatibility with legacy behavior.
 Note that this is usually *not* actually N^2 work, though, because of the
 descend_once tunable.  However, descend_once is unfortunately *not* the
 same thing as 1 chooseleaf try because it is only checked on a reject but
 not on a collision.  Sigh.

 In contrast, for indep, if tries is not specified we default to 1
 recursive attempt, because that is simply more sane, and we have the
 option to do so.  The descend_once tunable has no effect for indep.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
cb88763ccb crush/mapper: fix up the indep tests
Fix indentation.
Simplify+fix the changed vs moved calculation.
Use the new SET_CHOOSE_LEAF_TRIES command.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
580cf5f68c Merge pull request #886 from ceph/wip-6922
Fix some pg_num change return codes and make them more resistant to mis-use

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-06 14:15:56 -08:00
Sage Weil
63755c42f9 Merge pull request #909 from dachary/wip-crush-unittest
more CrushWrapper unittest
2013-12-06 12:35:52 -08:00
Loic Dachary
4e26cc0dac crush: unittest CrushWrapper::get_immediate_parent
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 20:40:48 +01:00
Loic Dachary
09938e6455 crush: unittest CrushWrapper::update_item
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 20:40:48 +01:00
Loic Dachary
16ac59042e crush: unittest s/std::string/string/
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 20:40:48 +01:00
Loic Dachary
b8190180c3 crush: unittest use const instead of define
And reduce the depth of the hierarchy because three levels of buckets
capture the same cases as four levels.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 20:40:48 +01:00
Loic Dachary
dc095214d3 crush: unittest CrushWrapper::check_item_loc
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 20:40:48 +01:00
Loic Dachary
000c59a9a2 crush: unittest remove useless c->create()
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 20:40:48 +01:00
Yehuda Sadeh
516788d15b Merge remote-tracking branch 'origin/next' 2013-12-06 11:24:06 -08:00
Sage Weil
cb26fbde52 osd: default to 3x replication
3x is the recommendation; it should be the default too.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 10:35:45 -08:00
Sage Weil
f4c16236b7 Merge pull request #901 from dachary/wip-crush-unittest
crush: check for invalid names in loc[]

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-06 08:29:01 -08:00
Loic Dachary
aedbc99ffc crush: check for invalid names in loc[]
Add the is_valid_crush_loc helper to test for invalid crush names in
insert_item and update_item, before performing any side
effect. Implement the associated unit tests.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-06 09:43:47 +01:00
Sage Weil
fe03ad2801 osd: queue pg deletion after on_removal txn
The removal is normally so slow that these don't really race, but they
could.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-05 23:13:28 -08:00
Sage Weil
aa63d6730a os/MemStore: implement reference 'memstore' backend
This is (as near to) a trivial ObjectStore backend for the OSD as we can
get at the moment.  Everything is stored in memory.  We are slightly
tricky with the locking, but not overly so.

On umount we dump everything out to disk, and on mount we load it all in
again, so we have some very coarse persistence/durability... just enough
to make this usable in a non-failure environment.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-05 23:13:28 -08:00