Commit Graph

30002 Commits

Author SHA1 Message Date
scuttlemonkey
85a024a6bd Merge pull request #865 from ceph/wip-doc-build-cluster
Wip doc build cluster
2013-12-10 10:14:59 -08:00
Sage Weil
d650474059 Merge pull request #923 from dachary/wip-crush-test
CrushTester patches and documentation

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-10 09:06:31 -08:00
Sage Weil
faaf546303 os/MemStore: do on_apply_sync callback synchronously
We can easily deadlock if we put this in the Finisher thread behind other
work; do it synchronously!

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-12-10 08:56:35 -08:00
Loic Dachary
a888a57f79 crush: implement --show-bad-mappings for indep
Support the presence of ITEM_NONE device numbers in the indep mapping as
proof of a bad mapping. Implement the associated unit tests.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 21:10:29 +01:00
Loic Dachary
20263dd30e crush: add unitest for crushtool --show-bad-mappings
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 21:10:29 +01:00
Loic Dachary
fbc4f99080 crush: remove scary message string
The string is no longer used and can be removed.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 21:10:29 +01:00
Loic Dachary
472f495e40 crush: document the --test mode of operations
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 21:10:23 +01:00
Sage Weil
7d000e3411 Merge pull request #918 from ceph/port/misc
Misc portability patches

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-09 11:16:49 -08:00
Sage Weil
4c5f7ba8ba Merge pull request #922 from dachary/wip-crush-choose-tries
crush: fix map->choose_tries boundary test

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-09 08:28:43 -08:00
Loic Dachary
41152a6317 crush: --show-utilization* implies --show-statistics
--show-utilization* outputs only if --show-statistics is set, which is
confusing. Instead of failing, set --show-statistics to avoid the
confusion.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-09 10:57:17 +01:00
Loic Dachary
7482d62f24 crush: add CrushTester accessors
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-08 22:17:26 +01:00
Loic Dachary
c928f077f7 crush: output --show-bad-mappings on err
Instead of using stdout so that it displays well when used in
conjunction with --show-statistics

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-08 22:17:26 +01:00
Loic Dachary
5e0722fab5 crush: fix map->choose_tries boundary test
CrushWrapper::start_choose_profile allocates map->choose_tries with
choose_total_tries elements. When crush_choose_firstn sets a value, it
tests against map->choose_local_tries which could lead to memory
corruption if map->choose_total_tries is smaller than
map->choose_local_tries.

Another indesirable but non fatal side effect is that the output crushtool
--show-choose-tries will be truncated to choose_local_tries which is
set to a lower value than choose_total_tries by the default tuneables.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-08 17:00:54 +01:00
Sage Weil
94da2153d1 Merge pull request #869 from ceph/wip-crush
crush changes for erasure coding

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-07 20:59:22 -08:00
Noah Watkins
ef4061f0ad librbd: remove unused private variable
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 18:07:03 -08:00
Noah Watkins
ad3825c608 TrackedOp: remove unused private variable
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 18:07:03 -08:00
Noah Watkins
3b39a8a9f1 librbd: rename howmany to avoid conflict
A howmany macro exists on some platforms in standard headers, but there
really isn't any sort of standard that I've found. We just avoid the
conflict entirely this way.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 18:07:03 -08:00
Sage Weil
096f9b3268 Merge pull request #917 from ceph/port/compat
compat: define replacement TEMP_FAILURE_RETRY

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-07 14:01:14 -08:00
Sage Weil
96068bfad6 Merge pull request #919 from ceph/port/fdatasync
wbthrottle: use feature check for fdatasync

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-07 14:00:40 -08:00
Noah Watkins
539fe26109 wbthrottle: use feature check for fdatasync
Checking for fdatasync uses the same approach as the qemu configure
script. The relevant commit is d1722a27f552a22561104210e0afad4577878e53.
Here is a copy of the commit message which explains the check:

Under Darwin, a symbol exists for the fdatasync() function, so that our
link test succeeds. However _POSIX_SYNCHRONIZED_IO is set to '-1'.

According to POSIX:2008, a value of -1 means the feature is not
supported.
A value of 0 means supported at compilation time, and a value greater 0
means supported at both compilation and run time.

Enable fdatasync() only if _POSIX_SYNCHRONIZED_IO is '>0'.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:37:00 -08:00
Noah Watkins
663da61c02 rados_sync: fix mismatched tag warning
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:46 -08:00
Noah Watkins
60a25093a4 rados_sync: remove unused private variable
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:46 -08:00
Noah Watkins
43c1676778 mon: check for sys/vfs.h existence
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:24:20 -08:00
Noah Watkins
c99cf265fd make: increase maximum template recursion depth
With clang on OSX spirit blows up without this.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:22:54 -08:00
Noah Watkins
e2be099118 compat: define replacement TEMP_FAILURE_RETRY
Not all platforms have it.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-07 10:18:51 -08:00
Sage Weil
a52ef1df49 Merge remote-tracking branch 'gh/wip-fix-3x'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-06 16:56:10 -08:00
Sage Weil
0386095ea0 Merge remote-tracking branch 'gh/wip-fix-tunables'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-06 16:55:54 -08:00
Sage Weil
3b3cbf52fb crush/CrushCompiler: make current set of tunables 'safe'
We can reenable this error the next time we add new tunables.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:16 -08:00
Sage Weil
8535ceda03 crushtool: remove scary tunables messages
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:15 -08:00
Sage Weil
4eb8891d8d crush/CrushCompiler: start with legacy tunables when compiling
Ensure that a crush file always compiled deterministically, even though
the default values for *new* maps has changed.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:24:15 -08:00
Sage Weil
e8fdef217f crush: add indep data set to cli tests
This will help us catch things if we break the mapping.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:22:59 -08:00
Sage Weil
564de6ea05 osdmaptool: fix cli tests for 3x
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:22:26 -08:00
Sage Weil
6704be68d4 osd: default to 3x replication
3x is the recommendation; it should be the default too.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 16:21:37 -08:00
Sage Weil
308e4f9def Merge pull request #913 from dachary/wip-crush-unittest
CrushWrapper::move_bucket unittest and minor fixes

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-06 16:10:00 -08:00
Sage Weil
38647f7627 Revert "osd: default to 3x replication"
This reverts commit cb26fbde52.

Fix unit tests and do integration tests first; this may have unexpected
consequences.
2013-12-06 15:48:39 -08:00
Loic Dachary
cbeb1f4510 crush: detach_bucket must test item >= 0 not > 0
Since detach_bucket is a private helper solely used by move_bucket which
contains another ( correct ) safeguard, the code cannot be reached and
the problem can never happen. If another function uses detach_bucket,
it may happen.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:31:54 +01:00
Loic Dachary
2cd73f9d3e crush: remove obsolete comments from link_bucket
Probably copy/pasted from move_bucket.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:27:09 +01:00
Loic Dachary
e00324b2bc crush: remove redundant code from move_bucket
The following was introduced in 2012 by a2d0cff1b0

  // un-set the device name so we can use add_item later
  build_rmap(name_map, name_rmap);
  name_map.erase(id);
  name_rmap.erase(id_name);

when insert_item refused to move a bucket for which a name already
exists. It was changed in 2013 by
4e2557a038 and now supports it. The
TestCrushWrapper unittest for move_bucket pass.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:21:16 +01:00
Loic Dachary
8ef80a4c67 crush: unittest CrushWrapper::move_bucket
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-07 00:20:31 +01:00
Sage Weil
865880b5b1 Merge pull request #888 from ceph/wip-crush-tunables
default to bobtail-era crush tunables.

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-12-06 14:45:57 -08:00
Sage Weil
650f896c4d Merge pull request #903 from ceph/wip-memstore
memstore: reference ObjectStore backend

Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-12-06 14:38:15 -08:00
Sage Weil
1d5427a790 Merge pull request #907 from ceph/wip-3x
osd: default to 3x replication
2013-12-06 14:25:38 -08:00
Sage Weil
384f01dfd3 crush/mapper: dump indep partial progression for debugging
...if DEBUG_INDEP is #defined.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
e632a79b3c PendingReleaseNotes: note change of CRUSH indep mode in release notes
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
c853019475 crush: add feature CRUSH_V2 for new indep mode and SET_*_TRIES rule steps
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
caa0e22e15 crush: CHOOSE_LEAF -> CHOOSELEAF throughout
This aligns the internal identifier names with the user-visible names in
the decompiled crush map language.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:03 -08:00
Sage Weil
431a13eb37 osd/OSDMap: fix feature calculation for CACHEPOOL
We need to include the faeture in the mask.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
03911b07e0 crush/CrushCompiler: [de]compile set_choose[leaf]_tries rule step
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
09ce7a2bd3 crush/CrushWrapper: set chooseleaf_tries to 5 for 'simple' indep rules
When making a generic indep rule, set the recursive retry to 5.  This gives
better overall results.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00
Sage Weil
d1b97462cf crush/mapper: add SET_CHOOSE_TRIES rule step
Since we can specify the recursive retries in a rule, we may as well also
specify the non-recursive tries too for completeness.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-06 14:24:02 -08:00