Commit Graph

37195 Commits

Author SHA1 Message Date
Sage Weil
89b2feea8d mon: 'osd crush reweight-all'
This corresponds to the crushtool --reweight command.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-09 16:09:26 -08:00
Sage Weil
dd7b58f3b1 crush: set straw_calc_version=1 for default+optimal; do not touch for presets
When using the presets for compatibility (i.e., based on version), do not
touch the straw behavior, as it does not affect mapping or compatibility.
However, make a point of setting it by default and for optimal.

For most users, this means that they will not see any change unless they
explicitly enable the new behavior, or switch to default or optimal
tunables.  The idea is that if they touched it, they shouldn't be
too surprised by the subsequent data movement.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-09 16:09:26 -08:00
Sage Weil
adf5c6de0a crush/builder: a note about the original crush_calc_straw()
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-09 16:09:25 -08:00
Sage Weil
9000068ae4 mon: add 'osd crush {get,set}-tunable <name> [value]' commands
For now, just add the straw_calc_version tunable.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-09 16:09:25 -08:00
Sage Weil
43d5c7caa7 crush: fix crush_calc_straw() scalers when there are duplicate weights
The straw bucket was originally tested with uniform weights and with a
few more complicated patterns, like a stair step (1,2,3,4,5,6,7,8,9).  And
it worked!

However, it does not behave with a pattern like
 1, 2, 2, 3, 3, 4, 4

Strangely, it does behave with
 1, 1, 2, 2, 3, 3, 4, 4

and more usefully it does behave with
 1, 2, 2.001, 3, 3.001, 4, 4.001

That is, the logic that explicitly copes with weights that are duplicates
is broken.

The fix is to simply remove the special handling for duplicate weights --
it isn't necessary and doesn't work correctly anyway.

Add a test that compares the mapping result of  [1, 2, 2, 3, 3, ...] with
[1, 2, 2.001, 3, 3.001, ...] and verifies that the difference is small.
With the fix, we get .00012, whereas the original implementation gets
.015.

Note that this changes the straw bucket scalar *precalculated* values that
are encoded with the map, and only when the admin opts into the new behavior.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 21:03:31 -08:00
Sage Weil
85498bc8f6 crush: fix distortion of straw scalers by 0-weight items
The presence of a 0-weight item in a straw bucket should have no effect
on the placement of other items.  Add a test validating that and fix
crush_calc_straw() to fix the distortion.

Note that this effects the *precalculation* of the straw bucket inputs and
does not effect the actually mapping process given a compiled or encoded
CRUSH map, and only when straw_calc_version == 1 (i.e., the admin opted in
to the new behavior).

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 21:02:54 -08:00
Sage Weil
7c1203635f crush/builder: break out new version 1 of crush_calc_straw
No change, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:25:25 -08:00
Sage Weil
f35a3d88cb crush: pass crush_map * to various builder methods
In particular, we will need it for crush_calc_straw().

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:25:25 -08:00
Sage Weil
9565621d60 crush: default to straw_calc_version 1
Stick with bobtail tunables where it affects compatibility.  Use v1 of
straw_calc, though, since that does not, and we want the best for new
clusters.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:25:25 -08:00
Sage Weil
50e2ceefee crush: add straw_calc_version tunable
It doesn't do anything, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:25:00 -08:00
Sage Weil
c133a83fe7 crush/CrushWrapper: dump chooseleaf_vary_r with other tunables
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:24:59 -08:00
Sage Weil
3877f00877 crush/CrushTester: add new --show-mappings option
This makes --show-utilization and --show-statistics usable.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:24:40 -08:00
Sage Weil
294b06c642 crushtool/CrushTester: output utilization even with 1 batch
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:24:04 -08:00
Sage Weil
35062937f1 crush: recalculate straw scalers during a reweight
The crushtool --reweight function triggers a fresh calculation of bucket
weights so that they are always the sum of the item weights.  In the
straw bucket case, the weights were updated but the corresponding straw
scalers were not being recalculated.  The result is that there was not
effect on placement in adjusted buckets until the next time a bucket item's
weight was adjusted.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:24:04 -08:00
Sage Weil
7a99b48990 osdmaptool: --test-map-pgs-dump
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-03 09:24:03 -08:00
Sage Weil
946bebd94b crush: add dprintk's for crush_calc_straw
These are compiled out by default.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-02 14:32:19 -08:00
Sage Weil
a4222b43ee Merge pull request #2992 from dachary/wip-10173-autogen
autogen.sh: git submodule sync

Reviewed-by: Sage Weil <sage@redhat.com>
2014-11-24 07:33:59 -08:00
Loic Dachary
61cae54137 autogen.sh: git submodule sync
It is not enough to git submodule update on a existing git clone: if the
URL of the repository has been changed, the

   git submodule sync

command must be called to synchronizes submodules' remote URL
configuration setting to the value specified in .gitmodules.

http://tracker.ceph.com/issues/10173 Fixes: #10173

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2014-11-24 12:35:50 +01:00
Loic Dachary
42974e12f6 Merge pull request #2936 from ceph/wip-mailmap
Update Inktank attribution to point to Red Hat

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-11-24 08:27:51 +01:00
Greg Farnum
12f1f21ace mailmap: Update Warren Usui's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:12:48 -08:00
Greg Farnum
43c75ee4ef mailmap: Update Tyler Brekke's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:12:18 -08:00
Greg Farnum
9df4624454 mailmap: Update Tamil Muthamizhan's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:11:34 -08:00
Greg Farnum
fb741524df mailmap: Update Sandon Van Ness' attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:10:22 -08:00
Greg Farnum
c5d5e0636d mailmap: Update Samuel Just's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:08:47 -08:00
Greg Farnum
a6a2a05518 mailmap: Update Patrick McGarry's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:06:48 -08:00
Greg Farnum
4d2f63e73d mailmap: Update Noah Watkins' attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:04:53 -08:00
Greg Farnum
4ee3a87691 mailmap: Update Neil Levine's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:02:38 -08:00
Greg Farnum
12e511b537 mailmap: Update Mark Nelson's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:01:56 -08:00
Greg Farnum
bdf534a543 mailmap: Update Ken Dreyer's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:00:55 -08:00
Greg Farnum
df19410610 mailmap: Update Josh Durgin's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 21:00:34 -08:00
Greg Farnum
d9ee8d601c mailmap: Update John Wilkins' attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 20:59:49 -08:00
Greg Farnum
c84aa39420 mailmap: Update Joao Luis' attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 20:56:00 -08:00
Greg Farnum
806a478edc mailmap: Update Ilya Dryomov's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 20:54:38 -08:00
Greg Farnum
eaa9889bde mailmap: Update Dan Mick's attribution
Point his addresses at Red Hat now, but revert the Inktank->Redhat mappings
so we have more historical accuracy.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 20:49:23 -08:00
Greg Farnum
330b09de1a mailmap: Update Alfredo Deza's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 20:47:01 -08:00
Greg Farnum
1853af3dcd mailmap: Update Greg Farnum's attribution to Red Hat
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2014-11-23 20:42:50 -08:00
Sage Weil
47bc8da329 Merge remote-tracking branch 'gh/next' 2014-11-23 18:55:32 -08:00
Sage Weil
ca13ce1b35 Merge pull request #2983 from tchaikov/wip-fix-lock-dep
lockdep: do not use $CEPH_LOCKDEP for g_lockdep

Reviewed-by: Sage Weil <sage@redhat.com>
2014-11-23 12:59:42 -08:00
Kefu Chai
5470914f73 lockdep: do not use $CEPH_LOCKDEP for g_lockdep
* a non-zero CEPH_LOCKDEP brings ceph down because g_lockdep_ceph_ctx
  is still being constructed when dout_impl() dereferences it.
* fix a typo in comment.
* remove dead code.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2014-11-24 03:19:40 +08:00
Yehuda Sadeh
9ba17a321d rgw-admin: create subuser if needed when creating user
Fixes: #10103
Backport: firefly, giant
This turned up after fixing #9973. Earlier we also didn't create the
subuser in this case, but we didn't really read the subuser info when it
was authenticating. Now we do that as required, so we end up failing the
authentication. This only applies to cases where a subuser was created
using 'user create', and not the 'subuser create' command.

Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-11-22 09:16:03 -08:00
Samuel Just
eb5d2117fd Merge pull request #2895 from yuyuyu101/wip-10022
AsyncMessenger: Fix several problems

Reviewed-by: Sage Weil <sage@redhat.com>
2014-11-21 16:18:25 -08:00
Josh Durgin
6070f4ad68 Merge pull request #2980 from ceph/wip-10123
librbd: protect list_children from invalid child pool IoCtxs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2014-11-21 14:35:24 -08:00
Haomai Wang
e0ab267c75 Merge pull request #2981 from yuyuyu101/quick-fix
GenericObjectMap: Avoid master branch compile error
2014-11-21 17:27:10 +08:00
Haomai Wang
25fe088b75 GenericObjectMap: Fix typo
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-11-21 17:25:06 +08:00
Sage Weil
4884d3e3d1 Merge pull request #2966 from yuyuyu101/wip-10119
KeyValueStore: Fix EC+ KV OSDs crashing

Reviewed-by: Sage Weil <sage@redhat.com>
2014-11-20 21:41:23 -08:00
Sage Weil
7d299528b5 Merge pull request #2978 from majianpeng/fix4
blkdev: using strncpy instead of strcpy.

Reviewed-by: Sage Weil <sage@redhat.com>
2014-11-20 20:51:38 -08:00
Jianpeng Ma
6c7bb8c068 blkdev: using strncpy instead of strcpy.
Coverity Scan reported this bug:
> New defect(s) Reported-by: Coverity Scan Showing 1 of 1 defect(s)
> *** CID 1255369:  Copy into fixed size buffer  (STRING_OVERFLOW)
> /common/blkdev.cc: 34 in block_device_support_discard(const char *)()
> 28     bool block_device_support_discard(const char *devname)
> 29     {
> 30       bool can_trim = false;
> 31       char *p = strstr((char *)devname, "sd");
> 32       char name[32] = {0};
> 33
> >>>     CID 1255369:  Copy into fixed size buffer  (STRING_OVERFLOW)
> >>>     You might overrun the 32 byte fixed-size string "name" by
> >>>     copying "p" without checking the length.
> 34       strcpy(name, p);
> 35       for (unsigned int i = 0; i < strlen(name); i++) {
> 36         if(isdigit(name[i])) {
> 37           name[i] = 0;
> 38           break;
> 39         }
>

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2014-11-21 09:55:02 +08:00
Sage Weil
cb001e7a6b Merge pull request #2974 from ceph/wip-sepia-update
Updated sepia hardware list.
2014-11-20 11:34:36 -08:00
Sandon Van Ness
627f138a72 Updated sepia hardware list.
Added missing details and additional list of hardware.

Signed-off-by: Sandon Van Ness <sandon@redhat.com>
2014-11-20 11:13:38 -08:00
Samuel Just
257ae0dc21 Merge pull request #2752 from dachary/wip-9817-deep-scrub-start
osd: log when scrub,deep scrub, repair starts

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-11-20 09:45:33 -08:00