Commit Graph

30929 Commits

Author SHA1 Message Date
Loic Dachary
a3eb935fa5 mailmap: add Yehuda Sadeh fixes for Reviewed-by:
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-09 13:49:15 +01:00
Loic Dachary
f6b52fd41f mailmap: add Sage Weil fixes for Reviewed-by:
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-09 13:49:12 +01:00
Sage Weil
ca9acb9634 mon: implement 'mds set max_mds|max_file_size'
First, add the ability to modify max_file_size.  While we are at it, move
to a more sensible interface for adjusting max_mds too.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-09 04:31:41 -08:00
Loic Dachary
45418e9762 Merge pull request #1056 from apeters1971/wip-jerasure-vectorop
EC-JERASURE: rewrite region-xor function using vector operations 

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-01-09 01:08:30 -08:00
Loic Dachary
5893469b98 Merge pull request #1057 from dachary/wip-buffer-aligned
erasure-code: ensure that coding chunks are page aligned

Reviewed-by: Andreas Peters <andreas.joachim.peters@cern.ch>
2014-01-09 01:06:32 -08:00
Sage Weil
7a89c678a2 Merge pull request #1045 from ceph/wip-7093
fix for #7093

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-01-08 17:05:46 -08:00
David Zafman
0940d8fa6c osd: Change waiting_on_backfill to a set<int>
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:58 -08:00
David Zafman
695255ecf9 osd: Interim working version with backfill reserve state changes
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:58 -08:00
David Zafman
a07d68205b osd: Recovery reservations need to include backfill peers
Caused by 86e4fd4f7a

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:57 -08:00
David Zafman
25a798edfa messages, os, osd: Clean-up g/hobject_t MAX handling
Add check of is_max() before updating an unset pool
Use efficient is_max() instead of compare with hobject::get_max()

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:57 -08:00
David Zafman
f89d4105f2 osd: Return false for func that returns bool
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:57 -08:00
David Zafman
7837490a9d osd: Fix problems in ReplicatedPG::do_op() logic
Fix assert(is_degraded_object(soid)) in ReplicatedPG::wait_for_degraded_object()
  Use last_backfill_started as the backfill line
  Handle uncommon case of multi op source after backfill line and target before
  backfill line and !is_degraded_object().
  Include backfill line itself for before_backfill (<= instead of <)

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:57 -08:00
David Zafman
f9a11ef2f2 osd: Determine if compatibility is required due to back level OSDs
fixes: #7039

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 16:33:57 -08:00
David Zafman
ba19006d06 osd: Config OSD when CEPH_FEATURE_OSD_ERASURE_CODES feature first seen
Set the OSD superblock CEPH_OSD_FEATURE_INCOMPAT_SHARDS and
notify the filestore through set_allow_sharded_objects() in that order.

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 14:53:28 -08:00
John Wilkins
09d021b4e8 doc: Updated for hostname -s and resolving to non-loopback IP address.
fixes: #6234

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-01-08 14:38:13 -08:00
John Wilkins
ed605f0dfb doc: Fixed hyperlink.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-01-08 14:37:20 -08:00
David Zafman
b780f4bf3f osd: Remove redundant incompat feature
We can remove this CompatSet bit without worry because the only
way it could have been set is if an erasure coded pool was create.
This isn't supported as of yet.

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-01-08 14:26:11 -08:00
Loic Dachary
294cdfb93b erasure-code: relax zero copy requirements in tests
The input buffer is copied when padding is required. Remove the tests
that ensured the input buffer was not copied entirely when padding was
added.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-08 20:14:52 +01:00
Loic Dachary
656de1b668 erasure-code: ensure that coding chunks are page aligned
When coding chunks are allocated for jerasure, their address must be
aligned to page boundaries. The requirement is actually to be aligned on
a long long boundary but bufferlist do not allow for fine tuning of the
alignment.

If padding is necessary because the total size of the data to be encoded
is not a multiple of the alignment requirements as returned by
get_alignment(), the buffer is not only padded but also rebuilt using
rebuild_page_aligned() to preserve the page alignment that is expected
of the input buffer.

The overhead of rebuilding the whole input buffer when padding is
necessary could be reduced by only reallocating one buffer for the last
data chunk, therefore reducing the amount of data being copied. However,
this optimization is not going to be used if the caller takes care of
the padding, which is likely to be the case most of the time.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-08 20:13:37 +01:00
Andreas Peters
9ba65996f1 EC-JERASURE: rewrite region-xor function using vector operations to get ~ x1.5 speedups for erasure code and guarantee proper 64-bit/128-bit buffer alignment 2014-01-08 15:15:24 +01:00
Loic Dachary
b6adf59708 Merge pull request #1055 from kri5/master
mailmap: Adds Christophe Courtaut

    git shortlog -nes 

shows only one Christophe Courtaut proof that the .mailmap addition is working.

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-01-08 03:13:20 -08:00
Christophe Courtaut
31f3745a1d mailmap: Adds Christophe Courtaut
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-01-08 11:46:24 +01:00
Kri5
65a15039c3 Merge pull request #1052 from dachary/wip-large-output-daemon-config
common: fix large output in unittest_daemon_config

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-01-08 02:26:03 -08:00
Sage Weil
6eea313346 Merge pull request #1044 from ceph/port/onexit
create on_exit replacement

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-07 23:57:08 -08:00
Josh Durgin
6fac2acc5e Merge pull request #1050 from ceph/wip-7083
radosgw-admin: fix object policy read op

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-01-07 11:23:58 -08:00
Noah Watkins
7acb0a17cb on_exit: remove side effects from asserts
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-07 11:14:42 -08:00
Loic Dachary
e3d0b0ae8c common: fix large output in unittest_daemon_config
All tests in daemon_config use the global g_ceph_context
object. Introducing an expansion loop in it will impact all tests and
generate a very large output.

Remove the SubstitutionLoop test case which is also covered in
test/common/test_config.cc.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-07 16:51:51 +01:00
Sage Weil
5300ed9428 Merge pull request #1043 from ceph/port/misc
misc portability patches

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-06 23:00:05 -08:00
Sage Weil
4eeee0e63f Merge pull request #1051 from kdreyer-inktank/automake-1.12
configure: support Automake 1.12
2014-01-06 22:29:03 -08:00
Sage Weil
ffc9d72d4f Merge pull request #1046 from dachary/wip-7103
common: recursive implementation of config::expand_meta …

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-06 21:30:18 -08:00
Jonathan Dieter
822ad58867 configure: support Automake 1.12
Signed-off-by: Jonathan Dieter <jdieter@lesbg.com>
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
2014-01-06 22:26:17 -07:00
Sage Weil
12a0f51360 Merge pull request #1047 from dachary/wip-duplicated-headers
autoconf: trim duplicated headers
2014-01-06 20:02:17 -08:00
Yehuda Sadeh
b1976dd00f radosgw-admin: fix object policy read op
Fixes: #7083
This was broken when we fixed #6940. We use the same function to both
read the bucket policy and the object policy. However, each needed to be
treated differently. Restore old behavior for objects.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-06 12:53:58 -08:00
Loic Dachary
3f34dc7d52 common: unit tests for config::expand_meta
Part of the config.cc tests are in test/confutils.cc but they do not
cover meta variable expansion. Create unittest_config for config.{h,cc}
specific tests.

The test_md_config_t is made a friend of md_config_t to allow testing
private and protected methods.

test/cli/ceph-conf/show-config-value.t is used to check that the human
readable message message shows as expected when there is an expansion
loop.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Loic Dachary
94854090fb common: recursive implementation of config::expand_meta
Using a recursive implementation of variable expansions make it easier
to protect against loops and provide human readable messages when they
happen.

It also enables one variable to be substituted multiple times in the
same configuration option instead of just once because it is confused
with a variable expansion loop.

http://tracker.ceph.com/issues/7103 fixes: #7103

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Loic Dachary
87db534cea common: cosmetic inversion of tests arguments
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Loic Dachary
5bb1545eed common: multiple variable occurence expansion test
http://tracker.ceph.com/issues/7103 refs #7103

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Sage Weil
98ed9ac5fe mon: only send messages to current OSDs
When choosing a random OSD to send a message to, verify not only that
the OSD id is up but that the session is for the same instance of that OSD
by checking that the address matches.

Fixes: #7093
Backport: emperor, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-04 22:44:57 -08:00
Sage Weil
f68de9f352 osd: ignore OSDMap messages while we are initializing
The mon may occasionally send OSDMap messages to random OSDs, but is not
very descriminating in that we may not have authenticated yet.  Ignore any
messages if that is the case; we will reqeust whatever we need during the
BOOTING state.

Fixes: #7093
Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-04 22:44:56 -08:00
Sage Weil
35da8f9d80 osd: do not send peering messages during init
Do not send any peering messages while we are still working our way
through init().

Fixes: #7093
Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-04 22:44:56 -08:00
Noah Watkins
7aa980528c log: use on exit manager to flush logs on exit
on_exit is not portable, and atexit doesn't allow parameters to be
passed to callbacks.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 12:54:03 -08:00
Noah Watkins
2181d25f7a onexit: add an on exit callback utility
Adds a class that executes registered callbacks in its destructor. Since
static duration objects have their destructors called when returning
from main or calling exit then this can be used as a replacement for
on_exit.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 12:54:03 -08:00
Noah Watkins
3d19f7c876 test: disable cross process sem tests on non-Linux
How to make this portable:

- MAP_ANONYMOUS -> MAP_ANON (OSX)

- sem_init (anonymous semaphore) needs to be replaced by named
semaphores using sem_open/sem_close. Use a memory address of the sem_t
variable to hack anonymous semaphore behavior.

- sem_getvalue isn't supported on OSX. it is used here to do
sem_wait/sem_post to bring a semaphore back to a specific value. to get
around this we may need to restructure the test so that the semaphore
can be destroyed and re-initialized rather than inspected as its
currently being done.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Noah Watkins
6342d05195 pipe: handle missing MSG_MORE and MSG_NOSIGNAL
On OSX (and currently any platform missing the MSG_MORE
macro) the MSG_MORE optimization is disabled. The MSG_NOSIGNAL flag is
available on OSX but is called SO_NOSIGPIPE and must be set via
setsockopt.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Noah Watkins
26c07d97ad ipaddr: use IN6_ARE_ADDR_EQUAL for comparison
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Noah Watkins
ef6a56aa26 keyutils: handle non-linux platform
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Josh Durgin
6663a5eee3 Merge pull request #1038 from ceph/wip-objectcacher-backoff
osdc/ObjectCacher: back off less during flush

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-01-03 16:58:28 -08:00
John Wilkins
ab2b2bb0fd doc: Fixed typo.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-01-03 16:30:12 -08:00
Sage Weil
a231f22032 Merge pull request #1041 from dachary/wip-mon-test
mon: get rid of --keyring /dev/null hack
2014-01-03 16:05:39 -08:00
Sage Weil
81d71f9df2 Merge pull request #1042 from kdreyer-inktank/gitignore-autotools-test-logs
add autotools-generated files to .gitignore
2014-01-03 16:05:24 -08:00