Commit Graph

30661 Commits

Author SHA1 Message Date
Yehuda Sadeh
56e4015cf4 rgw: handle temp url key authorization
Fixes: #3454
Implement auth handling code that checks temp url signature and
expiration. The key that is used for the signature is the bucket owner's
temp url key.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-09 14:28:25 -08:00
Yehuda Sadeh
0e43ac903d rgw: add optional tenant name for swift urls
In order to maintain compatibility with swift clients that expect the
url to also contain a reference to the tenant name, add an optional
param (empty by default) that would add it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-09 14:28:25 -08:00
Yehuda Sadeh
2626101f17 rgw: can set temp url key on user
Either through new through new rest api call, or via
radosgw-admin

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-09 14:28:24 -08:00
Loic Dachary
45418e9762 Merge pull request #1056 from apeters1971/wip-jerasure-vectorop
EC-JERASURE: rewrite region-xor function using vector operations 

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-01-09 01:08:30 -08:00
Loic Dachary
5893469b98 Merge pull request #1057 from dachary/wip-buffer-aligned
erasure-code: ensure that coding chunks are page aligned

Reviewed-by: Andreas Peters <andreas.joachim.peters@cern.ch>
2014-01-09 01:06:32 -08:00
Loic Dachary
294cdfb93b erasure-code: relax zero copy requirements in tests
The input buffer is copied when padding is required. Remove the tests
that ensured the input buffer was not copied entirely when padding was
added.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-08 20:14:52 +01:00
Loic Dachary
656de1b668 erasure-code: ensure that coding chunks are page aligned
When coding chunks are allocated for jerasure, their address must be
aligned to page boundaries. The requirement is actually to be aligned on
a long long boundary but bufferlist do not allow for fine tuning of the
alignment.

If padding is necessary because the total size of the data to be encoded
is not a multiple of the alignment requirements as returned by
get_alignment(), the buffer is not only padded but also rebuilt using
rebuild_page_aligned() to preserve the page alignment that is expected
of the input buffer.

The overhead of rebuilding the whole input buffer when padding is
necessary could be reduced by only reallocating one buffer for the last
data chunk, therefore reducing the amount of data being copied. However,
this optimization is not going to be used if the caller takes care of
the padding, which is likely to be the case most of the time.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-08 20:13:37 +01:00
Andreas Peters
9ba65996f1 EC-JERASURE: rewrite region-xor function using vector operations to get ~ x1.5 speedups for erasure code and guarantee proper 64-bit/128-bit buffer alignment 2014-01-08 15:15:24 +01:00
Loic Dachary
b6adf59708 Merge pull request #1055 from kri5/master
mailmap: Adds Christophe Courtaut

    git shortlog -nes 

shows only one Christophe Courtaut proof that the .mailmap addition is working.

Reviewed-by: Loic Dachary <loic@dachary.org>
2014-01-08 03:13:20 -08:00
Christophe Courtaut
31f3745a1d mailmap: Adds Christophe Courtaut
Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-01-08 11:46:24 +01:00
Kri5
65a15039c3 Merge pull request #1052 from dachary/wip-large-output-daemon-config
common: fix large output in unittest_daemon_config

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-01-08 02:26:03 -08:00
Sage Weil
6eea313346 Merge pull request #1044 from ceph/port/onexit
create on_exit replacement

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-07 23:57:08 -08:00
Josh Durgin
6fac2acc5e Merge pull request #1050 from ceph/wip-7083
radosgw-admin: fix object policy read op

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-01-07 11:23:58 -08:00
Noah Watkins
7acb0a17cb on_exit: remove side effects from asserts
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-07 11:14:42 -08:00
Loic Dachary
e3d0b0ae8c common: fix large output in unittest_daemon_config
All tests in daemon_config use the global g_ceph_context
object. Introducing an expansion loop in it will impact all tests and
generate a very large output.

Remove the SubstitutionLoop test case which is also covered in
test/common/test_config.cc.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-07 16:51:51 +01:00
Sage Weil
5300ed9428 Merge pull request #1043 from ceph/port/misc
misc portability patches

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-06 23:00:05 -08:00
Sage Weil
4eeee0e63f Merge pull request #1051 from kdreyer-inktank/automake-1.12
configure: support Automake 1.12
2014-01-06 22:29:03 -08:00
Sage Weil
ffc9d72d4f Merge pull request #1046 from dachary/wip-7103
common: recursive implementation of config::expand_meta …

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-06 21:30:18 -08:00
Jonathan Dieter
822ad58867 configure: support Automake 1.12
Signed-off-by: Jonathan Dieter <jdieter@lesbg.com>
Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
2014-01-06 22:26:17 -07:00
Sage Weil
12a0f51360 Merge pull request #1047 from dachary/wip-duplicated-headers
autoconf: trim duplicated headers
2014-01-06 20:02:17 -08:00
Yehuda Sadeh
b1976dd00f radosgw-admin: fix object policy read op
Fixes: #7083
This was broken when we fixed #6940. We use the same function to both
read the bucket policy and the object policy. However, each needed to be
treated differently. Restore old behavior for objects.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-06 12:53:58 -08:00
Loic Dachary
3f34dc7d52 common: unit tests for config::expand_meta
Part of the config.cc tests are in test/confutils.cc but they do not
cover meta variable expansion. Create unittest_config for config.{h,cc}
specific tests.

The test_md_config_t is made a friend of md_config_t to allow testing
private and protected methods.

test/cli/ceph-conf/show-config-value.t is used to check that the human
readable message message shows as expected when there is an expansion
loop.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Loic Dachary
94854090fb common: recursive implementation of config::expand_meta
Using a recursive implementation of variable expansions make it easier
to protect against loops and provide human readable messages when they
happen.

It also enables one variable to be substituted multiple times in the
same configuration option instead of just once because it is confused
with a variable expansion loop.

http://tracker.ceph.com/issues/7103 fixes: #7103

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Loic Dachary
87db534cea common: cosmetic inversion of tests arguments
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Loic Dachary
5bb1545eed common: multiple variable occurence expansion test
http://tracker.ceph.com/issues/7103 refs #7103

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-05 16:11:43 +01:00
Noah Watkins
7aa980528c log: use on exit manager to flush logs on exit
on_exit is not portable, and atexit doesn't allow parameters to be
passed to callbacks.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 12:54:03 -08:00
Noah Watkins
2181d25f7a onexit: add an on exit callback utility
Adds a class that executes registered callbacks in its destructor. Since
static duration objects have their destructors called when returning
from main or calling exit then this can be used as a replacement for
on_exit.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 12:54:03 -08:00
Noah Watkins
3d19f7c876 test: disable cross process sem tests on non-Linux
How to make this portable:

- MAP_ANONYMOUS -> MAP_ANON (OSX)

- sem_init (anonymous semaphore) needs to be replaced by named
semaphores using sem_open/sem_close. Use a memory address of the sem_t
variable to hack anonymous semaphore behavior.

- sem_getvalue isn't supported on OSX. it is used here to do
sem_wait/sem_post to bring a semaphore back to a specific value. to get
around this we may need to restructure the test so that the semaphore
can be destroyed and re-initialized rather than inspected as its
currently being done.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Noah Watkins
6342d05195 pipe: handle missing MSG_MORE and MSG_NOSIGNAL
On OSX (and currently any platform missing the MSG_MORE
macro) the MSG_MORE optimization is disabled. The MSG_NOSIGNAL flag is
available on OSX but is called SO_NOSIGPIPE and must be set via
setsockopt.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Noah Watkins
26c07d97ad ipaddr: use IN6_ARE_ADDR_EQUAL for comparison
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Noah Watkins
ef6a56aa26 keyutils: handle non-linux platform
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2014-01-04 09:18:04 -08:00
Josh Durgin
6663a5eee3 Merge pull request #1038 from ceph/wip-objectcacher-backoff
osdc/ObjectCacher: back off less during flush

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-01-03 16:58:28 -08:00
John Wilkins
ab2b2bb0fd doc: Fixed typo.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-01-03 16:30:12 -08:00
Sage Weil
a231f22032 Merge pull request #1041 from dachary/wip-mon-test
mon: get rid of --keyring /dev/null hack
2014-01-03 16:05:39 -08:00
Sage Weil
81d71f9df2 Merge pull request #1042 from kdreyer-inktank/gitignore-autotools-test-logs
add autotools-generated files to .gitignore
2014-01-03 16:05:24 -08:00
Ken Dreyer
bb8b7503b0 add autotools-generated files to .gitignore
When running "make check", Automake generates test-suite.log, along with
various *.log and *.trs files in the tree. Add these files to
.gitignore.

(It looks like this feature arrived in Automake 1.13.)

Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
2014-01-03 16:47:23 -07:00
Loic Dachary
f2e33e8129 mon: get rid of --keyring /dev/null hack
No longer necessary when --auth-supported=none

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-04 00:34:45 +01:00
Dan Mick
3a42d4116d Merge pull request #1026 from dmsimard/ceph_rest_api_discovery
Be more explicit how to discover available API calls

Reviewed-by: Dan Mick <dan.mick@inktank.com>
2014-01-03 15:00:49 -08:00
John Wilkins
d110c913c1 doc: Removed dash from --dmcrypt option.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-01-03 14:37:23 -08:00
Sage Weil
5de7e5622e Merge pull request #1040 from dachary/wip-ceph-disk-activate-data-dir
ceph-disk: fix false positive for gitbuilder
2014-01-03 14:00:57 -08:00
Loic Dachary
c772b6d148 ceph-disk: fix false positive for gitbuilder
The output of test/ceph-disk.sh is very verbose which is good for
debugging errors. However it sometime contains strings that match
/error:/i which is picked by gitbuilder as a sign that the test fail,
even when the exit code is zero.

Remove from the output the three strings triggering false positive in
gitbuilder.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 22:52:55 +01:00
Sage Weil
e2ee52879e osdc/ObjectCacher: back off less during flush
In cce990efc8 we added a limit to avoid
holding the lock for too long.  However, if we back off, we currently
wait for a full second, which is probably a bit much--we really just want
to give other threads a chance.

Backport: emperor
Signed-off-by: Sage Weil <sage@inktank.com>
2014-01-03 12:51:15 -08:00
Sage Weil
312107876c Merge pull request #1032 from dachary/wip-ceph-disk-activate-data-dir
ceph-disk : fixes and improvements for the --data-dir code path and associated tests

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-03 11:41:29 -08:00
David Moreau Simard
daefe8159f Be more explicit how to discover available API calls
Signed-off-by: David Moreau Simard <dmsimard@iweb.com>
2014-01-03 13:23:35 -05:00
Loic Dachary
de8522fa6c ceph-disk: tests for the --data-dir code path
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 16:33:52 +01:00
Loic Dachary
a71025d336 ceph-disk: implement --sysconfdir as /etc/ceph
Replace hardcoded /etc/ceph with the SYSCONFDIR global variable and
implement the --sysconfdir option to override the default value.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 16:30:41 +01:00
Loic Dachary
ca713f48ae ceph-disk: implement --statedir as /var/lib/ceph
Replace hardcoded /var/lib/ceph with the STATEDIR global variable and
implement the --statedir option to override the default value.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 16:30:41 +01:00
Loic Dachary
51ee3c0477 ceph-disk: add copyright notice
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 16:30:41 +01:00
Loic Dachary
306b099ab0 ceph-disk: create the data directory if it does not exist
Instead of failing if the OSD data directory does not exist, create
it. Only do so if the data directory is not enforced to be a device via
the use of the --data-dev flag. The directory is not recursively created.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 16:30:41 +01:00
Loic Dachary
0fcc081858 ceph-disk: run ceph-osd when --mark-init=none
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-03 16:30:40 +01:00