Commit Graph

32555 Commits

Author SHA1 Message Date
Yehuda Sadeh
dfd3cb5140 rgw: only look at prefetched data if we actually prefetched
Fixes: #7903
Since we didn't prefetch data then we couldn't rely on the data to
actually exist there. In that case just move on and read the object.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-03-28 13:25:47 -07:00
Dan Mick
c5682e78e9 Merge pull request #1546 from ceph/wip-fix-pools
fix pool ops test
2014-03-27 13:01:05 -07:00
Sage Weil
7cb1d3a43d qa/workunits/mon/pool_ops.sh: fix test
The pool create command doesn't take k/v pairs any more.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-27 12:57:40 -07:00
Sage Weil
233801c622 qa/workunits/mon/pool_ops.sh: use expect_false
Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-27 12:56:44 -07:00
Josh Durgin
ce59760aea Merge pull request #1545 from ceph/wip-7849-b
ceph-conf: do not log

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-03-27 12:35:50 -07:00
Sage Weil
72715b235a ceph-conf: no admin_socket
We don't need to worry about pidfile because that is done by the fork
functions, which ceph-conf doesn't call.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-27 12:30:39 -07:00
Josh Durgin
e91f5c8cc4 Merge pull request #1522 from themgt/patch-1
document adding dev key for custom Apache/FCGI install

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-03-27 12:03:25 -07:00
Sage Weil
fb208237a1 jerasure: fix up .gitignore
Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-27 11:41:57 -07:00
Sage Weil
acc31e75a3 ceph-conf: do not log
If you are querying the conf for an osd and it has a log configured, we
should not generate any log activity.

This isn't super pretty, but it is much less intrusive that wiring a 'do
not log' flag down into CephContext and a zillion other places.

Fixes: #7849
Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-27 11:36:42 -07:00
Josh Durgin
3f1417a850 Merge pull request #1542 from onlyjob/debian
logrotate: do not rotate empty logs (2nd logrotate file)

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-03-27 11:33:58 -07:00
Sage Weil
e21561e7f4 Merge pull request #1544 from ceph/wip-7876
rgw: use s->content_length instead of s->length

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-27 11:15:27 -07:00
Sage Weil
9f313109bc Merge pull request #1534 from dachary/wip-sse-fix
erasure code sse optimized jerasure plugin

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-27 11:14:30 -07:00
Yehuda Sadeh
ffd69ab3c0 rgw: use s->content_length instead of s->length
Fixes: #7876
Need to use the actual content length, not the pointer to the string.
This was probably working because there's correlation to when
content_length > 0 to whether s->length is not null.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-03-27 10:53:25 -07:00
Sage Weil
0935bb61b7 Merge pull request #1540 from ceph/wip-7860
test: Wait for tier removal before next test starts

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-27 10:21:10 -07:00
Dmitry Smirnov
501e31d94d logrotate: do not rotate empty logs (2nd logrotate file)
Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
2014-03-28 03:42:45 +11:00
Sage Weil
2d55316116 Merge pull request #1541 from onlyjob/debian
logrotate improvement: do not rotate empty logs

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-27 07:02:46 -07:00
Loic Dachary
91176f142c erasure-code: test encode/decode of SSE optimized jerasure plugins
If the machine running make check has the required CPU features
available, load the SSE optimized plugin and check that it can encode /
decode a simple payload. If the CPU features are not available, only
test the generic plugin and display an informative message about the
tests that were skipped.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:24 +01:00
Loic Dachary
b76ad972d1 erasure-code: test jerasure SSE optimized plugins selection
Test the selection of the plugin depending on the CPU features. The
prefix of the plugin is "jerasure" by default (jerasure_generic,
jerasure_sse3, jerasure_sse4) and can be modified with the
"jerasure-name" parameter. A test plugin is created for each
variant (test_jerasure_generic, test_jerasure_sse3, test_jerasure_sse4).
The flags set by ceph_probe are modified by the test to check if the
expected plugin suffix is appended.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:23 +01:00
Loic Dachary
30e714057c osd: increase osd verbosity during functional tests
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:23 +01:00
Loic Dachary
10fd6b3153 erasure-code: SSE optimized jerasure plugins
The jerasure plugin is compiled with three sets of flags:

* jerasure_generic with no SSE optimization
* jerasure_sse3 with SSE2, SSE3 and SSSE3 optimizations
* jerasure_sse4 with SSE2, SSE3, SSSE3, SSE41, SSE42 and PCLMUL optimizations

The jerasure plugin loads the appropriate plugin depending on the CPU
features detected at runtime.

http://tracker.ceph.com/issues/7826 fixes #7826

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:23 +01:00
Loic Dachary
e9878db230 arch: add SSE3, SSSE3, SSSE41 and PCLMUL intel features
And add a note about valgrind forcing a fake cpuid.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:23 +01:00
Loic Dachary
c07aedb6db autotools: intel cpu features detection
Rename SIMD to INTEL for clarity.

Instead of agregating all flags in INTEL_FLAGS, create individual flags
for each feature (INTEL_SSE2_FLAGS etc.) for finer control in the
makefiles.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:23 +01:00
Loic Dachary
cc0cc15212 erasure-code: gf-complete / jerasure modules updates
To avoid confusion, the jerasure v1 branch that contains commits pending
review upstream is named v2-ceph and the gf-complete v2 branch is named
v2-ceph.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:27:23 +01:00
Loic Dachary
12d4f382d6 erasure-code: allow loading a plugin from factory()
The Mutex scope is restricted to only protect the load() method and not
the factory() method. This allows a plugin to load another plugin from
within the factory() method.

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-03-27 14:26:54 +01:00
Sage Weil
d9a2dea755 Merge remote-tracking branch 'gh/firefly' 2014-03-26 21:44:45 -07:00
Dmitry Smirnov
506d2bbaeb logrotate improvement: do not rotate empty logs
Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
2014-03-27 12:12:19 +11:00
Sage Weil
b5702640cb Merge pull request #1537 from ceph/wip-7871
RadosModel: allow --no-omap to be specified seperately from --ec-pool

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-26 17:16:08 -07:00
Sage Weil
ec40196f4f Merge pull request #1536 from ceph/wip-7870
ReplicatedPG::do_osd_ops: only return ENOTSUP on OMAP write ops

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-26 17:14:07 -07:00
David Zafman
56974b91a2 test: Wait for tier removal before next test starts
Fixes: #7860

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-03-26 16:04:40 -07:00
Yehuda Sadeh
98654092fc rgw: configurable chunk size
Fixes: #7589

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-03-26 15:49:35 -07:00
Samuel Just
f171c93f18 Merge pull request #1535 from ceph/wip-7823
osd: trim copy-get backend read to object size

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-03-26 11:48:07 -07:00
Samuel Just
832c007d78 RadosModel: allow --no-omap to be specified seperately from --ec-pool
Fixes: #7871
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-03-26 11:44:32 -07:00
Samuel Just
555ae125ab ReplicatedPG::do_osd_ops: only return ENOTSUP on OMAP write ops
This way, redirected omap reads will work correctly by returning
empty maps and buffers.

Fixes: #7870
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-03-26 11:42:09 -07:00
Sage Weil
21fc535ea7 osd: trim copy-get backend read to object size
We are passing a big number to the backend to read and it is trimming it
to the stripe boundary, and then setting the cursor at a slightly smaller
offset bound by oi.size.  This is invalid, and will trigger an assert in
the _write_copy_chunk code:

     0> 2014-03-21 15:12:23.761509 7f8dd2324700 -1 osd/ReplicatedPG.cc: In function 'void ReplicatedPG::_write_copy_chunk(ReplicatedPG::CopyOpRef, PGBackend::PGTransaction*)' thread 7f8dd2324700 time 2014-03-21 15:12:23.758866
osd/ReplicatedPG.cc: 5554: FAILED assert(cop->data.length() + cop->temp_cursor.data_offset == cop->cursor.data_offset)

To fix this, trim the buffer to the correct length in the completion
context.

Fixes: #7823
Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-26 11:37:43 -07:00
Loic Dachary
ee217952c0 Merge pull request #1518 from dachary/wip-mailmap
mailmap updates

Reviewed-By: Christophe Courtaut <christophe.courtaut@gmail.com>
2014-03-26 16:47:08 +01:00
Sage Weil
f1a1ac8243 Merge remote-tracking branch 'gh/firefly' 2014-03-25 22:39:05 -07:00
Sage Weil
18c3e9ec53 osd: fix tests due to no default erasure-code ruleset
Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-25 22:38:05 -07:00
Sage Weil
29f7420269 Revert "osd: create the erasure-code ruleset in OSDMap::build_simple"
This reverts commit b85330eb1d.

We can't have this in the OSDMap by default or else all clients will require
the new features.

[Fixed up #include for strmap]

Signed-off-by: Sage Weil <sage@inktank.com>
2014-03-25 22:38:05 -07:00
Sage Weil
fbbdbf0aae Merge pull request #1530 from onlyjob/fix-defaultweight
fix bug in 'defaultweight' calculation on OSD start.

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-25 21:44:19 -07:00
Dmitry Smirnov
4cf9a73bac fix bug in 'defaultweight' calculation on OSD start.
awk: cmd. line:1: { d=$2/1073741824 ; r = sprintf(\"%.2f\", d); print r }
  awk: cmd. line:1:                                 ^ backslash not last character on line
  awk: cmd. line:1: { d=$2/1073741824 ; r = sprintf(\"%.2f\", d); print r }
  awk: cmd. line:1:                                 ^ syntax error

Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
2014-03-26 10:48:12 +11:00
Josh Durgin
f57cf56aea Merge branch 'wip-7676-firefly' into firefly
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-03-25 16:40:47 -07:00
SandonV
3f47f9e7a6 Merge pull request #1533 from ceph/wip-7863-wusui
Make sure s3_utilities are found.
2014-03-25 16:36:43 -07:00
Warren Usui
2779e2ab03 Make sure s3_utilities are found.
Add FindBin code to rgw/s3 tests to include s3_utilites.pm properly.

Fixes: 7863
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-03-25 16:30:03 -07:00
Sage Weil
b9e29caff3 Merge pull request #1527 from ceph/wip-6465
be nitpicky about missing references

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-25 15:29:18 -07:00
Sage Weil
1234f768e8 Merge pull request #1531 from onlyjob/debian
minor corrections from Debian

Reviewed-by: Sage Weil <sage@inktank.com>
2014-03-25 14:59:46 -07:00
Gregory Farnum
83661c273e Merge pull request #1363 from ceph/wip-3863
Add ceph-client-debug inode lookup

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-03-25 10:49:51 -07:00
Dmitry Smirnov
4ac78087c4 minor corrections to package descriptions 2014-03-25 22:23:13 +11:00
Dmitry Smirnov
012bb5fb5b minor init.d scripts lintianisation:
* init.d-script-does-not-source-init-functions
 * init.d-script-does-not-implement-required-option
 * init.d-script-missing-lsb-description
2014-03-25 22:10:48 +11:00
Yehuda Sadeh
14b743ba6f rgw: don't modify rgw_bucket for data extra pool
This cleans up the interface, as we don't have this ugly implicit
structure modification when we deal with object that resides on the data
extra pool. Replcae lot's of calls to get_obj_bucket_and_oid_key() to a
cleaner get_obj_ref() that uses a struct to hold the updated fields, and
update the relevant callers.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-03-24 14:59:13 -07:00
Yehuda Sadeh
7989cbd418 rgw: multipart meta object uses extra data pool
Fixes: #7676
We need the extra data pool since EC backends cannot handle the omap
operations that are needed for the multipart meta object.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-03-24 14:58:05 -07:00