Commit Graph

34700 Commits

Author SHA1 Message Date
Loic Dachary
90592e9d0e erasure-code: properly indent ErasureCodePluginSelectJerasure.cc
Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 11:42:00 +02:00
Loic Dachary
be3e1e4097 erasure-code: control jerasure plugin variant selection
The jerasure-variant parameter is interpreted as the name of the plugin
variant to be loaded regardless of the available CPU features. The
values can be sse3, sse4, generic. It is undocumented and meant for
benchmarking purposes, primarily to force the generic plugin to be
loaded when the sse4 would be chosen.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 11:41:58 +02:00
Loic Dachary
5fb4354f35 erasure-code: reduce jerasure verbosity
Only output a message about adjusting the buffer size when it is
adjusted, not when the size does not need adjustment.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 11:41:56 +02:00
Loic Dachary
c7daaaf5e6 erasure-code: implement alignment on chunk sizes
jerasure expects chunk sizes that are aligned on the largest possible
vector size that could be used by SSE instructions, when available (
LARGEST_VECTOR_WORDSIZE == 16 bytes ).

For techniques derived from Cauchy, encoding and decoding is done by
subdividing the chunk into packets of packetsize bytes. The operations
are done w * packetsize bytes at a time. It follows that each chunk must
have a size that is a multiple of w * packetsize bytes.

For techniques derived from Vandermonde, it is enough for a chunk to be
a multiple of w * LARGEST_VECTOR_WORDSIZE.

ErasureCodeJerasure::get_alignment returns a size alignment constraint
that has to be enforced as a multiple of the object size. The resulting
object size then has to match the chunk constraints described above
although they have no relationship with K. For Cauchy, it leads to
excessive padding, making it impossible to set sensible parameters for
when the object size is small.

When the per_chunk_alignement data member is true, the semantic of
ErasureCodeJerasure::get_alignment is changed to return a size alignment
constraint to be enforced as a multiple of the chunk size. The
ErasureCodeJerasure::get_chunk_size method is modified to use the new
semantic when appropriate.

The jerasure-per-chunk-alignement parameter is parsed to set
per_chunk_alignement for the Vandermonde and Cauchy techniques.

The memory address of a chunk is implicitly aligned to a page boundary
because it is allocated with buffer::create_page_aligned.

http://tracker.ceph.com/issues/8475 Fixes: #8475

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 10:54:21 +02:00
Loic Dachary
3987ac2a41 erasure-code: cauchy techniques allow w 8,16,32
Enforce the restriction at initialization time, the same way it is done
for Reed Solomon. Choosing a w value different from 8,16,32 will lead to
memory corruption that cannot easily be traced to the cause.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 10:54:21 +02:00
Sage Weil
3de7b7c52c doc/release-notes: fix syntax error
Attempt 2...

ERROR: /srv/autobuild-ceph/gitbuilder.git/build/doc/release-notes.rst:22: Unknown target name: "leveldb".

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-01 21:19:26 -07:00
Sage Weil
3caf1e3f8e Merge pull request #2188 from wonzhq/obj-mtime
osd: add local_mtime to struct object_info_t

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-01 19:27:01 -07:00
Sage Weil
c95e91ef1a os/KeyValueStore: clean up operator<< for KVSuperBlock
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-01 19:24:26 -07:00
Sage Weil
e408f98d89 Merge pull request #2174 from yuyuyu101/kvstore-superblock
Kvstore superblock

Reviewed-by: Sage Weil <sage@redhat.com>
2014-08-01 19:23:35 -07:00
Sage Weil
3a05ff9257 Merge pull request #2169 from ceph/wip-double-pc
mon: s/%%/%/

Realized where these came from; it was an accident.
2014-08-01 18:01:43 -07:00
Sage Weil
79d1aff182 Merge branch 'wip-cache-second'
Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-01 15:37:33 -07:00
Signed-off-by: Zhiqiang Wang
1417eded65 ceph_test_rados_api_tier: test promote-on-second-read behavior
Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-01 15:37:22 -07:00
Zhiqiang Wang
0ed3adc1e0 osd: promotion on 2nd read for cache tiering
Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
2014-08-01 15:37:22 -07:00
Samuel Just
52c2182fe8 Merge pull request #2183 from majianpeng/master
ECBackend: Using ROUND_UP_TO to refactor function get_recovery_chunk_size()

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-01 13:31:08 -07:00
Samuel Just
dad092c6e5 Merge pull request #2175 from majianpeng/fix1
ReplicatedPG: For async-read, set the real result after completing read.

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-01 13:30:04 -07:00
Sage Weil
f752ff49fe Merge pull request #2180 from ceph/wip-ec-isa
osd: add support for intel ISA-L EC library
2014-08-01 10:00:23 -07:00
Sage Weil
cd64a63db0 Merge remote-tracking branch 'gh/next' 2014-08-01 07:08:28 -07:00
Gregory Farnum
a00777f428 Merge pull request #2190 from ceph/wip-osd-leaks
osd: do not leak Session* ref in _send_boot()

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-08-01 10:08:16 -04:00
Gregory Farnum
fc2d18bb1e Merge pull request #2182 from ceph/wip-round
use llrintl when converting double to micro

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-08-01 08:11:08 -04:00
Zhiqiang Wang
13b9dc7084 osd: add local_mtime to struct object_info_t
This fixes a bug when the time of the OSDs and clients are not
synchronized (especially when client is ahead of OSD), and the cache
tier dirty ratio reaches the threshold, the agent skips the flush work
because it thinks the object is too young.

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
2014-08-01 16:09:50 +08:00
Sage Weil
c2fc1a9429 Merge branch 'wip-rocksdb' 2014-07-31 21:11:44 -07:00
Sage Weil
57fd60cdd2 rocksdb: -Wno-portability
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:25 -07:00
Sage Weil
c574e653e4 autogen.sh: debug with -x
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:25 -07:00
Sage Weil
213e0acf7e debian, rpm: build rocksdb statically if we can
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:25 -07:00
Sage Weil
a497a3b42e debian, rpm: build require bzip2 dev
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Sage Weil
da5edb8161 add rocksdb bz2 dep
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Sage Weil
8fcbb65901 configure: enable static rocksdb if check and c++11
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Sage Weil
7500b890fa Makefile: include rocksdb in dist tarball always
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Sage Weil
5502dd3e0f configure: verify c++11 is present for rocksdb
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Sage Weil
33293523da rocksdb: require less shiny autoconf
precise has 2.59

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Sage Weil
bcebf9c0ab do_autogen.sh: build with static librocksdb
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:11:24 -07:00
Josh Durgin
eb1cd7801e rocksdb: update submodule makefile
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-07-31 21:11:24 -07:00
Sage Weil
2e549b4f8e unittest_erasure_code_[plugin_]isa: conditionally compile
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 21:06:08 -07:00
Sage Weil
04037ef9dd osd: do not leak Session* ref in _send_boot()
The get_priv() call returns a ref; make sure we drop it if it exists.
This doesn't happen on every run because usually it is NULL and we take
the other path; it's only after the OSD has been marked down that we reach
the second path.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 20:59:49 -07:00
Yehuda Sadeh
9b05a87ec9 Merge pull request #2157 from ceph/wip-8937
rgw: call processor->handle_data() again if needed

Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-31 18:11:12 -07:00
Sage Weil
d00f23be32 erasure-code/isa: fix signed/unsigned comparison
warning: erasure-code/isa/ErasureCodeIsa.cc:313: comparison between signed and unsigned integer expressions

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 17:46:32 -07:00
Sage Weil
8a193ab033 erasure-code/isa: don't use typename outside of template
error: erasure-code/isa/ErasureCodeIsa.h:141: using 'typename' outside of template

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 17:45:49 -07:00
Ma Jianpeng
92d0bbe1cb ECBackend: Using ROUND_UP_TO to refactor function get_recovery_chunk_size()
Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
2014-08-01 08:32:23 +08:00
Ma Jianpeng
e4ed2cb266 ReplicatedPG: For async-read, set the real result after completing read.
When reading an object from replicated pool, ceph uses sync mode,
so it can set the results in execute_ctx correctly.
However, For the async-read in EC Pool, current code didn't set the
real results after read in complete_read_ctx.

Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
2014-08-01 08:25:03 +08:00
Sage Weil
6e52efabc2 doc/release-notes: tweak quoting
ERROR: /srv/autobuild-ceph/gitbuilder.git/build/doc/release-notes.rst:22: Unknown target name: "leveldb".

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 16:19:47 -07:00
Sage Weil
80911736bd use llrintl when converting double to micro
This avoids rounding error (noticeable on i386).

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 15:39:40 -07:00
Sage Weil
d8b291f30d configure: check for 'better' yasm that can build ISA-L
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 15:18:51 -07:00
Sage Weil
1520b47187 erasure-code/isa/Makefile: add missing \, and missing headers
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 13:43:06 -07:00
Sage Weil
e1f32d9f7f yasm-wrapper: fix -f ...
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 13:43:03 -07:00
Sage Weil
1670fd63fd yasm-wrapper: turn -I foo into -i foo
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 13:42:58 -07:00
Sage Weil
6d95797cef yasm-wrapper: echo original args
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 13:41:41 -07:00
Andreas-Joachim Peters
b7d0017d23 EC: add plugin for Intel ISA-L library 2014-07-31 11:40:17 -07:00
Samuel Just
fe99b25317 Merge pull request #2179 from ceph/wip-8982
osd: fix cache pools without hit_sets configured

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-07-31 11:28:04 -07:00
Sage Weil
383536a2ef mon/OSDMonitor: warn when cache pools do not have hit_sets configured
Give users a clue when cache pools are enabled but the hit_set is not
configured.  Note that technically this will work, but not well, so for
now let's just steer them away.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:19 -07:00
Sage Weil
caf554bf2f osd/ReplicatedPG: improve agent_choose_mode args
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-31 11:03:19 -07:00