Commit Graph

29282 Commits

Author SHA1 Message Date
Gregory Farnum
9f75a42336 Merge pull request #697 from dachary/master
This lets "make check" pass, so I like it.
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-10-07 14:15:19 -07:00
Loic Dachary
0f323bc4d6 common: unintended use of the wrong bloom_filter prototype
For encoding/decoding test purposes. The generate_test_instances should
be improved to cover more encoding/decoding cases.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-10-07 23:11:20 +02:00
Greg Farnum
4b911cf817 ReplicatedPG: copy: use aggregate return code instead of individual Op return
It appears that the OSD is not filling in the individual return codes, and they
should be equivalent for all purposes we care about here (the only Op we are
doing is the copy-get, and if it fails we are getting its failure code).

Reported-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 13:50:30 -07:00
Sage Weil
6da4b91c07 os/FileStore: fix ENOENT error code for getattrs()
In commit dc0dfb9e01 the omap xattrs code
moved up a block and r was no longer local to the block.  Translate
ENOENT -> 0 to compensate.

Fix the same error in _rmattrs().

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 13:50:08 -07:00
Gregory Farnum
13b80bb446 Merge pull request #610 from ceph/wip-optracker
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 08:56:46 -07:00
Sage Weil
f7c5d1ac94 Merge remote-tracking branch 'gh/next' 2013-10-07 04:37:20 -07:00
João Eduardo Luís
02da1a1d79 Merge pull request #695 from ceph/wip-mon-auth
fix mon double-free when dropping unhandled messages, and allow "get monmap" messages to go through without authenticating for MonCliente::get_monmap_privately()

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-07 04:31:36 -07:00
Sage Weil
901427c522 Merge pull request #693 from ceph/wip-bloom
bloom_filter improvements, cleanups

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-10-07 00:28:35 -07:00
Sage Weil
71ee6d7c8d mon: allow MMonGetMap without authentication
This is used by the MonClient::get_monmap_privately() helper.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 15:17:50 -07:00
Sage Weil
f279641894 mon: do not put() unhandle message
If we return false because we aren't handling a message, we should not put
the ref.  This fixes a double-free.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 15:17:50 -07:00
Sage Weil
0cae3a13fa common/bloom_filter: add a few assertions and checks for bit_table_ == NULL
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:28:20 -07:00
Sage Weil
cf34adb99b common/bloom_filter: simplify compressible_bloom_filter encode/decode/dump
Re-use bloom_filter for this, as per Loic's suggestion.  Much nicer!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:22:41 -07:00
Sage Weil
f45c3b98d3 common/bloom_filter: fix compress; improve argument
Fix bug in compress() when compressing to less than .5 or original.

Make the argument have sane units (target size relative to current size;
not a precentage reduction).

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:18:35 -07:00
Sage Weil
055dc4c1f2 common/bloom_filter: fix estimated element count for compressed filters
We need to compensate for the fact that there are fewer bits than there
used to be.

This is a crappy adjustment; it is non-linear.  It's clone enough for now,
though.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:18:30 -07:00
Alan Grosskurth
16dec28c02 ceph.spec.in: Always depend on junit4 (fixes bug #6216)
The RPM spec file currently lists `junit` as a dependency on all
platforms except recent versions of OpenSUSE. This is problematic
because `junit` refers to version 3.x, which doesn't satisfy the check
in the configure script:

    configure: Cannot find junit4.jar (apt-get install junit4)

As a result, rpmbuild doesn't build `libcephfs-test.jar` and fails:

    http://tracker.ceph.com/issues/6216

So let's depend on `junit4` in all cases. This package is provided by
all supported platforms (el6, fc17, fc18, fc19, opensuse12.2,
opensuse12, sles11), and my testing on CentOS 6 shows that rpmbuild
now succeeds and produces the expected `libcephfs_jni1` RPM.
2013-10-06 00:42:43 -07:00
Sage Weil
82f6ec596c doc/release-notes: v0.70
Note that a bunch of stuff we thought would go in 0.70 is actually in 0.69,
so the update/release notes were adjusted accordingly.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-05 21:08:55 -07:00
Sage Weil
aaabc65739 common/bloom_filter: methods for density, approx unique element counts
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:35 -07:00
Sage Weil
04091f87dd common/bloom_filter: remember original target size
This isn't strictly needed for core functionality, but it is convenient to
know.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:35 -07:00
Sage Weil
d07f5f3588 common/bloom_filter: drop raw_table_size_ member
We were storing table_size_ and raw_table_size_, where one is the size in
bits and the other is the size in bytes.  This is silly.  Store only the
size in bytes.

Also, bytes are always 8 bits, so use bit shifts and drop some of that
silliness too.

Move the member declarations to the top of the class so you read them
before the methods.

Fix some annoying whitespace.

Avoid allocating a 0-length array.

Mark the encoding incompatible with v1.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:18 -07:00
Sage Weil
ea2378e539 common/bloom_filter: make compressible_bloom_filter encodable
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:56:18 -07:00
Sage Weil
92a60a0584 mon/PGMap: make generated test instances obey new invariant
As of 091809b814 we keep an osd_map epoch
for any osd_stat update, and assert as much.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:06:04 -07:00
Gary Lowell
7efbebe20f Merge branch 'next' 2013-10-05 02:11:36 +00:00
Sage Weil
e927941fca doc/release-notes: v0.67.4
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 16:49:41 -07:00
Greg Farnum
3fc6cfbb23 Makefile: add include/histogram.h to noinst_HEADERS
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 16:45:38 -07:00
Gregory Farnum
c95d0a164f Merge pull request #676 from ceph/wip-start-copy
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-04 15:58:50 -07:00
David Zafman
5258c9c4e5 Merge pull request #667 from ceph/wip-6143
Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-04 14:59:11 -07:00
David Zafman
bb9b9c8953 common, os: Perform xattr handling based on detected fs type
In FileStore::_detect_fs() store discovered filesystem type in m_fs_type
Add per-filesystem filestore_max_inline_xattr_size_* variants
Add per-filesystem filestore_max_inline_xattrs_* variants
New function set_xattr_limits_via_conf()
  Set m_filestore_max_inline_xattr_size based on override or fs type
  Set m_filestore_max_inline_xattrs based on override or fs type
Handle conf change of any relevant value by calling set_xattr_limits_via_conf()
Change filestore_max_inline_xattr_size to override if non-zero
Change filestore_max_inline_xattrs to override if non-zero

Fixes: #6143

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-10-04 14:10:01 -07:00
athanatos
ce2d9ae617 Merge pull request #692 from ceph/wip-5992-2
Wip 5992 2

Reviewed-by: Sage Weil <sage@inktank.com>
2013-10-04 14:04:41 -07:00
Samuel Just
b87bc2311a ReplicatedPG: lock snapdir obc during write
Otherwise, we won't block properly in prep_push_backfill_object.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
0c2769d332 PGLog: on split, leave log head alone
This way last_update doesn't go backwards.

Fixes: 6447
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
391a885f70 FileStore: make _setattrs not return -ENOENT most of the time
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
0c1e251351 ReplicatedPG: add debugging in recover_replicas for objects added for backfill
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
2ae9ece1a8 ReplicatedPG,PG: use PGBackend methods for getattr
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
8a919fb41d ReplicatedPG,PG: adapt collection_list* users to PGBackend interface
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
ff17e45fe2 PG,ReplicatedPG: expose PGBackend to PG
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
9a10a801f0 PG.cc: remove leading empty space
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:09 -07:00
Samuel Just
4df481c2da PGBackend,ReplicatedBackend: add interfaces for scanning the pg
This will be important since the erasure coded pg will have a different
on-disk format than the replicated backend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:50:06 -07:00
Samuel Just
664b589b05 ReplicatedPG: don't rescan the local collection if we can avoid it
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
e73ec48371 common/hobject: add is_degenerate method
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
c8a4411db1 PGMap: calc_min_last_epoch_clean() will now also use osd_epochs
We don't want to trim past the current osd map for any up osd.
osd_epochs provides a lower bound for that epoch for each osd.

Fixes: 5869
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:55 -07:00
Samuel Just
091809b814 PGMap,PGMonitor: maintain mapping of osd to recent stat epoch
Also, osd_stat will be empty for out osd.

When an osd is marked out, rather than remove it from osd_stat,
we instead 0 out the structure.

This patch also makes osd_stat_updates and osd_stat_rm private.
This should make it simpler to enforce invariants on these
mappings.

Each up osd will have a mapping since out osds are now included as
empty stats.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-04 13:49:11 -07:00
Gary Lowell
e3bb0656d9 v0.70 2013-10-04 20:12:24 +00:00
Sage Weil
20043eba12 Merge pull request #690 from ceph/wip-fuse
a few ceph-fuse fixes

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-04 13:05:46 -07:00
Greg Farnum
806725a8b0 ReplicatedPG: copy: add op progression output
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:59:06 -07:00
Greg Farnum
639ff9f776 ReplicatedPG: copy: don't leak a ctx on failed copy ops
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:59:05 -07:00
Greg Farnum
469d471a8b ReplicatedPG: assert that we have succeeded in do_osd_ops on copyfrom repeats
Our callback is handling errors on its own at this point.

Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:59:05 -07:00
Greg Farnum
f3733a2052 ReplicatedPG: copy: switch CopyCallback to use a GenContext
Signed-off-by: Greg Farnum <greg@inktank.com>
2013-10-04 12:58:58 -07:00
David Zafman
dc0dfb9e01 common,os: Remove filestore_xattr_use_omap option
Now we operate just like when this was set to true

Fixes: #6143

Signed-off-by: David Zafman <david.zafman@inktank.com>
2013-10-04 12:40:49 -07:00
Yehuda Sadeh
10a1e9b371 rgw: update cache atomically
Use new lru_map::find_and_update() to update internal cache state.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-04 10:36:41 -07:00
Yehuda Sadeh
6ce762f641 lru_map: add find_and_update()
A new find_and_update() call to make atomic changes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-04 10:36:13 -07:00