Commit Graph

32012 Commits

Author SHA1 Message Date
Gregory Farnum
84decc119f Merge pull request #1331 from ceph/wip-cache-pool
mon/OSDMonitor: make default false-positive-probability 5%

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-28 14:09:49 -08:00
Samuel Just
fbb1ec88b2 ECBackend: don't leak transactions
Fixes: #7539
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-28 11:27:10 -08:00
Samuel Just
b0d426440b OSD::handle_misdirected_op: handle ops to the wrong shard
OSD recomputes op target based on current OSDMap. With an EC pg, we can get
this result:
1) client at map 512 sends an op to osd 3, pg_t 3.9 based on mapping
   [CRUSH_ITEM_NONE, 2, 3]/3
2) OSD 3 at map 513 remaps op to osd 3, spg_t 3.9s0 based on mapping [3, 2, 3]/3
3) PG 3.9s0 dequeues the op at epoch 512 and notices that it isn't
   primary -- misdirected op
4) client resends and this time PG 3.9s0 having caught up to 513 gets it and
   fulfils it

We can't compute the op target based on the sending map epoch due to
splitting.  The simplest thing is to detect such cases in
OSD::handle_misdirected_op and drop them without an error (the client
will resend anyway).

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-28 11:26:32 -08:00
David Zafman
123ff9e18a osd: stray pg ref on shutdown
Move agent_clear() from only being done when becoming replica
  Do it in clear_primary_state() whenever we stop being primary
  clear_primary_state() passed whether we are staying a primary
Add asserts in agent_stop() and don't need to clear agent_queue

Fixes: #7458

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-02-28 11:12:39 -08:00
Loic Dachary
07ddfcfa93 Merge pull request #1332 from ceph/wip-pg-msg
mon/OSDMonitor: missing space in string

Reviewed-by: Sage Weil <sage@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
2014-02-28 18:35:11 +01:00
John Spray
448fc0e91a mon/OSDMonitor: missing space in string
Minor glitch.  Was printing ..."exceeds per-OSD max of32)"

Signed-off-by: John Spray <john.spray@inktank.com>
2014-02-28 17:16:09 +00:00
Dan Mick
799cde0a7b Fix python-requests package dependencies.
python-ceph does not require requests, but ceph-common does (for ceph-brag).

Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 9a0ef6a181)
2014-02-28 08:34:43 -08:00
Josh Durgin
bfad17bfa9 librados: fix ObjectIterator::operator= for the end iterator
We can't set a shared_ptr to NULL, we need to reset it instead. Add
another test for various permutations of this.

Fixes: #7538
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2014-02-28 08:16:28 -08:00
Sage Weil
f0241c8ac8 mon/OSDMonitor: make default false-positive-probability 5%
This is a more conservative default (as in, less memory consumed) for
newly created cache pools.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-28 08:06:03 -08:00
Yan, Zheng
7ba3200f1e mds: fix nested_anchors update during journal replay
check if the inode is anchored/unanchored before updating the inode

Fixes: #7530
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-26 07:12:49 -08:00
Gregory Farnum
82aba4b02b Merge pull request #1319 from ceph/wip-primary-temp-fix
osd/OSDMap: respect temp primary without temp acting

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-26 07:12:04 -08:00
Samuel Just
1040d1b08a osd/OSDMap: respect temp primary without temp acting
be2748c6d5 ensured that
if the temp acting mapping contains only CRUSH_ITEM_NONE,
that the acting_primary is left at -1.  However, even if
acting.empty(), we need to respect a temp_primary mapping.
Thus, use _acting_primary unless acting.empty() &&
acting_primary == -1.

Bug introduced in be2748c6d5.
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-25 23:09:57 -08:00
Samuel Just
5a6cb3da20 Merge pull request #1317 from ceph/wip-7537
Wip 7537

Reviewed-by: David Zafman <david.zafman@inktank.com>
2014-02-25 20:42:18 -08:00
Samuel Just
be2748c6d5 OSDMap::_pg_to_up_acting_osds: use _acting_primary unless acting is empty
If the temp set for whatever reason has only CRUSH_ITEM_NONE,
we need primary to be -1.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-25 16:47:24 -08:00
Samuel Just
f93bf33b99 Merge pull request #1311 from ceph/wip-dz-scrub-fixes
Wip dz scrub fixes

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-02-25 15:28:08 -08:00
Samuel Just
dc079eb3c5 OSDMonitor: when thrashing, only generate valid temp pg mappings
Since backfill peers are no longer placed into the acting set,
temp mappings will never exceed the pool size.  Also, for ec
pools, temp mappings will never be less than the pool size.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-25 15:27:02 -08:00
David Zafman
9f7f4edad3 Revert "osd/PG: fix assert when deep repair finds no errors"
This reverts commit e3e3328ec8.
2014-02-24 19:56:48 -08:00
David Zafman
728e391112 osd: Don't include primary's shard in repair result message
Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-02-24 19:56:02 -08:00
Gregory Farnum
60c9aafaf0 Merge pull request #1308 from ceph/wip-osdmap-inc
mon/OSDMonitor: fix osdmap encode feature logic

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-24 09:17:42 -08:00
Gregory Farnum
1717601537 Merge pull request #1302 from ceph/wip-create-null
client: fix possible null dereference in create

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-02-24 09:08:36 -08:00
Sage Weil
27968a74d2 ceph_test_objectstore: fix i386 build (again)
test/objectstore/store_test.cc: In member function ‘void SyntheticWorkloadState::read()’:
error: test/objectstore/store_test.cc:462:23: no matching function for call to ‘swap(uint64_t&, size_t&)’

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 19:54:34 -08:00
Sage Weil
5f53cf132b Merge pull request #1307 from ceph/wip-7517
Wip 7517
2014-02-23 19:49:18 -08:00
Sage Weil
14ea8157eb mon/OSDMonitor: fix osdmap encode feature logic
If we are encoding a full map based on an old Incremental that does not
encode the features, fall back to the quorum features or (barring that)
all features.  Do *not* do no features or else we will end up with
encode_client_old which does not even include the extended info and will
cause the mon to crash when decoding.

This was observed when upgading a 0.76 cluster to 0.77 (all mons stopped,
upgraded, and then started)

Reported-by: Aaron Ten Clay <aarontc@aarontc.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 18:23:55 -08:00
Samuel Just
7357b6ed4b PG: skip pg_whoami.osd, not pg_whoami.shard in scrub feature check
Caused by typo in 68184d4574.

Fixes: #7517
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-23 16:20:00 -08:00
Samuel Just
ae59ba3cef Merge pull request #1304 from ceph/wip-last-epoch-clean
mon: fix min_last_epoch_clean handling

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-02-23 15:32:18 -08:00
Sage Weil
dcb9946b1a Merge pull request #1306 from ceph/wip-7512
wip-7512

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-23 14:48:10 -08:00
Samuel Just
696fa36d3f OSD::project_pg_history needs to account for acting_primary/up_primary
Fixes: #7512
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-23 13:35:16 -08:00
Samuel Just
0442b456b9 Objecter/OSDMap: factor out primary_changed() into static OSDMap method
We need to reuse this logic in OSD::project_pg_history.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-23 13:35:16 -08:00
Samuel Just
d0359f7c1c PG: clarify same_primary_since updates regarding primary rank
pg_shard_t includes the position, so these checks handle
same osd/different rank properly.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-23 13:35:16 -08:00
Sage Weil
73c59a022f mon/PGMap: fix osd_epochs update
The insert() call here does not overwrite a previous entry, which means
that the osd_epochs map is never moving forward in time.  This seems to
have been broken since it was introduced in 091809b814.

Backport: emperor, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 12:15:27 -08:00
Sage Weil
df3ba724d5 mon/PGMap: add unit test for min_last_epoch_clean
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 12:15:27 -08:00
Samuel Just
8913ab4bb5 ECBackend: skip canceled xattr reads as well
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-23 12:14:23 -08:00
Sage Weil
23e538efa4 client: fix possible null dereference in create
There are two paths that jump to the out label for which 'in' can be
NULL and outp can be non-NULL.  For those cases we want to fill in the
caller's pointer value (they asked for it) but we clearly cannot take
a reference.

Backport: emperor, dumpling
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 10:18:02 -08:00
Sage Weil
bcc18d4877 osd: increase default leveldb write buffer, cache size
The FileStore's leveldb currently uses libleveldb's defaults for cache and
write buffer size, which are both 4 MB. Increase the cache size to 128MB and
the write buffer to 8MB.

Tested-by: Dmitry Smirnov <onlyjob@member.fsf.org>
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 10:05:39 -08:00
Sage Weil
c4a3a86a33 ceph_test_objectstore: fix i386 build error
We can't swap size_t and uint64_t; just change the len type to match.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 09:05:56 -08:00
Sage Weil
b25c22a20a ceph_test_objectstore: fix signed/unsigned warning
In file included from test/objectstore/store_test.cc:33:0:
../src/gtest/include/gtest/gtest.h: In function ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = int, T2 = unsigned int]’:
../src/gtest/include/gtest/gtest.h:1300:30: instantiated from ‘static testing::AssertionResult testing::internal::EqHelper::Compare(const char*, const char*, const T1&, const T2&) [with T1 = int, T2 = unsigned int, bool lhs_is_null_literal = false]’
test/objectstore/store_test.cc:484:193: instantiated from here
warning: ../src/gtest/include/gtest/gtest.h:1263:3: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-23 09:03:42 -08:00
Loic Dachary
1340b360b5 erasure-code: test that changing the pool size is not allowed
http://tracker.ceph.com/issues/7360 fixes #7360

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-02-23 07:52:59 -08:00
Sage Weil
24c89bf01c Merge pull request #1294 from ceph/wip-librados-object-iterator
fix bugs in librados::ObjectIterator

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-23 07:25:13 -08:00
Sage Weil
5c4129dfb8 Merge pull request #1295 from yuyuyu101/keyvaluestore-enhance
Keyvaluestore enhance(backport to firely)

Pulling this into firefly because it doesn't (substantiatively) touch anything outside of KeyValueStore.

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-23 07:24:43 -08:00
Sage Weil
8ef0d7e6e8 Merge pull request #1297 from ceph/wip-ec-bugs
Wip ec bugs

Reviewed-by: Sage Weil <sage@inktank.com>
2014-02-23 07:12:40 -08:00
Sage Weil
6d8cb22e58 unittest_ecbackend: fix signed/unsigned warnings
In file included from test/osd/TestECBackend.cc:20:0:
../src/gtest/include/gtest/gtest.h: In instantiation of ‘testing::AssertionResult testing::internal::CmpHelperEQ(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int]’:
../src/gtest/include/gtest/gtest.h:1300:30: required from ‘static testing::AssertionResult testing::internal::EqHelper::Compare(const char*, const char*, const T1&, const T2&) [with T1 = long unsigned int; T2 = int; bool lhs_is_null_literal = false]’
test/osd/TestECBackend.cc:30:281: required from here
warning: ../src/gtest/include/gtest/gtest.h:1263:3: comparison between signed and unsigned integer expressions [-Wsign-compare]

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-22 22:00:55 -08:00
Sage Weil
bff4a3bf42 Merge pull request #1301 from ceph/wip-scrub-primary
mon/PGMonitor: fix primary osd check on deep-scrub

gitbuilders are now passing
2014-02-22 21:56:53 -08:00
Sage Weil
fed655e3e9 mon/PGMonitor: fix primary osd check on deep-scrub
s/!=/==/.  Logic was reversed.

Broken in 40bdcb8850.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-22 21:31:48 -08:00
Samuel Just
5516f8c1ff Merge pull request #1300 from ceph/wip-7513
PGLog::activate_not_complete typo

Reviewed-by: David Zafman <david.zafman@inktank.com>
2014-02-22 17:35:46 -08:00
Samuel Just
7b0f3d9092 PGLog::activate_not_complete: fix log.complete_to increment typo
info.last_complete should be the entry before log.complete_to.

This appears to have been a typo introduced in
dd71051a8f.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-22 17:24:29 -08:00
Samuel Just
885d19631e Merge pull request #1298 from ceph/wip-subscribe
osd: fix off-by-one is boot subscription

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-02-22 17:18:55 -08:00
David Zafman
bfb3a2479a Merge pull request #1299 from ceph/wip-repair
osd/PG: fix assert when a shallow repair has fixed all errors.

Reviewed-by: David Zafman <david.zafman@inktank.com>
2014-02-22 14:55:08 -08:00
David Zafman
e3e3328ec8 osd/PG: fix assert when deep repair finds no errors
If there are no deep repairs, we don't want to assert.
Fixes:

    -1> 2014-02-21 21:13:56.393087 7f0258ff9700  0 log [INF] : 0.0 repair ok, 0 fixed
     0> 2014-02-21 21:13:56.428703 7f0258ff9700 -1 osd/PG.cc: In function 'void PG::scrub_finish()' thread 7f0258ff9700 time 2014-02-21 21:13:56.393127
osd/PG.cc: 4294: FAILED assert(deep_scrub)

Signed-off-by: David Zafman <david.zafman@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2014-02-22 13:51:18 -08:00
Samuel Just
9da4d40ecc PGLog: test for missing using the correct shard value
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-22 12:37:55 -08:00
Samuel Just
ace65fec8b OSD: fix query for ec pgs
We need to parse a pg_t and translate it into an spg_t.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-02-22 12:37:55 -08:00