Commit Graph

38171 Commits

Author SHA1 Message Date
Ning Yao
852aab64f6 PG:: filter_snapc: Return immediately if no snapc need to trim
we can return immediately if no snapc need to trim. Do not iterater snapc vector and do extra judgement and ops.

Signed-off-by: Ning Yao <zay11022@gmail.com>
2014-12-26 04:20:35 +00:00
Jianpeng Ma
45fb9a3b5d FileJournal: Don't cache journal data if journal w/o directio mode
Only read journal data in osd starting.So no need to keep journal
data in kernel cache/buffer.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2014-12-26 09:58:24 +08:00
Kefu Chai
e4e1777894 AsyncMessenger: fix the leak of file_events
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2014-12-25 21:45:00 +08:00
Sage Weil
aa56ee40c0 mon: provide encoded canonical full OSDMap from primary
Currently we make each monitor apply the incremental and encode the full
map locally.  The original motivation was to save bandwidth, but the
savings are minimal to modest and the complexity associated with doing this
is huge.

This strategy also causes problems now that we have OSDMap crc's and old
mons/clusters may have diverging full OSDMaps due to mixed version
clusters.  See #10422

Instead, include the encoded full map in the paxos transaction.  We will
still apply the incremental and check the crc, but if it fails and we have
the correct version, reload it from disk and move on.  If we don't, we
will continue as we have before--the primary mon doesn't have support for
crc's yet.  When it does we will start verifying and/or get our
full map back into sync.

Fixes: #10422
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-23 17:01:53 -08:00
Sage Weil
d7fd6fccc9 osdc/Objecter: improve pool deletion detection
Currently we can have a race like so:

 - send op in epoch X
 - osd replies
 - pool deleted in epoch X+1
 - client gets X+1, sends map epoch check
 - client gets reply
   -> fails assert that there is no map check in flight

Avoid this situation by inferring that the pool is deleted when we see
that we previously sent the request but the pool is no longer present.
Since pool ids are not reused there is no point in doing a synchronous
map check at all.

Backport: giant
Fixes: #10372
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-23 15:59:19 -08:00
Haomai Wang
a540ac3385 librados: only call watch_flush if necessary
Fix bug #10424
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

(cherry picked from commit 926a1b7468)
2014-12-23 14:24:27 -08:00
Loic Dachary
6b030aa8ba mds: add default ctor for quota_info_t
http://tracker.ceph.com/issues/10400 Fixes: #10400

Signed-off-by: Loic Dachary <ldachary@redhat.com>
(cherry picked from commit 7f1e510165)
2014-12-23 12:52:46 -08:00
Sage Weil
203d549ddc Merge pull request #3241 from ceph/wip-10415
libcephfs/test.cc: close fd before umount

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-23 08:17:34 -08:00
Sage Weil
678a5cf78b Merge pull request #3245 from trociny/fix-10132
10132: osd: tries to set ioprio when the config option is blank

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-23 08:13:35 -08:00
Haomai Wang
ee6529b47d AsyncMessenger: Fix leak memory
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 21:09:09 +08:00
Mykola Golub
b41a739f68 10132: osd: tries to set ioprio when the config option is blank
According to documentation, ioprio params will only be used if both
osd disk thread ioprio class and osd disk thread ioprio priority are
set to a non default value.

So, add a proper check and do not generate "set_disk_tp_priority(22)
Invalid argument" warning for the default settings.
2014-12-23 13:39:33 +02:00
Haomai Wang
9aa65aa9aa Makefile: Restore check
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 19:22:10 +08:00
Haomai Wang
69dbfce5f3 AsyncConnection: Init peer_global_seq
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 19:22:10 +08:00
Haomai Wang
917ed9665b test_msgr: Avoid deadlock between send_message and dispatch
If connection holds Connection's lock and try to acquire
FakeDispatcher's lock while gtest thread try to send_message with
FakeDispatcher's lock and try to acquire Connection's lock,
it will be deadlock.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 19:22:09 +08:00
Haomai Wang
1cb78eeb73 AsyncMessenger: Make learn_addr async to avoid destroying lock rule
Make learn_addr become a async op, otherwise holding connection's
lock then acquire messenger's lock will destroy lock rule.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 19:22:09 +08:00
Haomai Wang
1c0af36c0f AsyncConnection:: Avoid getting stopping connection
Now AsyncConnection::_stop may consume a little time on deleting time events,
it may occur that accepting a connection get this stopping connection
because unregister call isn't met.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 19:22:09 +08:00
Haomai Wang
6b01b5e06f AsyncConnection: Ensure reply connect_seq larger than sent
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 19:22:09 +08:00
Loic Dachary
bb01f9e930 Merge pull request #3243 from yuyuyu101/wip-10424
librados: only call watch_flush if necessary

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-23 12:12:12 +01:00
Haomai Wang
926a1b7468 librados: only call watch_flush if necessary
Fix bug #10424
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-12-23 18:57:34 +08:00
Yan, Zheng
d3fb563cee libcephfs/test.cc: close fd before umount
Fixes: #10415
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2014-12-23 10:22:00 +08:00
Sage Weil
b95c73eed6 librados: warn about rados_watch_flush() prior to ioctx shutdown
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 1fbe9b6f3a)
2014-12-22 17:41:37 -08:00
Sage Weil
93825bf05a librados: watch_flush() on shutdown
Users can easily forget this. It makes shutdown potentially block, but if
they have racing callbacks they get what they ask for.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 30678f6daf)
2014-12-22 17:41:32 -08:00
Sage Weil
7de1b4d4ff librados: add rados_watch_flush() call
Add a call so that callers can make sure all queued callbacks have
completed before shutting down the ioctx.  This avoids a segv triggered
by the LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1
test due to the ioctx being destroyed when the in-progress callback
does a notify_ack.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4ebd4b4280)
2014-12-22 17:41:28 -08:00
Sage Weil
5cf4483cc4 osdc/Objecter: do notify completion callback in fast-dispatch context
The notify completion has exactly one user, the librados caller which
does nothing but take a local (inner) lock and signal a Cond.  Do this
in the fast-dispatch context for simplicity.

Notably, this makes the notify completion (and timeout) trigger a
notify2() return (with ETIMEDOUT) even when the finisher queue that
normally delivers notify is busy.. for example with a notify that is
being very slow.  In our case, the unit test is doing a sleep(3) to
test timeouts but also prevented the ETIMEDOUT notification from
being delivered to the caller.  This patch resolves that.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9b78dafd4a)
2014-12-22 17:41:21 -08:00
Loic Dachary
e1384afd5a Merge pull request #3200 from boydc2014/master
fix unmatched op code

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-23 00:56:17 +01:00
Sage Weil
3b98b8301f Merge pull request #3222 from ceph/wip-watch-notify-2
librados: fix watch/notify event flushing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2014-12-22 13:57:52 -08:00
Sage Weil
1fbe9b6f3a librados: warn about rados_watch_flush() prior to ioctx shutdown
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-22 13:56:07 -08:00
Samuel Just
a3c29f44c1 Merge pull request #3114 from xiaoxichen/drop_incorrect_pool
Cleanup: Drop hobject_incorrect_pool

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-22 11:41:16 -08:00
Samuel Just
ece079de58 Merge pull request #3138 from ceph/wip-10029
SimpleMessenger: Retry binding on addresses if binding fails

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2014-12-22 11:40:22 -08:00
Samuel Just
f76e0e4598 Merge pull request #3229 from ceph/wip-osd-ctor
osd/ReplicatedPG: initialize new_backfill in ctor

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-22 11:39:16 -08:00
John Spray
2fbad1e46c Merge pull request #3239 from ceph/wip-stop.sh-me
stop.sh: killall -u takes username, not uid

Reviewed-by: John Spray <john.spray@redhat.com>
2014-12-22 19:23:41 +00:00
Dan Mick
4ce6d25211 stop.sh: killall -u takes username, not uid
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2014-12-22 11:18:13 -08:00
Samuel Just
bc4e269b61 Merge pull request #3183 from ceph/wip-10262
osd: fix handling of create+delete transaction

Reviewed-by: Samuel Just <sjust@redhat.com>
2014-12-22 10:16:58 -08:00
Gregory Farnum
547ee783d4 Merge pull request #3238 from ceph/wip-10414
client: fix uninitialized member

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2014-12-22 10:00:18 -08:00
John Spray
dab5391807 client: fix uninitialized member
Fixes: #10414

Signed-off-by: John Spray <john.spray@redhat.com>
2014-12-22 17:57:57 +00:00
Sage Weil
59305ddcd3 Merge pull request #3237 from dachary/wip-10408-something-is-better-than-nothing
osd: be_compare_scrubmaps uses incorrect j iterator

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-22 08:21:42 -08:00
Sage Weil
10e6e2227a Merge pull request #3236 from ceph/wip-10409
osd: fix 10409 (object checksums for ec backends)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2014-12-22 08:21:06 -08:00
Loic Dachary
3f3f2fa172 osd: be_compare_scrubmaps uses incorrect j iterator
The code moved from be_select_auth_object to be_compare_scrubmaps
74bd8708df but the j iterator is use
differently although it has the same type. Use map.begin() as a
fallback instead.

http://tracker.ceph.com/issues/10408 Fixes: #10408

Signed-off-by: Loic Dachary <ldachary@dachary.org>
2014-12-22 17:05:50 +01:00
Sage Weil
d87918a6a6 osd: scrub: only assume shard digest == oi digest for replicated pools
For an EC object, the digest we get from scrub is for the *shard*, and that
is not the same as the *object* digest in the object_info_t.  Skip these
checks; we already have the per-shard digest that is verified in the EC
backend.

Fixes: #10409
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-22 07:32:36 -08:00
Sage Weil
a25429c9bc osd: clean up use of hex for digests
Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-22 07:27:10 -08:00
Sage Weil
9c96fbb68d osd/ECBackend: use correct seed for (empty) omap digest
This will be 0 on mixed OSD version PGs, -1 on current PGs.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-22 07:12:44 -08:00
Sage Weil
2bfcf74d76 Merge branch 'wip-10409-removed-object' of git://github.com/dachary/ceph 2014-12-22 06:58:19 -08:00
Sage Weil
20a4c57c4b Merge remote-tracking branch 'gh/next'
Conflicts:
	PendingReleaseNotes
2014-12-22 06:51:14 -08:00
Sage Weil
4047f2ddef Merge pull request #3230 from ceph/wip-pg-stat
mon/PGMap: restructure 'pg stat' formatted output

Reviewed-by: John Spray <jspray@redhat.com>
2014-12-22 06:46:21 -08:00
Sage Weil
7f9c03d1bf mon/PGMap: restructure 'pg stat' formatted output
The + character, which appears in state names, is not a valid XML token.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-12-22 06:41:25 -08:00
Sage Weil
97c8156916 Merge pull request #3235 from dachary/wip-10410-port-7110
tests: use port 7111 for osd-copy-from.sh

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-22 06:39:20 -08:00
Sage Weil
864509ff5f Merge pull request #3234 from majianpeng/bug-fix
objectstore: add fadvise_flags operations in ObjectStore::Transaction::append func.

Reviewed-by: Sage Weil <sage@redhat.com>
2014-12-22 06:38:30 -08:00
Lei Dong
e89bafb550 remove unmatched op code comparasion
Signed-off-by: Dong Lei <leidong@yahoo-inc.com>
2014-12-22 21:33:34 +08:00
Loic Dachary
4bb6e29054 tests: use port 7111 for osd-copy-from.sh
Because port 7110 is already used by mkfs.sh

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2014-12-22 14:20:58 +01:00
Loic Dachary
6f8aad0514 tests: recovery of a lost object in erasure coded pools
http://tracker.ceph.com/issues/10409 Refs: #10409

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2014-12-22 13:56:24 +01:00