Commit Graph

35830 Commits

Author SHA1 Message Date
John Spray
386f2d7c82 mds: update segment references during journal rewrite
... to avoid leaving log events that reference log
segments by offsets which no longer exist.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-09-12 23:05:38 +01:00
Loic Dachary
62ee5fc1f4 Merge pull request #2464 from dachary/wip-9429-bench
erasure-code: fix erasure_code_benchmark goop

Reviewed-by: Janne Grunau <j@jannau.net>
2014-09-12 11:30:31 +02:00
Sage Weil
e01e0c274e Merge pull request #2416 from xiaoxichen/make_crush_private
Change CrushWrapper::crush to private

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-11 20:41:07 -07:00
Sage Weil
69d3594f9e Merge pull request #2450 from dachary/wip-9413-erasure-code-version-check
erasure-code: mon, osd etc. depend on the plugins
2014-09-11 19:56:04 -07:00
Loic Dachary
b7bdb93ff7 erasure-code: fix erasure_code_benchmark goop
Using a stringstream that is only displayed on error when calling the
erasure code factory, instead of cerr. The user expects the output to be
clean when there is no error.

http://tracker.ceph.com/issues/9429 Fixes: #9429

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-11 22:24:53 +02:00
Loic Dachary
60d65daa96 Merge pull request #2409 from apeters1971/wip-ec-isa-table-cache-refac-master
EC-ISA: add intelligent table cache

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-09-11 21:56:19 +02:00
John Wilkins
2fbe82dfeb doc: Fixed syntax error.
Signed-off-by: John Wilkins <jowilki@redhat.com>
2014-09-11 10:50:42 -07:00
John Wilkins
7dbf750fc5 doc: Updated authentication notes. Fixed syntax error.
Signed-off-by: John Wilkins <jowilki@redhat.com>
2014-09-11 10:50:22 -07:00
Loic Dachary
9e6d05afb4 Merge pull request #2459 from ceph/wip-7934
test: Fix ceph_test_rados_watch_notify to delete the pools it creates

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-09-11 19:33:46 +02:00
John Spray
dcc608d5d3 Merge pull request #2463 from ceph/wip-mds-beacon
mds: a couple fixes for the beacons

Reviewed-by: John Spray <john.spray@redhat.com>
2014-09-11 16:45:21 +01:00
Sage Weil
7281638d4a mds: sleep in progress thread if laggy and waiting_for_nolaggy waiters
If we have nolaggy waiters but are laggy we want to sleep.  Otherwise,
we will just spin and spam the log ...

Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-11 08:35:49 -07:00
Sage Weil
6fb5769aee mds/Beacon: do not reconnect to mon in quick succession
Wait at least one beacon interval between mon session resets.

Fixes: #9428
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-11 08:35:49 -07:00
Yan, Zheng
c930624163 Merge pull request #2460 from ceph/wip-client-ll-ref
client: include ll_ref when printing inode
2014-09-11 17:06:39 +08:00
Yan, Zheng
4ad5db0050 client: include ll_ref when printing inode
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2014-09-11 17:03:55 +08:00
Sage Weil
2235aa990a Merge pull request #2444 from wonzhq/read-recency
osd: set min_read_recency_for_promote to default 1 when doing upgrade

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-10 20:44:13 -07:00
Sage Weil
59f1a49b3e Merge pull request #2449 from majianpeng/fix3
fix two bugs about perfcounter

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-10 20:37:03 -07:00
David Zafman
cf70b906ae test: Fix ceph_test_rados_watch_notify to delete the pools it creates
Fixes: #7934

Signed-off-by: David Zafman <dzafman@redhat.com>
2014-09-10 19:19:08 -07:00
Ma Jianpeng
2e4e98bd08 ReplicatedPG: Make perfcounter record the read-size for
async-read.

Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
2014-09-11 08:32:06 +08:00
Ma Jianpeng
cf34e00d49 ReplicatedPG: record correctly subop for perfcounter.
In log_subop_stats, it omit to record the counter of subop.

Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
2014-09-11 08:09:47 +08:00
Sage Weil
e50c426bef Merge pull request #2454 from athanatos/wip-9269
FileStore: report l_os_j_lat as commit latency

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-10 12:15:19 -07:00
Sage Weil
6846811ed3 Merge pull request #2453 from athanatos/wip-9220
Objecter::_recalc_linger_op: resend for any acting set change

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-10 12:09:34 -07:00
Samuel Just
dc5e98d3e4 Merge pull request #2443 from ceph/wip-9241
osdc/Objecter: drop bad session nref assert

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-09-10 12:09:04 -07:00
Samuel Just
d0feeaa150 Merge pull request #2293 from ceph/wip-hitset-bytes
osd: improve agent calculation by factoring out hit_set bytes used properly

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-09-10 12:02:56 -07:00
Loic Dachary
117de007ef erasure-code: mon, osd etc. depend on the plugins
Since the erasure code plugin version check has been introduced,
whenever a library/binary that can load plugin needs to be recompiled,
the erasure code plugins must also be considered. If the reason for
recompiling the library/binary is a new commit, the plugins will fail to
load.

The dependency is not based on source compilation and a shared library
dependency on liberasure-code.la is added instead. This library is
uniformly used whenever a plugin is to be loaded and therefore covers
all library/binaries that need it.

http://tracker.ceph.com/issues/9413 Fixes: #9413

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-10 20:49:00 +02:00
Sage Weil
c6678eb57a Merge pull request #2451 from ceph/wip-osdc-leak
osdc/Objecter: fix leak of MStatfsReply

Reviewed-by: John Spray <john.spray@redhat.com>
2014-09-10 11:31:27 -07:00
Yehuda Sadeh
bead44cd98 Merge pull request #2447 from reclosedev/s3_colon_in_access_key
[rgw][s3] Allow colon ':' in access key

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2014-09-10 09:59:53 -07:00
Roman Haritonov
5a05e6b8b9 [rgw][s3] Allow colon ':' in access key
When access key contains ':', e.g. `some_info:for_user',
authorization header looks like:

"AWS some_info:for_user:request_signature"

so `auth_str.find(':')` result in auth_id = "some_info",
auth_sign = "for_user:request_signature".

auth_str.rfind(':') solves this issue.

Signed-off-by: Roman Haritonov <reclosedev@gmail.com>
2014-09-10 20:45:52 +04:00
Sage Weil
46bbe30e68 osdc/Objecter: fix leak of MStatfsReply
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-10 06:57:12 -07:00
John Spray
136c5e1104 Merge pull request #2448 from ceph/wip-replay-locking
mds: fix replay locking

Reviewed-by: John Spray <john.spray@redhat.com>
2014-09-10 14:56:31 +01:00
Andreas-Joachim Peters
1ff94ccf89 EC-ISA: avoid usage of table cache lock outside the class implementation by introducing the setEncodingTable/setEncodingCoefficient methods 2014-09-10 11:25:05 +02:00
Andreas-Joachim Peters
5b41dacfd9 EC-ISA: add intelligent table cache 2014-09-10 11:25:05 +02:00
Yan, Zheng
83bd3430e3 mds: fix replay locking
When replaying EImportFinish/EFragment event, the replay thread may call
MDS::queue_waiters. MDS::queue_waiters() requires its caller to hold the
mds_lock. Otherwise assert(waiter_mutex == __null || waiter_mutex->is_locked())
in Cond::Signal() will be tiggered.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2014-09-10 16:48:47 +08:00
Zhiqiang Wang
b40cce7ad0 osd: set min_read_recency_for_promote to default 1 when doing upgrade
When upgrading from a build without the promotion on 2nd read feature,
should set min_read_recency_for_promote to the default value 1, instead
of 0.

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
2014-09-10 11:58:32 +08:00
Xiaoxi Chen
d734600f92 Change CrushWrapper::crush to private
Currently in CrushWrapper, the member "struct crush_map *crush"  is a public member,
so people can break the encapsulation and manipulate directly to the crush structure.

This is not a good practice for encapsulation and will lead to inconsistent if code
mix use the CrushWrapper API and crush C API.A simple example could be:
1.some code use crush_add_rule(C-API) to add a rule, which will not set the have_rmap flag to false in CrushWrapper
2.another code using CrushWrapper trying to look up the newly added rule by name will get a -ENOENT.

This patch move CrushWrapper::crush to private, together with three reverse map(type_rmap, name_rmap, rule_name_rmap)
and also change codes accessing the CrushWrapper::crush to make it compile.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
2014-09-10 09:08:22 +08:00
Sage Weil
5a784cd3eb osdc/Objecter: drop bad session nref assert
This is a bad assert.  Specifically, handle_osd_op_reply may still be
holding the session ref while it is calling the completion for a previous
request.  This is safe: it is only holding the session ref after it dropped
the global map rwlock because of the per-session completion locks.  The
request in question was already marked completed by the time our thread
took the session lock.

Fixes: #9241
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-09 17:28:54 -07:00
Josh Durgin
e11f0617b1 Merge pull request #2433 from ceph/wip-rbd-force-write-back
rbd should use write-back when caching is enabled

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-09-09 16:40:07 -07:00
Sage Weil
3c6e8884df osd/ClassHandler: fix build
Broken by 70ce400a8b.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-09 14:45:28 -07:00
Samuel Just
d165238b4e FileStore: report l_os_j_lat as commit latency
l_os_commit_lat is actually the commit cycle latency.

Fixes: #9269
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-09-09 14:07:17 -07:00
Samuel Just
1bce730be2 Merge pull request #2441 from ceph/wip-9365
osd/ClassHandler: improve error logging

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-09-09 13:53:11 -07:00
Sage Weil
70ce400a8b osd/ClassHandler: improve error logging
Fixes: #9365
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-09 13:38:49 -07:00
Sage Weil
1388f35bf8 Merge pull request #2412 from dachary/wip-9370-flush-logs
tests: flush logs before grepping them
2014-09-09 13:36:53 -07:00
Sage Weil
0c24bff6cd Merge pull request #2434 from dachary/wip-9381-erasure-code-rpm
packaging: add to RPM packages isa and lrc

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-09 13:14:42 -07:00
Gregory Farnum
0d66922cfb Merge pull request #2427 from ceph/wip-9362
librados: do not write to user buffer after timeout

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-09-09 13:11:04 -07:00
Samuel Just
4597401ec2 Merge pull request #2437 from athanatos/wip-9339
ReplicatedPG: create max hitset size

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-09 13:10:29 -07:00
Sage Weil
2305b2897a osdc/Objecter: revoke rx_buffer on op_cancel
If we cancel a read, revoke the rx buffers to avoid a use-after-free and/or
other undefined badness by using user buffers that may no longer be
present.

Fixes: #9362
Backport: firefly, dumpling
Reported-by: Matthias Kiefer <matthias.kiefer@1und1.de>
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-09 13:01:41 -07:00
Sage Weil
f295c1fee4 ceph_test_rados_api_io: add read timeout test
Verify we don't receive data after a timeout.

Based on reproducer for #9362 written by
Matthias Kiefer <matthias.kiefer@1und1.de>.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-09 13:01:41 -07:00
Sage Weil
977d289055 ceph_test_rados_api_*: expose nspace
Signed-off-by: Sage Weil <sage@redhat.com>
2014-09-09 13:01:41 -07:00
Samuel Just
1349383ac4 Objecter::_recalc_linger_op: resend for any acting set change
Fixes: #9220
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-09-09 12:58:07 -07:00
Sage Weil
b477abbf50 Merge pull request #2439 from athanatos/wip-9054
Wip 9054

Reviewed-by: Sage Weil <sage@redhat.com>
2014-09-09 12:43:57 -07:00
Samuel Just
4843fd510b ReplicatedPG:start_flush send a second delete
Suppose we start with the following in the cache pool:

30:[29,21,20,15,10,4]:[22(21), 15(15,10), 4(4)]+head

The object doesn't exist at 29 or 20.

First, we flush 4 leaving the backing pool with:

3:[]+head

Then, we begin to flush 15 with a delete with snapc 4:[4] leaving the
backing pool with:

4:[4]:[4(4)]

Then, we finish flushing 15 with snapc 9:[4] with leaving the backing
pool with:

9:[4]:[4(4)]+head

Next, snaps 10 and 15 are removed causing clone 10 to be removed leaving
the cache with:

30:[29,21,20,4]:[22(21),4(4)]+head

We next begin to flush 22 by sending a delete with snapc 4(4) since
prev_snapc is 4 <---------- here is the bug

The backing pool ignores this request since 4 < 9 (ORDERSNAP) leaving it
with:

9:[4]:[4(4)]

Then, we complete flushing 22 with snapc 19:[4] leaving the backing pool
with:

19:[4]:[4(4)]+head

Then, we begin to flush head by deleting with snapc 22:[21,20,4] leaving
the backing pool with:

22[21,20,4]:[22(21,20), 4(4)]

Finally, we flush head leaving the backing pool with:

30:[29,21,20,4]:[22(21*,20*),4(4)]+head

When we go to flush clone 22, all we know is that 22 is dirty, has snaps
[21], and 4 is clean. As part of flushing 22, we need to do two things:
1) Ensure that the current head is cloned as cloneid 4 with snaps [4] by
sending a delete at snapc 4:[4].
2) Flush the data at snap sequence < 21 by sending a copyfrom with snapc
20:[20,4].

Unfortunately, it is possible that 1, 1&2, or 1 and part of the flush
process for some other now non-existent clone have already been
performed.  Because of that, between 1) and 2), we need to send
a second delete ensuring that the object does not exist at 20.

Fixes: #9054
Backport: firefly
Related: 66c7439ea0
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-09-09 12:37:41 -07:00