Commit Graph

36713 Commits

Author SHA1 Message Date
Adam Crume
65be257e92 Fix read performance regression in ObjectCacher
The regression was introduced in commit
4fc9fffc49.  The problem is that the cache
thinks it's full (when it's not), so it defers the read.  This change
frees up cache space if necessary and only defers the read if enough
space cannot be freed.

Fixes: 9513
Signed-off-by: Adam Crume <adamcrume@gmail.com>
(cherry picked from commit 82175ec94a)
2014-10-20 13:49:04 -07:00
Loic Dachary
349eb518ba tests: mon id is incorrectly skipped
The argument is shifted and $1 should be used to get the mon id instead
of $2.

http://tracker.ceph.com/issues/9836 Fixes: #9836

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-20 11:20:35 -07:00
Sage Weil
27d5ae61b1 Merge pull request #2758 from ceph/wip-9820
qa/workunits: cephtool: don't remove self's key on auth tests

Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-20 10:46:48 -07:00
Joao Eduardo Luis
ac3c1cb5d0 qa/workunits: cephtool: don't remove self's key on auth tests
Suites run with CEPH_TEST_CLI_DUP_COMMAND=1, which will send a duplicate
command for every command issued with the 'ceph' tool.  Behavior is to
get a reply from the command and then send a duplicate, looking for the
same outcome (guaranteeing idempotency of the operations).  However, it
so happens that if you remove the entity's own key from the keyring and
you happen to be unlucky enough so that the client's connection gets
failed (we also run tests with connection failure injections), the
'ceph' tool won't be able to reconnect to the cluster to send the
duplicate command (as it's entity no longer exists in the cluster's
keyring).

We rewrite the test instead of resorting to ugly hacks to work around
this behavior, simply having a new 'role-definer' added by the existing
'role-definer' (which we weren't testing anyway, so bonus points for
that) and then have one removing the other (to test the procedure) and
finally using 'client.admin' to remove the last 'role-definer'.

Fixes: #9820

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-20 18:00:15 +01:00
Sebastien Ponce
98d7760020 Added missing semaphore in libradosstriper AIO tests leading to potential memory corruption and thus test failures
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-10-20 18:46:31 +02:00
Sebastien Ponce
f26cd1b037 Fixed remaining part of the seg fault described in bug 9356 by adding reference counting to CompletionData
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-10-20 18:46:31 +02:00
Sebastien Ponce
783fc66a00 Fixed part of the seg fault described in bug 9356 by adding reference counting on RadosReadCompletionData
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-10-20 18:46:31 +02:00
Sebastien Ponce
52c97b0bea Fixed part of the seg fault described in bug 9356 by cleaning reference counting on MultiAioCompletionImpl
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-10-20 18:46:31 +02:00
Sebastien Ponce
5299db341b Cleanup up useless method release in MultiAioCompletionImpl. It was exactly equivalent to put
Signed-off-by: Sebastien Ponce <sebastien.ponce@cern.ch>
2014-10-20 18:46:31 +02:00
Sage Weil
dbe11d1e9b Merge pull request #2757 from fgimenez/update-pool-names-in-dev-guide
doc: updated names of default pools created at startup
2014-10-20 08:34:45 -07:00
fgimenez
777fe211b0 Updated names of default pools created at startup 2014-10-20 13:43:56 +02:00
John Spray
680b2e09ad Merge pull request #2751 from dachary/wip-warnings
client: fix signed/unsigned compilation warnings

Reviewed-by: John Spray <john.spray@redhat.com>
2014-10-20 09:42:37 +01:00
Haomai Wang
ea100ac715 KeyValueStore: Add clone op detect to promote error
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-10-20 13:26:20 +08:00
Loic Dachary
21dc6af60b mailmap: Ilya Dryomov transition from Inktank to Red Hat
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-19 17:40:15 -07:00
Loic Dachary
1037193239 mailmap: Greg Farnum name normalization
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-19 17:40:14 -07:00
Loic Dachary
f2074164a0 mailmap: Cheng Cheng name normalization
Reviewed-by: Cheng Cheng <ccheng.leo@gmail.com>
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-19 17:39:50 -07:00
Loic Dachary
64d977b947 client: fix signed/unsigned compilation warnings
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-19 09:04:02 -07:00
Loic Dachary
4fc12b66d7 Merge pull request #2744 from ceph/wip-9794
mon: MDSMonitor: wait for osdmon to be writable when requesting proposal

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-10-18 08:43:40 -07:00
Loic Dachary
e5fc8f7f22 Merge pull request #2746 from ceph/wip-adamcrume
osdc: Fix compiler warning

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Loic Dachary <loic-201408@dachary.org>
2014-10-17 15:13:28 -07:00
Adam Crume
cb290a117a osdc: Fix compiler warning
Was getting:

./osdc/ObjectCacher.h:169:51: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       uint64_t overlap_start = MAX(start, ex.start);
                                                   ^
./osdc/ObjectCacher.h:170:59: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       uint64_t overlap_end = MIN(end, ex.start + ex.length);
                                                           ^

Signed-off-by: Adam Crume <adamcrume@gmail.com>
2014-10-17 12:09:44 -07:00
Joao Eduardo Luis
2ae1cba595 mon: MDSMonitor: wait for osdmon to be writable when requesting proposal
Otherwise we may end up requesting the osdmon to propose while it is
mid-proposal.  We can't simply return EAGAIN to the user either because
then we would have to expect the user to be able to successfully race
with the whole cluster in finding a window in which 'mds fs new' command
would succeed -- which is not a realistic expectation.  Having the
command to osdmon()->wait_for_writable() guarantees that the command
will be added to a queue and that we will, eventually, tend to it.

Fixes: #9794

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-17 19:08:20 +01:00
Joao Eduardo Luis
0dd473cbad mon: MDSMonitor: have management_command() returning int instead of bool
We can more easily differentiate between errors we get out of this
function, which makes the code a bit more versatile and readable.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-17 18:59:51 +01:00
Samuel Just
22877ba735 Merge pull request #2708 from ceph/wip-9718
osd/osd_types: consider CRUSH_ITEM_NONE in check_new_interval() min_size

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-10-17 10:38:43 -07:00
Samuel Just
234f5a2347 Merge pull request #2711 from guangyy/wip-9614-followup
Follow-up fix for 9614

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-10-17 10:38:30 -07:00
Joao Eduardo Luis
91b2a2b69b mon: MDSMonitor: don't return -EINVAL if function is bool
Returning -EINVAL on a function that expects bool and the error code to
be in a variable 'r' can only achieve one thing: if this path is ever
touched, instead of returning an error as it was supposed to, we're
returning 'true' with 'r = 0' and, for no apparent reason, the user will
think everything went smoothly but with no new fs created.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-17 18:01:05 +01:00
Joao Eduardo Luis
1d1ae413e9 mon: MDSMonitor: check all conditions are met *before* osdmon proposal
We should not allow ourselves to request the osdmon to propose before we
know for sure that we meet the required conditions to go through with
our own state change.  Even if we still can't guarantee that our
proposal is going to be committed, we shouldn't just change the osdmon's
state just because we can.  This way, at least, we make sure that our
checks hold up before doing anything with side-effects.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-17 17:43:33 +01:00
Joao Eduardo Luis
07b7f10105 mon: MDSMonitor: return if fs exists on 'fs new'
We were just setting return code to -EINVAL, while allowing the logic to
continue regardless.  If we are to return error, then we should abort
the operation as well and let the user know it went wrong instead of
continuing as if nothing had happened.

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
2014-10-17 17:37:03 +01:00
Loic Dachary
66cffd72f5 Merge pull request #2736 from dachary/wip-backfill-documentation
doc: updates on Backfill Reservation

Reviewed-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2014-10-17 09:02:10 -07:00
Sage Weil
fdf534dd89 Merge pull request #2740 from ceph/giant-unknown-locktype
mds: reply -EOPNOTSUPP for unknown lock type
2014-10-17 08:21:16 -07:00
Sage Weil
6fa686c8c4 Merge pull request #2743 from ceph/wip-mds-coverity
mds: fix null dereference

Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-17 08:16:18 -07:00
Sage Weil
1334be7484 Merge pull request #2738 from majianpeng/fix1
Fix1 compile warning

Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-17 08:11:45 -07:00
Yan, Zheng
2c064137d3 mds: fix null dereference
Server::handle_client_file_setlock() should return after calling
reply_request()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
2014-10-17 22:09:22 +08:00
Yan, Zheng
9e6ae73a13 mds: reply -EOPNOTSUPP for unknown lock type
Signed-off-by: Yan, Zheng <zyan@redhat.com>
(cherry picked from commit 675392335c)
2014-10-17 22:02:49 +08:00
Jianpeng Ma
bb9add6cdc test: fix compile warning in bufferlist.cc
test/bufferlist.cc: In member function ‘virtual void
Buffer_constructors_Test::TestBody()’:
test/bufferlist.cc:154:36: warning: ignoring return value of ‘int
system(const char*)’, declared with attribute warn_unused_result
[-Wunused-result]
     ::system("echo ABC > testfile");
                                    ^
test/bufferlist.cc: In member function ‘virtual void
TestRawPipe::SetUp()’:
test/bufferlist.cc:182:36: warning: ignoring return value of ‘int
system(const char*)’, declared with attribute warn_unused_result
[-Wunused-result]
     ::system("echo ABC > testfile");
                                    ^
test/bufferlist.cc: In member function ‘virtual void
BufferList_read_file_Test::TestBody()’:
test/bufferlist.cc:1768:53: warning: ignoring return value of ‘int
system(const char*)’, declared with attribute warn_unused_result
[-Wunused-result]
   ::system("echo ABC > testfile ; chmod 0 testfile");
                                                     ^
test/bufferlist.cc:1770:32: warning: ignoring return value of ‘int
system(const char*)’, declared with attribute warn_unused_result
[-Wunused-result]
   ::system("chmod +r testfile");
                                ^
test/bufferlist.cc: In member function ‘virtual void
BufferList_read_fd_Test::TestBody()’:
test/bufferlist.cc:1781:34: warning: ignoring return value of ‘int
system(const char*)’, declared with attribute warn_unused_result
[-Wunused-result]
   ::system("echo ABC > testfile");

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2014-10-17 14:04:40 +08:00
Jianpeng Ma
a49255f34c librbd: fix compile warning in librbd/internal.cc.
librbd/internal.cc: In function 'void
librbd::readahead(librbd::ImageCtx*, const std::vector<std::pair<long
unsigned int, long unsigned int> >&, const md_config_t*)':
librbd/internal.cc:3150:38: warning: comparison between signed and
unsigned integer expressions [-Wsign-compare]
       ictx->total_bytes_read > conf->rbd_readahead_disable_after_bytes;
                                      ^

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2014-10-17 13:19:59 +08:00
Loic Dachary
69b5c72cbe doc: updates on Backfill Reservation
The logic was changed by:

0985ae71bc
osd: prioritize backfill based on *how* degraded

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-16 18:36:11 -07:00
Josh Durgin
a9f302da08 Merge remote-tracking branch 'origin/wip-rbd-readahead'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>

Conflicts:
	src/client/Fh.h
2014-10-16 14:33:42 -07:00
Josh Durgin
0bb51db92f Merge pull request #2588 from xinxinsh/wip-9595
Fix #9595

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-10-16 13:15:33 -07:00
Gregory Farnum
ca44f503cc Merge pull request #2628 from ceph/wip-client-flock
Wip client flock

Add support for file locking to the userspace client, and improve blocked-lock cancellation so that it doesn't remove locks that succeeded when racing.

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-10-16 06:57:34 -07:00
Loic Dachary
8e7fb9056e Merge pull request #2732 from dachary/wip-9526-crush-rename-bucket
ceph osd crush rename-bucket srcname dstname

Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-15 20:22:02 -07:00
Loic Dachary
42e5c40a42 Merge pull request #2731 from dachary/wip-9790-display-auid
auth: add display of auid to auth get / list when it is not the default

Reviewed-by: Sage Weil <sage@redhat.com>
2014-10-15 20:15:29 -07:00
Loic Dachary
4606af7901 mon: unit tests for osd crush rename-bucket
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-15 17:22:49 -07:00
Loic Dachary
aa675604c9 mon: add the osd crush rename-bucket command
The synopsis is:

 osd crush rename-bucket name1 name2

It is made idempotent by interpreting -EALREADY as returned by
CrushWrapper::rename_bucket return as success.

The crush_rename_bucket method first checks for errors with
CrushWrapper::can_rename_bucket if there is no pending crush so that it
can return early and avoid the creation of a pending crush map.

If renaming is possible, CrushWrapper::rename_bucket is called on the
pending crush map (and creates it indirectly if it does not already
exists).

http://tracker.ceph.com/issues/9526 Fixes: #9526

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-15 17:22:12 -07:00
Loic Dachary
29d13d41c5 crush: unit tests for CrushWrapper rename item/bucket methods
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-15 17:13:59 -07:00
Loic Dachary
30f3ac3fb6 crush: add CrushWrapper::rename_bucket and can_rename_bucket
Rename a bucket and return -ENOTDIR if trying to rename an item. The
behavior is otherwise the same as rename_item and can_rename_item.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-15 17:13:59 -07:00
Loic Dachary
5e2a82675e crush: add CrushWrapper::rename_item and can_rename_item
The can_rename_item is a const method checking if renaming an item could
succeed. If not it returns a unique -errno code and a human readable
message message.

Trying to rename a non existent item into an existent item returns
-EALREADY which can be treated as success if renaming is to be
idempotent.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-15 17:12:36 -07:00
xinxin shu
f8107102bf add unittest for Striper::get_num_objects
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2014-10-16 08:09:52 +08:00
Loic Dachary
236895eea6 crush: improve constness of CrushWrapper methods
A number of CrushWrapper get methods or predicates were not const
because they need to maintain transparently the rmaps. Make the rmaps
mutable and update the constness of the methods to match what the caller
would expect.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-10-15 17:02:58 -07:00
xinxin shu
7b66ee4928 when non-default striping is used, internal methods can operate on extra objects
Fixes: #9595

Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2014-10-16 07:58:41 +08:00
Sage Weil
bf8fe2e3d0 Merge pull request #2712 from lpabon/bug-8766
Update vstart to setup users for s3-tests
2014-10-15 15:18:48 -07:00