Commit Graph

25016 Commits

Author SHA1 Message Date
Danny Al-Gaaf
8c962b5e3f MDS.cc: remove double is_clientreplay() check in active_start()
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-27 11:05:11 +01:00
Sage Weil
3f2e71169a mon: fix crush_ops.sh tests
Make it work.  Also, make note that these aren't handled idempotently by
the mon currently.  Doh!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-26 22:47:37 -07:00
Sage Weil
ae993c47c4 Merge remote-tracking branch 'gh/next' 2013-03-26 18:19:27 -07:00
Samuel Just
76b296f01f ReplicatedPG: send entire stats on OP_BACKFILL_FINISH
Otherwise, we update the stat.stat structure, but not the
stat.invalid_stats part.  This will result in a recently
split primary propogating the invalid stats but not the
invalid marker.  Sending the whole pg_stat_t structure
also mirrors MOSDSubOp.

Fixes: #4557
Backport: bobtail
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-26 17:21:51 -07:00
Sage Weil
8eb2274011 Merge remote-tracking branch 'gh/next' 2013-03-26 17:05:48 -07:00
Joe Buck
503ed9ee49 testing: fix hadoop-internal-test
Remove now superfluous directory changes
that are causing tests to fail.
This code should have been removed when we transitioned
from running tests with Ant to using Java to run the tests.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
2013-03-26 16:08:51 -07:00
Sage Weil
75bf193764 Merge pull request #149 from ceph/wip-4530
#4530

Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-26 15:13:24 -07:00
Joe Buck
ebc3abaf6d Merge pull request #139 from ceph/wip-topo-java
Merging in Noah's branch for adding topology calls. This passes existing libcephfs, libcephfs-java and hadoop tests.
2013-03-26 15:09:19 -07:00
Sam Lang
7c1e9be0e9 client: Don't signal requests already handled
The assertion failure reported in #4530 is triggered
by the following:

1. client sends request
2. mds sends unsafe reply
3. before request gets journaled, mds is killed
4. mds restarts
5. client receives session close (from close request before restart)
6. session close does kick_requests()
7. kick_requests tries to signal caller that doesn't exist.

This fix avoids signaling a caller if the unsafe reply
has been received and the make_request() function has completed.
We do this by setting the caller_cond to null once the caller
is woken up, and only signal the caller in kick_requests if
caller_cond is non-null.  This avoids trying to resend requests
listed in mds_request but that have already received unsafe replies.
The unsafe requests are handled by resend_unsafe_requests() code,
so skipping those requests is allowable.

Fixes #4530.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-26 17:08:49 -05:00
Sage Weil
b8160018bd Merge pull request #144 from dalgaaf/wip-da-ceph-disk
Fix some issues in ceph-dsk

Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-26 12:06:41 -07:00
Sage Weil
1baf66b9ee Merge pull request #143 from ceph/wip-mds-health
improve mds health checks

Reviewed-by: Sam Lang <sam.lang@inktank.com>
2013-03-26 11:44:29 -07:00
Gary Lowell
9eda8e5d5a ceph-disk: udevadm settle before partprobe
After changing the partition table, allow the udev event to be
processed before calling partprobe.  This helps prevent partprobe
from getting a resource busy error on some platforms.

Signed-off-by: Gary Lowell  <gary.lowell@inktank.com>
2013-03-26 11:31:16 -07:00
Sage Weil
23faa9f050 Merge pull request #147 from ceph/wip-4537
mds: CInode::build_backtrace() always incr iter

Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-26 09:29:42 -07:00
Sam Lang
14cef276f1 mds: CInode::build_backtrace() always incr iter
Always increment the iterator when adding old pools
to the backtrace.  This fixes a bug on files where
the layout had been set to a different pool and then
back to the same pool, causing continuous looping in
the build_backtrace() function.

Fixes #4537.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-26 11:14:52 -05:00
Noah Watkins
bb68a20fb4 java: fix test name typo
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-03-26 09:06:14 -07:00
Sage Weil
67c696dfac Merge pull request #145 from dalgaaf/wip-da-c_str
CrushWrapper.cc: remove some std::string::c_str() calls

Reviewed-by: Sage Weil <sage@inktank.com>
2013-03-26 08:15:53 -07:00
Sam Lang
b9141e9493 client: Cleanup request signaling
Split up the conditionals handling unsafe reply
and signaling the caller to improve readability.
The overall behavior of the code remains the same.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-26 09:25:19 -05:00
Sam Lang
3b520b5173 client: Handle duplicate safe replies
If the mds sends a duplicate safe reply, the mds_requests
map won't contain a matching request id (tid).  Instead of
assert failing, we log a message that we saw a reply without
a matching request.

Also remove redundant mds_requests->erase(tid) line.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-26 09:25:18 -05:00
Sam Lang
dfb46b9a32 client: Always cleanup request after safe
The client MetaRequest should always be cleaned up
and removed from the mds_requests map once the client
gets a safe reply.  This patch avoids a leak where the
mds does not send back an unsafe reply and the request
is never cleaned up.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-26 09:25:18 -05:00
Sam Lang
4dda138b07 client: Remove got_safe from MetaRequest
Once a safe reply is received, we remove the
request from the mds_requests map, so checking that
it might be a duplicate won't succeed.  This patch
removes the got_safe checks in the reply handling code
and the got_safe field on the MetaRequest to avoid confusion.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-26 09:25:18 -05:00
Danny Al-Gaaf
f957e72345 CrushWrapper.cc: remove some std::string::c_str() calls
Passing the result of c_str() to a function that takes
std::string as argument is slow and redundant.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-26 12:46:46 +01:00
Sage Weil
ffe8235f34 Merge remote-tracking branch 'gh/wip-crush'
The non-crush bits
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-03-25 16:29:56 -07:00
Sage Weil
7f65c5129a doc/release-notes: extra note for v0.56.4
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-25 16:24:48 -07:00
Sage Weil
82b129ff0d doc/release-notes: v0.56.4
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-25 16:09:24 -07:00
David Zafman
491795e234 Improve test by getting cloneid from my_snaps vector
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sam Just <sam.just@inktank.com>
2013-03-25 11:42:35 -07:00
Yehuda Sadeh
70e0ee8ba9 rgw: bucket index ops on system buckets shouldn't do anything
Fixes: #4508
Backport: bobtail
On certain bucket index operations we didn't check whether
the bucket was a system bucket, which caused the operations
to fail. This triggered an error message on bucket removal
operations.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-03-25 09:55:35 -07:00
Danny Al-Gaaf
b7d7e6894c ceph-disk: rename some local variabels in list_*partitions
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-25 17:45:32 +01:00
Danny Al-Gaaf
0b5fcfffe6 ceph-disk: fix naming of a local variable in find_cluster_by_uuid
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-25 16:24:00 +01:00
Danny Al-Gaaf
8a999ded08 ceph-disk: rename some constants to upper case variable names
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-25 16:18:17 +01:00
Danny Al-Gaaf
86e55f5448 ceph-disk: add some more docstrings
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-25 16:15:29 +01:00
Danny Al-Gaaf
543327b1f2 ceph-disk: print subprocess.CalledProcessError on error
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-25 14:36:41 +01:00
Danny Al-Gaaf
2d26bcc071 ceph-disk: fix indention
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-03-25 13:55:56 +01:00
Noah Watkins
9db05a2541 java: pretty print Ceph extent
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-03-24 13:03:56 -07:00
Noah Watkins
486e5b9927 java: support ceph_get_osd_addr
Adds a few JNI utilities from the Android project (license: Apache) to
help with IP address conversions. These functions are also updated to
work in our environment (use Ceph exception utilities, edit header
paths).

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-03-24 10:51:06 -07:00
Noah Watkins
bc65ace09f java: support ceph_get_osd_crush_location
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-03-24 10:51:06 -07:00
Noah Watkins
60fe1367fe java: support ceph_get_file_extent_osds
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-03-24 10:51:06 -07:00
Sage Weil
8befbca77a Merge branch 'next' 2013-03-23 17:31:29 -07:00
Sam Lang
ece4348807 client: don't set other if lookup fails in rename
On rename, only set the other inode if the
lookup for the destination succeeds, otherwise we hit
a segv in set_other_inode().

Fixes #4517.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
Tested-by: Noah Watkins <jayhawk@cs.ucsc.edu>
2013-03-23 17:30:53 -07:00
Sage Weil
84738791fe Merge branch 'next' 2013-03-23 11:03:58 -07:00
Sam Lang
836b97f386 test/libcephfs: Test rename error cases
Make sure that rename fails with the ENOENT
if the source path doesn't exist.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-23 11:03:55 -07:00
Sam Lang
8e6a970c45 client: Fix rename returning ENOENT for dest
Introduced by fc80c1dc6e,
the client should _not_ fail if the lookup for the
destination path on rename returns ENOENT.

The previous code also did not check that the lookup
returned ENOENT or success.  We add the check and fail
if we get any other errors.

Fixes #4517.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-23 11:01:50 -07:00
Sage Weil
3f5f432617 MDSMap: improve health check
Note if the cluster is degraded.  If so, indicate specifically which MDSs
are degraded and what state they are in.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 21:04:43 -07:00
Sage Weil
e77cd59b4e MDSMap:: constify a bunch of methods
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 18:22:21 -07:00
Sage Weil
838f1cde94 preserve /var/lib/ceph on deb/rpm purge
We should clobber configuration and log data, but *not* user data.  Leave
/var/lib/ceph alone.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Gary Lowell <gary.lowell@inktank.com>
2013-03-22 15:24:51 -07:00
Sage Weil
4f2051c4a1 mon: factor out _get_pending_crush() helper
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 15:15:38 -07:00
Sage Weil
eae1532ca1 mon, crush: add some tests to build a DAG via the cli
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 15:15:38 -07:00
Sage Weil
a60d7df419 crush, mon: unlink vs remove
Make an 'unlink' mode of remove that will remove a link to a bucket but
not remove the bucket itself.  This refactors remove_item[_under] and moves
some of the checks into common helpers where they are not duplicated.  Fix
callers to pass the extra arg.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 15:15:38 -07:00
Sage Weil
b54db77ede crush: fix remove_item on bucket removal
Remove the bucket if there are no references left.

Remove the name from the map even if it is a bucket (not sure why that
condition was there in the first place!).

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 15:15:38 -07:00
Sage Weil
d8f4be6e6d mon: add 'osd crush add-bucket <name> <type>'
This is (I think) the last missing piece to let you construct an entire
map via the CLI.  The add/set commands will construct intervening ancestor
nodes provide there is an existing ancestor to stick them under, but this
is needed to create the initial root node.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 15:15:38 -07:00
Sage Weil
96e08ef1d4 mon: allow removal of buckets via 'osd crush rm ...'
No reason to limit this to leaves.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-22 15:15:37 -07:00