Commit Graph

21956 Commits

Author SHA1 Message Date
Sage Weil
e287296584 osdc/ObjectCacher: do read completions after assimilating read result
Wait until we have applied the entire read result to the cache before we
trigger any read completion events.  This is a cleaner and safer approach
since we can be sure that the callback won't get blocked again on data we
have but haven't applied yet.  It also fixes a crash I just observed where
the completion did a read, called trim(), and invalidated/destroyed the
iterator/bh p was referencing.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:45 -07:00
Sage Weil
9407046dc7 osdc/ObjectCacher: do not close objects explicitly
Let the trimmer do that.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:45 -07:00
Sage Weil
8920f417bf osdc/ObjectCacher: make trim() trim Objects
Pull unpinned objects off the LRU in trim().  This never happens currently
due to all the explicit calls to close_object()...

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:45 -07:00
Sage Weil
f241e22f52 osdc/ObjectCacher: check lru_is_expireable() in can_close()
We assert that if can_close(), the Object isn't pinned in the LRU.  This
assumes we did yur get/put refcounting properly, such that the pins are
at least as restrictive as can_close().

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
cd8e991af8 osdc/ObjectCacher: add LRU for Object
Incomplete; we aren't trimming yet.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
46897fd4ff osdc/ObjectCacher: take Object ref for bh writes
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
21dc0e0262 osdc/ObjectCacher: take refs for inflight lock ops
These are all dead/unused; should probably just rip out this code!

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
57e18a74fe osdc/ObjectCacher: take Object ref when there are buffers
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
a34a8b8bae osdc/ObjectCacher: add ref count to Object
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
24d07e8727 osdc/ObjectCacher: rename lru_* -> bh_lru_*
We'll be adding LRUs for objects, too.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-26 11:31:44 -07:00
Sage Weil
57a4cbbfb2 librbd: fix race in AioCompletion that are still being built
When caching is enabled, it is possible for the io completion to happen
faster than we call ->finish_adding_requests() (e.g., on cache read).
When that happens, the final read request completion doesn't see a
pending_count == 0 and thus doesn't do all the final buffer construction
that is necessary to return correct data.  In particular, users will see
zeroed buffers.  test_librbd_fsx is turning this up consistently after
several thousand ops with an image size of ~100MB and cloning disabled.

This was introduced with the extra logic added here with striping.

Fix this by making a separate flag to indicate the completion is under
construction, and make sure we call complete() when both pending_count==0
and building==false.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-26 11:30:38 -07:00
Noah Watkins
e572b4b4cf Merge branch 'wip-client-unmount'
Signed-off-by: Noah Watkins <noah.watkins@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-10-26 09:07:30 -07:00
Noah Watkins
67bc92aa54 client: add ceph_release, ceph_shutdown
Notes that ceph_shutdown() is now deprecated.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2012-10-26 08:38:26 -07:00
Noah Watkins
f1eef53200 client: double mount returns -EISCONN
Change error code from -EDOM to -EISCONN when mounting an already
mounted ceph_mount_info instance.  The current convention is to return
-ENOTCONN when using the libcephfs interface in an unmounted state.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2012-10-26 08:38:26 -07:00
Sage Weil
1152656c62 Merge branch 'wip-mds'
Reviewed-by: Greg Farnum <greg@inktank.com>
2012-10-25 20:49:25 -07:00
Sage Weil
c9ca3c997c client: do not reset session state on reopened sessions
We can have a sequence one the MDS like:

 - queue REQUEST_CLOSE to journal
 - force_open, queue open to journal
 - request_close acked, do nothing
 - force_open acked, send OPEN

In this case, the MDS never actually closed the session, and all of the
state remained valid.  The client, however, gets a suprious OPEN
message and resets the session state.

Fix this by not resetting that state.

A nicer fix might be to not send the second OPEN at all, but that would
require a REOPENING state on the MDS which is more complicated; this is
good enough.  Also, that approach might not give the client an
appropriate opportunity to say "um, no..." and resend the
REQUEST_CLOSE.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-25 20:49:04 -07:00
Sage Weil
3153ec740d mds: fix handling of cache_expire export
During export, between the warning stage and the final notify, we may
get cache expire messages because the replicas are sending to both us
and the new auth.  This check should look for >= WARNING so that it
includes the EXPORTING states as well as the portion of WARNING after
we heard from that replica.  This aligns the conditional with the
following assert such that they are properly mutually exclusive.

Fixes: #1527
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-25 20:49:04 -07:00
Sage Weil
4ac45200f1 mds: do not mark closed connections disposable
These will get reused when the client reconnects.  If we are going to
clean these up, we need a different strategy.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-25 20:49:03 -07:00
Sage Weil
ad839c70dc mds: use connection on closed sessions in force_open_sessions
If the have a Connection*, use it.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-25 20:49:03 -07:00
Ross Turk
53b354dfaa doc: update copyright to include contributors, CC
Signed-off-by: Ross Turk <ross@inktank.com>
2012-10-25 17:51:04 -07:00
Sage Weil
b668ee503b Merge branch 'next' 2012-10-25 12:00:02 -07:00
Sage Weil
c0df832877 osd: fix populate_obc_watchers() assert
There is one case where populate_obc_watchers gets called when the object
is missing: during a revert.  And in that case we *should* do the populate,
since all that is getting reverted is the object version.

Fixes: #3405
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Sam Just <sam.just@inktank.com>
2012-10-25 11:59:33 -07:00
Sage Weil
3eda37aaab mds: check getcwd() return value
warning: mds/MDS.cc:1586:27: ignoring return value of ‘char* getcwd(char*, size_t)’, declared with attribute warn_unused_result [-Wunused-result]

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-24 21:34:42 -07:00
Sage Weil
1afa8ff0ba leveldb: fix accidental submodule revert
a5eb72ba01

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-24 17:04:42 -07:00
Josh Durgin
a5eb72ba01 doc: note raw format required for openstack/rbd
A couple people have been confused by this omission.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-24 16:19:21 -07:00
Sage Weil
345bcac0d0 mds: fix #include for getcwd(), simplify
Always use getcwd().. it's more portable.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-24 09:38:58 -07:00
John Wilkins
8c16a838cc doc: Added hyperlink.
fixes: #3393

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-10-24 09:27:34 -07:00
John Wilkins
da8bb88f7b doc: Fixed usage and deleted legacy values.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-10-24 09:14:12 -07:00
Sam Lang
e510677033 vstart.sh: Use ./init-ceph instead of CEPH_BIN
This effectively reverts faddb80c42
which prevented vstart.sh from being used in an enviroment where
CEPH_BIN pointed to a make install target.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-10-23 16:21:02 -05:00
tamil
1921a06fdf Script that generates perl wrapper for teuthology tests
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
2012-10-23 13:11:53 -07:00
caleb miles
18731b4bb5 rgw_admin: do not throw error when start-date and end-date are not
passed to usage::trim()

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-10-23 11:25:11 -07:00
Sage Weil
63cbb2bdd1 Merge branch 'wip-assert-exists-2'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2012-10-23 05:37:54 -07:00
Sage Weil
7c67230969 osd/osd_types: make stat osd_op print nicely
No 0~0 crap.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 21:04:30 -07:00
Sage Weil
4240846f5c librbd: use assert_exists() to simplify copyup check
Previously we would explicitly STAT the object to see if it exists before
sending the write to the OSD.  Instead, send the write optimistically, and
assert that the object already exists.  This avoids an extra round trip in
the optimistic/common case, and makes the existence check in the initial
first-write case more expensive because we send the data payload along.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 21:04:30 -07:00
Sage Weil
e609c8ac54 librados: add assert_exists guard operation
Add a guard operation for writes that asserts that the object already
exists.  To avoid requiring new functionality on the OSD side, implement
this by including a STAT operation, and discard the results on the
client side.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 21:04:30 -07:00
Dan Mick
59839a8f90 librbd: validate copy size against proper snapshot id
Fixes: #3383
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2012-10-22 20:32:23 -07:00
Sage Weil
eed28daaf8 osd: return EOPNOTSUPP on bad class or method name
Currently we return EIO, which isn't particularly informative.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 17:39:47 -07:00
Sage Weil
31260a3530 leveldb: fix accidental revert
In b85cde576e

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 17:25:03 -07:00
Yehuda Sadeh
b85cde576e rgw: dump an error message if FCGX_Accept fails
Adding missing debug info.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-22 17:07:23 -07:00
Sage Weil
d4f0a8897d workqueue: make debug output include active threads
Include active thread count in threadpool debug output.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 15:50:58 -07:00
Sage Weil
74ef52bcff Merge branch 'wip-msgr' 2012-10-22 14:47:24 -07:00
Sage Weil
4d20b60970 msg/Pipe: fix tight reconnect loop on connect failure
The fault() call in connect should not set onread=true since connect is
effectively a write path.  This was forcing the writer() into a tight
loop that repeatedly would call connect(); not very polite.

Changing that, we want to avoid treating this as a normal fault (with the
failure callback) and instead back off.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2012-10-22 14:47:01 -07:00
Sage Weil
54cab4de56 throttle: less default log noise
This tends to drown out anything interesting in the log buffer.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 14:41:49 -07:00
Dan Mick
3dffa6dd25 rbd: document defaults for rbd bench-write
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2012-10-22 13:34:23 -07:00
Sam Lang
ac07afa205 test: Dont check initial permissions
We can't check the initial permissions of the
file because the umask may be set to something
other than 0022.  The check isn't needed to check
for chmod correctness anyway.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2012-10-22 15:27:12 -05:00
Sage Weil
84a35f80e5 Makefile: include java in dist tarball
Include all the java goodness in the dist tarball.  Fixes package builds.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-22 12:51:10 -07:00
Yehuda Sadeh
3cc39f558d rgw: check client write status on swift get_obj
Fixes: #3381
We check the return code of the cio->write() operation
when doing get_obj(). This makes sure that we don't
continue processing the request if client has disconnected.
This commit complements another commit that does the same
for the specific s3 operation.

backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-22 12:41:30 -07:00
Yehuda Sadeh
a75567479c rgw: don't continue processing of GET request on error
Fixes #3381
We continued processing requests long after the client has died.

backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-22 11:01:30 -07:00
Sage Weil
f3b5254206 Merge remote-tracking branch 'gh/next' 2012-10-22 10:46:04 -07:00
Sage Weil
2248822b2c osd: drop conditional check in populate_obc_watchers
Turn these into asserts.  The only two callers are create_object_context()
and get_object_context(), and they only get called when the object is no
longer missing.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2012-10-22 10:45:36 -07:00