Commit Graph

25982 Commits

Author SHA1 Message Date
Josh Durgin
13ae13a906 librbd: add options to enable balanced or localized reads for snapshots
Since snapshots never change, it's safe to read from replicas for them.
A common use for this would be reading from a parent snapshot shared by
many clones.

Convert LibrbdWriteback and AioRead to use the ObjectOperation api
so we can set flags. Fortunately the external wrapper holds no data,
so its lifecycle doesn't need to be managed.

Include a simple workunit that sets the flags in various combinations
and looks for their presence in the logs from 'rbd export'.

Fixes: #3064
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-12 19:31:22 -07:00
Josh Durgin
0c7414b1de ReplicatedPG: send -EAGAIN for both balanced and localized reads
This logic for localized reads applies to balanced reads too.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-12 19:31:22 -07:00
Josh Durgin
4ddaea70b2 librados: add per-ObjectOperation flags for balanced and localized reads
These need to apply to the entire ObjectOperation, not just a subop,
so use a new enum and a new aio_operate() call that takes them.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-12 19:31:22 -07:00
Josh Durgin
442f0588a5 librados: add sparse_read() to the C++ bindings for an ObjectOperation
This will allow it to be used with general aio_operate() so we don't have
to add new versions of each operation when we want to add new per-op
arguments, like flags, namespaces, or explicit snapshot contexts/ids.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-12 19:31:22 -07:00
Josh Durgin
d5193460bf Objecter, librados: use only ObjectOperation form of sparse_read internally
This will be used when exposing an ObjectOperation version of sparse_read()
to the librados user, and there's no reason to duplicate code for creating
and handling it. Add a wrapper Context for handling the lifecycle of the
::ObjectOperation.

This cleans up the synchronous version of sparse_read quite a bit by
using the general operate_read() instead of duplicating decoding and
a bunch of sync boilerplate.

Move handling the decoding of a sparse_read into the Objecter, with
the rest of the decoding of rados operations. librados shouldn't be
the only user of the Objecter that can understand sparse_reads.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-12 19:31:22 -07:00
Josh Durgin
ed76824c47 Objecter: fix error handling for decoding stat
r is just a local variable, changing it has no effect.
Set the per-operation return value if provided when a decoding
error occurs.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-12 19:26:19 -07:00
Sage Weil
b5e9b56fc9 Merge pull request #272 from ceph/wip-rbd-parallel
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-10 17:13:12 -07:00
David Zafman
ea0e0c7e34 Merge branch 'wip-4273'
Reviewed-by: Sam Just <sam.just@inktank.com>
2013-05-10 16:45:34 -07:00
Josh Durgin
93f2794233 Throttle: move start_op() to C_SimpleThrottle constructor
This is done by all callers right before constructing this.
Since C_SimpleThrottle is already responsible for calling ->end_op(),
it makes sense to call start_op() there too.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 16:17:11 -07:00
Josh Durgin
613d7471a2 librbd: run copy in parallel
Instead of using read_iterate(), loop over each period of objects in
the source, read from them asynchronously, and then asynchronously
write to the destination.

The callbacks make this a bit more complex, but it can perform much
better.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 16:17:10 -07:00
Josh Durgin
fb299d3819 librbd: move completion release into rbd_ctx_cb()
All the users of rbd_ctx_cb() do this separately right now, but
there's no reason to keep the completion around after the nested
completion has been called. Also declare rbd_ctx_cb() in the header
so it can be used before its definition.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 16:17:10 -07:00
Josh Durgin
a6d0a25435 librbd: parallelize and simplify flatten
Flattening reads the logical child object from the parent image, and
then does a copyup operation if the data is non-zero. This is
equivalent to doing a zero-length write to each object in the
child image. Do this instead, so that we can easily control how
many are in flight, and eliminate some code as well.

Since we no longer read from the parent within the flatten function,
the buffer is not needed. It would be leaked in some error conditions,
but since's it's unecessary we can just get rid of it.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 16:17:10 -07:00
Josh Durgin
bfa106694d librbd: only send non-zero copyup data
If the parent image is logically zero for the range of a child object,
it's equivalent to the object not existing. Save some I/O and network
bandwidth and don't send the useless zeroes.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 16:17:10 -07:00
Josh Durgin
cfece23d5c librbd: parallelize rollback
Use a SimpleThrottle like trim_image() to limit the number of
requests in flight.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 16:17:09 -07:00
Sage Weil
7b408537c4 Merge pull request #273 from dalgaaf/wip-da-CID-fixes-v2
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-10 15:08:57 -07:00
Danny Al-Gaaf
437d69ef71 mds/CDir.cc: fix possible dereference after NULL check
CID 1019553 Dereference after null check (FORWARD_NULL, CWE-476)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
c006151c05 ceph-monstore-tool.cc: check if open() was successful
Should fix: "fd" is passed to a parameter that cannot be negative.

CID 1019566 Improper use of negative value (NEGATIVE_RETURNS, CWE-394)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
d1e0fc64e5 kv_flat_btree_async.cc: fix resource leak
Call AioCompletion::release() if the completion is no longer
needed to free the resources.

CID 727976 Resource leak (CWE-404)
CID 727977 Resource leak (CWE-404)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
f6635cb30a client/SyntheticClient.cc: check return value of describe_layout()
Fix SyntheticClient::chunk_file(): check return value of
describe_layout() and handle the error.

CID 966615 Unchecked return value (CWE-252)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
cb91f0fd6e client/Client.cc: fix possible NULL pointer dereference
CID 751332 Dereference null return value (CWE-476)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
5af2cbfea2 rados.cc: fix leaking of Formatter*
Make sure Formatter* is deleted in error case.

717096 Resource leak (CWE-404) (25 of 25 cases)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
4b3a7dcb54 client/SyntheticClient.cc: fix another memory leak
Fix memory leak in read_random: call delete[] on buf before
call new[] again in the for-loop.

CID 717071 Resource leak (CWE-404)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:56 +02:00
Danny Al-Gaaf
b9fbc8212e client/SyntheticClient.cc: fix memory leak
Fix memory leak in read_random: call delete[] on buf before
call new[] again in the for-loop.

CID 717070 Resource leak (CWE-404)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
447f318619 mds/MDCache.cc: fix dereference NULL pointer
Fix possible NULL pointer dereference. Change return value of
CInode::get_dirfrag() to return NULL instead of 0 since it's a
pointer returned.

CID 716991 Dereference null return value (CWE-478)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
d258df4da3 mds/MDCache.cc: add NULL pointer check
Check for result of get_inode() for NULL before use the pointer.

716990 Dereference null return value (CWE-476)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
5392e9709e mds/Server.cc: remove dead default in switch
The default switch can't get executed since the switch value
can only have the two values already checked.

CID 716894 Dead default in switch (CWE-561)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
f56cb98482 osd/OSD.h: add missing unlock of osd_lock
CID 1019560 Missing unlock (CWE-667)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
b097f65627 rgw/rgw_rest.cc: remove dead and unneeded code
Since origin and meth are already checked to be true there is
no need to check again in s->cio->print() after the initial check.

1019551 Logically dead code (CWE-561)
1019552 Logically dead code (CWE-561)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
5babc816e0 rbd.cc: fix error handling
Fix undead code. Get error code from write_fd() before check
the result against < 0.

CID 1019550 Logically dead code (CWE-561)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
4908079c20 rados_sync.cc: remove dead and not needed code
The first if handles all chars < 32, the last 2 if's check for
'\n' (10) and '\r' (13). This code will never be reached and
is already covered. Remove unneeded code.

CID 1019549 Logically dead code (CWE-561)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:55 +02:00
Danny Al-Gaaf
077cdb0487 test/omap_bench.cc: remove dead code
CID 716900 Logically dead code (CWE-561)
CID 716901 Logically dead code (CWE-561)
CID 727968 Logically dead code (CWE-561)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:54 +02:00
Danny Al-Gaaf
0c70e44630 libcephfs/test.cc: add assert for result of ceph_getxattr()
Check result of ceph_getxattr() before pass it as parameter to
strncmp(). Make sure it's not negative.

CID 739411 Argument cannot be negative (CWE-687)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:54 +02:00
Danny Al-Gaaf
8d614665ec rgw/rgw_user.cc: add missing break in switch
Fix switch handling for case KEY_TYPE_SWIFT, add break after the
case to prevent fall through into KEY_TYPE_S3 case.

CID 1019563 Missing break in switch (CWE-484)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:54 +02:00
Danny Al-Gaaf
d9c5b5b7ff client/SyntheticClient.cc: add missing break in switch
Fix switch handling for case SYNCLIENT_MODE_OVERLOAD_OSD_0, add break
after the case to prevent fall through into next case.

CID 716958 Missing break in switch (CWE-484)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:54 +02:00
Danny Al-Gaaf
6e241b97bb ObjectStore.cc: add missing break in switch
Fix switch handling for case OP_SPLIT_COLLECTION2, add break after
the case to prevent fall through into default case.

CID 1019562 Missing break in switch (CWE-484)

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-11 00:02:54 +02:00
John Wilkins
27fb38bb4d doc: Fixed typos. Somehow got a merge error.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-10 14:39:23 -07:00
Sage Weil
65768a6837 mds: be more explicit about path_traverse completion
Coverity turned up

CID 717085 (#1 of 1): Resource leak (RESOURCE_LEAK)
10. leaked_storage: Variable "c" going out of scope leaks the storage it points to.

from _find_ino_dir(), but for all r > 0 cases fin is consumed.  Make this
a bit more explicit by using the helper in the mdr-only case, and by
asserting it is zero in the return 2 path.  Hopefully coverity will notice.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-05-10 14:30:23 -07:00
Ross Turk
8eaa15f2a6 doc: fix broken link to ceph-deploy in release notes
Signed-off-by: Ross Turk <ross@inktank.com>
2013-05-10 13:53:21 -07:00
Josh Durgin
3b2c5fb8ab librados: add selfmanaged_snap_rollback as an ObjectOperation
This allows it to be done asynchronously, or in conjunction with
other operations.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 12:00:11 -07:00
Josh Durgin
4095641016 librbd: delete more than one object at once
Speed up deletions when resizing down or removing an image by keeping
up 10 operations in flight by default.

Refs: #2256
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 12:00:11 -07:00
Josh Durgin
537386d906 Throttle: add a simpler throttle that just blocks above a threshold
This is convenient to use to turn synchronous calls into asynchronous
calls with a limited number of operations in flight concurrently.
It assumes the caller will wait for all operations to complete at the
end.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-05-10 12:00:11 -07:00
John Wilkins
261aaba117 doc: Added entry for the RGW Admin Ops API.
fixes: #5002

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-05-10 11:30:48 -07:00
Sage Weil
49d22aa649 Merge pull request #271 from dalgaaf/wip-da-sca-cppcheck-v2.1
Reviewed-by: Sage Weil <sage@inktank.com>
2013-05-10 10:26:05 -07:00
Danny Al-Gaaf
54e7a6ffe6 libcephfs_jni.cc: reduce scope of ret variable
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:59 +02:00
Danny Al-Gaaf
cd0f461fd5 include/ceph_hash.cc: reduce scope of a var in ceph_str_hash_rjenkins()
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:59 +02:00
Danny Al-Gaaf
ad2990c48f include/addr_parsing.c: reduce scope of port_str in safe_cat()
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:59 +02:00
Danny Al-Gaaf
cc3376cf91 global/global_init.cc: reduce scope of ret in global_init_daemonize()
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:58 +02:00
Danny Al-Gaaf
73a10ac9cc crush/builder.c: reduce scope of oldsize in crush_add_bucket()
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:58 +02:00
Danny Al-Gaaf
e064e67c2b common/safe_io.c: reduce scope of some ssize_t variables
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:58 +02:00
Danny Al-Gaaf
8c97e77e57 common/obj_bencher.cc: reduce scope of avg_bandwidth
Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
2013-05-10 19:19:58 +02:00