Commit Graph

21782 Commits

Author SHA1 Message Date
Sage Weil
a736cb028e librbd: allow striping to be specified for clones
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:06 -07:00
Sage Weil
819f93916a filer: add debug printability for striped read result stuff
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:06 -07:00
Sage Weil
80ebceea58 filer: uninline StripedReadResult
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:06 -07:00
Sage Weil
fd8013a97b librbd: avoid read copy for c++ api read
Read directly into the user's bufferlist instead of copying the buffer
contents.

NOTE: this potentially exposes cached buffers to C++ API users.  Document
this!

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
ecb040c926 librbd: read_from_parent into a bufferlist
This avoids a copy.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
b1b332486f librbd: use bufferlist target for read_iterate()
Save ourselves a copy for read_iterate.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
a8042c704e librbd: make read methods target buffers or bufferlists
This will let us transition much of the read code to move bufferlists
around instead of copying data between buffers.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
94de63a9ef librbd: kill unused get_block_osd()
Replaced by ImageCtx::get_object_name().

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
e9e197bca1 librbd: kill legacy striping helpers
These are replaced by ImageCtx accessors or the use of the Filer striping
helper.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
d556eda485 librbd: reimplement flatten in terms of child objects
Iterate over child *objects*, and map each one to the parent extents it
represents.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
cdb165e90f librbd: write sync read over an extent list
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
2660129593 librbd: make rollback use new striping accessors
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
113f698a34 librbd: make trim_image() behave with new world striping order
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:05 -07:00
Sage Weil
0aeedf657c librbd: generate format_string in ImageCtx
Prebuild the format string on image load so that it is handy at all times.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
3ae9f5723c librbd: make read_iterate iterate over periods
Object sizes no longer make sense since we are iterating over the logical
device image space, and the striping may make that unrelated to the size
of the individual objects.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
e114cbffcc librbd/ImageCtx: accessors
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
6151899664 librbd: reimplement striping
This replaces most of the existing striping code with use of the Filer
striping helper methods and a more general ceph_file_layout that can
handle more sophisticated striping patterns that the previous uniform
object approach.

This patch is not fully complete; there are a few additional patches that
follow that clean up some of the support functions.  However, most of the
IO path is covered here.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
9e109e187c filer: add sparse result into a striped read result
Add a helper to assimilate a sparse read result into the destriper
helper class.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
08d1eb5b0b filer: make tail zeroing in stripe result assembly optional
Objecter doesn't want it, librbd will.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
6cd51feced librbd: remove callback from handle_sparse_read
We always used simple_read_cb, so put it inline.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
3d010fc4c8 objecter: use StripedReadResult for scatter/gather read
This vastly simplifies the code, and uses the more generic helper class.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
3c2024de44 filer: helper to assemble striped read results into a single result
This is intended to replace the Objecter::_sg_read_finish() monstrosity,
and be reused by the librbd striping code.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
36a2245d50 buffer: add claim_prepend()
Steal another bufferlist's content and prepend it to our own.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
827ad53aba filer: add optional buffer offset
It may be that some range is being mapped that is intended for some offset
within the result buffer.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:04 -07:00
Sage Weil
c85b9016ea filer: add helper to map from object extent to file extents
Given an extent in an object and a layout, map it back to extent(s) in the
original file.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
c3f186e678 filer: use vector<pair<>> instead of map<> for buffer extent map
This is less inefficient.  :)

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
058c24ffea filer: include (numeric) objectno in ObjectExtent
This lets us map backwards, if we need to.  Arguably the oid and oloc
don't belong here at all.  That can come later.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
a7f634a8c8 librbd: update python bindings for striped image creation
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
ae7271e13f rbd: document stripe unit options on man page
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
d7b912f5c0 librbd: document create3
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
79a365ac76 librbd: populate struct ceph_file_layout for image
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
d54ca6720e filer: generalize file_to_extents to take a format string
Allow any format string to be used, so long as it takes a long long
unsigned.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
848a94217f rbd: show striping parameters with 'info' command
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
efa1bea17d librbd: expose image striping properties
Fix C and C++ APIs.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
e0b4638e3d cls_rbd: fix snap key filtering
Ignore any omap keys that don't begin with "snap_".  We got away with this
previously because it was (lexiographically) the last key for the object.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
8143206253 rbd: allow striping to be specified on image creation
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
d6501dbed5 librbd: allow striping to be specified on image creation
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:03 -07:00
Sage Weil
4ef9a3d2c4 librbd: load striping information when opening image
Silently tolerate an -ENOEXEC or -EINVAL (which means striping is not
present or is not supported by the osd class).

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:02 -07:00
Sage Weil
2e403ef2c9 cls_rbd: add stripe_unit, stripe_count metadata
* add STRIPINGV2 feature
* add stripe_unit, stripe_count keys
* add accessor methods

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 15:34:02 -07:00
Sage Weil
18a3cee29e client: avoid possible null deref
CID 716910 (#1 of 1): Explicit null dereferenced (FORWARD_NULL)
At (6): Dereferencing null pointer "mds_session".

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 14:22:31 -07:00
Sage Weil
0095a13824 client: fix shadowing in inode ctor
CID 728080 (#1 of 1): Incorrect sizeof expression (BAD_SIZEOF)
Taking the size of pointer parameter "layout" is suspicious.

At (2): Non-static class member field "layout.fl_stripe_unit" is not initialized in this constructor nor in any functions that it calls.
At (4): Non-static class member field "layout.fl_stripe_count" is not initialized in this constructor nor in any functions that it calls.
At (6): Non-static class member field "layout.fl_object_size" is not initialized in this constructor nor in any functions that it calls.
At (8): Non-static class member field "layout.fl_cas_hash" is not initialized in this constructor nor in any functions that it calls.
At (10): Non-static class member field "layout.fl_object_stripe_unit" is not initialized in this constructor nor in any functions that it calls.
At (12): Non-static class member field "layout.fl_unused" is not initialized in this constructor nor in any functions that it calls.
CID 717206 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (14): Non-static class member field "layout.fl_pg_pool" is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 14:20:51 -07:00
Sage Weil
d8bb685d84 client: init readdir fields
At (2): Non-static class member "readdir_offset" is not initialized in this constructor nor in any functions that it calls.
At (4): Non-static class member "readdir_end" is not initialized in this constructor nor in any functions that it calls.
At (6): Non-static class member "readdir_num" is not initialized in this constructor nor in any functions that it calls.
CID 717207 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (8): Non-static class member "tid" is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 14:19:10 -07:00
Sage Weil
a1d8267c57 cls_rgw: init var in ctor
CID 727992 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
At (2): Non-static class member "tag_timeout" is not initialized in this constructor nor in any functions that it calls.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-15 14:14:28 -07:00
Yehuda Sadeh
8d7c8e3b86 rgw: don't add port to url if already has one
Fixes: #3296
Specifically, is host name string already has ':', then
don't try to append theport (swift auth).

backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-15 09:59:14 -07:00
Tommi Virtanen
662c69e525 ceph-disk-prepare, debian/control: Support external journals.
Previously, ceph-disk-* would only let you use a journal that was a
file inside the OSD data directory. With this, you can do:

  ceph-disk-prepare /dev/sdb /dev/sdb

to put the journal as a second partition on the same disk as the OSD
data (might save some file system overhead), or, more interestingly:

  ceph-disk-prepare /dev/sdb /dev/sdc

which makes it create a new partition on /dev/sdc to use as the
journal. Size of the partition is decided by $osd_journal_size.
/dev/sdc must be a GPT-format disk. Multiple OSDs may share the same
journal disk (using separate partitions); this way, a single fast SSD
can serve as journal for multiple spinning disks.

The second use case currently requires parted, so a Recommends: for
parted has been added to Debian packaging.

Closes: #3078
Closes: #3079
Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-10-15 09:34:36 -07:00
Chris Dunlop
4db12511f7 logrotate: fix bash syntax
Introduced by 32a6394be0.

Signed-off-by: Chris Dunlop <chris@onthe.net.au>
2012-10-13 22:34:29 -07:00
Sage Weil
251649cdfa doc: remove cephfs warning
Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-13 20:13:35 -07:00
Sage Weil
168bd10e4d doc: fix file system recs
- drop xattr warning; this is not an issue with the leveldb stuff.
- the ext3 vs xattr discussion was somewhat inaccurate.  also, no longer
  relevant.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-10-13 20:12:48 -07:00
Yehuda Sadeh
389fac7a82 rgw: replace bucket creation with explicit pool creation
Following a recent cleanup, usage should create a pool and
not a bucket.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-10-12 14:03:02 -07:00
Sam Lang
5c5feaec64 Merge branch 'wip-fix-getdents' 2012-10-11 14:19:04 -05:00