Commit Graph

41344 Commits

Author SHA1 Message Date
Josh Durgin
e1d0f2f193 Merge pull request #4422 from ceph/wip-10154
librbd: deep-flatten of cloned images

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-07 08:29:25 -07:00
Loic Dachary
ac89b8fccf Merge pull request #4603 from ceph/wip-11557
librbd: fast diff should treat all _EXISTS objects as dirty

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-07 17:09:18 +02:00
John Spray
a7ffd8d19b Merge pull request #4460 from yuyuyu101/wip-fix-pycephfs
Libcephfs: Add python bindings

Reviewed-by: John Spray <john.spray@redhat.com>
2015-05-07 11:58:26 +02:00
Jason Dillaman
6b0bdc3891 librbd: copyup now optionally updates the snapshot object maps
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
a1eab0f839 librbd: allow updates to snapshot object maps
The modified copyup behavior will need to update snapshot object
maps when performing a copyup since the object will now exist within
all snapshots of the image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
6511eb071f librbd: AsyncRequest is now optional for AsyncObjectThrottle
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
d5b4e7827d librbd: progress context is now optional for AsyncObjectThrottle
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
f76f772006 tests: verify object map is updated after full copyup
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
e2be3416ea tests: verify new librbd snapshot copyup behavior
Copyup should make the object available for all existing snapshots in
addition to the HEAD object.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
64cbd901ae tests: add new feature code to rbd test list
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
e01a064f44 tests: verify cls_rbd::remove_parent handles snapshots
When the deep-flatten feature is enabled, the remove_parent
class method should also remove the parent from all snapshots.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
9632cbac7b cls_rbd: remove_parent now updates snapshots if deep-flatten enabled
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
bc66b7849a librbd: always deregister child clone from parent with deep flatten
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
12719478ad librados_test_stub: support AIO snapshot context
AIO operations can be executed with a different snapshot
context from the IoCtx.  librbd takes advantage of this
capability to perform a deep object copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
9fd708b1a0 librbd: copyup should use empty snapshot context
This allows child object to exist through all existing
snapshots as well as HEAD.  If the operation was a CoW,
issue the write ops as a separate RADOS op with the
correct snapshot context.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:44 -04:00
Yan, Zheng
d20a387788 Merge pull request #4523 from ceph/wip-mds-flushsnapack
mds: send FLUSHSNAP_ACK even if FLUSHSNAP message is unexpected
2015-05-07 09:57:03 +08:00
Jason Dillaman
7be3df6780 librbd: move copyup class method call to CopyupRequest
Move AbstractWrite's invocation of copyup to the CopyupRequest
class.  The AioRequest write path will now always create a
CopyupRequest, which will now append the actual write ops to the
copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 21:51:31 -04:00
Jason Dillaman
b556d3173a librbd: add new deep-flatten RBD feature
Copy-on-read/write will now use an empty snapshot context to
deep-copy the object through all existing snapshots.  With the
deep-flatten feature enabled, the flatten operation will now
disconnect a child image from its parent even if it has existing
snapshots.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 21:51:31 -04:00
Jason Dillaman
819c9800df librbd: fast diff should treat all _EXISTS objects as dirty
With the fast-diff feature enabled, if the object map marks the
object as _EXISTS, it can only occur due to a write -- otherwise
it would be _EXISTS_CLEAN.  Therefore, ensure it is properly
flagged as an updated object for diff_iterate.

Fixes: #11553
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 21:28:25 -04:00
Josh Durgin
ca5c4a34ce Merge pull request #4593 from ceph/wip-librbd-fast-diff
librbd: issues with fast diff discovered via rbd du tool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-06 15:50:17 -07:00
Jason Dillaman
1fc4b1bbbc Merge pull request #4594 from liewegas/wip-librbd-warning
librbd: fix warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 18:11:21 -04:00
Sage Weil
fc428372a3 librbd: fix warning
librbd/librbd.cc: In member function 'int librbd::Image::metadata_get(const string&, std::string*)':
librbd/librbd.cc:845:76: warning: suggest braces around empty body in an 'else' statement [-Wempty-body]

Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-06 14:41:38 -07:00
Jason Dillaman
1b6a5e951e librbd: fast diff of first snapshot loads incorrect object map
The ImageCtx::snaps collection is stored in newest->oldest order,
so the first snapshot id should be retrieved from the back of the
array.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 17:40:44 -04:00
Jason Dillaman
baebe74492 librbd: fast diff is incorrectly flagging objects as updated
If the diff_iterate call doesn't start with the first snapshot,
unmodified objects in future snapshots / HEAD are incorrectly
flagged as modified.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 17:40:44 -04:00
Josh Durgin
1c8978fdbd Merge pull request #4589 from ceph/wip-11549
librbd: ignore lack of support for metadata on older OSDs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-06 14:20:30 -07:00
Jason Dillaman
f25aa5f393 librbd: ignore lack of support for metadata on older OSDs
If an Infernalis librbd attempts to open an image stored on a
pre-Infernalis OSD, the new config metadata operations won't
be supported.  This error can be safely ignored.

Fixes: #11549
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 16:30:27 -04:00
Jason Dillaman
5b8c7662ba Merge pull request #4558 from wonzhq/detect-format
librbd: fix the image format detection

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 14:55:21 -04:00
Loic Dachary
4ca500d840 Merge pull request #4234 from dmitryya/bug_10163
obj_bencher rados cli: rados bench producing wrong values when different blocksize used in writes and reads

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 20:13:44 +02:00
Dmitry Yatsushkevich
70585a6f7b rados cli: fix documentation for -b option
Fix focumentation for block size option for rados cli

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-06 10:24:35 -07:00
Kefu Chai
b65e93b125 Merge pull request #4527 from SUSE/wip-hack-utf8-into-json-parser
json_spirit: use utf8 intenally when parsing \uHHHH

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 22:29:36 +08:00
John Spray
e8aff1c3ae Merge pull request #4572 from ceph/wip-11540
test_libcephfs: Fix zero length read tests

Reviewed-by: John Spray <john.spray@redhat.com>
2015-05-06 15:51:54 +02:00
Haomai Wang
fbd6646898 test_libcephfs: Fix zero length read tests
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-06 18:45:44 +08:00
Tim Serong
8add15b86e json_spirit: use utf8 intenally when parsing \uHHHH
When the python CLI is given non-ASCII characters, it converts them to
\uHHHH escapes in JSON.  json_spirit parses these internally into 16 bit
characters, which could only work if json_spirit were built to use
std::wstring, which it isn't; it's using std::string, so the high byte
ends up being zero'd, leaving the low byte which is effectively garbage.

This hack^H^H^H^H change makes json_spirit convert to utf8 internally
instead, which can be stored just fine inside a std::string.

Note that this implementation still assumes \uHHHH escapes are four hex
digits, so it'll only cope with characters in the Basic Multilingual
Plane.  Still, that's rather a lot more characters than it could cope
with before ;)

(For characters outside the BMP, Python seems to generate escapes in the
form \uHHHHHHHH, i.e. 8 hex digits, which the current implementation
doesn't expect to see)

Fixes: #7387

Signed-off-by: Tim Serong <tserong@suse.com>
2015-05-06 20:18:34 +10:00
Loic Dachary
8f23382064 Merge pull request #4353 from ceph/wip-11376-packaging-objstore-tool
#11376: packaging: mv ceph-objectstore-tool to main ceph pkg

Reviewed-by: Sage Weil <sage@redhat.com>
2015-05-06 11:04:24 +02:00
Loic Dachary
de1cd2d97c Merge pull request #4508 from dachary/wip-enoent-offset
osd: For write a non-exist erasure object don't offset > 0.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-05-06 10:54:10 +02:00
Kefu Chai
d0c76c3946 Merge pull request #4548 from dachary/wip-docker-ubuntu-12.04
tests: install sudo on ubuntu-12.04 container

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 16:44:40 +08:00
Zhiqiang Wang
835b12ff6d librbd: fix the image format detection
If the detection of the old format fails with reasons other than
-ENOENT, we should return with this error. Otherwise, if we continue the
new format detection and fail with -ENOENT, the caller will get the
missleading failure information.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-05-06 16:14:29 +08:00
Loic Dachary
62ec3b7f30 Merge pull request #4544 from ceph/wip-with-man-pages
Wip with man pages

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 09:47:49 +02:00
Kefu Chai
cd1a9c0101 Merge pull request #4555 from zzxuanyuan/wip-11534-incorrect-osdsum-size
mon: Total size of OSDs is a magnitude less than it is supposed to be.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 14:32:21 +08:00
Kefu Chai
899dd23766 configure.ac: no use to add "+" before ac_ext=c
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-06 10:17:48 +08:00
Kefu Chai
5b2357e663 configure.ac: add an option: --with-man-pages
* do not require sphinx-build if --without-man-pages
* dist rst files for man pages
* build and dist .8 files if sphinx-build is found

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-06 10:17:42 +08:00
Zhe Zhang
73d16f69d6 mon: Total size of OSDs is a maginitude less than it is supposed to be.
When dumping statistics of OSDs such as running command "ceph osd df",
the sum of OSDs' size is 2^10 times less than their real size.

Signed-off-by: Zhe Zhang <zzxuanyuan@gmail.com>
2015-05-05 18:08:48 -05:00
Dmitry Yatsushkevich
a87ac4d36b obj_bencher: aio_bench - rename op_size to object_size
Rename aio_bench argument 'op_size' to 'object_size' to reflect the reality of
how it is used.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
0dea11d907 rados cli: add preventing using --block-size with bench seq and rand
Add handling of situation when '-b|--block-size' option is using with
'bench seq' or 'bench rand' which is erroneous according to the help message:
     -b op_size
            set the size of write ops for put or benchmarking

Fixes: #10163

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
1d1d0aa01b obj_bencher: remove excess 'object_size = op_size'
Remove excess branch 'else {object_size = op_size;}' from
'if(operation != OP_WRITE){...}else{object_size = op_size;}' because
'object_size' variable already initialized with 'op_size'

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
31d16e9010 obj_bencher: remove 'trans_size' as obsolete
'trans_size' description from header: "size of the write/read to perform"
But really 'object_size' is used in write/read operations. 'trans_size' is used
only in ObjBencher::status_printer for calc current and average bandwidth.
As result - bad statistics in case 'trans_size' and and 'object_size' are different.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Sage Weil
e324578b1e Merge pull request #4554 from liewegas/wip-cleanup
9.0.0 release notes, scripts cleanup

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 14:47:42 -07:00
Sage Weil
538e6eafb9 doc/release-notes: 9.0.0
Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-05 14:37:04 -07:00
Sage Weil
c9a6e60eee src/script: remove obsolete scripts
These are mostly from gather graph data for very early versions of ceph and
are not useful anymore.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-05 13:35:29 -07:00
Loic Dachary
2e9e3ba421 Merge pull request #4550 from ktdreyer/wip-submittingpatches-backports
SubmittingPatches: clarify backport procedure

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 19:42:21 +02:00