1
0
mirror of https://github.com/ceph/ceph synced 2024-12-18 17:37:38 +00:00
Commit Graph

41698 Commits

Author SHA1 Message Date
Jason Dillaman
bc66b7849a librbd: always deregister child clone from parent with deep flatten
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
12719478ad librados_test_stub: support AIO snapshot context
AIO operations can be executed with a different snapshot
context from the IoCtx.  librbd takes advantage of this
capability to perform a deep object copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:46 -04:00
Jason Dillaman
9fd708b1a0 librbd: copyup should use empty snapshot context
This allows child object to exist through all existing
snapshots as well as HEAD.  If the operation was a CoW,
issue the write ops as a separate RADOS op with the
correct snapshot context.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 22:07:44 -04:00
Yan, Zheng
d20a387788 Merge pull request from ceph/wip-mds-flushsnapack
mds: send FLUSHSNAP_ACK even if FLUSHSNAP message is unexpected
2015-05-07 09:57:03 +08:00
Jason Dillaman
7be3df6780 librbd: move copyup class method call to CopyupRequest
Move AbstractWrite's invocation of copyup to the CopyupRequest
class.  The AioRequest write path will now always create a
CopyupRequest, which will now append the actual write ops to the
copyup.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 21:51:31 -04:00
Jason Dillaman
b556d3173a librbd: add new deep-flatten RBD feature
Copy-on-read/write will now use an empty snapshot context to
deep-copy the object through all existing snapshots.  With the
deep-flatten feature enabled, the flatten operation will now
disconnect a child image from its parent even if it has existing
snapshots.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 21:51:31 -04:00
Jason Dillaman
819c9800df librbd: fast diff should treat all _EXISTS objects as dirty
With the fast-diff feature enabled, if the object map marks the
object as _EXISTS, it can only occur due to a write -- otherwise
it would be _EXISTS_CLEAN.  Therefore, ensure it is properly
flagged as an updated object for diff_iterate.

Fixes: 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 21:28:25 -04:00
Josh Durgin
ca5c4a34ce Merge pull request from ceph/wip-librbd-fast-diff
librbd: issues with fast diff discovered via rbd du tool

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-06 15:50:17 -07:00
Jason Dillaman
1fc4b1bbbc Merge pull request from liewegas/wip-librbd-warning
librbd: fix warning

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 18:11:21 -04:00
Sage Weil
fc428372a3 librbd: fix warning
librbd/librbd.cc: In member function 'int librbd::Image::metadata_get(const string&, std::string*)':
librbd/librbd.cc:845:76: warning: suggest braces around empty body in an 'else' statement [-Wempty-body]

Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-06 14:41:38 -07:00
Jason Dillaman
1b6a5e951e librbd: fast diff of first snapshot loads incorrect object map
The ImageCtx::snaps collection is stored in newest->oldest order,
so the first snapshot id should be retrieved from the back of the
array.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 17:40:44 -04:00
Jason Dillaman
baebe74492 librbd: fast diff is incorrectly flagging objects as updated
If the diff_iterate call doesn't start with the first snapshot,
unmodified objects in future snapshots / HEAD are incorrectly
flagged as modified.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 17:40:44 -04:00
Josh Durgin
1c8978fdbd Merge pull request from ceph/wip-11549
librbd: ignore lack of support for metadata on older OSDs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-06 14:20:30 -07:00
Jason Dillaman
f25aa5f393 librbd: ignore lack of support for metadata on older OSDs
If an Infernalis librbd attempts to open an image stored on a
pre-Infernalis OSD, the new config metadata operations won't
be supported.  This error can be safely ignored.

Fixes: 
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 16:30:27 -04:00
Jason Dillaman
5b8c7662ba Merge pull request from wonzhq/detect-format
librbd: fix the image format detection

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 14:55:21 -04:00
Loic Dachary
4ca500d840 Merge pull request from dmitryya/bug_10163
obj_bencher rados cli: rados bench producing wrong values when different blocksize used in writes and reads

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 20:13:44 +02:00
Samuel Just
3f2946aa9f OSD: add op_wq suicide timeout
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-06 11:02:19 -07:00
Samuel Just
f2fbfa32a1 OSD: add remove_wq suicide timeout
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-06 10:54:31 -07:00
Samuel Just
547a7041ed OSD: add scrub_wq suicide timeout
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-06 10:53:28 -07:00
Samuel Just
e1073a4a57 OSD: add snap_trim_wq suicide timeout
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-06 10:51:28 -07:00
Samuel Just
85311b6568 OSD: add recovery_wq suicide timeout
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-06 10:50:19 -07:00
Samuel Just
df4e5de819 OSD: add command_wq suicide timeout
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-06 10:49:07 -07:00
Dmitry Yatsushkevich
cd0f2b754d obj_bencher: does not accumulate bandwidth that is zero
We never want to accumulate a bandwidth that is zero. A bandwidth of
zero means that the object could not be written in one second (or that
it's the first time in the loop). Even if writing an object takes a very
long time, the bandwidth cannot be zero, it will just be very small.

Fix suggested by @dachary
Fixes: 

Signed-off-by: Dmitry Yatsushkevich <dmitry.yatsushkevich@gmail.com>
2015-05-06 10:29:40 -07:00
Dmitry Yatsushkevich
70585a6f7b rados cli: fix documentation for -b option
Fix focumentation for block size option for rados cli

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-06 10:24:35 -07:00
Kefu Chai
b65e93b125 Merge pull request from SUSE/wip-hack-utf8-into-json-parser
json_spirit: use utf8 intenally when parsing \uHHHH

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 22:29:36 +08:00
John Spray
e8aff1c3ae Merge pull request from ceph/wip-11540
test_libcephfs: Fix zero length read tests

Reviewed-by: John Spray <john.spray@redhat.com>
2015-05-06 15:51:54 +02:00
Haomai Wang
fbd6646898 test_libcephfs: Fix zero length read tests
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-06 18:45:44 +08:00
Tim Serong
8add15b86e json_spirit: use utf8 intenally when parsing \uHHHH
When the python CLI is given non-ASCII characters, it converts them to
\uHHHH escapes in JSON.  json_spirit parses these internally into 16 bit
characters, which could only work if json_spirit were built to use
std::wstring, which it isn't; it's using std::string, so the high byte
ends up being zero'd, leaving the low byte which is effectively garbage.

This hack^H^H^H^H change makes json_spirit convert to utf8 internally
instead, which can be stored just fine inside a std::string.

Note that this implementation still assumes \uHHHH escapes are four hex
digits, so it'll only cope with characters in the Basic Multilingual
Plane.  Still, that's rather a lot more characters than it could cope
with before ;)

(For characters outside the BMP, Python seems to generate escapes in the
form \uHHHHHHHH, i.e. 8 hex digits, which the current implementation
doesn't expect to see)

Fixes: 

Signed-off-by: Tim Serong <tserong@suse.com>
2015-05-06 20:18:34 +10:00
Loic Dachary
8f23382064 Merge pull request from ceph/wip-11376-packaging-objstore-tool
: packaging: mv ceph-objectstore-tool to main ceph pkg

Reviewed-by: Sage Weil <sage@redhat.com>
2015-05-06 11:04:24 +02:00
Loic Dachary
de1cd2d97c Merge pull request from dachary/wip-enoent-offset
osd: For write a non-exist erasure object don't offset > 0.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-05-06 10:54:10 +02:00
Kefu Chai
d0c76c3946 Merge pull request from dachary/wip-docker-ubuntu-12.04
tests: install sudo on ubuntu-12.04 container

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 16:44:40 +08:00
Zhiqiang Wang
835b12ff6d librbd: fix the image format detection
If the detection of the old format fails with reasons other than
-ENOENT, we should return with this error. Otherwise, if we continue the
new format detection and fail with -ENOENT, the caller will get the
missleading failure information.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-05-06 16:14:29 +08:00
Loic Dachary
62ec3b7f30 Merge pull request from ceph/wip-with-man-pages
Wip with man pages

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 09:47:49 +02:00
Kefu Chai
cd1a9c0101 Merge pull request from zzxuanyuan/wip-11534-incorrect-osdsum-size
mon: Total size of OSDs is a magnitude less than it is supposed to be.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 14:32:21 +08:00
Kefu Chai
899dd23766 configure.ac: no use to add "+" before ac_ext=c
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-06 10:17:48 +08:00
Kefu Chai
5b2357e663 configure.ac: add an option: --with-man-pages
* do not require sphinx-build if --without-man-pages
* dist rst files for man pages
* build and dist .8 files if sphinx-build is found

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-06 10:17:42 +08:00
Loic Dachary
69248bc4bf ceph-disk: use ceph-detect-init instead of hard coded values
The ceph-detect-init implements the same detection logic as ceph-deploy.
It is better than the current hardcoded values that assume Ubuntu is
upstart and all the rest is sysvinit.

Signed-off-by: Owen Synge <osynge@suse.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
1dad037532 tests: ceph-disk.sh must exit 1 on error
The test for error was reversed and ceph-disk.sh always returned on the
the first successfull test, with no error.

And add verbosity to help with debugging.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
b4a441a0ed ceph-detect-init: package for rpm and debian
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
35a16ffc1e ceph-detect-init: integration with automake / make check
* Add the files to the distribution tarbal via EXTRA_DIST
* Build the module via the all-local target
* Add run-tox.sh to the tests run via make check
* Add manual page

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
8ee5a82832 ceph-disk: implement activate --no-start-daemon
And improve the --mark-init documentation as well.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
9b6e1cdadb ceph-disk: pep8 conformance
And remove misleading comment.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
73e2c74262 ceph-detect-init: ceph-disk helper to select the init system
On Ubuntu 14.04

$ ceph-detect-init --default=sysvinit
upstart

The ceph-detect-init helper is a copy/paste of the ceph-deploy init
system detection found at:

4992ef0993/ceph_deploy/hosts/__init__.py (L15)

It is meant to be used by both ceph-disk and ceph-deploy to avoid
duplicating the logic. Some operating systems implement more than one
init system and ceph-detect-init will not return the default one, it
will return the init system suitable for ceph deployment.

ceph-detect-init is implemented as a standalone python module suitable
to be published at https://pypi.python.org/pypi/ceph-detect-init, with
unit tests and integration tests (based on docker) for the following
platforms:

centos-6
centos-7
debian-jessie
debian-sid
debian-squeeze
debian-wheezy
fedora-21
opensuse-13.1
opensuse-13.2
ubuntu-12.04
ubuntu-14.04
ubuntu-15.04

The tests can be run without network access with run-tox.sh, provided
pip wheel previously populated the wheelhouse directory.

Signed-off-by: Owen Synge <osynge@suse.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:32 +02:00
Zhe Zhang
73d16f69d6 mon: Total size of OSDs is a maginitude less than it is supposed to be.
When dumping statistics of OSDs such as running command "ceph osd df",
the sum of OSDs' size is 2^10 times less than their real size.

Signed-off-by: Zhe Zhang <zzxuanyuan@gmail.com>
2015-05-05 18:08:48 -05:00
Dmitry Yatsushkevich
a87ac4d36b obj_bencher: aio_bench - rename op_size to object_size
Rename aio_bench argument 'op_size' to 'object_size' to reflect the reality of
how it is used.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
0dea11d907 rados cli: add preventing using --block-size with bench seq and rand
Add handling of situation when '-b|--block-size' option is using with
'bench seq' or 'bench rand' which is erroneous according to the help message:
     -b op_size
            set the size of write ops for put or benchmarking

Fixes: 

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
1d1d0aa01b obj_bencher: remove excess 'object_size = op_size'
Remove excess branch 'else {object_size = op_size;}' from
'if(operation != OP_WRITE){...}else{object_size = op_size;}' because
'object_size' variable already initialized with 'op_size'

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
31d16e9010 obj_bencher: remove 'trans_size' as obsolete
'trans_size' description from header: "size of the write/read to perform"
But really 'object_size' is used in write/read operations. 'trans_size' is used
only in ObjBencher::status_printer for calc current and average bandwidth.
As result - bad statistics in case 'trans_size' and and 'object_size' are different.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Sage Weil
e324578b1e Merge pull request from liewegas/wip-cleanup
9.0.0 release notes, scripts cleanup

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 14:47:42 -07:00
Sage Weil
538e6eafb9 doc/release-notes: 9.0.0
Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-05 14:37:04 -07:00