Commit Graph

41464 Commits

Author SHA1 Message Date
Jason Dillaman
1b6a5e951e librbd: fast diff of first snapshot loads incorrect object map
The ImageCtx::snaps collection is stored in newest->oldest order,
so the first snapshot id should be retrieved from the back of the
array.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 17:40:44 -04:00
Jason Dillaman
baebe74492 librbd: fast diff is incorrectly flagging objects as updated
If the diff_iterate call doesn't start with the first snapshot,
unmodified objects in future snapshots / HEAD are incorrectly
flagged as modified.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 17:40:44 -04:00
Josh Durgin
1c8978fdbd Merge pull request #4589 from ceph/wip-11549
librbd: ignore lack of support for metadata on older OSDs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-06 14:20:30 -07:00
Jason Dillaman
f25aa5f393 librbd: ignore lack of support for metadata on older OSDs
If an Infernalis librbd attempts to open an image stored on a
pre-Infernalis OSD, the new config metadata operations won't
be supported.  This error can be safely ignored.

Fixes: #11549
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 16:30:27 -04:00
Jason Dillaman
5b8c7662ba Merge pull request #4558 from wonzhq/detect-format
librbd: fix the image format detection

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2015-05-06 14:55:21 -04:00
Loic Dachary
4ca500d840 Merge pull request #4234 from dmitryya/bug_10163
obj_bencher rados cli: rados bench producing wrong values when different blocksize used in writes and reads

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 20:13:44 +02:00
Dmitry Yatsushkevich
70585a6f7b rados cli: fix documentation for -b option
Fix focumentation for block size option for rados cli

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-06 10:24:35 -07:00
Kefu Chai
b65e93b125 Merge pull request #4527 from SUSE/wip-hack-utf8-into-json-parser
json_spirit: use utf8 intenally when parsing \uHHHH

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 22:29:36 +08:00
John Spray
e8aff1c3ae Merge pull request #4572 from ceph/wip-11540
test_libcephfs: Fix zero length read tests

Reviewed-by: John Spray <john.spray@redhat.com>
2015-05-06 15:51:54 +02:00
Haomai Wang
fbd6646898 test_libcephfs: Fix zero length read tests
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-06 18:45:44 +08:00
Tim Serong
8add15b86e json_spirit: use utf8 intenally when parsing \uHHHH
When the python CLI is given non-ASCII characters, it converts them to
\uHHHH escapes in JSON.  json_spirit parses these internally into 16 bit
characters, which could only work if json_spirit were built to use
std::wstring, which it isn't; it's using std::string, so the high byte
ends up being zero'd, leaving the low byte which is effectively garbage.

This hack^H^H^H^H change makes json_spirit convert to utf8 internally
instead, which can be stored just fine inside a std::string.

Note that this implementation still assumes \uHHHH escapes are four hex
digits, so it'll only cope with characters in the Basic Multilingual
Plane.  Still, that's rather a lot more characters than it could cope
with before ;)

(For characters outside the BMP, Python seems to generate escapes in the
form \uHHHHHHHH, i.e. 8 hex digits, which the current implementation
doesn't expect to see)

Fixes: #7387

Signed-off-by: Tim Serong <tserong@suse.com>
2015-05-06 20:18:34 +10:00
Loic Dachary
8f23382064 Merge pull request #4353 from ceph/wip-11376-packaging-objstore-tool
#11376: packaging: mv ceph-objectstore-tool to main ceph pkg

Reviewed-by: Sage Weil <sage@redhat.com>
2015-05-06 11:04:24 +02:00
Loic Dachary
de1cd2d97c Merge pull request #4508 from dachary/wip-enoent-offset
osd: For write a non-exist erasure object don't offset > 0.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-05-06 10:54:10 +02:00
Kefu Chai
d0c76c3946 Merge pull request #4548 from dachary/wip-docker-ubuntu-12.04
tests: install sudo on ubuntu-12.04 container

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 16:44:40 +08:00
Zhiqiang Wang
835b12ff6d librbd: fix the image format detection
If the detection of the old format fails with reasons other than
-ENOENT, we should return with this error. Otherwise, if we continue the
new format detection and fail with -ENOENT, the caller will get the
missleading failure information.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-05-06 16:14:29 +08:00
Loic Dachary
62ec3b7f30 Merge pull request #4544 from ceph/wip-with-man-pages
Wip with man pages

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 09:47:49 +02:00
Kefu Chai
cd1a9c0101 Merge pull request #4555 from zzxuanyuan/wip-11534-incorrect-osdsum-size
mon: Total size of OSDs is a magnitude less than it is supposed to be.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-06 14:32:21 +08:00
Kefu Chai
899dd23766 configure.ac: no use to add "+" before ac_ext=c
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-06 10:17:48 +08:00
Kefu Chai
5b2357e663 configure.ac: add an option: --with-man-pages
* do not require sphinx-build if --without-man-pages
* dist rst files for man pages
* build and dist .8 files if sphinx-build is found

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-06 10:17:42 +08:00
Loic Dachary
69248bc4bf ceph-disk: use ceph-detect-init instead of hard coded values
The ceph-detect-init implements the same detection logic as ceph-deploy.
It is better than the current hardcoded values that assume Ubuntu is
upstart and all the rest is sysvinit.

Signed-off-by: Owen Synge <osynge@suse.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
1dad037532 tests: ceph-disk.sh must exit 1 on error
The test for error was reversed and ceph-disk.sh always returned on the
the first successfull test, with no error.

And add verbosity to help with debugging.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
b4a441a0ed ceph-detect-init: package for rpm and debian
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
35a16ffc1e ceph-detect-init: integration with automake / make check
* Add the files to the distribution tarbal via EXTRA_DIST
* Build the module via the all-local target
* Add run-tox.sh to the tests run via make check
* Add manual page

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
8ee5a82832 ceph-disk: implement activate --no-start-daemon
And improve the --mark-init documentation as well.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
9b6e1cdadb ceph-disk: pep8 conformance
And remove misleading comment.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:36 +02:00
Loic Dachary
73e2c74262 ceph-detect-init: ceph-disk helper to select the init system
On Ubuntu 14.04

$ ceph-detect-init --default=sysvinit
upstart

The ceph-detect-init helper is a copy/paste of the ceph-deploy init
system detection found at:

4992ef0993/ceph_deploy/hosts/__init__.py (L15)

It is meant to be used by both ceph-disk and ceph-deploy to avoid
duplicating the logic. Some operating systems implement more than one
init system and ceph-detect-init will not return the default one, it
will return the init system suitable for ceph deployment.

ceph-detect-init is implemented as a standalone python module suitable
to be published at https://pypi.python.org/pypi/ceph-detect-init, with
unit tests and integration tests (based on docker) for the following
platforms:

centos-6
centos-7
debian-jessie
debian-sid
debian-squeeze
debian-wheezy
fedora-21
opensuse-13.1
opensuse-13.2
ubuntu-12.04
ubuntu-14.04
ubuntu-15.04

The tests can be run without network access with run-tox.sh, provided
pip wheel previously populated the wheelhouse directory.

Signed-off-by: Owen Synge <osynge@suse.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-06 01:54:32 +02:00
Zhe Zhang
73d16f69d6 mon: Total size of OSDs is a maginitude less than it is supposed to be.
When dumping statistics of OSDs such as running command "ceph osd df",
the sum of OSDs' size is 2^10 times less than their real size.

Signed-off-by: Zhe Zhang <zzxuanyuan@gmail.com>
2015-05-05 18:08:48 -05:00
Dmitry Yatsushkevich
a87ac4d36b obj_bencher: aio_bench - rename op_size to object_size
Rename aio_bench argument 'op_size' to 'object_size' to reflect the reality of
how it is used.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
0dea11d907 rados cli: add preventing using --block-size with bench seq and rand
Add handling of situation when '-b|--block-size' option is using with
'bench seq' or 'bench rand' which is erroneous according to the help message:
     -b op_size
            set the size of write ops for put or benchmarking

Fixes: #10163

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
1d1d0aa01b obj_bencher: remove excess 'object_size = op_size'
Remove excess branch 'else {object_size = op_size;}' from
'if(operation != OP_WRITE){...}else{object_size = op_size;}' because
'object_size' variable already initialized with 'op_size'

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Dmitry Yatsushkevich
31d16e9010 obj_bencher: remove 'trans_size' as obsolete
'trans_size' description from header: "size of the write/read to perform"
But really 'object_size' is used in write/read operations. 'trans_size' is used
only in ObjBencher::status_printer for calc current and average bandwidth.
As result - bad statistics in case 'trans_size' and and 'object_size' are different.

Signed-off-by: Dmitry Yatsushkevich <dyatsushkevich@mirantis.com>
2015-05-05 14:50:01 -07:00
Sage Weil
e324578b1e Merge pull request #4554 from liewegas/wip-cleanup
9.0.0 release notes, scripts cleanup

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 14:47:42 -07:00
Sage Weil
538e6eafb9 doc/release-notes: 9.0.0
Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-05 14:37:04 -07:00
Sage Weil
c9a6e60eee src/script: remove obsolete scripts
These are mostly from gather graph data for very early versions of ceph and
are not useful anymore.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-05-05 13:35:29 -07:00
Loic Dachary
2e9e3ba421 Merge pull request #4550 from ktdreyer/wip-submittingpatches-backports
SubmittingPatches: clarify backport procedure

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 19:42:21 +02:00
Ken Dreyer
9282f15d05 SubmittingPatches: clarify backport procedure
Developers should not add "Backport: " fields to Git commits, because
this data is immutable after the commits are merged. It makes more sense
to handle this information in Redmine instead.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2015-05-05 11:27:46 -06:00
Loic Dachary
2b5f0fc6ae automake: allow multiple {install,all,...}-local targets
Replace install-data-local: with install-data-local:: (two :) so that
other Makefile.am can have the same target and all get concatenated
instead of being overridden.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 19:16:56 +02:00
Loic Dachary
8c7a781ef2 build: make-debs.sh NPROC overrides make -j
Building a package is disk intensive and running make -j8 on a machine
with a spinner can actually be slower than make. NPROC=1 make-deb.sh
overrides the default value of make -jX.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 19:16:56 +02:00
Loic Dachary
8b7953affc install-deps.sh: pip wheel for python dependencies
For all tox.ini in the source tree which also have requirement.txt
files, create a wheelhouse with the dependencies. This allows tox to
setup test environment with no access to the network with something
like (in tox.ini):

        deps =
          --use-wheel
          --find-links={toxinidir}/wheelhouse
          -r{toxinidir}/requirements.txt
          -r{toxinidir}/test-requirements.txt

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 19:16:50 +02:00
Sage Weil
7f209b4bc8 Merge remote-tracking branch 'gh/next' 2015-05-05 09:53:51 -07:00
Josh Durgin
56be5d6ffc Merge pull request #4530 from ceph/wip-librbd-clean-object
librbd: object map is not properly updated when deleting clean objects

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-05 09:15:49 -07:00
Loic Dachary
9784e5d23d tests: install sudo on ubuntu-12.04 container
Contrary to Ubuntu 14.04 it is not installed by default.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 18:05:01 +02:00
Loic Dachary
125c59bcda tests: erasure coded pools do not allow offset on creation
If a write operation implicitly creates an object on an erasure coded
pool, there cannot be an offset > 0, even if it is properly aligned.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 14:42:08 +02:00
Jianpeng Ma
a4f1256c21 osd: refuse to write a new erasure coded object with an offset > 0
Even if the offset is properly aligned.

http://tracker.ceph.com/issues/11507 Fixes: #11507

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 14:42:08 +02:00
Wido den Hollander
5c9e9da879 rados: Tell that pool removal failed. Not that it doesn't exist.
If removing a pool fails it could have various reasons. The pool
might be protected from removal (nodelete flag).
2015-05-05 14:22:43 +02:00
Loic Dachary
3f50b5e19e Merge pull request #4542 from liewegas/wip-sepia
sepia

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-05 09:14:06 +02:00
Haomai Wang
752a16fa35 AsyncConnection: Avoid lockdep detect failed
Previously we used a atomic field to avoid deadlock, but it still let lockdep
detect failed. So we remove the atomic field and unlock connection firstly.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-05 14:20:48 +08:00
Haomai Wang
c26b21fa79 AsyncConnection: Don't dispatch event when connection is stopped
When marking down connection, previously we will call stop which will dispatch event
regardless of the staus of the connection. If this connection is already down and
its events all has cleaned, we will hit NULL event.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-05 13:53:23 +08:00
Josh Durgin
393b736505 Merge pull request #4387 from wonzhq/async-comp
librbd: don't notify_change on r == -ERESTART when notify async complete

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-04 19:15:34 -07:00
Zhiqiang Wang
63eb4326db librbd: don't notify_change on error when notify async complete
Don't need to increment the refresh_seq since the async op fails.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-05-05 09:07:57 +08:00