Commit Graph

41773 Commits

Author SHA1 Message Date
Ilya Dryomov
ea4f9421ab tests: a couple tweaks to osd-pool-create.sh
In TEST_default_deprectated_*(), make expected/unexpected vars local
and actually check that rbd, being a default created pool, is set to
use the ruleset specified by conf.

INVALIDRULESET thing in TEST_replicated_pool() is redundant - it is
checked in TEST_replicated_pool_with_ruleset() a bit earlier.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-27 15:05:10 +03:00
Loic Dachary
770bd756b1 Merge pull request #4777 from SUSE/wip-git_ignore_systemd_autotools_template_systemd_ceph_osd
.gitignore: systemd/ceph-osd@.service

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-27 13:21:23 +02:00
Loic Dachary
93653f5983 Merge pull request #4779 from theanalyst/scripts/ceph-release-notes
scripts: Add a helper to make release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-27 11:54:08 +02:00
Abhishek Lekshmanan
cc92872b6f scripts: Add a helper to make release notes
First cut for a `ceph-release-notes` script added which looks at merge
commits and picks out issue numbers. Though this ideally suits for
backport releases workflow where the commit messages always follow a
specific pattern, it is partly useful for preparing release notes for
normal releases as well.

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
2015-05-27 15:12:58 +05:30
Owen Synge
2879b0cd64 .gitignore: systemd/ceph-osd@.service
systemd/ceph-osd@.service is now auto-generated by autotools.

This means the file should be added to gitignored list.

Signed-off-by: Owen Synge <osynge@suse.com>
2015-05-27 10:25:03 +02:00
Loic Dachary
9721d422fa Merge pull request #4708 from dachary/wip-erasure-code-profile
erasure-code: fix error reporting

Reviewed-by: Takeshi Miyamae <miyamae.takeshi@jp.fujitsu.com>
2015-05-27 01:17:30 +02:00
Sage Weil
5a216b1470 Merge pull request #4768 from SUSE/wip-systemd_autotools_template_systemd_ceph_osd
Template systemd/ceph-osd@.service with autotools,

Reviewed-by: Sage Weil <sage@redhat.com>
2015-05-26 15:37:20 -07:00
Loic Dachary
e2f392efdb Merge pull request #4761 from dachary/wip-doc-workflow
doc: update the development workflow

Reviewed-by: Nathan Cutler <ncutler@suse.cz>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-26 21:07:05 +02:00
Loic Dachary
101287d8ac Merge pull request #4772 from theanalyst/doc/release/v0.94.2-updates
doc: updates to v0.94.2 draft release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-26 20:50:24 +02:00
Josh Durgin
d925aeb071 Merge pull request #4318 from ceph/wip-11056
librbd: avoid blocking AIO API methods

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-26 11:43:29 -07:00
Jason Dillaman
93af38290f PendingReleaseNotes: document changes to librbd's aio_read methods
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:30:22 -04:00
Jason Dillaman
948b15eb52 tests: update librbd AIO tests to remove result code
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
c77bce3311 librbd: AioRequest::send no longer returns a result
The librados calls used by AioRequest::send should always return
zero unless there is a bug.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
9ab42d6131 librbd: internal AIO methods no longer return result
All failures should be returned via the AioCompletion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
b88b88c5df Throttle: added pending_error method to SimpleThrottle
Allow the client of SimpleThrottle to detect an async error
so that it can exit early.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
6d1d0c8678 librbd: add new fail method to AioCompletion
Helper method to handle passing fatal errors generated within
librbd (not from the OSDs) back to the client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
3a7b5e30ef librbd: avoid blocking AIO API methods
Enqueue all AIO API methods within the new librbd thread pool to
reduce the possibility of any blocking operations. To maintain
backwards compatibility with the legacy return codes of the API's
AIO methods, it's still possible to block attempting to acquire
the snap_lock.

Fixes: #11056
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
afb896d91f librbd: add task pool / work queue for requests / callbacks
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
b3f5a75332 WorkQueue: added virtual destructor
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
24a33e977f WorkQueue: add new ContextWQ work queue
The queue holds a collection of Context pointers that will
be completed by the thread pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Abhishek Lekshmanan
a7bce20341 doc: updates to v0.94.2 draft release notes
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
2015-05-26 23:53:50 +05:30
Loic Dachary
d7a2349188 doc: update the development workflow
* use HOWTO_monitor_the_automated_tests_AKA_nightlies to explain the nightlies
* replace references to Serverity with Backport tracker
* add links to the backporter manual and the release page
* s/0.95/9.0.0/
* unify release names to be lowercase
* replace lifecycle with release cycle and end of life with retirement
* Prefer LTS or Long Term Stable over Long Term Support

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-26 19:11:50 +02:00
Josh Durgin
62b0f88ee0 Merge pull request #4756 from ceph/wip-mount-timeout-doc
rbd: document mount_timeout in the man page

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-26 10:07:11 -07:00
Owen Synge
ac347dc340 Template systemd/ceph-osd@.service with autotools,
The libexec path is different for different distributions.
systemd. This path is defined by a new variable on the
configure path.

This variable can be set with enviroment SYSTEMD_LIBEXEC_DIR.
The parameter --with-systemd-libexec-dir overrides the enviroment
variable.

Appropriate conditionals are set for SUSE and RHEL derivatives.

This is then used to template out systemd/ceph-osd@.service

Signed-off-by: Owen Synge <osynge@suse.com>
2015-05-26 19:04:22 +02:00
Loic Dachary
b37807c8a8 Merge pull request #4746 from ceph/wip-blacklist-json
osd: fix blacklist field in OSDMap::dump

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-26 13:14:14 +02:00
John Spray
38a319d515 qa/cephtool: add blacklist json output check
...not very elegantly because this is bash, but
at least check the expected value is somewhere
present in the JSON output.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-26 10:58:32 +01:00
John Spray
8ef6f8600e osd: fix blacklist field in OSDMap::dump
This was using an array_section so we were getting
a list of only the times, instead of an array
mapping addr to time.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-26 10:58:32 +01:00
Haomai Wang
e805b944b9 Merge pull request #4757 from xinxinsh/wip-kv-cleanup
os : remove GenericObjectMap::sync() function

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-26 11:57:10 +08:00
Yan, Zheng
70198708c5 Merge pull request #4755 from ceph/wip-11752
mds: fix use-after-free in SessionMap::remove_session
2015-05-26 09:18:05 +08:00
xinxin shu
81130516f1 os : remove unused GenericObjectMap::sync() funtion since no caller invoke this function
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2015-05-26 03:38:50 +08:00
Loic Dachary
db7936ae1c erasure-code: implement consistent error stream
The error stream in the erasure code path is broken and the error
message is sometime not reported back to the user. For instance the
ErasureCodePlugin::factory method has no error stream: when an error
happens the user is left with a cryptic error code that needs lookup in
the sources to figure it out.

The error stream is made more systematic by:

  * always pass it as ostream *ss (instead of something passing it as
    a reference and sometime as a stringstream)

  * ostream *ss is added to ErasureCodePlugin::factory

  * define the ErasureCodeInterface::init pure virtual. It is
    already implemented by all plugins, only in slightly different
    ways. The ostream *ss is added so the init function has a way to
    report error in a human readable way to the caller, in addition to
    the error code.

The ErasureCodePluginJerasure::init return value was incorrectly ignored
when called from ErasureCodePluginJerasure::factory and now returns when
it fails.

The ErasureCodeLrc::layers_init method is given ostream *ss for error
messages instead of printing them via derr.

The ErasureCodePluginLrc::factory method no longer prints errors via
derr: this workaround is made unnecessary by the ostream *ss argument.

The ErasureCodeShec::init ostream *ss argument is ignored. The
ErasureCodeShec::parse method entirely relies on derr to report errors
and converting it goes beyond the scope of this cleanup. There is a
slight risk of getting it wrong and it deserves a separate commit and
careful and independent review.

The PGBackend, OSDMonitor.{cc,h} changes are only about prototype
changes.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:59:02 +02:00
Loic Dachary
0822922566 erasure-code: do not leak shec instance on failure
If the shec plugin fails to initialize the instance, it must be deleted
before returning to the caller, otherwise it will be leaked.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:59:02 +02:00
Loic Dachary
6ca60061bb erasure-code: lrc size test depends on layer semantic
When the lrc layers are defined, the semantic of the D,c and _
characters are defined, the rest is undefined. The test that verifies
the guard against layers of different size uses the A character which
is undefined. Depending on the implementation, the size test could fail
because the A character is undefined and a guard to forbid undefined
characters is added. Replace A with D to make sure the undefined
character A will not interfere with the test.

This may seem nitpicking but it actually caused problems after a code
refactor that will appear in a few commits from here.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:59:02 +02:00
Loic Dachary
21036cf086 erasure-code: define the ErasureCodeProfile type
Instead of map<string,string>. Make it a non const when initializing
an ErasureCodeInterface instance so that it can be modified.

Rename parameters into profile for consistency with the user
documentation. The parameters name was chosen before the user interface
was defined. This cosmetic update is made in the context of larger
functional changes to improve error reporting and user interface
consistency.

The init() method are made to accept non const parameters.  It is
desirable for them to be able to modify the profile so that is
accurately reflects the values that are used. The caller may use this
information for better error reporting.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:49:44 +02:00
Ilya Dryomov
52440c4b97 rbd: document mount_timeout in the man page
With "rbd: timeout watch teardown on unmap with mount_timeout" going
into kernel 4.2, document its effect in the man page.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-25 15:41:51 +03:00
Yehuda Sadeh
51bf619b5e Merge pull request #4745 from jmunhoz/object-copy-bug
rgw: Use attrs from source bucket on copy

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-05-25 00:05:11 -07:00
Yan, Zheng
2daaa61bc2 mds: fix use-after-free in SessionMap::remove_session
Fixes: #11752
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-05-25 11:46:06 +08:00
Loic Dachary
d49d8166a0 Merge pull request #4709 from dachary/wip-shec-corpus
erasure-code: update ceph-erasure-code-corpus for shec
2015-05-23 10:54:12 +02:00
Loic Dachary
4507cb235d Merge pull request #4751 from islepnev/wip-11612
Support NVMe device partitions by ceph-disk

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-23 09:05:03 +02:00
islepnev
9b62cf254d ceph-disk: support NVMe device partitions
Linux nvme kernel module v0.9 enumerate devices as following:

/dev/nvme0 - characted revice
/dev/nvme0n1 - whole block device
/dev/nvme0n1p1 - first partition
/dev/nvme0n1p2 - second partition

http://tracker.ceph.com/issues/11612 Fixes: #11612

Signed-off-by: Ilja Slepnev <islepnev@gmail.com>
2015-05-23 00:51:45 +03:00
Kefu Chai
f36344f07f Merge pull request #4736 from tchaikov/wip-11693-only-restart-crashed-osds
test/test-erasure-code: spin off EIO tests to avoid lingering OSDs after tests finish

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-23 03:56:34 +08:00
Josh Durgin
87ec95fb37 Merge pull request #4748 from ceph/wip-11562
dev/rbd-diff: clarify encoding of image size

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-22 12:43:46 -07:00
Kefu Chai
f417eda9a4 tests/test-erasure-code: spin off eio tests into another testsuite
* since the eio tests crashes some of the OSD nodes, before the
  change, the tests try to undo the crash before moving on, so it
  won't interfere with following tests. a more robust/clean way to
  do this is to isolate individual tests in a sandbox, so each eio
  test will have its own:
    setup + inject + verify crash + teardown
  cycle. this change helps to remove the cleanup/undo steps in
  invidual test.
* update the disabled tests accordingly.
* use a minimum set of OSDs and R-S(2,1) for the testing to speed
  up the test.
* add the new testsuite to check_SCRIPTS

Fixes: #11693
Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-23 03:22:41 +08:00
Kefu Chai
2230deffce tests: fix the get_config()
* the "daemon" parameter was not respected.
* update the test_get_config() to check the overrided option instead of
  the default one.
* add set_config()

Signed-off-by: Kefu Chai <kchai@redhat.com>
2015-05-23 03:19:23 +08:00
Loic Dachary
09a6457296 Merge pull request #4749 from ddiss/ceph_disk_test_fix
tests: don't choke on deleted losetup paths

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-22 20:45:16 +02:00
David Disseldorp
7c1bae5624 tests: don't choke on deleted losetup paths
If a file has been deleted with a loopback device attached, then the
`losetup --all` output will carry:
/dev/loopX: [0032]:344213 (/.../src/test-ceph-disk/vdf.disk (deleted))

This causes the losetup parsing in reset_leftover_dev() to throw an
error, e.g.:
rreset_leftover_dev: 430: test
'(/home/ddiss/ceph/src/test-ceph-disk/vdf.disk' '(deleted))' =
'(/home/ddiss/ceph/src/test-ceph-disk/vdf.disk)'
test/ceph-disk.sh: line 430: test: too many arguments

Fix this by quoting the path variable for the string comparison.

Signed-off-by: David Disseldorp <ddiss@suse.de>
2015-05-22 17:22:51 +02:00
Jason Dillaman
f9ba711c30 dev/rbd-diff: clarify encoding of image size
Fixes: #11562
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-22 11:18:09 -04:00
Loic Dachary
7defc06962 Merge pull request #4512 from hjwsm1989/init-ceph
init-ceph.in: Create osd data dir if don't exists.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-22 16:19:28 +02:00
Loic Dachary
9f0d2da72f Merge pull request #4740 from ktdreyer/wip-11688-doc-firewall-ports
#11688: doc: update OSD/MDS firewall port list

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-22 14:32:04 +02:00
Kefu Chai
6a7aa6c552 Merge pull request #4734 from wonzhq/aio-completion
test/aio: fix the leak of aio completion

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-22 17:57:48 +08:00