Commit Graph

41891 Commits

Author SHA1 Message Date
Loic Dachary
933df0369c tests: fix the main() documentation of ceph-helpers.sh
It was incorrectly documented as calling the setup function. It was
also vague on how the environment was reset.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-27 16:54:25 +02:00
Loic Dachary
a8e4b4fdad tests: remove unused CEPH_HELPER_VERBOSE in ceph-helpers.sh
And undocumented. It may be useful for debugging but it would have to be
properly implemented instead of being half commented out.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-27 16:16:19 +02:00
Ilya Dryomov
eb9dbef2fe doc: fix crush-ruleset-name param description
Specified crush-ruleset-name is required to exist, implicit creation is
going to happen only if crush-ruleset-name wasn't specified on the
command line.  While at it, pool-name is very much a required param.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-27 15:06:37 +03:00
Ilya Dryomov
c00e3933cf OSDMonitor: fix prepare_pool_crush_ruleset() retval
CrushWrapper::get_osd_pool_default_crush_replicated_ruleset() returns
-1 on error.  Returning -EPERM for "No suitable CRUSH ruleset exists"
is wrong.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-27 15:06:37 +03:00
Ilya Dryomov
29f11c7b67 CrushWrapper: validate default replicated ruleset config opt
Validate osd_pool_default_crush_{replicated_ruleset,rule} config
options, in particular when creating pools.  Otherwise "ceph osd pool
create foo <pg_num>" may end up creating pools with non-existent
rulesets.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-27 15:05:10 +03:00
Ilya Dryomov
f032c4098e OSDMap: respect default replicated ruleset config opt in build_simple()
Use id provided by osd_pool_default_crush_{replicated_ruleset,rule}
config options when creating a simple replicated ruleset for an initial
osdmap instead of always making it ruleset 0.  Not doing so may leave
default created pools (currently "rbd") in a broken state with their
crush_ruleset pointing to a non-existent ruleset.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-27 15:05:10 +03:00
Ilya Dryomov
ea4f9421ab tests: a couple tweaks to osd-pool-create.sh
In TEST_default_deprectated_*(), make expected/unexpected vars local
and actually check that rbd, being a default created pool, is set to
use the ruleset specified by conf.

INVALIDRULESET thing in TEST_replicated_pool() is redundant - it is
checked in TEST_replicated_pool_with_ruleset() a bit earlier.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-27 15:05:10 +03:00
Loic Dachary
770bd756b1 Merge pull request #4777 from SUSE/wip-git_ignore_systemd_autotools_template_systemd_ceph_osd
.gitignore: systemd/ceph-osd@.service

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-27 13:21:23 +02:00
Haomai Wang
81faac75b5 debian/copyright: update copyright for test/perf*
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-27 19:03:30 +08:00
Loic Dachary
93653f5983 Merge pull request #4779 from theanalyst/scripts/ceph-release-notes
scripts: Add a helper to make release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-27 11:54:08 +02:00
Abhishek Lekshmanan
cc92872b6f scripts: Add a helper to make release notes
First cut for a `ceph-release-notes` script added which looks at merge
commits and picks out issue numbers. Though this ideally suits for
backport releases workflow where the commit messages always follow a
specific pattern, it is partly useful for preparing release notes for
normal releases as well.

Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
2015-05-27 15:12:58 +05:30
Owen Synge
2879b0cd64 .gitignore: systemd/ceph-osd@.service
systemd/ceph-osd@.service is now auto-generated by autotools.

This means the file should be added to gitignored list.

Signed-off-by: Owen Synge <osynge@suse.com>
2015-05-27 10:25:03 +02:00
Haomai Wang
f814298008 packaging: include ceph_perf_local
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-27 12:51:50 +08:00
Haomai Wang
61d70db320 ceph_perf_local: rename from ceph_perf
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-27 12:51:36 +08:00
Loic Dachary
9721d422fa Merge pull request #4708 from dachary/wip-erasure-code-profile
erasure-code: fix error reporting

Reviewed-by: Takeshi Miyamae <miyamae.takeshi@jp.fujitsu.com>
2015-05-27 01:17:30 +02:00
Sage Weil
5a216b1470 Merge pull request #4768 from SUSE/wip-systemd_autotools_template_systemd_ceph_osd
Template systemd/ceph-osd@.service with autotools,

Reviewed-by: Sage Weil <sage@redhat.com>
2015-05-26 15:37:20 -07:00
Samuel Just
ff79959c03 osd_types::is_new_interval: size change triggers new interval
Fixes: 11771
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-05-26 12:58:07 -07:00
Loic Dachary
e2f392efdb Merge pull request #4761 from dachary/wip-doc-workflow
doc: update the development workflow

Reviewed-by: Nathan Cutler <ncutler@suse.cz>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-05-26 21:07:05 +02:00
Loic Dachary
101287d8ac Merge pull request #4772 from theanalyst/doc/release/v0.94.2-updates
doc: updates to v0.94.2 draft release notes

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-26 20:50:24 +02:00
Josh Durgin
d925aeb071 Merge pull request #4318 from ceph/wip-11056
librbd: avoid blocking AIO API methods

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-26 11:43:29 -07:00
Jason Dillaman
93af38290f PendingReleaseNotes: document changes to librbd's aio_read methods
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:30:22 -04:00
Jason Dillaman
948b15eb52 tests: update librbd AIO tests to remove result code
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
c77bce3311 librbd: AioRequest::send no longer returns a result
The librados calls used by AioRequest::send should always return
zero unless there is a bug.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
9ab42d6131 librbd: internal AIO methods no longer return result
All failures should be returned via the AioCompletion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
b88b88c5df Throttle: added pending_error method to SimpleThrottle
Allow the client of SimpleThrottle to detect an async error
so that it can exit early.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
6d1d0c8678 librbd: add new fail method to AioCompletion
Helper method to handle passing fatal errors generated within
librbd (not from the OSDs) back to the client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
3a7b5e30ef librbd: avoid blocking AIO API methods
Enqueue all AIO API methods within the new librbd thread pool to
reduce the possibility of any blocking operations. To maintain
backwards compatibility with the legacy return codes of the API's
AIO methods, it's still possible to block attempting to acquire
the snap_lock.

Fixes: #11056
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
afb896d91f librbd: add task pool / work queue for requests / callbacks
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
b3f5a75332 WorkQueue: added virtual destructor
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Jason Dillaman
24a33e977f WorkQueue: add new ContextWQ work queue
The queue holds a collection of Context pointers that will
be completed by the thread pool.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-05-26 14:25:19 -04:00
Abhishek Lekshmanan
a7bce20341 doc: updates to v0.94.2 draft release notes
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@ril.com>
2015-05-26 23:53:50 +05:30
Loic Dachary
d7a2349188 doc: update the development workflow
* use HOWTO_monitor_the_automated_tests_AKA_nightlies to explain the nightlies
* replace references to Serverity with Backport tracker
* add links to the backporter manual and the release page
* s/0.95/9.0.0/
* unify release names to be lowercase
* replace lifecycle with release cycle and end of life with retirement
* Prefer LTS or Long Term Stable over Long Term Support

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-26 19:11:50 +02:00
Josh Durgin
62b0f88ee0 Merge pull request #4756 from ceph/wip-mount-timeout-doc
rbd: document mount_timeout in the man page

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-05-26 10:07:11 -07:00
Owen Synge
ac347dc340 Template systemd/ceph-osd@.service with autotools,
The libexec path is different for different distributions.
systemd. This path is defined by a new variable on the
configure path.

This variable can be set with enviroment SYSTEMD_LIBEXEC_DIR.
The parameter --with-systemd-libexec-dir overrides the enviroment
variable.

Appropriate conditionals are set for SUSE and RHEL derivatives.

This is then used to template out systemd/ceph-osd@.service

Signed-off-by: Owen Synge <osynge@suse.com>
2015-05-26 19:04:22 +02:00
Loic Dachary
b37807c8a8 Merge pull request #4746 from ceph/wip-blacklist-json
osd: fix blacklist field in OSDMap::dump

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-05-26 13:14:14 +02:00
John Spray
38a319d515 qa/cephtool: add blacklist json output check
...not very elegantly because this is bash, but
at least check the expected value is somewhere
present in the JSON output.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-26 10:58:32 +01:00
John Spray
8ef6f8600e osd: fix blacklist field in OSDMap::dump
This was using an array_section so we were getting
a list of only the times, instead of an array
mapping addr to time.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-05-26 10:58:32 +01:00
Haomai Wang
4cc0f2f42c KeyValueStore: Add collect_metadata support
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-26 12:40:19 +08:00
Haomai Wang
7b5fc50005 KeyValueStore: Avoid extra lookup for map
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-26 12:34:08 +08:00
Haomai Wang
e805b944b9 Merge pull request #4757 from xinxinsh/wip-kv-cleanup
os : remove GenericObjectMap::sync() function

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>
2015-05-26 11:57:10 +08:00
Yan, Zheng
70198708c5 Merge pull request #4755 from ceph/wip-11752
mds: fix use-after-free in SessionMap::remove_session
2015-05-26 09:18:05 +08:00
xinxin shu
81130516f1 os : remove unused GenericObjectMap::sync() funtion since no caller invoke this function
Signed-off-by: xinxin shu <xinxin.shu@intel.com>
2015-05-26 03:38:50 +08:00
Loic Dachary
db7936ae1c erasure-code: implement consistent error stream
The error stream in the erasure code path is broken and the error
message is sometime not reported back to the user. For instance the
ErasureCodePlugin::factory method has no error stream: when an error
happens the user is left with a cryptic error code that needs lookup in
the sources to figure it out.

The error stream is made more systematic by:

  * always pass it as ostream *ss (instead of something passing it as
    a reference and sometime as a stringstream)

  * ostream *ss is added to ErasureCodePlugin::factory

  * define the ErasureCodeInterface::init pure virtual. It is
    already implemented by all plugins, only in slightly different
    ways. The ostream *ss is added so the init function has a way to
    report error in a human readable way to the caller, in addition to
    the error code.

The ErasureCodePluginJerasure::init return value was incorrectly ignored
when called from ErasureCodePluginJerasure::factory and now returns when
it fails.

The ErasureCodeLrc::layers_init method is given ostream *ss for error
messages instead of printing them via derr.

The ErasureCodePluginLrc::factory method no longer prints errors via
derr: this workaround is made unnecessary by the ostream *ss argument.

The ErasureCodeShec::init ostream *ss argument is ignored. The
ErasureCodeShec::parse method entirely relies on derr to report errors
and converting it goes beyond the scope of this cleanup. There is a
slight risk of getting it wrong and it deserves a separate commit and
careful and independent review.

The PGBackend, OSDMonitor.{cc,h} changes are only about prototype
changes.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:59:02 +02:00
Loic Dachary
0822922566 erasure-code: do not leak shec instance on failure
If the shec plugin fails to initialize the instance, it must be deleted
before returning to the caller, otherwise it will be leaked.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:59:02 +02:00
Loic Dachary
6ca60061bb erasure-code: lrc size test depends on layer semantic
When the lrc layers are defined, the semantic of the D,c and _
characters are defined, the rest is undefined. The test that verifies
the guard against layers of different size uses the A character which
is undefined. Depending on the implementation, the size test could fail
because the A character is undefined and a guard to forbid undefined
characters is added. Replace A with D to make sure the undefined
character A will not interfere with the test.

This may seem nitpicking but it actually caused problems after a code
refactor that will appear in a few commits from here.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:59:02 +02:00
Loic Dachary
21036cf086 erasure-code: define the ErasureCodeProfile type
Instead of map<string,string>. Make it a non const when initializing
an ErasureCodeInterface instance so that it can be modified.

Rename parameters into profile for consistency with the user
documentation. The parameters name was chosen before the user interface
was defined. This cosmetic update is made in the context of larger
functional changes to improve error reporting and user interface
consistency.

The init() method are made to accept non const parameters.  It is
desirable for them to be able to modify the profile so that is
accurately reflects the values that are used. The caller may use this
information for better error reporting.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-05-25 16:49:44 +02:00
Ilya Dryomov
52440c4b97 rbd: document mount_timeout in the man page
With "rbd: timeout watch teardown on unmap with mount_timeout" going
into kernel 4.2, document its effect in the man page.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2015-05-25 15:41:51 +03:00
Yehuda Sadeh
51bf619b5e Merge pull request #4745 from jmunhoz/object-copy-bug
rgw: Use attrs from source bucket on copy

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2015-05-25 00:05:11 -07:00
Yan, Zheng
2daaa61bc2 mds: fix use-after-free in SessionMap::remove_session
Fixes: #11752
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2015-05-25 11:46:06 +08:00
Loic Dachary
d49d8166a0 Merge pull request #4709 from dachary/wip-shec-corpus
erasure-code: update ceph-erasure-code-corpus for shec
2015-05-23 10:54:12 +02:00