Commit Graph

38535 Commits

Author SHA1 Message Date
Samuel Just
733ce25ccb Merge pull request #3302 from ceph/wip-9956
os/FileStore: verify kernel is new enough before using extsize ioctl

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-13 10:54:21 -08:00
Samuel Just
c82f8177f7 Merge pull request #3305 from majianpeng/fix5
fix bugs about sync_filesystem

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-13 10:53:34 -08:00
Sage Weil
9264d2508a common/Formatter: new_formatter -> Formatter::create
Also make the default formatter selection more explicit.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-13 08:32:45 -08:00
Sage Weil
617ad5ddc1 common/Formatter: improve json-pretty whitespace
The whitespace for the current pretty mode is awkward and weird.  Move
to a more standard approach.  It's more newlines, but *much* more
readable.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-13 08:20:08 -08:00
Sage Weil
83c3b1305e common/Formatter: add newline to flushed output if m_pretty
This applies to json-pretty and xml-pretty modes.

Signed-off-by: Sage Weil <sage@inktank.com>
2015-01-13 08:06:17 -08:00
Gregory Farnum
a6f5682d55 Merge pull request #3364 from ceph/wip-quota-test
qa: set -e explicitly in quota test

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-01-13 07:08:30 -08:00
John Spray
b578a53b03 qa: set -e explicitly in quota test
Previously was set in hashbang, which meant
that "./quota.sh" was OK, but "sh ./quota.sh" would
just run through ignoring errors.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-01-13 14:58:57 +00:00
Gregory Farnum
d8e1f675f6 Merge pull request #3336 from ceph/wip-fs-reset
mon: implement `fs reset`

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-01-13 06:47:04 -08:00
Frank Yu
7c664fadff Doc: Fix the indentation in doc/rbd/rbd-snapshot.rst 2015-01-13 18:27:08 +08:00
Loic Dachary
fc7fa3087d Merge pull request #3343 from dachary/wip-10505-centos-parted
tests: install parted in centos Dockerfile

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
2015-01-13 11:07:55 +01:00
Jason Dillaman
1c01c3a456 tests: create unittest_librbd
The existing ceph_test_librbd integration tests are now used
within unittest_librbd as unit tests run during 'make check'.
The unit tests use librados_test_stub to simulate RADOS.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-13 04:49:33 -05:00
Jason Dillaman
c105003bd6 tests: add mock librados API for supporting unit tests
The new librados_test_stub library can be used by unit tests
as a replacement for the librados shared library.  The library
currently provides an in-memory RADOS simulator and will
allow future unit tests to be developed using mocking.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-13 04:49:32 -05:00
Jason Dillaman
1e9da43f36 include: moved RBD's generic FunctionContext wrapper
FunctionContext uses boost::function to allow the Context
interface to invoke arbitrary functions.  This eliminates
the need to create derived Context classes for every use
case.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-13 03:34:56 -05:00
Jianpeng Ma
9ef6ce5647 FileJournal: check fsync/fdatasync result.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-13 15:51:51 +08:00
Jianpeng Ma
3cbe5da1c2 mon: check size must larger than zero.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-13 14:27:36 +08:00
Jianpeng Ma
3f0300358b test/mon: Add test case for ceph osd pool set size/min_size.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-13 14:27:36 +08:00
Jianpeng Ma
e3678f41e3 mon: check min_size range.
When set min_size value:
A:for replicated pool, the value must be between in 1 and size
B:for erasure pool, the value must be between in data_chunk_count)
 and size(data_chunk_count + code_chunk_count).

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2015-01-13 14:27:36 +08:00
Jason Dillaman
7945f8d688 librbd: flush pending AIO requests under all existing flush scenarios
AIO requests that are waiting on the image lock should be flushed
during all existing RBD flush scenarios.  A few flush cases were
missed in the original implementation.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 23:17:50 -05:00
Jason Dillaman
2dd0f038d6 librbd: AIO requests should retry lock requests
Added a timer to support retrying AIO lock requests until
they are successful.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 23:14:11 -05:00
Jason Dillaman
6176ec5fde librbd: differentiate between R/O vs R/W RBD features
The new RBD exclusive lock feature should be treated as a
feature that is only applied when the image is opened in
R/W mode.

Older clients will need to handle the updated
cls_rbd::get_features method in order to properly determine
the incompatible features for an image depending on the
current mode.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 20:02:25 -05:00
Jason Dillaman
544ed961c6 librbd: Add internal unit test cases
The new unit tests cover the modifications made to integrate
the internal librbd functionality with the new ImageWatcher.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 20:02:24 -05:00
Jason Dillaman
17f22d98df librbd: Add ImageWatcher unit test cases
Directly unit test the new ImageWatcher class to complement
the existing librbd integration tests of exclusive lock
handling.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 20:02:06 -05:00
Jason Dillaman
5160281b1a librbd: Add convenience library to support unit tests
Unit tests need access to the private symbols of librbd no
longer exported from librbd.so.  A new librbd_internal
convenience library was created to allow access.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 20:01:21 -05:00
Jason Dillaman
4297e3bf8e rbd: Allow CLI to optionally create shared images
Images that are flagged as shared cannot use the RBD
object map nor RBD mirroring features.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 20:01:21 -05:00
Jason Dillaman
ccadff1445 librbd: Integrate librbd with new exclusive lock feature
Operations that update the image now require the exclusive lock
if the feature is enabled.  AIO write and discard operations will
automatically request the exclusive lock from the current leader
to support live-migration.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 20:01:19 -05:00
Matt Richards
9ee80b3322 librados: bump rados version number
As a follow-on to 49d114f1ff,
increment the "extra" version field so clients can easily
determine if they have a version of librados that properly
translates C API operation flags.

Signed-off-by: Matthew Richards <mattjrichards@gmail.com>
2015-01-12 17:00:52 -08:00
Josh Durgin
d784bc47c4 Merge pull request #3316 from ceph/wip-10471
rgw: index swift keys appropriately

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-01-12 16:20:28 -08:00
Jason Dillaman
f5668d6b9d librbd: Create image exclusive lock watch/notify handler
The new watch/notify handler replaces the existing header
update watch/notify handler and adds support for managing
image exclusive lock leadership.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-01-12 19:01:07 -05:00
Sage Weil
2ecd8748d2 osd: enable filestore_extsize by default
Note that this will only get used if the kernel is new enough; if it is
older than 3.5 the option will get disabled and extsize will not be used
even if the option is set to true.

This partially reverts 01cd3cdc72.

Fixes: #9956
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-12 14:00:21 -08:00
Sage Weil
b76d0dc0f2 os/FileStore: verify kernel is new enough before using extsize ioctl
Old kernels have an XFS bug that exposes uninitialized data when the
extsize hint is set and only partially written.  This is fixed by Linux
commit aff3a9edb7080f69f07fe76a8bd089b3dfa4cb5d, documented in XFS bug
http://oss.sgi.com/bugzilla/show_bug.cgi?id=874, and tested by XFS
test xfs/229 to prevent regressions.

Notably the original bug affects kernel 3.2, which is widely deployed with
ubuntu precise 12.04.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-12 13:59:39 -08:00
Gregory Farnum
c219c43cc2 Merge pull request #3352 from kylinstorage/fix-10503
Fix bug 10503: http://tracker.ceph.com/issues/10503

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-01-12 11:33:02 -08:00
Sage Weil
f4ff12aebc drop ceph_mon_store_converter
This was needed for upgrades in the cuttlefish era; no longer useful.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-12 09:37:08 -08:00
Samuel Just
4ecea7e54e Merge pull request #3203 from majianpeng/fix1
avoid memcopy from librados to caller buffer

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-12 08:39:48 -08:00
Samuel Just
187c85a5c4 Merge pull request #3034 from dachary/wip-10017-erasure-code-repair
erasure code repair when there are two failures

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-12 08:26:08 -08:00
Samuel Just
fb72b9e4b9 Merge pull request #3148 from mslovy/optimazation_wbthrottle
os: WBThrottle: optimize the map to unordered_map

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-01-12 08:23:26 -08:00
John Spray
a4152dbfa1 mon/MDSMonitor: add confirm flag to fs reset
This was already in the command map but was not
being checked.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-01-12 15:00:15 +00:00
John Spray
8630696681 qa: add fs reset to cephtool tests
This is just a superficial "I can call it" test,
it's actual behaviour is checked elsewhere.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-01-12 15:00:15 +00:00
John Spray
1713ffd898 mon: implement fs reset
This is for use in CephFS disaster recovery.  When
the metadata pool has been forcibly reset to a single-MDS
metadata tree, we would like to reset the MDSMap to match.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-01-12 15:00:14 +00:00
Yunchuan Wen
1f694767c3 Fix bug 10503: http://tracker.ceph.com/issues/10503
ceph-fuse: quota code is not 32-bit safe for vxattr output

Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
2015-01-12 05:49:32 +00:00
Yunchuan Wen
c80c2cf3f9 test: modify test/cli/rbd/help.t for merge-diff
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
2015-01-12 01:05:26 +00:00
Yunchuan Wen
d300220aaf rbd: test case for rbd merge-diff
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: MingXin Liu <mingxinliu@ubuntukylin.com>
2015-01-12 01:05:19 +00:00
Yunchuan Wen
337f82197b rbd: merge two diff files into one single file
Note: It does not do the merging work if
either of the source diff files is stripped,
since which complicates the process and is
rarely used

Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: MingXin Liu <mingxinliu@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>
2015-01-12 01:04:39 +00:00
Yunchuan Wen
9c82efcc79 rbd: parse diff file body for diff merging
Signed-off-by: MingXin Liu <mingxinliu@ubuntukylin.com>
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>
2015-01-12 01:03:41 +00:00
Yunchuan Wen
b4720ae6a5 rbd: parse diff file header for diff merging
Signed-off-by: MingXin Liu <mingxinliu@ubuntukylin.com>
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Reviewed-by: Li Wang <liwang@ubuntukylin.com>
2015-01-12 01:02:51 +00:00
Sage Weil
538bad38d3 Merge pull request #2948 from ceph/wip-promote
osd: promote_object separation; proxy read

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-01-11 07:55:08 -08:00
Sage Weil
f42ef1e370 ceph_test_rados: add some debug output
Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-10 20:35:31 -08:00
Sage Weil
1554fbc87e osd/ReplicatedPG: improve proxy read cancelation
Avoid taking the PG lock for a canceled read op (if we are lucky).  Recheck
after the lock is taken for good measure.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-10 20:35:31 -08:00
Sage Weil
1c8b679d0c osd/ReplicatedPG: put proxy read completion on finisher
We can't use the synchronous completion callbacks (in fast dispatch
context) do to the proxy read completion work.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-01-10 20:35:30 -08:00
Zhiqiang Wang
650f1d989b osd: tiering: avoid duplicate promotion on proxy read
Do not promote if it is already undergoing in maybe_handle_cache.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-01-10 20:35:30 -08:00
Zhiqiang Wang
43f13af614 osd: tiering: proxy instead of redirect read in writeback mode when the
cache pool is full

To preserve read op order

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
2015-01-10 20:35:30 -08:00