Commit Graph

30527 Commits

Author SHA1 Message Date
Loic Dachary
0b40bbd495 common: evaluate --show-config* after CEPH_ARGS
The content of CEPH_ARGS is appended to the list of arguments. When
--show-config or --show-config-value is also set, it should be evaluated
after all arguments are parsed to accurately reflect the value that
would be visible to the program.

It failed to do so because the action for --show-config* was carried out
immediately. It is postponed until all options are parsed instead.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-30 11:12:41 +01:00
Sage Weil
38d9613bcb Merge pull request #1008 from ceph/port/spinlock
spinlock: add generic spinlock implementation

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-29 21:09:04 -08:00
Sage Weil
f9b50cee94 Merge pull request #1014 from ceph/port/misc
misc portability patches

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-29 20:58:17 -08:00
Sage Weil
9a7e85dba1 Merge pull request #1015 from yuyuyu101/wip-fix-tests
Fix some test codes problems

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-29 20:44:13 -08:00
Sage Weil
fb941d82e0 Merge pull request #1013 from ceph/port/public-hdrs
use portable types in public headers

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-29 20:41:24 -08:00
Haomai Wang
d7d7ca8451 Fix qa/workunits/rados/test_cache_pool.sh typos
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2013-12-30 12:07:56 +08:00
Haomai Wang
cc67b7b2aa Fix test/filestore/store_test.cc error
Commit
d2b661d0ef
change the test result, just change it.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2013-12-30 12:06:37 +08:00
Noah Watkins
e8e174e791 rados: include struct timeval definition
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:35:58 -08:00
Noah Watkins
356d71a24d inttypes: use portable types in public headers
loff_t -> int64_t
  __u8 -> uint8_t

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:35:58 -08:00
Noah Watkins
76ad85d868 test: include headers for struct statfs
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
93c125ced9 test: remove platform specific header
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
a625b2804d test: print warning about missing fs features
- sync
- posix_fadvise

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
9a7a33ecab test: only build some tests on linux
These tests have portability problems that have yet to be solved on
non-Linux systems.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
a48d038689 test: fix VLA of non-POD type
Variable length array of non-POD type is not supported by clang.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
424d7c7813 gtest: disable tr1/tuple
Not all compilers are supporting tr1/tuple. This forces libgtest to use
an internal implementation of tuple. Alternatively, the newer 1.6
version of gtest may correctly handle this case automatically.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
0c09abd9cc make: use c++ flags for building librados
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
891801ed39 warning: fix typo and -Wmismatched-tags
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:32:00 -08:00
Noah Watkins
c7e1c4bfdb c++11: fix std::lock naming conflicts
Unfortunately, 'using namespace std;' is in pretty widespread use in the Ceph
tree, so we need to rename to avoid the conflict.

Example error output:

test/streamtest.cc:37:19: error: reference to 'lock' is ambiguous
  Mutex::Locker l(lock);
                  ^
test/streamtest.cc:32:7: note: candidate found by name lookup is 'lock'
Mutex lock("streamtest.cc lock");
      ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/mutex:346:1: note: candidate found by name lookup is 'std::__1::lock'
lock(_L0& __l0, _L1& __l1)

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:31:59 -08:00
Noah Watkins
bbcb022319 kvstore: only build on linux
There are several non-standard errno values used. There is still work to
do on addressing errno portability in Ceph, and this disables kvstore on
non-Linux platforms until that work is complete.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-29 13:31:59 -08:00
Sage Weil
f881157e37 Merge pull request #1007 from ceph/wip-misc-fixes
misc fixes

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-12-29 12:04:09 -08:00
Loic Dachary
16699f4032 Merge pull request #1004 from ceph/wip-snaps
make ceph_test_rados read from snaps; resulting bugs found

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-12-29 00:31:39 -08:00
Noah Watkins
1fec818f7c spinlock: add generic spinlock implementation
Adds a ceph_spinlock_t implementation that will use pthread_spinlock_t
if available, and otherwise reverts to pthread_mutex_t. Note that this
spinlock is not intended to be used in process-shared memory.

Switches implementation in:

  ceph_context
  SimpleMessenger
  atomic_t

Only ceph_context initialized its spinlock with PTHREAD_PROCESS_SHARED.
However, there does not appear to be any instance in which CephContext
is allocated in shared memory, and thus can use the default private
memory space behavior.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2013-12-28 14:43:14 -08:00
Sage Weil
12f4631586 qa/workunits/rest/test.py: rbd pool ruleset is now 0
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-28 10:36:27 -08:00
Sage Weil
b286e4fed3 ceph_test_rados_api_tier: retry EBUSY race checks
...or else these will occasionally fail against a thrashing cluster.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-28 10:34:56 -08:00
Sage Weil
b88af07ef5 libcephfs: get osd location on -1 should return EINVAL
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-28 10:25:00 -08:00
Sage Weil
250ecf6655 qa/workunits/mon/crush_ops.sh: fix in-use rule rm test
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-28 10:22:18 -08:00
Sage Weil
d4f07cd90b crush: fix get_full_location_ordered
This should return -ENOENT when an id is not present.  Broken by
746069ee62.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-28 08:55:02 -08:00
Yehuda Sadeh
23f715ba82 Merge pull request #1005 from ceph/wip-rgw-leak
rgw: fix leak of RGWProcess

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
2013-12-27 16:52:30 -08:00
Sage Weil
f9f5c37149 rgw: fix leak of RGWProcess
Introduced by a3e50b09a1.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-27 16:36:21 -08:00
Sage Weil
96fe80dbd7 osd: preserve user_version in snaps/clones
Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-27 16:32:22 -08:00
Sage Weil
80b5487671 ceph_test_rados: test read from snapshots
This was disabled back in 2011, c54aa7db3b.
Whoops!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-27 16:32:21 -08:00
Sage Weil
2f8b602910 osd/OSDMap: observe 'osd crush chooseleaf type' option for initial rules
This option was dropped by 2a7fcc35b8.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-27 13:45:34 -08:00
Josh Durgin
9c068939c4 Merge branch 'rbd-map-options'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-27 09:56:55 -08:00
Ilya Dryomov
9b7364d245 rbd: expose options available to rbd map
Add a -o / --options option, which would allow users to specify
rbd-specific and generic ceph client and osd options available at
mapping time in a comma separated list (similar to mount(8) mount
options).

Exposed options are:

- fsid=%s
- ip=%s
- share
- noshare
- crc
- nocrc
- osdkeepalive=%d
- osd_idle_ttl=%d
- rw
- ro (equivalent to existing --read-only flag)

The rw/ro < 3.7 kernels compatibility kludge added in commit
fb0f198644 is preserved.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2013-12-27 09:56:24 -08:00
Sage Weil
b8f42b6b4d Merge pull request #1001 from dachary/wip-forward-tid
messages: add tid to string form of MForward

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-27 07:59:43 -08:00
Loic Dachary
542f8d307c Merge pull request #1002 from yuyuyu101/wip-7062
Lack of "start" member function declare in WBThrottle.h
make check runs ok

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-12-27 04:28:59 -08:00
Haomai Wang
b3bda085b6 Lack of "start" member function declare in WBThrottle.h
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2013-12-27 18:11:09 +08:00
Loic Dachary
4a9c770953 messages: add tid to string form of MForward
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 07:30:04 +01:00
Sage Weil
b8fb366eab Merge pull request #837 from ceph/port/fallocate
FileJournal: zero-fill in-lieu of posix_fallocate

We may want to change that to a #warning later...

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-26 21:33:39 -08:00
Sage Weil
f5b698cb1d Merge pull request #982 from dachary/wip-default-crush-rule
osd: add default crush rule for erasure pools

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-26 21:29:36 -08:00
Sage Weil
39a9c323dc Merge pull request #974 from dachary/wip-build-depends
packaging: make check needs argparse and uuidgen

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-26 21:26:02 -08:00
Sage Weil
a39226ac9a Merge pull request #994 from yuyuyu101/wip-7062
Fix WBThrottle thread disappear problem

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-26 21:25:08 -08:00
Loic Dachary
67f99f3455 packaging: make check needs argparse and uuidgen
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 06:14:02 +01:00
Josh Durgin
8f3ad4e3b9 Merge pull request #1000 from ceph/wip-rbd-tinc-5426
fix #5426 race in librbd

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-26 18:53:02 -08:00
Josh Durgin
4cea7895da librbd: call user completion after incrementing perfcounters
The perfcounters (and the ictx) are only valid while the image is
still open.  If the librbd user gets the callback for its last I/O,
then closes the image, the ictx and its perfcounters will be
invalid. If the AioCompletion object is has not run the rest of its
complete() method yet, it will access these now-invalid addresses,
possibly leading to a crash.

The AioCompletion object is independent of the ictx and does not
access it again after incrementing perfcounters, so avoid this race by
calling the user's callback after this step. The AioCompletion object
will be cleaned up by the rest of complete_request(), independent of
the ImageCtx.

Fixes: #5426
Backport: dumpling, emperor
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-26 17:40:34 -08:00
Loic Dachary
f8a4001440 osd: create default ruleset for erasure pools
The ruleset --osd_pool_default_crush_erasure_ruleset is created to be
suitable for erasure coded pools when OSDMap::build_simple is required
to build the default OSD map of a new cluster.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 00:28:04 +01:00
Loic Dachary
8b2b5a33bd mon: implement --osd-pool-default-crush-erasure-ruleset
It must be different from the replicated default.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 00:13:47 +01:00
Loic Dachary
dd81858ca6 mon: implement --osd-pool-default-crush-replicated-ruleset
--osd-pool-default-crush-replicated-ruleset replaces
--osd-pool-default-crush-rule

If --osd-pool-default-crush-rule is set it takes precedence over
--osd-pool-default-crush-replicated-ruleset and a deprecation warning is
displayed.

The CrushWrapper::get_osd_pool_default_crush_replicated_ruleset helper is
used to implement this behaviour.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 00:13:47 +01:00
Loic Dachary
2a7fcc35b8 osd: use CrushWrapper::add_simple_ruleset
Replace the manually crafted ruleset in OSDMap::build_simple_crush_map*
with calls to add_simple_ruleset. The generated ruleset do not have the
same behavior but that presumably do not cause any backward
compatibility problem because they are only created when a new cluster
is being initialized.

The prototypes of OSDMap::build_simple* are modified to allow for a
return code and display of a human readable error message.

The --osd-min-rep and --osd-max-rep configuration options are removed :
they were only used in the code that was removed.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 00:13:47 +01:00
Loic Dachary
a10fc025d7 osd: build_simple creates a single rule
The three rules created by build_simple are identical. They are replaced
by a single rule named replicated_rule which is set to be used by the
data, rbd and metadata pools.

Instead of hardcoding the ruleset number to zero, it is read from
osd_pool_default_crush_ruleset which defaults to zero.

The CEPH_DEFAULT_CRUSH_REPLICATED_RULESET enum is moved from osd_type.h to
config.h because it may be needed when osd_type.h is not included.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-12-27 00:13:47 +01:00