Commit Graph

56083 Commits

Author SHA1 Message Date
Jason Dillaman
2666d36664 journal: replay should only read from a single object set
Previously it was prefetching up to 2 object sets worth of journal
data objects which consumed too much memory.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-21 12:52:57 -04:00
Jason Dillaman
8c1877b82f journal: optionally restrict maximum entry payload size
Journal playback will need to read at least a full entry which was
currently limited to the maximum object size. In memory constrained
environment, this new optional limit will set a fix upper bound on
memory usage.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-21 12:52:57 -04:00
Jason Dillaman
f7362e9a57 journal: optionally fetch entries in small chunks during replay
Support fetching the full object or incremental chunks (with a
minimum of at least a single decoded entry if available).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-21 12:52:57 -04:00
Jason Dillaman
dad8328f2d journal: helper class for organizing optional settings
Additional runtime configuration settings will be needed. The
new class will avoid the need to expand the constructor.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-07-21 12:52:57 -04:00
Daniel Gryniewicz
a0294e6450 Revert RGW torrent
Revert the RGW torrent commit for now, it was causing issues.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2016-07-21 12:02:58 -04:00
Kefu Chai
0555abadf0 Merge pull request #10377 from chhabaramesh/extent_alloc
os/bluestore:Fix size calculation in bitallocator

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-21 23:01:35 +08:00
Daniel Gryniewicz
f2bea590f7 Merge pull request #9589 from zhouruisong/master
rgw:add a s3 API of make torrent for a object
2016-07-21 09:54:19 -04:00
John Spray
a2e7ea9c78 Merge pull request #10244 from jcsp/wip-cephfs-health-doc
doc/cephfs: explain the various health messages

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-07-21 12:32:45 +01:00
John Spray
ef1405ab19 doc/cephfs: explain the various health messages
Signed-off-by: John Spray <john.spray@redhat.com>
2016-07-21 12:32:05 +01:00
Ramesh Chander
0be5d5f973 os/bluestore:Fix size calculation in bitallocator
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>
2016-07-21 02:47:15 -07:00
Kefu Chai
88f383c7ed Merge pull request #10149 from wjwithagen/patch-12
CMakeList.txt: link ceph_objectstore_tool against fuse only if WITH_FUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-21 17:22:10 +08:00
Willem Jan Withagen
2f7264e815 src/tools/CMakeLists.txt: conditionalise the FUSE usage
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-21 09:59:08 +02:00
Kefu Chai
278ea12b12 Merge pull request #10366 from ceph/wip-cmake-debian
debian: bump compat to 9

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>
2016-07-21 11:00:55 +08:00
Kefu Chai
390ac1d630 Merge pull request #10361 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-21 10:42:15 +08:00
Yehuda Sadeh
1b1c86f25c Merge pull request #10355 from yehudasa/wip-16742
rgw: back off bucket sync on failures, don't store marker

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-20 15:43:40 -07:00
Yehuda Sadeh
aacc31f72c rgw: fix collection of object sync errors
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 14:50:18 -07:00
Yehuda Sadeh
f5801c726e rgw: fix marker tracker completion handling
Was not tracking high markers correctly. Could only work if there was a single
hole in the completion range. Just keep a map of all the complete entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 14:50:18 -07:00
Yehuda Sadeh
5528932e4c rgw: collect() stops if error encoutered
and returns true if needed to be called again

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 12:42:52 -07:00
Kefu Chai
c1c6cfc49c debian: bump compat to 9
dh_auto_configure set -DCMAKE_BUILD_TYPE=None so the default does not
take effect at all. this is on purpose, see [1].

and dpkg-buildflags is able to produce the suggested compiling flags by
debian policy. dh_* can pass these flags to cmake only if it works at
compat>=9, see [2].

and we are safe to move to compat 9, as jewel's supported debian based
distro is jessie (with debhelper 9.20150101), ubuntu trusty (with
debhelper 9.20131227ubuntu1)

---
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701233#35
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653916

Fixes: http://tracker.ceph.com/issues/16744
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-21 02:02:58 +08:00
Casey Bodley
d21a3cd3f2 Merge pull request #10368 from oritwas/wip-rgw-upgrade-period-commit
rgw: fix upgrade from old multisite to new multisite configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-20 13:53:56 -04:00
Ali Maredia
31b2f483d5 Merge pull request #10360 from ceph/wip-include-include
cmake: add_subdirectory(include)

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-20 12:56:34 -04:00
John Spray
e0a1321ced Merge pull request #10315 from onyb/wip-cephfs-pypi
pybind: Rework cephfs/setup.py for PyPI

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-20 17:34:05 +01:00
John Spray
297500399e Merge pull request #10316 from batrick/mds-standby-assertions
mds: add assertions for standby_daemons invariant

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:43:43 +01:00
John Spray
92c589032f Merge pull request #10347 from renhwztetecs/renhw-wip-test-add-fsdump
test/mds: add fs dump in test_ceph_argparse.py

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:43:00 +01:00
John Spray
27574785df Merge pull request #10202 from xiaoxichen/fix_16588
mds/MDSMap default metadata pool to -1 (was: output None instead of 0 when no fs present.)

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:36:05 +01:00
John Spray
6e056f6b2f Merge pull request #9864 from ajarr/wip-15615
ceph_volume_client: store authentication metadata

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:15:18 +01:00
Mark Nelson
12257c2df2 Merge pull request #10327 from xiexingguo/xxg-wip-bluestore-2016-07-18
os/bluestore: misc fixes/cleanups

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR may have contributed to write performance improvements along with #10257.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-20 08:28:40 -05:00
Mark Nelson
c3b632e4d5 Merge pull request #10257 from chhabaramesh/extent_alloc
os/bluestore: extent alloc functionality for stupid and bitmap allocator

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR appears to result in a large sequential write performance increase (Up to ~3.5X), and may contribute to a small random write performance increase as well.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-20 08:22:31 -05:00
Orit Wasserman
6d7841c6f0 rgw: fix upgrade from old multisite to new multisite configuration
We need to store the updated current period after adding the old converted regions

Fixes: http://tracker.ceph.com/issues/16751
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2016-07-20 14:00:43 +02:00
Kefu Chai
73f46a9066 Merge pull request #9082 from stiopaa1/osd_TestOpsSocketHook_removeUnneededParam
pass string by const ref and add override to virtual function

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-20 18:07:40 +08:00
Boris Ranto
ce74e3bdc2 Merge pull request #9667 from SUSE/wip-15984
ceph-osd-prestart.sh: drop Upstart-specific code

Reviewed-by: Boris Ranto <branto@redhat.com>
2016-07-20 11:01:43 +02:00
Kefu Chai
28819d35a7 Merge pull request #10339 from tchaikov/wip-16675
selinux: allow read /proc/<pid>/cmdline

Reviewed-by: Boris Ranto <branto@redhat.com>
2016-07-20 17:00:38 +08:00
Kefu Chai
e8a16b7fda cmake: remove unused Findfio.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:22:29 +08:00
Kefu Chai
b5607f88e8 cmake: Findfcgi.cmake cleanups
FindPackageHandleStandardArgs() takes care of the find_package()
boilderplate stuff. so no need to repeat them. and remove the checkings
in env variables.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:21:06 +08:00
Kefu Chai
21a852fd65 cmake: do not include $lrc_srcs in unittest_erasure_code_lrc
as ${lrc_srcs} is not defined in the scope where
unittest_erasure_code_lrc is added as a target. and instead we link
unittest_erasure_code_lrc against ec_lrc dynamic library. so we can
safely remove ${lrc_srcs} from unittest_erasure_code_lrc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:01:55 +08:00
Kefu Chai
1df15fd833 cmake: s/EXTRA_LIBS/EXTRALIBS/
it's a typo, i think. as EXTRA_LIBS is never defined anywhere.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:56:25 +08:00
Kefu Chai
c3bcb25b5b cmake: s/HAVE_FUSE/WITH_FUSE/ s/HAVE_LIBFUSE/WITH_FUSE/
we don't define HAVE_FUSE, and HAVE_LIBFUSE is defined to be compatible
with autotools. so use WITH_FUSE in cmake whenever possible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:29:43 +08:00
Kefu Chai
48da9f94cf cmake: remove $UNITTEST_CXX_FLAGS from ceph-kvstore-tool's linked libs
it does not make sense to link against CXX_FLAGS.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:16:39 +08:00
Kefu Chai
ec84d7169b cmake: CMAKE_MODULE_PATH is a list and empty by default
we should not reference it using ${CMAKE_MODULE_PATH} before assigning
to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:10:48 +08:00
Kefu Chai
8582aa8608 cmake: add_subdirectory(include)
fixes the packaging failure introduced by 741f990

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
25c76b19a6 cmake: s/PTHREAD_LIBS/CMAKE_THREAD_LIBS_INIT/
${CMAKE_THREAD_LIBS_INIT} is defined by find_package(Thread), while
PTHREAD_LIBS is defined by the acx_pthread.m4.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
b482cf083f cmake: use HAVE_ARMV8_CRC not ${HAVE_ARMV8_CRC}
as this variable could be undefined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
636522b669 cmake: use FindBacktrace instead of Findexecinfo
always better to use upstream cmake modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
cf6dcf0ae7 cmake: add "WITH_PROFILER" option
this option matches '--with-profiler' option in autoconf. and it is off
by default. we should not link against libprofiler unless asked to do
so. this change fixes this problem.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Nathan Cutler
0a44aaa74c Merge pull request #10353 from rjfd/wip-fix-ceph-mount
rpm: Fix creation of mount.ceph symbolic link for SUSE distros

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2016-07-20 04:22:21 +02:00
Kefu Chai
f0426fbf54 Merge pull request #10351 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-20 10:06:29 +08:00
Yehuda Sadeh
1f3fec8070 rgw: back off bucket sync on failures, don't store marker
Fixes: http://tracker.ceph.com/issues/16742

If we fail on any single entry in bucket, skip updating the marker tracker
so that next time we'll go over that entry, and back off. This will trigger
a report to the data sync error repo and eventually a retry on the failing
object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-19 15:37:41 -07:00
Mark Nelson
472a5d8d6c Merge pull request #10295 from xiexingguo/xxg-wip-bluestore-2016-07-14
kv/MemDB: misc fixes and cleanups

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:16:29 -05:00
Mark Nelson
dd4de0f786 Merge pull request #10294 from xiexingguo/xxg-wip-bluestore-2016-07-13
os/bluestore: use BE for gifting and reclaiming from bluefs

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:15:25 -05:00
Mark Nelson
81c10210af Merge pull request #10277 from xiexingguo/xxg-wip-fix-posix-fallocate
os/bluestore: fix error handling of posix_fallocate()

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:12:13 -05:00