Commit Graph

56407 Commits

Author SHA1 Message Date
Yehuda Sadeh
aacc31f72c rgw: fix collection of object sync errors
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 14:50:18 -07:00
Yehuda Sadeh
f5801c726e rgw: fix marker tracker completion handling
Was not tracking high markers correctly. Could only work if there was a single
hole in the completion range. Just keep a map of all the complete entries.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 14:50:18 -07:00
Yehuda Sadeh
5528932e4c rgw: collect() stops if error encoutered
and returns true if needed to be called again

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-20 12:42:52 -07:00
Kefu Chai
c1c6cfc49c debian: bump compat to 9
dh_auto_configure set -DCMAKE_BUILD_TYPE=None so the default does not
take effect at all. this is on purpose, see [1].

and dpkg-buildflags is able to produce the suggested compiling flags by
debian policy. dh_* can pass these flags to cmake only if it works at
compat>=9, see [2].

and we are safe to move to compat 9, as jewel's supported debian based
distro is jessie (with debhelper 9.20150101), ubuntu trusty (with
debhelper 9.20131227ubuntu1)

---
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701233#35
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653916

Fixes: http://tracker.ceph.com/issues/16744
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-21 02:02:58 +08:00
Casey Bodley
d21a3cd3f2 Merge pull request #10368 from oritwas/wip-rgw-upgrade-period-commit
rgw: fix upgrade from old multisite to new multisite configuration

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2016-07-20 13:53:56 -04:00
Ali Maredia
31b2f483d5 Merge pull request #10360 from ceph/wip-include-include
cmake: add_subdirectory(include)

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-20 12:56:34 -04:00
John Spray
e0a1321ced Merge pull request #10315 from onyb/wip-cephfs-pypi
pybind: Rework cephfs/setup.py for PyPI

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-20 17:34:05 +01:00
John Spray
297500399e Merge pull request #10316 from batrick/mds-standby-assertions
mds: add assertions for standby_daemons invariant

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:43:43 +01:00
John Spray
92c589032f Merge pull request #10347 from renhwztetecs/renhw-wip-test-add-fsdump
test/mds: add fs dump in test_ceph_argparse.py

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:43:00 +01:00
John Spray
27574785df Merge pull request #10202 from xiaoxichen/fix_16588
mds/MDSMap default metadata pool to -1 (was: output None instead of 0 when no fs present.)

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:36:05 +01:00
John Spray
6e056f6b2f Merge pull request #9864 from ajarr/wip-15615
ceph_volume_client: store authentication metadata

Reviewed-by: John Spray <john.spray@redhat.com>
2016-07-20 15:15:18 +01:00
Willem Jan Withagen
062bfc7584 README.FreeBSD: Cmake instructions
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-20 15:59:57 +02:00
Willem Jan Withagen
b4b00f43b1 FindNSS.cmake: find includes in /usr/local/includes/nss/nss
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-20 15:57:42 +02:00
Willem Jan Withagen
a38a7fa198 CMakeLists.txt: Add FreeBSD isms
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-20 15:57:41 +02:00
Willem Jan Withagen
e1dcab2eca do_freebsd-cmake.sh: script to cmake on FreeBSD
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-20 15:57:41 +02:00
Mark Nelson
12257c2df2 Merge pull request #10327 from xiexingguo/xxg-wip-bluestore-2016-07-18
os/bluestore: misc fixes/cleanups

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR may have contributed to write performance improvements along with #10257.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-20 08:28:40 -05:00
Mark Nelson
c3b632e4d5 Merge pull request #10257 from chhabaramesh/extent_alloc
os/bluestore: extent alloc functionality for stupid and bitmap allocator

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR appears to result in a large sequential write performance increase (Up to ~3.5X), and may contribute to a small random write performance increase as well.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-20 08:22:31 -05:00
Haomai Wang
6ea50dabd6 msg/async/Event: ensure not refer to member variable which may destroyed
If nonwait is false, another thread is waiting for complete. After calling f()
this thread will notify for finishing and at this moment another thread will
destroy this C_submit_event right now. So we may refer to nonwait when
C_submit_event is disappeared.

Fixes: http://tracker.ceph.com/issues/16714
Signed-off-by: Haomai Wang <haomai@xsky.com>
2016-07-20 20:44:32 +08:00
Orit Wasserman
6d7841c6f0 rgw: fix upgrade from old multisite to new multisite configuration
We need to store the updated current period after adding the old converted regions

Fixes: http://tracker.ceph.com/issues/16751
Signed-off-by: Orit Wasserman <owasserm@redhat.com>
2016-07-20 14:00:43 +02:00
Willem Jan Withagen
4dd92f99ee src/test/cli/* tests: POSIX Convert grep -P to grep -E (v2)
- FreeBSD does not have grep -P
 - [Posix grep]: replace \t by [[:space:]]

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2016-07-20 13:42:19 +02:00
Kefu Chai
73f46a9066 Merge pull request #9082 from stiopaa1/osd_TestOpsSocketHook_removeUnneededParam
pass string by const ref and add override to virtual function

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-20 18:07:40 +08:00
Boris Ranto
ce74e3bdc2 Merge pull request #9667 from SUSE/wip-15984
ceph-osd-prestart.sh: drop Upstart-specific code

Reviewed-by: Boris Ranto <branto@redhat.com>
2016-07-20 11:01:43 +02:00
Kefu Chai
28819d35a7 Merge pull request #10339 from tchaikov/wip-16675
selinux: allow read /proc/<pid>/cmdline

Reviewed-by: Boris Ranto <branto@redhat.com>
2016-07-20 17:00:38 +08:00
Kefu Chai
e8a16b7fda cmake: remove unused Findfio.cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:22:29 +08:00
Kefu Chai
b5607f88e8 cmake: Findfcgi.cmake cleanups
FindPackageHandleStandardArgs() takes care of the find_package()
boilderplate stuff. so no need to repeat them. and remove the checkings
in env variables.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:21:06 +08:00
Kefu Chai
21a852fd65 cmake: do not include $lrc_srcs in unittest_erasure_code_lrc
as ${lrc_srcs} is not defined in the scope where
unittest_erasure_code_lrc is added as a target. and instead we link
unittest_erasure_code_lrc against ec_lrc dynamic library. so we can
safely remove ${lrc_srcs} from unittest_erasure_code_lrc.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 16:01:55 +08:00
Kefu Chai
1df15fd833 cmake: s/EXTRA_LIBS/EXTRALIBS/
it's a typo, i think. as EXTRA_LIBS is never defined anywhere.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:56:25 +08:00
Kefu Chai
c3bcb25b5b cmake: s/HAVE_FUSE/WITH_FUSE/ s/HAVE_LIBFUSE/WITH_FUSE/
we don't define HAVE_FUSE, and HAVE_LIBFUSE is defined to be compatible
with autotools. so use WITH_FUSE in cmake whenever possible.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:29:43 +08:00
Kefu Chai
48da9f94cf cmake: remove $UNITTEST_CXX_FLAGS from ceph-kvstore-tool's linked libs
it does not make sense to link against CXX_FLAGS.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:16:39 +08:00
Kefu Chai
ec84d7169b cmake: CMAKE_MODULE_PATH is a list and empty by default
we should not reference it using ${CMAKE_MODULE_PATH} before assigning
to it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 15:10:48 +08:00
Kefu Chai
8582aa8608 cmake: add_subdirectory(include)
fixes the packaging failure introduced by 741f990

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
25c76b19a6 cmake: s/PTHREAD_LIBS/CMAKE_THREAD_LIBS_INIT/
${CMAKE_THREAD_LIBS_INIT} is defined by find_package(Thread), while
PTHREAD_LIBS is defined by the acx_pthread.m4.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
b482cf083f cmake: use HAVE_ARMV8_CRC not ${HAVE_ARMV8_CRC}
as this variable could be undefined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
636522b669 cmake: use FindBacktrace instead of Findexecinfo
always better to use upstream cmake modules.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
Kefu Chai
cf6dcf0ae7 cmake: add "WITH_PROFILER" option
this option matches '--with-profiler' option in autoconf. and it is off
by default. we should not link against libprofiler unless asked to do
so. this change fixes this problem.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-20 14:47:52 +08:00
xie xingguo
a16ba7d0ae kv/MemDB: assert to confirm that we don't access violation or underflow
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-20 13:48:07 +08:00
xie xingguo
3c3b78f113 kv/MemDB: fix wrong output stream of error message
The origin logic choose the wrong target and has no effect.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-20 13:40:09 +08:00
xie xingguo
3bc31746b5 os/bluestore: add a boundary check of cache read
By trapping the "==" case and break, we can finish
the cache read just in time. As a result, we can avoid
two extra operations(adjusting "offset" and "length") and
the final loop check.

Since this is one of key methods of cache, it deserves this
fix.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-07-20 10:37:16 +08:00
Nathan Cutler
0a44aaa74c Merge pull request #10353 from rjfd/wip-fix-ceph-mount
rpm: Fix creation of mount.ceph symbolic link for SUSE distros

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2016-07-20 04:22:21 +02:00
Kefu Chai
f0426fbf54 Merge pull request #10351 from ceph/wip-cmake
cmake changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2016-07-20 10:06:29 +08:00
songbaisen
76b5451dea logmon: check is_leader() before doing any work on get_trim_to()
Signed-off-by: song baisen <song.baisen@zte.com.cn>
2016-07-20 08:37:02 +08:00
Yehuda Sadeh
1f3fec8070 rgw: back off bucket sync on failures, don't store marker
Fixes: http://tracker.ceph.com/issues/16742

If we fail on any single entry in bucket, skip updating the marker tracker
so that next time we'll go over that entry, and back off. This will trigger
a report to the data sync error repo and eventually a retry on the failing
object.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-07-19 15:37:41 -07:00
Mark Nelson
472a5d8d6c Merge pull request #10295 from xiexingguo/xxg-wip-bluestore-2016-07-14
kv/MemDB: misc fixes and cleanups

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:16:29 -05:00
Mark Nelson
dd4de0f786 Merge pull request #10294 from xiexingguo/xxg-wip-bluestore-2016-07-13
os/bluestore: use BE for gifting and reclaiming from bluefs

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:15:25 -05:00
Mark Nelson
81c10210af Merge pull request #10277 from xiexingguo/xxg-wip-fix-posix-fallocate
os/bluestore: fix error handling of posix_fallocate()

Mark's Comments:
This passed "ceph_test_objectstore --gtest_filter=*/2".
This PR did not appear to have a significant impact on performance tests.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2016-07-19 16:12:13 -05:00
Mykola Golub
ee75375f3c Merge pull request #10249 from dillaman/wip-16622
rbd-mirror: snap rename does not properly replicate to peers

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-07-19 20:16:04 +03:00
Kefu Chai
6199d2ef05 Merge pull request #10313 from ceph/objclass-perm-feedback
Objclass perm feedback

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:43:29 +08:00
Ricardo Dias
976c9d4f31
rpm: Fix creation of mount.ceph symbolic link for SUSE distros
Signed-off-by: Ricardo Dias <rdias@suse.com>
2016-07-19 15:40:36 +01:00
Kefu Chai
eb71afdbe3 cmake: disable VTA for osdcap.cc
this silences the warning of:
```
In file included from
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:58:0,
                 from
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:20:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:
In member function ‘virtual void
OSDCap_AllowClassMulti_Test::TestBody()’:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:766:6:
note: variable tracking size limit exceeded with
-fvar-tracking-assignments, retrying without
 TEST(OSDCap, AllowClassMulti) {
      ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/internal/gtest-internal.h:1211:3:
note: in definition of macro ‘GTEST_TEST_CLASS_NAME_’
   test_case_name##_##test_name##_Test
   ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2181:3:
note: in expansion of macro ‘GTEST_TEST_’
   GTEST_TEST_(test_case_name, test_name, \
   ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/googletest/googletest/include/gtest/gtest.h:2187:42:
note: in expansion of macro ‘GTEST_TEST’
 # define TEST(test_case_name, test_name) GTEST_TEST(test_case_name,
 # test_name)
                                          ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osdcap.cc:766:1:
note: in expansion of macro ‘TEST’
 TEST(OSDCap, AllowClassMulti) {
 ^
```

see also b668051

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:25:45 +08:00
Kefu Chai
3173d2ea12 cmake: remove util.cc from lib{rados,cephfs},ceph-objectstore-tool
util.cc is included by both librados and libcephfs, the `lvm` static
variable in `lsb_release_parse()` will be free twice by them. this
could lead to double free issue. and util.cc is not used by client at all, so
remove it from them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-07-19 22:25:36 +08:00