Kefu Chai
792d6c53fe
test/librbd: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
efee3ae5fb
test/journal: s/Mutex/ceph::mutex/
...
as we cannot create a `smart_pointer<ceph::mutex>` due to limitation of
`ceph::make_mutex()`, we need to refactor `TestObjectRecorder` to remove
its `TearDown()` method, as when `TearDown()` is called, all local
variables have been destroyed, including the `ceph::mutex` instances. so
we need to introduce a helper of `ObjectRecorderFlusher`, to flush the
objects before `mutex` instances are destroyed. to simplify the
interfaces, it flushes in its dtor. so its lifecycle should be shorter
than those of mutexes. that's why, mutexes are created before `flusher`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
02292616a3
rgw: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
a79837bcee
librbd: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
f39b32ebfa
journal: s/Mutex/ceph::mutex/
...
* FutureImpl::m_lock is an exception. as, before this change, the lock
was initialized like `m_lock("FutureImpl::m_lock", false, false)`, see
the declaration of
`Mutex(const std::string &n, bool r = false, bool ld=true, bool bt=false)`
so `m_lock` is actually not using the extra features offered by
`Mutex` like runtime lockdeps check. and `mutex_debugging_base` does
not allow us to disable lockdeps individually. but it does use the `is_locked()`
method. so instead of using `ceph::mutex` directly, a cutomized
`ceph::mutex` is added for `CEPH_DEBUG_MUTEX` build.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
2dbd31eb0b
common/mutex_debug: add lockdep parameter to mutex's ctor
...
so we can disable lockdep when constructing a ceph::mutex. otherwise
it's difficult to use the locks from standard library and to use
`ceph_assert(ceph_mutex_is_locked(lock))` at the same time.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
bb2a97000b
common/mutex_debug.h: pass "no_lockdep" to try_lock()
...
there is chance that we want to disable lockdep, so we shold pass
`no_lockdep` down to `try_lock()` in `lock()`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
2507fb64c3
include/Context: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
0a75f504cc
erasure-code: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
307ccc42f0
common/ceph_mutex: add more ceph_mutex_is_*() helpers
...
we have caller sites like
ceph_assert(!client->client_lock.is_locked_by_me())
in `src/client/Client.cc` and `src/librados/RadosClient.cc`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
cc67c34a9e
common/ceph_mutex: improve ceph::shared_mutex
...
add helpers for supporting `ceph_mutex_is_locked()` and
`ceph_mutex_is_wlocked()` macros
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
99f31705b6
common/Timer: add necessary #include
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
3f612c45e3
common/RWLock: add RWLock::{lock,unlock}_shared()
...
so RWLock can be used with std::shared_lock,
rgw is using RWLock'prioritize_write feature, which is not offered by
std::shared_mutex, but we want to unify the way we use mutex, so add
these helper functions.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
d5cbb94bcb
common: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
79a722402a
common/HeartbeatMap.cc: include necessary header
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
f4ac81502a
common/Cond.h: include necessary header
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
6e6faf080a
cls/rgw: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
675606bf71
mds: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:19 +08:00
Kefu Chai
20b1ac6e09
osdc: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:18 +08:00
Kefu Chai
35d0ce394f
osd: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:18 +08:00
Kefu Chai
939ae2d378
os: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 11:27:18 +08:00
Kefu Chai
079fc7b3fa
msg: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
85157d5aae
mon: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
c93dc88491
mgr: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
ed75406c4f
librados: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
118be25980
include/utime: add utime_t<>(duration)
...
to ease the conversion from chrono durations to utime_t
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
33ba265b79
include/C_Lock: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
fcc6ae58c3
common/condition_variable_debug: fix wait_until()
...
to_timespec() is not a member function of time_point, it is a member
funcion of real_clock / coarse_real_clock.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
649891ae71
common/timer: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
bcabe52695
client: s/Mutex/ceph::mutex/
...
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-03 09:34:50 +08:00
Kefu Chai
d20a13b553
Merge pull request #29291 from tchaikov/wip-cmake-cleanups
...
cmake: require CMake v3.10.2
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-08-03 08:46:17 +08:00
Neha Ojha
75e950ce0b
Merge pull request #29408 from neha-ojha/wip-bs-fsck-mnt
...
common/options.cc: change default value of bluestore_fsck_on_mount_deep to false
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-08-02 11:51:22 -07:00
Matt Benjamin
f9d64e6501
Merge pull request #28637 from linuxbox2/wip-rgw-blorderflat
...
rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered
2019-08-02 13:42:22 -04:00
Matt Benjamin
d60dee0f0a
rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered
...
(RGWRados and CLS).
Probably faster, allocating less. Definitely not slower.
Examples from single-OSD vstart.sh cluster, Ceph built at -O2
BEFORE
[mbenjamin@lemon python]$ time s3cmd -c s3cfg_userx ls s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null
real 4m48.991s
user 3m45.260s
sys 0m7.174s
(2nd run)
radosgw
Samples: 81K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 3189324729
Overhead Shared Object Symbol
7.06% libtcmalloc.so.4.5.1 [.] tcmalloc::CentralFreeList::FetchFromOneSpans
6.85% libstdc++.so.6.0.25 [.] std::__ostream_insert<char, std::char_traits<char> >
6.15% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
4.12% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::ptr::copy_out
4.11% libstdc++.so.6.0.25 [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
3.49% libc-2.27.so [.] __memmove_avx_unaligned_erms
3.33% libtcmalloc.so.4.5.1 [.] tc_deletearray_aligned_nothrow
3.04% radosgw [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
2.46% radosgw [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare
2.45% libtcmalloc.so.4.5.1 [.] operator new[]
2.39% libstdc++.so.6.0.25 [.] std::ostream::sentry::sentry
2.36% radosgw [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
2.07% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
1.94% libc-2.27.so [.] __memcmp_avx2_movbe
1.93% radosgw [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
1.85% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
1.79% radosgw [.] rgw_bucket_dir::decode
1.76% libceph-common.so.0 [.] operator<<
1.42% radosgw [.] ceph::decode
1.35% libstdc++.so.6.0.25 [.] std::_Rb_tree_insert_and_rebalance
1.33% libtcmalloc.so.4.5.1 [.] tcmalloc::CentralFreeList::ReleaseToSpans
1.31% librados.so.2.0.0 [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
1.24% [kernel] [k] copy_user_enhanced_fast_string
For a higher level overview, try: perf top --sort comm,dso
osd
Samples: 23K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 5059851086
Overhead Shared Object Symbol
8.57% libc-2.27.so [.] vfprintf
4.90% ceph-osd [.] ceph::logging::Log::_flush
4.66% libc-2.27.so [.] _IO_default_xsputn
3.49% libtcmalloc.so.4.5.1 [.] operator new[]
2.93% ceph-osd [.] StackStringBuf<4096ul>::xsputn
2.70% libstdc++.so.6.0.25 [.] std::__ostream_insert<char, std::char_traits<char> >
2.11% libpthread-2.27.so [.] __pthread_mutex_unlock_usercnt
1.91% libc-2.27.so [.] __memmove_avx_unaligned_erms
1.75% libstdc++.so.6.0.25 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
1.64% libc-2.27.so [.] _itoa_word
1.58% libtcmalloc.so.4.5.1 [.] tc_deletearray_aligned_nothrow
1.39% libstdc++.so.6.0.25 [.] std::ostream::sentry::sentry
1.33% [kernel] [k] stackleak_erase
1.27% [kernel] [k] copy_user_enhanced_fast_string
1.22% libc-2.27.so [.] __strlen_avx2
1.15% ceph-osd [.] ceph::buffer::v14_2_0::list::append
1.12% libstdc++.so.6.0.25 [.] std::ostream::_M_insert<unsigned long>
1.10% ceph-osd [.] ceph::buffer::v14_2_0::ptr::append
1.04% libpthread-2.27.so [.] __pthread_mutex_lock
1.01% libc-2.27.so [.] __tz_convert
0.94% [kernel] [k] entry_SYSCALL_64
0.94% ceph-osd [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
0.93% ceph-osd [.] ceph::logging::Log::submit_entry
For a higher level overview, try: perf top --sort comm,dso
AFTER
[mbenjamin@lemon python]$ time s3cmd -c s3cfg_userx ls s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null
real 4m51.488s
user 3m36.785s
sys 0m5.689s
(1st run)
radosgw
Samples: 52K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 4426952205
Overhead Shared Object Symbol
6.11% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
5.83% libstdc++.so.6.0.25 [.] std::__ostream_insert<char, std::char_traits<char> >
3.89% radosgw [.] rgw_bucket_dir::decode
3.73% radosgw [.] rgw_bucket_dir_entry::rgw_bucket_dir_entry
3.68% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::ptr::copy_out
3.37% libstdc++.so.6.0.25 [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
3.14% libtcmalloc.so.4.5.1 [.] tcmalloc::CentralFreeList::FetchFromOneSpans
2.55% libc-2.27.so [.] __memmove_avx_unaligned_erms
2.22% libtcmalloc.so.4.5.1 [.] tc_deletearray_aligned_nothrow
2.02% libstdc++.so.6.0.25 [.] std::ostream::sentry::sentry
1.79% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
1.77% [kernel] [k] n_tty_write
1.74% libc-2.27.so [.] vfprintf
1.71% libtcmalloc.so.4.5.1 [.] operator new[]
1.65% librados.so.2.0.0 [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
1.58% radosgw [.] ceph::decode
1.38% librados.so.2.0.0 [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
1.22% [kernel] [k] stackleak_erase
1.12% libceph-common.so.0 [.] operator<<
1.09% libc-2.27.so [.] _IO_default_xsputn
1.06% libc-2.27.so [.] __memcmp_avx2_movbe
1.04% [kernel] [k] copy_user_enhanced_fast_string
0.93% librados.so.2.0.0 [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append
For a higher level overview, try: perf top --sort comm,dso
osd
Samples: 134K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 26819176020
Overhead Shared Object Symbol
8.82% libc-2.27.so [.] vfprintf
6.88% ceph-osd [.] ceph::logging::Log::_flush
4.80% libc-2.27.so [.] _IO_default_xsputn
4.15% libpthread-2.27.so [.] __pthread_mutex_unlock_usercnt
2.69% ceph-osd [.] StackStringBuf<4096ul>::xsputn
2.54% libstdc++.so.6.0.25 [.] std::__ostream_insert<char, std::char_traits<char> >
2.40% libtcmalloc.so.4.5.1 [.] operator new[]
2.39% libc-2.27.so [.] __memmove_avx_unaligned_erms
2.13% libcls_rgw.so.1.0.0 [.] boost::container::dtl::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
1.61% libtcmalloc.so.4.5.1 [.] tc_deletearray_aligned_nothrow
1.54% libc-2.27.so [.] _itoa_word
1.49% libstdc++.so.6.0.25 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
1.41% libpthread-2.27.so [.] __pthread_mutex_lock
1.41% ceph-osd [.] ceph::logging::Log::submit_entry
1.23% libstdc++.so.6.0.25 [.] std::ostream::sentry::sentry
1.19% libc-2.27.so [.] __tz_convert
1.08% libc-2.27.so [.] __strlen_avx2
1.08% [kernel] [k] stackleak_erase
1.06% libcls_rgw.so.1.0.0 [.] rgw_bucket_list
1.02% ceph-osd [.] ceph::buffer::v14_2_0::list::append
0.99% ceph-osd [.] ceph::buffer::v14_2_0::ptr::append
0.96% [kernel] [k] copy_user_enhanced_fast_string
0.93% [kernel] [k] entry_SYSCALL_64
For a higher level overview, try: perf top --sort comm,dso
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-02 12:39:48 -04:00
Kefu Chai
bbb69fe793
cmake: require CMake v3.10.2
...
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.
- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5
so in this change,
* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
`VERSION` explicitly defined.
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-02 22:09:12 +08:00
Casey Bodley
69062ee705
Merge pull request #25962 from greenx/master
...
rgw: swift: bugfix: https://tracker.ceph.com/issues/37765
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-08-02 10:04:07 -04:00
Kefu Chai
ddacfcc0e0
install-deps.sh: install cmake 3.10.2 and up on xenial
...
to enable us to build on xenial, install newer cmake.
cmake 3.10.2 is the version offered by bionic.
with this change, we can safely require cmake 3.10.2 in our cmake
script. as EPEL7 offers cmake 3.13
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-08-02 20:23:33 +08:00
Jan Fajerski
45a4d53012
Merge pull request #29401 from jan--f/c-v-inventory-parttype
...
ceph-volume: detect ceph-disk osd if PARTLABEL is missing
2019-08-02 11:17:17 +02:00
Ricardo Dias
29edead1c6
Merge pull request #29046 from p-na/wip-pna-disable-user
...
mgr/dashboard: Provide user enable/disable capability
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-08-02 10:05:21 +01:00
Ricardo Dias
1fb9c64692
Merge pull request #29088 from votdev/issue_24662
...
mgr/dashboard: Allow disabling redirection on standby Dashboards
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-08-02 10:02:38 +01:00
Sage Weil
647d80437b
Merge PR #29379 into master
...
* refs/pull/29379/head:
os/bluestore: s/align_down/p2align/
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-08-02 01:19:57 -05:00
Sage Weil
e8bca54f4e
Merge PR #28881 into master
...
* refs/pull/28881/head:
os/Transaction: dump alloc hint flags in op
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 01:17:48 -05:00
Kefu Chai
0e35c6f641
Merge pull request #29349 from smithfarm/wip-40975
...
rpm: put librgw lttng SOs in the librgw-devel package
Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:52:07 +08:00
Kefu Chai
8d497f1ed2
Merge pull request #29397 from wjwithagen/wjw-fix-malloc.h
...
common: use of malloc.h is deprecated
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:50:29 +08:00
Kefu Chai
95fd792be1
Merge pull request #29396 from wjwithagen/wjw-fix-boost-1.70
...
cmake: update FindBoost.cmake
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-08-02 11:49:34 +08:00
J. Eric Ivancich
b4e4acb213
Merge pull request #29181 from liewegas/wip-rgw-meta-pool-priority
...
rgw: add rgw_rados_pool_recovery_priority (default 5)
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-01 20:46:39 -04:00
J. Eric Ivancich
61fc5c7df5
Merge pull request #28587 from zhangsw/rgw-bugfix-versioning
...
rgw: fix a bug that lifecycle expiraton generates delete marker continuously
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-01 19:33:40 -04:00
J. Eric Ivancich
fa0dd4003d
Merge pull request #28789 from taodd/rgw-fix-remove-bucket
...
rgw: fix drain handles error when deleting bucket with bypass-gc option
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-08-01 19:30:43 -04:00
J. Eric Ivancich
a9a3e59abc
Merge pull request #29178 from ivancich/wip-small-rgw-efficiency
...
rgw: add a small efficiency
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-01 18:36:30 -04:00
J. Eric Ivancich
092ef6085c
Merge pull request #29179 from ivancich/wip-bucket-list-excessive-max-entries
...
rgw: mitigate bucket list with max-entries excessively high
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-08-01 18:34:35 -04:00