RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-03-11 02:39:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,599 Commits 219 Branches 396 Tags 1.4 GiB
0527d9c8cd
Commit Graph

100599 Commits

Author SHA1 Message Date
Kefu Chai
816709cda7 test/omap_bench: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
bfb8c741cd test/msgr: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
60de46ef04 test/admin_socket: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
b1f5565c2f kv: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
de68b2cbbc tools/rados: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
42f6597231 tools/immutable_object_cache: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
1fb8a60f23 tools/cephfs: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
4b64f16fe4 test/os/filestore: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
bf0142c6c9 tools/rbd_mirror: s/Mutex/ceph::Formatter/ 
* s/Formatter/ceph::Formatter/ in tools/rbd_mirror/ImageSyncThrottler,
  as we cannot rely on `using ceph` or `using ceph::Formatter` in
  some other included header files.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
34cb0ebe5d test/perf_local: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:20 +08:00
Kefu Chai
9e1e84d9f7 test/osdc: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
eb1dce5e4c test/objectstore: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
ed5fc085b2 test/mon: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
792d6c53fe test/librbd: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
efee3ae5fb test/journal: s/Mutex/ceph::mutex/ 
as we cannot create a `smart_pointer<ceph::mutex>` due to limitation of
`ceph::make_mutex()`, we need to refactor `TestObjectRecorder` to remove
its `TearDown()` method, as when `TearDown()` is called, all local
variables have been destroyed, including the `ceph::mutex` instances. so
we need to introduce a helper of `ObjectRecorderFlusher`, to flush the
objects before `mutex` instances are destroyed. to simplify the
interfaces, it flushes in its dtor. so its lifecycle should be shorter
than those of mutexes. that's why, mutexes are created before `flusher`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
02292616a3 rgw: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
a79837bcee librbd: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
f39b32ebfa journal: s/Mutex/ceph::mutex/ 
* FutureImpl::m_lock is an exception. as, before this change, the lock
  was initialized like `m_lock("FutureImpl::m_lock", false, false)`, see
  the declaration of
  `Mutex(const std::string &n, bool r = false, bool ld=true, bool bt=false)`
  so `m_lock` is actually not using the extra features offered by
  `Mutex` like runtime lockdeps check. and `mutex_debugging_base` does
  not allow us to disable lockdeps individually. but it does use the `is_locked()`
  method. so instead of using `ceph::mutex` directly, a cutomized
  `ceph::mutex` is added for `CEPH_DEBUG_MUTEX` build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
2dbd31eb0b common/mutex_debug: add lockdep parameter to mutex's ctor 
so we can disable lockdep when constructing a ceph::mutex. otherwise
it's difficult to use the locks from standard library and to use
`ceph_assert(ceph_mutex_is_locked(lock))` at the same time.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
bb2a97000b common/mutex_debug.h: pass "no_lockdep" to try_lock() 
there is chance that we want to disable lockdep, so we shold pass
`no_lockdep` down to `try_lock()` in `lock()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
2507fb64c3 include/Context: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
0a75f504cc erasure-code: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
307ccc42f0 common/ceph_mutex: add more ceph_mutex_is_*() helpers 
we have caller sites like

ceph_assert(!client->client_lock.is_locked_by_me())

in `src/client/Client.cc` and `src/librados/RadosClient.cc`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
cc67c34a9e common/ceph_mutex: improve ceph::shared_mutex 
add helpers for supporting `ceph_mutex_is_locked()` and
`ceph_mutex_is_wlocked()` macros

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
99f31705b6 common/Timer: add necessary #include 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
3f612c45e3 common/RWLock: add RWLock::{lock,unlock}_shared() 
so RWLock can be used with std::shared_lock,

rgw is using RWLock'prioritize_write feature, which is not offered by
std::shared_mutex, but we want to unify the way we use mutex, so add
these helper functions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
d5cbb94bcb common: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
79a722402a common/HeartbeatMap.cc: include necessary header 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
f4ac81502a common/Cond.h: include necessary header 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
6e6faf080a cls/rgw: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
675606bf71 mds: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
20b1ac6e09 osdc: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:18 +08:00
Kefu Chai
35d0ce394f osd: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:18 +08:00
Kefu Chai
939ae2d378 os: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:18 +08:00
Kefu Chai
079fc7b3fa msg: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
85157d5aae mon: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
c93dc88491 mgr: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
ed75406c4f librados: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
118be25980 include/utime: add utime_t<>(duration) 
to ease the conversion from chrono durations to utime_t

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
33ba265b79 include/C_Lock: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
fcc6ae58c3 common/condition_variable_debug: fix wait_until() 
to_timespec() is not a member function of time_point, it is a member
funcion of real_clock / coarse_real_clock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
649891ae71 common/timer: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
bcabe52695 client: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
d20a13b553
Merge pull request from tchaikov/wip-cmake-cleanups 
cmake: require CMake v3.10.2

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
 2019-08-03 08:46:17 +08:00
Ernesto Puerta
7f767ab48e
install-deps.sh: remove failing error catching 
Fixes: https://tracker.ceph.com/issues/41013
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
 2019-08-02 21:26:40 +02:00
Neha Ojha
75e950ce0b
Merge pull request from neha-ojha/wip-bs-fsck-mnt 
common/options.cc: change default value of bluestore_fsck_on_mount_deep to false

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
 2019-08-02 11:51:22 -07:00
Matt Benjamin
f9d64e6501
Merge pull request from linuxbox2/wip-rgw-blorderflat 
rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered
 2019-08-02 13:42:22 -04:00
Matt Benjamin
d60dee0f0a rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered 
(RGWRados and CLS).

Probably faster, allocating less.  Definitely not slower.

Examples from single-OSD vstart.sh cluster, Ceph built at -O2

BEFORE

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real	4m48.991s
user	3m45.260s
sys	0m7.174s

(2nd run)

radosgw
Samples: 81K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 3189324729
Overhead  Shared Object         Symbol
   7.06%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   6.85%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   6.15%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   4.12%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   4.11%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.49%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   3.33%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   3.04%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.46%  radosgw               [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare
   2.45%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   2.36%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.07%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.94%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.93%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   1.85%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.79%  radosgw               [.] rgw_bucket_dir::decode
   1.76%  libceph-common.so.0   [.] operator<<
   1.42%  radosgw               [.] ceph::decode
   1.35%  libstdc++.so.6.0.25   [.] std::_Rb_tree_insert_and_rebalance
   1.33%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::ReleaseToSpans
   1.31%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.24%  [kernel]              [k] copy_user_enhanced_fast_string
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 23K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 5059851086
Overhead  Shared Object         Symbol
   8.57%  libc-2.27.so          [.] vfprintf
   4.90%  ceph-osd              [.] ceph::logging::Log::_flush
   4.66%  libc-2.27.so          [.] _IO_default_xsputn
   3.49%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.93%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.70%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.11%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   1.91%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   1.75%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.64%  libc-2.27.so          [.] _itoa_word
   1.58%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.33%  [kernel]              [k] stackleak_erase
   1.27%  [kernel]              [k] copy_user_enhanced_fast_string
   1.22%  libc-2.27.so          [.] __strlen_avx2
   1.15%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   1.12%  libstdc++.so.6.0.25   [.] std::ostream::_M_insert<unsigned long>
   1.10%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   1.04%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.01%  libc-2.27.so          [.] __tz_convert
   0.94%  [kernel]              [k] entry_SYSCALL_64
   0.94%  ceph-osd              [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   0.93%  ceph-osd              [.] ceph::logging::Log::submit_entry
For a higher level overview, try: perf top --sort comm,dso

AFTER

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real	4m51.488s
user	3m36.785s
sys	0m5.689s

(1st run)

radosgw

Samples: 52K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 4426952205
Overhead  Shared Object         Symbol
   6.11%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   5.83%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   3.89%  radosgw               [.] rgw_bucket_dir::decode
   3.73%  radosgw               [.] rgw_bucket_dir_entry::rgw_bucket_dir_entry
   3.68%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   3.37%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.14%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   2.55%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.22%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   2.02%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.79%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.77%  [kernel]              [k] n_tty_write
   1.74%  libc-2.27.so          [.] vfprintf
   1.71%  libtcmalloc.so.4.5.1  [.] operator new[]
   1.65%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.58%  radosgw               [.] ceph::decode
   1.38%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.22%  [kernel]              [k] stackleak_erase
   1.12%  libceph-common.so.0   [.] operator<<
   1.09%  libc-2.27.so          [.] _IO_default_xsputn
   1.06%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.04%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 134K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 26819176020
Overhead  Shared Object         Symbol
   8.82%  libc-2.27.so          [.] vfprintf
   6.88%  ceph-osd              [.] ceph::logging::Log::_flush
   4.80%  libc-2.27.so          [.] _IO_default_xsputn
   4.15%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   2.69%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.54%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.40%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.13%  libcls_rgw.so.1.0.0   [.] boost::container::dtl::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
   1.61%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.54%  libc-2.27.so          [.] _itoa_word
   1.49%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.41%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.41%  ceph-osd              [.] ceph::logging::Log::submit_entry
   1.23%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.19%  libc-2.27.so          [.] __tz_convert
   1.08%  libc-2.27.so          [.] __strlen_avx2
   1.08%  [kernel]              [k] stackleak_erase
   1.06%  libcls_rgw.so.1.0.0   [.] rgw_bucket_list
   1.02%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   0.99%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   0.96%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  [kernel]              [k] entry_SYSCALL_64
For a higher level overview, try: perf top --sort comm,dso

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
 2019-08-02 12:39:48 -04:00
Kefu Chai
bbb69fe793 cmake: require CMake v3.10.2 
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 22:09:12 +08:00
Casey Bodley
69062ee705
Merge pull request from greenx/master 
rgw: swift: bugfix: https://tracker.ceph.com/issues/37765

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
 2019-08-02 10:04:07 -04:00