RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-01-11 05:29:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,549 Commits 214 Branches 394 Tags 1.3 GiB
ce613b955a
Commit Graph

100549 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kefu Chai
2507fb64c3 include/Context: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
0a75f504cc erasure-code: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
307ccc42f0 common/ceph_mutex: add more ceph_mutex_is_*() helpers 
we have caller sites like

ceph_assert(!client->client_lock.is_locked_by_me())

in `src/client/Client.cc` and `src/librados/RadosClient.cc`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
cc67c34a9e common/ceph_mutex: improve ceph::shared_mutex 
add helpers for supporting `ceph_mutex_is_locked()` and
`ceph_mutex_is_wlocked()` macros

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
99f31705b6 common/Timer: add necessary #include 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
3f612c45e3 common/RWLock: add RWLock::{lock,unlock}_shared() 
so RWLock can be used with std::shared_lock,

rgw is using RWLock'prioritize_write feature, which is not offered by
std::shared_mutex, but we want to unify the way we use mutex, so add
these helper functions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
d5cbb94bcb common: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
79a722402a common/HeartbeatMap.cc: include necessary header 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
f4ac81502a common/Cond.h: include necessary header 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
6e6faf080a cls/rgw: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
675606bf71 mds: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:19 +08:00
Kefu Chai
20b1ac6e09 osdc: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:18 +08:00
Kefu Chai
35d0ce394f osd: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:18 +08:00
Kefu Chai
939ae2d378 os: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 11:27:18 +08:00
Kefu Chai
079fc7b3fa msg: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
85157d5aae mon: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
c93dc88491 mgr: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
ed75406c4f librados: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
118be25980 include/utime: add utime_t<>(duration) 
to ease the conversion from chrono durations to utime_t

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
33ba265b79 include/C_Lock: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
fcc6ae58c3 common/condition_variable_debug: fix wait_until() 
to_timespec() is not a member function of time_point, it is a member
funcion of real_clock / coarse_real_clock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
649891ae71 common/timer: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
bcabe52695 client: s/Mutex/ceph::mutex/ 
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-03 09:34:50 +08:00
Kefu Chai
d20a13b553
Merge pull request #29291 from tchaikov/wip-cmake-cleanups 
cmake: require CMake v3.10.2

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
 2019-08-03 08:46:17 +08:00
Ernesto Puerta
7f767ab48e
install-deps.sh: remove failing error catching 
Fixes: https://tracker.ceph.com/issues/41013
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
 2019-08-02 21:26:40 +02:00
Neha Ojha
75e950ce0b
Merge pull request #29408 from neha-ojha/wip-bs-fsck-mnt 
common/options.cc: change default value of bluestore_fsck_on_mount_deep to false

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
 2019-08-02 11:51:22 -07:00
Matt Benjamin
f9d64e6501
Merge pull request #28637 from linuxbox2/wip-rgw-blorderflat 
rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered
 2019-08-02 13:42:22 -04:00
Matt Benjamin
d60dee0f0a rgw: s/std::map/boost::container::flat_map/ cls_bucket_list_ordered 
(RGWRados and CLS).

Probably faster, allocating less.  Definitely not slower.

Examples from single-OSD vstart.sh cluster, Ceph built at -O2

BEFORE

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real	4m48.991s
user	3m45.260s
sys	0m7.174s

(2nd run)

radosgw
Samples: 81K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 3189324729
Overhead  Shared Object         Symbol
   7.06%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   6.85%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   6.15%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   4.12%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   4.11%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.49%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   3.33%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   3.04%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.46%  radosgw               [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare
   2.45%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   2.36%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   2.07%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.94%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.93%  radosgw               [.] std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pa
   1.85%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.79%  radosgw               [.] rgw_bucket_dir::decode
   1.76%  libceph-common.so.0   [.] operator<<
   1.42%  radosgw               [.] ceph::decode
   1.35%  libstdc++.so.6.0.25   [.] std::_Rb_tree_insert_and_rebalance
   1.33%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::ReleaseToSpans
   1.31%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.24%  [kernel]              [k] copy_user_enhanced_fast_string
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 23K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 5059851086
Overhead  Shared Object         Symbol
   8.57%  libc-2.27.so          [.] vfprintf
   4.90%  ceph-osd              [.] ceph::logging::Log::_flush
   4.66%  libc-2.27.so          [.] _IO_default_xsputn
   3.49%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.93%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.70%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.11%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   1.91%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   1.75%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.64%  libc-2.27.so          [.] _itoa_word
   1.58%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.39%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.33%  [kernel]              [k] stackleak_erase
   1.27%  [kernel]              [k] copy_user_enhanced_fast_string
   1.22%  libc-2.27.so          [.] __strlen_avx2
   1.15%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   1.12%  libstdc++.so.6.0.25   [.] std::ostream::_M_insert<unsigned long>
   1.10%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   1.04%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.01%  libc-2.27.so          [.] __tz_convert
   0.94%  [kernel]              [k] entry_SYSCALL_64
   0.94%  ceph-osd              [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   0.93%  ceph-osd              [.] ceph::logging::Log::submit_entry
For a higher level overview, try: perf top --sort comm,dso

AFTER

[mbenjamin@lemon python]$ time  s3cmd -c s3cfg_userx ls  s3://DOCREQUEST_750/CSV/SUB1/ > /dev/null

real	4m51.488s
user	3m36.785s
sys	0m5.689s

(1st run)

radosgw

Samples: 52K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 4426952205
Overhead  Shared Object         Symbol
   6.11%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   5.83%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   3.89%  radosgw               [.] rgw_bucket_dir::decode
   3.73%  radosgw               [.] rgw_bucket_dir_entry::rgw_bucket_dir_entry
   3.68%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::ptr::copy_out
   3.37%  libstdc++.so.6.0.25   [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
   3.14%  libtcmalloc.so.4.5.1  [.] tcmalloc::CentralFreeList::FetchFromOneSpans
   2.55%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.22%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   2.02%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.79%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::advance
   1.77%  [kernel]              [k] n_tty_write
   1.74%  libc-2.27.so          [.] vfprintf
   1.71%  libtcmalloc.so.4.5.1  [.] operator new[]
   1.65%  librados.so.2.0.0     [.] ceph::buffer::v14_2_0::list::iterator_impl<true>::copy
   1.58%  radosgw               [.] ceph::decode
   1.38%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_mutate
   1.22%  [kernel]              [k] stackleak_erase
   1.12%  libceph-common.so.0   [.] operator<<
   1.09%  libc-2.27.so          [.] _IO_default_xsputn
   1.06%  libc-2.27.so          [.] __memcmp_avx2_movbe
   1.04%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  librados.so.2.0.0     [.] std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append
For a higher level overview, try: perf top --sort comm,dso

osd

Samples: 134K of event 'cycles:ppp', 4000 Hz, Event count (approx.): 26819176020
Overhead  Shared Object         Symbol
   8.82%  libc-2.27.so          [.] vfprintf
   6.88%  ceph-osd              [.] ceph::logging::Log::_flush
   4.80%  libc-2.27.so          [.] _IO_default_xsputn
   4.15%  libpthread-2.27.so    [.] __pthread_mutex_unlock_usercnt
   2.69%  ceph-osd              [.] StackStringBuf<4096ul>::xsputn
   2.54%  libstdc++.so.6.0.25   [.] std::__ostream_insert<char, std::char_traits<char> >
   2.40%  libtcmalloc.so.4.5.1  [.] operator new[]
   2.39%  libc-2.27.so          [.] __memmove_avx_unaligned_erms
   2.13%  libcls_rgw.so.1.0.0   [.] boost::container::dtl::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<ch
   1.61%  libtcmalloc.so.4.5.1  [.] tc_deletearray_aligned_nothrow
   1.54%  libc-2.27.so          [.] _itoa_word
   1.49%  libstdc++.so.6.0.25   [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned l
   1.41%  libpthread-2.27.so    [.] __pthread_mutex_lock
   1.41%  ceph-osd              [.] ceph::logging::Log::submit_entry
   1.23%  libstdc++.so.6.0.25   [.] std::ostream::sentry::sentry
   1.19%  libc-2.27.so          [.] __tz_convert
   1.08%  libc-2.27.so          [.] __strlen_avx2
   1.08%  [kernel]              [k] stackleak_erase
   1.06%  libcls_rgw.so.1.0.0   [.] rgw_bucket_list
   1.02%  ceph-osd              [.] ceph::buffer::v14_2_0::list::append
   0.99%  ceph-osd              [.] ceph::buffer::v14_2_0::ptr::append
   0.96%  [kernel]              [k] copy_user_enhanced_fast_string
   0.93%  [kernel]              [k] entry_SYSCALL_64
For a higher level overview, try: perf top --sort comm,dso

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
 2019-08-02 12:39:48 -04:00
Kefu Chai
bbb69fe793 cmake: require CMake v3.10.2 
since we dropped the support of xenial, we now have the luxury of using
newer CMake! and by using CMake 3.10.2, we can prevent libfmt from
assuming that we are using C++11, and hence set `CMAKE_CXX_STANDARD` to
11, which will literally append `-std=gnu++11` to `CMAKE_CXX_FLAGS`.
the last `-std` option passed to `g++` takes precendence.
since we've switched over to C++17, and we are using C++17 features.
so, using cmake older than 3.8 breaks the build. because it is CMake 3.8
which stared support `CMAKE_CXX_STANDARD` 17.

- for bionic: https://packages.ubuntu.com/bionic/cmake : 3.10.2
- for CentOS7:
https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/c/ : 3.13.5

so in this change,

* bump up the required version to v3.10.2
* cleanups to wipe out the workaround for lower CMake versions
* use `PROJECT_VERSION` defined by `project()` command instead of
  `VERSION` explicitly defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 22:09:12 +08:00
Casey Bodley
69062ee705
Merge pull request #25962 from greenx/master 
rgw: swift: bugfix: https://tracker.ceph.com/issues/37765

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
 2019-08-02 10:04:07 -04:00
Tiago Melo
7f7f8a443c mgr/dashboard: Update all npm packages 
This includes a major upgrade of angular from v7 to v8.

Using NG_CLI_ANALYTICS to disable angular cli to query about allowing analytics.

Fixes: https://tracker.ceph.com/issues/41059

Signed-off-by: Tiago Melo <tmelo@suse.com>
 2019-08-02 13:15:55 +00:00
Kefu Chai
ddacfcc0e0 install-deps.sh: install cmake 3.10.2 and up on xenial 
to enable us to build on xenial, install newer cmake.
cmake 3.10.2 is the version offered by bionic.

with this change, we can safely require cmake 3.10.2 in our cmake
script. as EPEL7 offers cmake 3.13

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 20:23:33 +08:00
Jan Fajerski
45a4d53012
Merge pull request #29401 from jan--f/c-v-inventory-parttype 
ceph-volume: detect ceph-disk osd if PARTLABEL is missing
 2019-08-02 11:17:17 +02:00
Jan Fajerski
1f8f1e95de ceph-volume: print most logging messages to stderr 
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Fixes: http://tracker.ceph.com/issues/38548
 2019-08-02 11:10:00 +02:00
Ricardo Dias
29edead1c6
Merge pull request #29046 from p-na/wip-pna-disable-user 
mgr/dashboard: Provide user enable/disable capability

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
 2019-08-02 10:05:21 +01:00
Ricardo Dias
1fb9c64692
Merge pull request #29088 from votdev/issue_24662 
mgr/dashboard: Allow disabling redirection on standby Dashboards

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
 2019-08-02 10:02:38 +01:00
Jiaying Ren
08d3ef1e22 rgw: drop dead flush_read_list declaration 
Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>
 2019-08-02 15:21:46 +08:00
Sage Weil
647d80437b Merge PR #29379 into master 
* refs/pull/29379/head:
	os/bluestore: s/align_down/p2align/

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
 2019-08-02 01:19:57 -05:00
Sage Weil
e8bca54f4e Merge PR #28881 into master 
* refs/pull/28881/head:
	os/Transaction: dump alloc hint flags in op

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 01:17:48 -05:00
Kefu Chai
0e35c6f641
Merge pull request #29349 from smithfarm/wip-40975 
rpm: put librgw lttng SOs in the librgw-devel package

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 11:52:07 +08:00
Kefu Chai
8d497f1ed2
Merge pull request #29397 from wjwithagen/wjw-fix-malloc.h 
common: use of malloc.h is deprecated

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 11:50:29 +08:00
Kefu Chai
95fd792be1
Merge pull request #29396 from wjwithagen/wjw-fix-boost-1.70 
cmake: update FindBoost.cmake

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2019-08-02 11:49:34 +08:00
J. Eric Ivancich
b4e4acb213
Merge pull request #29181 from liewegas/wip-rgw-meta-pool-priority 
rgw: add rgw_rados_pool_recovery_priority (default 5)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2019-08-01 20:46:39 -04:00
J. Eric Ivancich
61fc5c7df5
Merge pull request #28587 from zhangsw/rgw-bugfix-versioning 
rgw: fix a bug that lifecycle expiraton generates delete marker continuously

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2019-08-01 19:33:40 -04:00
J. Eric Ivancich
fa0dd4003d
Merge pull request #28789 from taodd/rgw-fix-remove-bucket 
rgw: fix drain handles error when deleting bucket with bypass-gc option

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
 2019-08-01 19:30:43 -04:00
J. Eric Ivancich
a9a3e59abc
Merge pull request #29178 from ivancich/wip-small-rgw-efficiency 
rgw: add a small efficiency

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2019-08-01 18:36:30 -04:00
J. Eric Ivancich
092ef6085c
Merge pull request #29179 from ivancich/wip-bucket-list-excessive-max-entries 
rgw: mitigate bucket list with max-entries excessively high

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2019-08-01 18:34:35 -04:00
J. Eric Ivancich
0ef6bee2d5
Merge pull request #29240 from mdw-at-linuxbox/wip-master-rgw-swift-metadata-versioning 
rgw: Fix bucket versioning vs. swift metadata bug.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
 2019-08-01 18:28:19 -04:00
Matt Benjamin
57df918a1a
Merge pull request #28331 from linuxbox2/wip-rgw-uinfo-by-acc 
rgw: permit rgw-admin to populate user info by access-key
 2019-08-01 16:07:34 -04:00
Kefu Chai
d835e9a926
Merge pull request #29413 from tchaikov/wip-remove-test/python 
test/python: remove stale tests

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>
 2019-08-02 00:38:28 +08:00