Sage Weil
da887c82ce
qa/standalone/osd/pg-merge.sh: add merge vs pg import test
...
- You can't import the source half a PG that's since merged. Sorry! We
could implement this later.
- You can import the target half, but the result will then be incomplete,
and you rely on backfill to clean it up.
- Map gaps don't affect this behavior.
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-17 12:52:46 -05:00
Mykola Golub
5fc71af857
Merge pull request #24105 from dillaman/wip-24516
...
librbd: object map improperly flagged as invalidated
Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-17 20:31:36 +03:00
Matt Benjamin
9cfdf3d7e8
Merge pull request #23981 from cbodley/wip-35830
...
rgw: raise debug level on redundant data sync error messages
2018-09-17 11:26:50 -04:00
Kefu Chai
5bcc179bc4
rpm: bump up required GCC version to 7.3.1
...
so we can include the fix of
https://gcc.gnu.org/ml/libstdc++/2017-07/msg00077.html and the fix at
7c78f2e672
2018-09-17 22:51:26 +08:00
Kefu Chai
338612ad88
Merge pull request #24088 from dzafman/wip-35982
...
qa/standalone: Standalone test corrections
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-09-17 22:35:43 +08:00
Radoslaw Zarzynski
b71cbb97b7
common: get_data() of buffer_raw isn't virtual anymore.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:34:33 +02:00
Radoslaw Zarzynski
a9ff787071
common: make the get_data() of buffer_raw interface final.
...
This is just to ensure the just dropped buffer::raw_pipe
was the solely user of this facility. After successful
validation, we can drop `virtual` on the method entirely.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:34:33 +02:00
Radoslaw Zarzynski
2fa78f59ad
common: drop can_zero_copy() from ceph::buffer interfaces.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:34:33 +02:00
Radoslaw Zarzynski
e81d55803f
common: drop zero_copy_to_fd() from ceph::buffer interfaces.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:33:26 +02:00
Radoslaw Zarzynski
df90885ed1
common: drop write_fd_zero_copy() from bufferlist.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:31:45 +02:00
Radoslaw Zarzynski
6fc91d1a68
common: drop buffer::raw_pipe altogether with its factory method.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:31:45 +02:00
Radoslaw Zarzynski
7518eed56b
common: drop buffer::list::read_fd_zero_copy.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:31:45 +02:00
Radoslaw Zarzynski
53e491b6a6
common, test: drop unit tests for 0c in ceph::bufferlist.
...
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-17 15:31:45 +02:00
Rick Chen
4abb79f159
mgr/diskprediction: add prototype diskprediction module
...
This module is written by Rick Chen <rick.chen@prophetstor.com> and
provides both a built-in local predictor and a cloud mode that queries
a cloud service (provided by ProphetStor) to predict device failures.
Signed-off-by: Rick Chen <rick.chen@prophetstor.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-17 08:20:57 -05:00
John Spray
153a8cb025
pybind/cephfs: enable passing filesystem name to mount
...
Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-17 09:14:27 -04:00
planetbeing
50d13b6f5a
arch/arm: Allow ceph_crc32c_aarch64 to be chosen only if it is compiled in
...
If the build host does not support the crc32cx instruction, HAVE_ARMV8_CRC will not be set and ceph_crc32c_aarch64 will be a dummy function that always returns zero. However, if HWCAP_CRC32 is supported on the actual target machine, that dummy function will be selected anyway. This patch ensures ceph_crc32c_aarch64 will never be chosen if it is not compiled in.
Fixes: https://tracker.ceph.com/issues/36030
Signed-off-by: David Wang <planetbeing@gmail.com>
2018-09-17 01:07:56 -07:00
John Spray
24ef847c35
libcephfs: add ceph_select_filesystem
...
Because we're a C API, amending function
prototypes is painful. Instead, give an extra
function that can be used to configure a filesystem
name before calling mount.
Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-17 03:55:20 -04:00
John Spray
4b50e12b8e
common: add doc strings to client_mds_namespace
...
...and remove it from legacy options, as it's
now accessed by get_val.
Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-17 03:55:20 -04:00
John Spray
eae7e2d3b4
client: allow passing fs name to mount()
...
Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-17 03:55:20 -04:00
Kefu Chai
f557a78540
Merge pull request #24087 from rzarzynski/wip-common-buffer_raw-drop-page-align-getters
...
common: drop/mark-as-final getters of buffer::raw for palign.
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-09-17 15:55:10 +08:00
Kefu Chai
237bc951c4
Merge pull request #24061 from liewegas/wip-35923
...
osd/PG: write pg epoch when resurrecting pg after delete vs merge race
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-09-17 15:52:49 +08:00
Kefu Chai
3d1d337428
Merge pull request #23306 from penglaiyxy/msg_branch
...
msg: ceph_abort() when there are enough accepter errors in msg server
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-09-17 15:50:57 +08:00
Kefu Chai
f46523e464
Merge pull request #23955 from wjwithagen/wjw-fix-ceph-helpers.sh
...
test: Start using GNU awk and fix archiving directory
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-09-17 15:44:06 +08:00
John Spray
f162fa5365
mon: fix fs add_data_pool setting wrong pool metadata
...
Fixes: http://tracker.ceph.com/issues/36028
Signed-off-by: John Spray <john.spray@redhat.com>
2018-09-17 08:19:33 +01:00
Patrick Nawracay
f95f231037
mgr/dashboard: Cleanup
...
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-09-17 08:25:41 +02:00
Patrick Nawracay
8ba41c5e2b
mgr/dashboard: Refactoring of DeletionModalComponent
...
- Simpler variable names:
Examples:
- `actionDescription` and `itemDescription` instead of `metaType`
- `bodyTemplate` instead of `description`
- `validationPattern` instead of `pattern`
Some of these variable names have been generalized to ease the
unification/generalization of dialog components:
- `submitAction` instead of `deletionMethod`
- Removed unique `setUp` method.
Benefits:
- Creation of the component is done as intended by the developers of
the `ngx-boostrap` package and as expected by developers which use
the package. The `setUp` method does not have to be called anymore
on the `DeletionModalComponent` exclusively but instead the
component is instantiated as all other modals. Property assignment
on the instantiated object isn't handled by the `setUp` method
anymore but by the `modalService`.
- With the removal of the `setUp` method, some tests could be
removed as well.
- No need to pass the reference of the created modal to the modal
manually.
Preserved:
- The provided check within the `setUp` method, which checked if the
component had been correctly instantiated, has been moved to the
`ngOnInit` method of the component.
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-09-17 08:25:41 +02:00
Kefu Chai
e69e50a859
rpm: use updated gperftools
...
make sure we only build with the higher version of gperftools on
distros where both 2.4 and 2.6.1 are packaged. see
https://git.centos.org/summary/rpms!gperftools.git . at the time of
writing, gperftools 2.6.1 is packaged for CentOS/RHEL 7, if gperftools
(>= 2.4) is required by Ceph, and user already has this version
installed, when new Ceph packages are installed, the updated gperftools
2.6.1 version won't be installed as a dependency. when launching
Ceph compiled with tcmalloc enabled, we will have
symbol lookup error: ceph-osd: undefined symbol: _ZdaPvm
so, by bumping up the required version of gperftools, the updated
gperftools will be installed.
see https://software.opensuse.org/package/gperftools , openSUSE/SLE offer
2.5. so they are safe at this moment.
Fixes: http://tracker.ceph.com/issues/35969
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-09-17 14:20:51 +08:00
李丙洋 10208981
64613065ac
doc: Fix Typos of Developer Guide
...
Signed-off-by: Li Bingyang <li.bingyang1@zte.com.cn>
2018-09-17 09:59:29 +08:00
Patrick Donnelly
0ac789c435
Merge PR #24023 into master
...
* refs/pull/24023/head:
mds: prevent rctime from going back
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-16 15:01:01 -07:00
Patrick Donnelly
b43f26c1d9
Merge PR #24022 into master
...
* refs/pull/24022/head:
client: update ctime when modifying file content
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-16 14:58:38 -07:00
Patrick Donnelly
1b3b2744f9
Merge PR #24048 into master
...
* refs/pull/24048/head:
mds: use common send message mechanisms
mds: remove dead code
Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-09-16 14:51:17 -07:00
Patrick Donnelly
cd1ce232ba
Merge PR #24053 into master
...
* refs/pull/24053/head:
mds: translate CEPH_MDS_STATE_NULL state name as null
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-16 14:33:55 -07:00
Patrick Donnelly
ea43ce7aae
Merge PR #21992 into master
...
* refs/pull/21992/head:
qa: extend timeout for SessionMap flush
Reviewed-by: John Spray <john.spray@redhat.com>
2018-09-16 14:30:28 -07:00
Jason Dillaman
0a31c55ea8
librbd: do not invalidate object map when attempting to delete non-existent snapshot
...
If duplicate snapshot remove requests are received by the lock owner from a peer
client, the first request will remove the object map. If the second request
arrives while the first is in-progress, it will again attempt to remove the
object map but fail to load it since it's already been deleted. This incorrectly
results in the next object map being flagged as invalid.
Fixes: http://tracker.ceph.com/issues/24516
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-16 12:46:25 -04:00
Jason Dillaman
58770188ab
librbd: converted object map snapshot remove state machine to new style
...
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-09-16 12:46:25 -04:00
Kefu Chai
28d9982bde
Merge pull request #24029 from smithfarm/wip-35922
...
cmake: no libradosstriper headers if WITH_LIBRADOSSTRIPER=OFF
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
2018-09-17 00:04:03 +08:00
Mykola Golub
a68be2b452
Merge pull request #24036 from dillaman/wip-24688
...
rbd-mirror: optionally extract peer secrets from config-key
Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-16 11:53:04 +03:00
Mykola Golub
fa7bb832a3
Merge pull request #24063 from dillaman/wip-24140
...
rbd-mirror: prevent creation of clones when parents are syncing
Reviewed-by: Mykola Golub <mgolub@suse.com>
2018-09-16 11:52:32 +03:00
Patrick Donnelly
9c10509092
log: use smaller buffer for ConcreteEntry
...
This brings down the static size of the memory used by the logging infrastructure:
If we used 1024, we'd have 1088*10000 = 10880000 = 10MB in use by the ring
buffer and 2*1088*100 = 2*108800 = 2*106KB for the m_new and m_flush
vectors.
In my testing, 1024 covers most log entries.
Note, I've kept 4096 for the StackStringStream via MutableEntry as these are
already allocated on the heap and cached in a thread local vector. Generally
there should only be about a dozen of these allocated so it's worth keeping a
larger buffer.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-14 19:46:12 -07:00
Patrick Donnelly
b86994e3f9
common: remove obsolete buffer classes
...
In favor of the simpler StackStreamBuffer.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-14 19:46:11 -07:00
Patrick Donnelly
a747aeac13
log: avoid heap allocations for most log entries
...
Each log Entry now exists on the stack and uses a large (4k) buffer for its log
stream. This Entry is std::move'd to the queues (std::vector and
boost::circular_buffer) in the Log, involving only memory copies in the general
case. There are two memory copies (std::move) for any given Entry, once in
Log::submit_entry and again in Log::_flush
In practice, this eliminates 100% of allocations outside of startup
allocations
I've run a simple experiment with the MDS that copies /usr/bin to CephFS. I got
measurements for the number of allocations from the heap profiler and the
profile of CPU usage in the MDS.
** Before this patch **
== Heap profile: ==
$ google-pprof --alloc_objects --text bin/ceph-mds out/mds.a.profile.0001.heap
Total: 1105048 objects
433329 39.2% 39.2% 433329 39.2% ceph::logging::Log::create_entry
209311 18.9% 58.2% 209311 18.9% __gnu_cxx::__aligned_membuf::_M_addr (inline)
192963 17.5% 75.6% 192963 17.5% __gnu_cxx::new_allocator::allocate (inline)
61774 5.6% 81.2% 61774 5.6% std::__cxx11::basic_string::_M_mutate
37689 3.4% 84.6% 37689 3.4% ceph::buffer::raw_combined::create (inline)
22773 2.1% 86.7% 22773 2.1% mempool::pool_allocator::allocate (inline)
17761 1.6% 88.3% 20523 1.9% std::pair::pair (inline)
15795 1.4% 89.7% 15797 1.4% std::swap (inline)
11011 1.0% 90.7% 130061 11.8% std::__cxx11::list::_M_insert (inline)
10822 1.0% 91.7% 10822 1.0% std::__cxx11::basic_string::reserve
9108 0.8% 92.5% 32721 3.0% __gnu_cxx::new_allocator::construct (inline)
8608 0.8% 93.3% 8610 0.8% std::_Deque_base::_M_initialize_map (inline)
7694 0.7% 94.0% 7694 0.7% std::__cxx11::basic_string::_M_capacity (inline)
6160 0.6% 94.5% 6160 0.6% Journaler::wrap_finisher
6084 0.6% 95.1% 70892 6.4% std::map::operator[] (inline)
5347 0.5% 95.6% 5347 0.5% MutationImpl::add_projected_fnode
4381 0.4% 96.0% 7706 0.7% mempool::pool_allocator::construct (inline)
3588 0.3% 96.3% 182966 16.6% Locker::_do_cap_update
3049 0.3% 96.6% 5280 0.5% std::__shared_count::__shared_count (inline)
3043 0.3% 96.9% 3043 0.3% MDSLogContextBase::MDSLogContextBase (inline)
3038 0.3% 97.1% 14763 1.3% std::__uninitialized_copy::__uninit_copy (inline)
So approximately 430k heap allocations for Entry were created! The
basic_string::_M_mutate is also another allocation via PrebufferedStreambuf
== Profile data ==
Selecting interesting functions
Samples: 798K of event 'cycles:pp', Event count (approx
Children Self Com Shared Object Symbol
+ 1.04% 1.04% log libceph-common.so.0 [.] ceph::logging::Log::_flush
+ 0.05% 0.05% log libceph-common.so.0 [.] ceph::logging::Log::flush
+ 0.00% 0.00% log libceph-common.so.0 [.] ceph::logging::Log::_log_safe_write
+ 0.00% 0.00% log libceph-common.so.0 [.] ceph::logging::Log::entry
+ 0.00% 0.00% log libceph-common.so.0 [.] ceph::logging::Log::_flush_logbuf
...
Children Self Command Shared Object Symbol
+ 3.69% 0.00% safe_timer libceph-common.so.0 [.] CachedPrebufferedStreambuf::~CachedPrebufferedStreambuf
+ 0.53% 0.00% ms_dispatch libceph-common.so.0 [.] CachedPrebufferedStreambuf::~CachedPrebufferedStreambuf
+ 0.13% 0.00% fn_anonymous libceph-common.so.0 [.] CachedPrebufferedStreambuf::~CachedPrebufferedStreambuf
+ 0.00% 0.00% ms_dispatch libceph-common.so.0 [.] CachedPrebufferedStreambuf::create
+ 0.00% 0.00% fn_anonymous libceph-common.so.0 [.] CachedPrebufferedStreambuf::create
Children Self Command Shared Object Symbol
+ 0.07% 0.07% fn_anonymous libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% ms_dispatch libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% ms_dispatch ceph-mds [.] _ZN4ceph7logging3Log12create_entryEiiPm@plt
+ 0.00% 0.00% md_submit libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% fn_anonymous ceph-mds [.] _ZN4ceph7logging3Log12create_entryEiiPm@plt
+ 0.00% 0.00% safe_timer libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% mds_rank_progr libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% mds_rank_progr ceph-mds [.] _ZN4ceph7logging3Log12create_entryEiiPm@plt
+ 0.00% 0.00% msgr-worker-0 libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% msgr-worker-2 libceph-common.so.0 [.] ceph::logging::Log::create_entry
+ 0.00% 0.00% md_submit ceph-mds [.] _ZN4ceph7logging3Log12create_entryEiiPm@plt
+ 0.00% 0.00% msgr-worker-1 libceph-common.so.0 [.] ceph::logging::Log::create_entry
Children Self Command Shared Object Symbol
+ 8.29% 0.00% ms_dispatch libstdc++.so.6.0.24 [.] virtual thunk to std::basic_ostream<char, std::char_traits<char> >::~basic_ostream
+ 7.55% 1.46% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::_M_insert<long>
+ 3.87% 0.00% fn_anonymous libstdc++.so.6.0.24 [.] std::basic_ostream<char, std::char_traits<char> >::~basic_ostream
+ 2.92% 0.00% md_submit libstdc++.so.6.0.24 [.] virtual thunk to std::basic_ostream<char, std::char_traits<char> >::~basic_ostream
+ 2.38% 0.00% fn_anonymous libstdc++.so.6.0.24 [.] virtual thunk to std::basic_ostream<char, std::char_traits<char> >::~basic_ostream
+ 2.22% 2.22% fn_anonymous libstdc++.so.6.0.24 [.] std::ostream::sentry::sentry
+ 1.89% 0.13% fn_anonymous libstdc++.so.6.0.24 [.] std::__ostream_insert<char, std::char_traits<char> >
+ 0.71% 0.00% ms_dispatch libstdc++.so.6.0.24 [.] std::basic_ostream<char, std::char_traits<char> >::~basic_ostream
+ 0.39% 0.06% fn_anonymous libstdc++.so.6.0.24 [.] std::ostream::_M_insert<long>
+ 0.29% 0.21% ms_dispatch libstdc++.so.6.0.24 [.] std::__ostream_insert<char, std::char_traits<char> >
+ 0.27% 0.27% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::sentry::~sentry
+ 0.27% 0.27% fn_anonymous libstdc++.so.6.0.24 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>
+ 0.22% 0.22% ms_dispatch libstdc++.so.6.0.24 [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
+ 0.20% 0.20% ms_dispatch libstdc++.so.6.0.24 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>
+ 0.15% 0.15% fn_anonymous libstdc++.so.6.0.24 [.] std::ostream::sentry::~sentry
+ 0.14% 0.14% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::sentry::sentry
+ 0.13% 0.00% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::_M_insert<unsigned long>
+ 0.13% 0.13% fn_anonymous libstdc++.so.6.0.24 [.] std::basic_streambuf<char, std::char_traits<char> >::xsputn
+ 0.00% 0.00% fn_anonymous libstdc++.so.6.0.24 [.] std::ostream::_M_insert<unsigned long>
+ 0.00% 0.00% ms_dispatch libstdc++.so.6.0.24 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<unsigned long>
And the unittest_log time:
$ bin/unittest_log
[==========] Running 15 tests from 1 test case
[----------] Global test environment set-up
[----------] 15 tests from Log
[ RUN ] Log.Simple
[ OK ] Log.Simple (0 ms)
[ RUN ] Log.ReuseBad
[ OK ] Log.ReuseBad (1 ms)
[ RUN ] Log.ManyNoGather
[ OK ] Log.ManyNoGather (0 ms)
[ RUN ] Log.ManyGatherLog
[ OK ] Log.ManyGatherLog (12 ms)
[ RUN ] Log.ManyGatherLogStringAssign
[ OK ] Log.ManyGatherLogStringAssign (27 ms)
[ RUN ] Log.ManyGatherLogStringAssignWithReserve
[ OK ] Log.ManyGatherLogStringAssignWithReserve (27 ms)
[ RUN ] Log.ManyGatherLogPrebuf
[ OK ] Log.ManyGatherLogPrebuf (15 ms)
[ RUN ] Log.ManyGatherLogPrebufOverflow
[ OK ] Log.ManyGatherLogPrebufOverflow (15 ms)
[ RUN ] Log.ManyGather
[ OK ] Log.ManyGather (8 ms)
[ RUN ] Log.InternalSegv
[WARNING] /home/pdonnell/cephfs-shell/src/googletest/googletest/src/gtest-death-test.cc:836:: Death tests use fork(), which is unsafe particularly in a threaded context. For this test, Google Test detected 3 threads
[ OK ] Log.InternalSegv (8 ms)
[ RUN ] Log.LargeLog
[ OK ] Log.LargeLog (43 ms)
[ RUN ] Log.TimeSwitch
[ OK ] Log.TimeSwitch (1 ms)
[ RUN ] Log.TimeFormat
[ OK ] Log.TimeFormat (0 ms)
[ RUN ] Log.Speed_gather
[ OK ] Log.Speed_gather (1779 ms)
[ RUN ] Log.Speed_nogather
[ OK ] Log.Speed_nogather (64 ms)
[----------] 15 tests from Log (2000 ms total)
[----------] Global test environment tear-down
[==========] 15 tests from 1 test case ran. (2000 ms total)
[ PASSED ] 15 tests
** After Patch **
The StackStreamBuf uses 4k for its default buffer. This appears to be more than
reasonable for preventing allocations for logging
== Heap profile: ==
$ google-pprof --alloc_objects --text bin/ceph-mds out/mds.a.profile.0001.heap
Total: 1052127 objects
384957 36.6% 36.6% 384957 36.6% __gnu_cxx::new_allocator::allocate (inline)
274720 26.1% 62.7% 274720 26.1% __gnu_cxx::__aligned_membuf::_M_addr (inline)
68496 6.5% 69.2% 68496 6.5% std::__cxx11::basic_string::_M_mutate
44140 4.2% 73.4% 51828 4.9% std::pair::pair (inline)
43091 4.1% 77.5% 43091 4.1% ceph::buffer::raw_combined::create (inline)
27706 2.6% 80.1% 236407 22.5% std::__cxx11::list::_M_insert (inline)
25699 2.4% 82.6% 25699 2.4% std::__cxx11::basic_string::reserve
23183 2.2% 84.8% 23183 2.2% mempool::pool_allocator::allocate (inline)
19466 1.9% 86.6% 81716 7.8% __gnu_cxx::new_allocator::construct (inline)
17606 1.7% 88.3% 17606 1.7% std::__cxx11::basic_string::_M_capacity (inline)
16879 1.6% 89.9% 16881 1.6% std::swap (inline)
8572 0.8% 90.7% 8574 0.8% std::_Deque_base::_M_initialize_map (inline)
8477 0.8% 91.5% 11808 1.1% mempool::pool_allocator::construct (inline)
6166 0.6% 92.1% 6166 0.6% Journaler::wrap_finisher
6080 0.6% 92.7% 134975 12.8% std::map::operator[] (inline)
6079 0.6% 93.3% 6079 0.6% MutationImpl::add_projected_fnode
== Profile data ==
Samples: 62K of event 'cycles:u', Event count (approx.)
Overhead Command Shared Object Symbol
+ 5.91% log libc-2.23.so [.] vfprintf
+ 5.75% ms_dispatch libstdc++.so.6.0.24 [.] std::__ostream_insert<char, std::char_traits<char> >
+ 4.59% ms_dispatch ceph-mds [.] StackStringBuf<4096ul>::xsputn
+ 4.26% ms_dispatch libc-2.23.so [.] __memmove_ssse3_back
+ 4.07% log libceph-common.so.0 [.] ceph::logging::Log::_flush
+ 2.48% ms_dispatch libstdc++.so.6.0.24 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<l
+ 2.13% fn_anonymous libstdc++.so.6.0.24 [.] std::__ostream_insert<char, std::char_traits<char> >
+ 2.09% ms_dispatch ceph-mds [.] CDir::check_rstats
+ 2.06% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::sentry::sentry
+ 1.98% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::sentry::~sentry
+ 1.87% log libc-2.23.so [.] __strcpy_sse2_unaligned
+ 1.60% fn_anonymous ceph-mds [.] StackStringBuf<4096ul>::xsputn
+ 1.46% log libc-2.23.so [.] _IO_default_xsputn
+ 1.45% log libc-2.23.so [.] _itoa_word
+ 1.43% fn_anonymous libc-2.23.so [.] __memmove_ssse3_back
+ 1.40% ms_dispatch libstdc++.so.6.0.24 [.] std::ostream::_M_insert<long>
+ 0.98% fn_anonymous libstdc++.so.6.0.24 [.] std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<l
+ 0.89% ms_dispatch libstdc++.so.6.0.24 [.] 0x
+ 0.88% ms_dispatch libstdc++.so.6.0.24 [.] std::_Rb_tree_increment
And the unittest_log time:
$ bin/unittest_log
[==========] Running 13 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 13 tests from Log
[ RUN ] Log.Simple
[ OK ] Log.Simple (1 ms)
[ RUN ] Log.ReuseBad
[ OK ] Log.ReuseBad (0 ms)
[ RUN ] Log.ManyNoGather
[ OK ] Log.ManyNoGather (0 ms)
[ RUN ] Log.ManyGatherLog
[ OK ] Log.ManyGatherLog (83 ms)
[ RUN ] Log.ManyGatherLogStringAssign
[ OK ] Log.ManyGatherLogStringAssign (79 ms)
[ RUN ] Log.ManyGatherLogStackSpillover
[ OK ] Log.ManyGatherLogStackSpillover (81 ms)
[ RUN ] Log.ManyGather
[ OK ] Log.ManyGather (80 ms)
[ RUN ] Log.InternalSegv
[WARNING] /home/pdonnell/ceph/src/googletest/googletest/src/gtest-death-test.cc:836:: Death tests use fork(), which is unsafe particularly in a threaded context. For this test, Google Test detected 3 threads.
[ OK ] Log.InternalSegv (7 ms)
[ RUN ] Log.LargeLog
[ OK ] Log.LargeLog (55 ms)
[ RUN ] Log.TimeSwitch
[ OK ] Log.TimeSwitch (4 ms)
[ RUN ] Log.TimeFormat
[ OK ] Log.TimeFormat (1 ms)
[ RUN ] Log.Speed_gather
[ OK ] Log.Speed_gather (1441 ms)
[ RUN ] Log.Speed_nogather
[ OK ] Log.Speed_nogather (63 ms)
[----------] 13 tests from Log (1895 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 1 test case ran. (1895 ms total)
[ PASSED ] 13 tests.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-09-14 19:46:08 -07:00
Kefu Chai
9e7a0111da
Merge pull request #24108 from clebergnu/rm_qa_btrfs_binary
...
qa/btrfs/test_rmdir_async_snap: remove binary file
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-09-15 10:17:58 +08:00
Josh Durgin
53c3119d42
Merge pull request #23989 from dzafman/wip-35846
...
tools: ceph-objectstore-tool: Allow target level as first positional argument
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-09-14 16:07:09 -07:00
Robin H. Johnson
3b864482d6
rgw: fix chunked-encoding for chunks >1MiB
...
For HTTP responses sent with chunked-encoding, and greater than 1MiB in
size, the chunk-size field was being printed wrong.
Specifically, the chunk-size field was being sent with a mangled or
missing trailer of '\r\n'.
This bug manifested as HTTP clients being unable to read the response:
Chrome generates ERR_INCOMPLETE_CHUNKED_ENCODING
Python/boto generates httplib.LineTooLong: got more than 65536 bytes when reading chunk size
The wrong variable was being used to determine the size of the buffer
used for the chunk-size field.
Fix it by using the correct variable, and rename the variables to
clearly reflect their purpose.
Prior to PR#23940, this would only have been seen in some Swift
operations. PR#23940 changed some S3 operations to also use chunked
encoding to get responses sent faster, and made the bug easier to
detect. It was initially reported for a ListBucket call with a high
max-keys argument.
Backport: luminous, mimic
Reference: https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
Reference: https://github.com/ceph/ceph/pull/23940
Fixes: http://tracker.ceph.com/issues/35990
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
2018-09-14 14:38:24 -07:00
Radoslaw Zarzynski
184d0a75f9
osd: avoid unncecessary ref-counting in OSD::enqueue_op.
...
```
- 4,30% 0,32% msgr-worker-0 ceph-osd [.] OSD::enqueue_op
- 3,98% OSD::enqueue_op
+ 2,40% OSD::ShardedOpWQ::_enqueue
+ 0,42% OpRequest::mark_flag_point
0,37% operator new
0,36% PerfCounters::tinc
+ 0,21% Mutex::Unlock
```
Annotate:
```
OSD::enqueue_op /home/perf/src/ceph-rzarzynski3/build/bin/ceph-osd
│93 if( px != 0 ) intrusive_ptr_add_ref( px );
│ test %rbx,%rbx
│95 _ZN13PGOpQueueableC4E5spg_t():
│175 explicit PGOpQueueable(spg_t pg) : pgid(pg) {}
0,70 │ mov %cl,0x18(%rax)
│177 _ZN8PGOpItemC4E5spg_tN5boost13intrusive_ptrI9OpRequestEE():
│204 PGOpItem(spg_t pg, OpRequestRef op) : PGOpQueueable(pg), op(op) {
│ mov %rdx,(%rax)
│206 _ZN5boost13intrusive_ptrI9OpRequestEC4ERKS2_():
│ ↓ je 166
│94 _ZNSt13__atomic_baseIiEppEv():
48,25 │ lock addl $0x1,0x30(%rbx)
│166: mov 0x158(%rbp),%ecx
│ mov 0x15c(%rbp),%edx
│299 _ZN17ShardedThreadPool9ShardedWQI11OpQueueItemE5queueEOS1_():
│669 tp->set_wq(this);
│670 }
│671 ~ShardedWQ() override {}
│
│673 void queue(T&& item) {
│674 _enqueue(std::move(item));
2,80 │ mov %r12,%rsi
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-09-14 21:39:10 +02:00
Sage Weil
3e791d1f9e
doc: update docs for device management
...
Move this out of mgr section and into rados operations.
Leave a placeholder for failure prediction, which is still a work in
progress.
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-14 14:06:41 -05:00
Sage Weil
5d68f77360
mgr: make devicehealth always-on
...
Signed-off-by: Sage Weil <sage@redhat.com>
2018-09-14 14:05:57 -05:00
Casey Bodley
764d6a8599
rgw: abort_bucket_multiparts() ignores individual NoSuchUpload errors
...
if the bucket index lists multipart meta objects that don't actually
exist in rados, this error prevents the bucket from being deleted
Fixes: http://tracker.ceph.com/issues/35986
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-09-14 15:05:07 -04:00
Casey Bodley
138be4fc53
crimson/net: messenger uses dispatch gate for authorizer callbacks
...
this also removes the checks for null dispatcher - start() must be
called with a valid dispatcher before getting here
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-09-14 14:47:32 -04:00
Casey Bodley
cabc60aaec
crimson/net: messenger uses dispatch gate for accept/connect callbacks
...
the dispatcher can throw to reject a new connection, so these callbacks
were moved ahead of the exception handling
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-09-14 14:47:32 -04:00
