Commit Graph

79794 Commits

Author SHA1 Message Date
Alfredo Deza
4674da9c07 ceph-volume tests add support for bluestore list verification
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:07 -04:00
Alfredo Deza
5b9f0853f7 ceph-volume tests activate with fsid works for bluestore and filestore
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:06 -04:00
Alfredo Deza
15586715f5 ceph-volume tests arg validator can accept absolute paths
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:06 -04:00
Alfredo Deza
39bdc2e3f6 ceph-volume tests update changes from the help menu in lvm
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:06 -04:00
Alfredo Deza
9be1012072 ceph-volume tests update to get_lv from get_journal_lv
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:06 -04:00
Alfredo Deza
d5984bad97 ceph-volume tests.util new path/device mount interface
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:10:06 -04:00
Alfredo Deza
e8a1863753 ceph-volume tests update api.lvm tests
The create_lv signature changed to require full size description and
tags need to be an actual dictionary (vs. keyword args)

Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-20 12:09:43 -04:00
Casey Bodley
1a2db61de2 Merge pull request #18335 from linuxbox2/wip-rgw-readdir2
rgw_file: implement variant offset readdir processing

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-10-20 11:55:17 -04:00
Jason Dillaman
a3a40413f7 common/common_init: disable ms subsystem log gathering for clients
The log gathering causes large performance degradation to clients
with high message throughputs. This is hopefully a short-term
workaround until per-message logging can be replaced with an
efficient data recording system for post-incident analysis
use-cases.

Fixes: http://tracker.ceph.com/issues/21860
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-20 11:15:07 -04:00
Sage Weil
4324c8bc7e os/bluestore/BlueFS: fix race with log flush during async log compaction
During async log compaction we rely on _flush-and_sync_log to update the
log_writer to jump_to.  However, if racing threads are also trying to flush
the log and manage to flush our new log events for us, then our flush will
turn into a no-op, and we won't update jump_to correctly at all.  This
results in a corrupted log size a bit later one.

Fix by ensuring that there are no in-progress flushes before we add our
log entries.  Also, add asserts to _flush_and_sync_log to make sure we
never bail out early if jump_to is set (which would indicate this or
another similar bug is still present).

Fixes: http://tracker.ceph.com/issues/21878
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 08:51:17 -05:00
Kefu Chai
38e789dc3b Merge pull request #18392 from tchaikov/wip-21842
os/bluestore: implement BlueRocksEnv::AreFilesSame()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-20 21:37:44 +08:00
Jason Dillaman
eca4fafbcf osdc/Objecter: delay initialization of hobject_t in _send_op
Fixes: http://tracker.ceph.com/issues/21845
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-20 09:32:14 -04:00
Enming Zhang
7b8502c9a2 rgw: copy object add response error message
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
2017-10-20 20:20:24 +08:00
Sage Weil
9b92d87d37 buffer: remove list _mempool member
This broke the C++ ABI by changing the list structure size.  Also, it's
not necessary as we can infer the mempool by looking at the other list
contents.  We don't (currently) have a need to map an empty list to a
particular mempool and have that state stick.

Fixes: http://tracker.ceph.com/issues/21573
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-20 07:16:22 -05:00
John Spray
8c45855325 Merge pull request #18251 from jan--f/mgr-py-module-constants
pybind/mgr_module: move PRIO_* and PERFCOUNTER_* to MgrModule class

Reviewed-by: John Spray <john.spray@redhat.com>
2017-10-20 12:58:22 +01:00
John Spray
bd8814bf58 Merge pull request #18021 from jcsp/wip-prometheus-disk-occupation
mgr/prometheus: add ceph_disk_occupation series

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2017-10-20 12:28:12 +01:00
Enming Zhang
6f3fe82646 common/options.cc: cleanup readable literals for default sizes
For better readablity.

Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
2017-10-20 18:41:38 +08:00
Jianpeng Ma
73fb6eef73 Revert "submodule: make libmpem as a submodule."
This reverts commit 6a14159ee4.
Using ExternalProject_Add::GIT to download nvml source .

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-20 17:55:48 +08:00
Yan, Zheng
edd76fc883 mds: disable early reply for dir layout and quota related requests
These requests impacts whole subtree tree, replaying them when
mds recovers may break order of requests in multimds cluster.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2017-10-20 17:13:31 +08:00
Mykola Golub
8eeae328cb Merge pull request #18250 from dillaman/wip-21763
doc/rbd: tweaks for the LIO iSCSI gateway

Reviewed-by: Mykola Golub <to.my.trociny@gmail.com>
2017-10-20 11:55:44 +03:00
Kefu Chai
5d31a5aee4 os/bluestore: implement BlueRocksEnv::AreFilesSame()
it is used by the "repair" feature to dedup the files to be searched for
MANIFEST-* files. the default implementation is the POSIX one, which
tries to look at the local fs, but we should be looking for the files in
the bluefs. in this very use case, wal and db do not share the same device,
so we can just compare the paths. actually, it should aways return
"false". as the files being compared are always "db" and "db.wal".

Fixes: http://tracker.ceph.com/issues/21842
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-20 12:25:14 +08:00
Kefu Chai
d7244dc030 Merge pull request #18387 from tchaikov/wip-dmclock-client-info-ptr
osd,dmclock: use pointer to ClientInfo instead of a copy of it

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2017-10-20 11:47:59 +08:00
Kefu Chai
e956c9a195 Merge pull request #18350 from ovh/bp-ceph-conf-dump-all
tools/ceph-conf: dump parsed config in plain text or as json

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-20 11:46:26 +08:00
Kefu Chai
7425a8830c Merge pull request #18367 from zealoussnow/wip-10180
ceph-bluestore-tool: create out_dir before create full path  of kvdb

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-20 11:45:38 +08:00
kungf
87be7c70a1 osd: add scrub week day constraint
if add week day constraint, we can set scrub permit time
more flexible. eg. we can set scurb in Monday-Wednesday 0-12 o'clock
according set this parameter:
osd_scrub_begin_week_day = 1
osd_scrub_end_week_day = 3
osd_scrub_begin_hour = 0
osd_scrub_end_hour = 12

Signed-off-by: kungf <yang.wang@easystack.cn>
2017-10-20 10:33:21 +08:00
Jason Dillaman
dc9b309d03 osdc/Objecter: skip sparse-read result decode if bufferlist is empty
If the OSD does not execute sub-ops due to errors encountered prior to
the sub-op, the sub-op result remains zeroed with empty out data.
Attempting to decode the empty bufferlist results in large exception
handling CPU overhead.

Fixes: http://tracker.ceph.com/issues/21844
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-19 22:33:01 -04:00
Kefu Chai
74f147b849 Merge pull request #17441 from yanyixing/update-umcloud-affiliation
mailmap: Update umcloud affiliation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-20 09:58:49 +08:00
chunmei
857b58dd1b fix two assert happend in dpdk library.
1.RTE_ASSERT(mp->elt_size >= sizeof(struct rte_mbuf) +
		user_mbp_priv->mbuf_data_room_size +
		user_mbp_priv->mbuf_priv_size); in rte_pktmbuf_pool_init
2.RTE_ASSERT(rte_mbuf_refcnt_read(m) == 0) in rte_mbuf_raw_alloc

Signed-off-by: chunmei <chunmei.liu@intel.com>
2017-10-19 14:46:58 -07:00
Sage Weil
481277b8c9 buffer: allow mempool to be passed into raw* ctors and create methods
This is more convenient, and also faster than initializing it in
buffer_anon and the immediately moving it elsewhere.

Drop the optionality of the alignment argument.

No users yet.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-19 16:30:11 -05:00
Adam C. Emerson
14c3f3c214 zstd: Upgrade to v1.3.2
This buys us fixes for a few memory errors and asserts and also
improves the CMake build script so it doesn't mostly ignore the
CXXFLAGS we pass in.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-19 16:08:07 -04:00
Josh Durgin
34951266fe Merge pull request #18374 from tchaikov/wip-ceph-C-d
ceph.in: do not panic at control+d in interactive mode

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Amit Kumar <amitkuma@redhat.com>
2017-10-19 11:52:13 -07:00
Adam C. Emerson
bda3c2673b OSD: Silence unused variable warnings
Even though extending the lifetime of this temporary delays execution
of its destructor, the compiler does not consider the reference to
have a side-effecting constructor.

The compiler is wrong, but we can silence its complaints.

This will cease to be an issue once we adopt C++17, since mandatory
RVO will solve the inability to bind a variable to a returned
move-only object and the need to extend a temporary.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-19 14:43:57 -04:00
Adam C. Emerson
87b4d1646e kvstore_tool: Create default constructor
Clang4 does not recognize a constructor with all arguments defaulted
as a default constructor in some circumstances.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-19 13:59:50 -04:00
Adam C. Emerson
c40953f696 test/cls_journal: Fix reversed std::string constructor arguments
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2017-10-19 13:15:32 -04:00
Ali Maredia
73f75c5365 cmake: add cython_rbd as a dependency to vstart target
Without cython_rbd, there are import errors
in the mgr log and the ceph-mgr dashboard cannot be
viewed by building just the vstart target.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-10-19 13:04:46 -04:00
Jens Rosenboom
bd0576652d Update docs for manual deployment
Fixes: http://tracker.ceph.com/issues/20309
Signed-off-by: Jens Rosenboom <j.rosenboom@x-ion.de>
2017-10-19 17:03:16 +00:00
Kefu Chai
cc09e30d42 Merge pull request #17462 from majianpeng/pmem-work
cmake: compile nvml as an external project

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-10-19 23:38:10 +08:00
Jianpeng Ma
b4cc9a9aa8 cmake: Build libpmem from source code.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2017-10-20 00:44:09 +08:00
Alfredo Deza
f3bb616bcf ceph-volume lvm.activate ceph-bluestore-tool should populate the osd dir
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-19 11:31:38 -04:00
Alfredo Deza
472aa0b199 ceph-volume lvm.activate auto detect objectstore type
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-19 11:31:38 -04:00
Alfredo Deza
f579ef0702 ceph-volume lvm.trigger enable objectstore auto detection for activate
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-19 11:31:38 -04:00
Alfredo Deza
5696fed09b ceph-volume lvm.activate add bluestore support for activation
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-19 11:31:38 -04:00
Alfredo Deza
f61007cc65 ceph-volume util.system create mappings of mounts
This makes it easier for consumers to detect a path, or a device that
might be mounted or might have more than one mount, including tmpfs and
devtmpfs support, which was just not possible before.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-19 11:31:38 -04:00
Jason Dillaman
4f227e108d doc/rbd: tweaks for the LIO iSCSI gateway
Fixes: http://tracker.ceph.com/issues/21763
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-10-19 09:40:22 -04:00
amitkuma
17df8c979b rgw: Initialization of epoch,len
Fixes the coverity issues:

** 1402628 Uninitialized scalar field
CID 1402628 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member epoch is not initialized
in this constructor nor in any functions that it calls.

** 1409841 Uninitialized scalar field
CID 1409841 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member len is not initialized
in this constructor nor in any functions that it calls.

** 1416594 Uninitialized scalar field
CID 1416594 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
2. uninit_member: Non-static class member field fh.fh_hk is not
initialized in this constructor nor in any functions that it calls.

Signed-off-by: Amit Kumar <amitkuma@redhat.com>
2017-10-19 19:09:27 +05:30
kungf
1ea6fab5ed osd: make scrub no deadline when max interval is zero
some times, we only want scrub at out permitted time, avoid
scrub mainly deep scrub affecting business io. this patch
make no scrub deadline when osd_scrub_max_interval == 0, then
we can make sure scrub can only do in expect time.

Signed-off-by: kungf <yang.wang@easystack.cn>
2017-10-19 20:15:39 +08:00
Xie Xingguo
3e38616297 Merge pull request #18389 from yaozongyou/fix-type-in-bluestore-migration
doc: fix typo in bluestore-migration.rst

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-10-19 18:36:25 +08:00
Jos Collin
33d99b91c1 Merge pull request #18395 from gaosibei/mailmap
mailmap: Add Sibei, XueYu Affiliation

Reviewed-by: Jos Collin <jcollin@redhat.com>
2017-10-19 10:21:40 +00:00
gaosibei
d6c3e8676b mailmap, organizationmap: add Sibei, XueYu
Signed-off-by: Sibei Gao <gaosb@inspur.com>
2017-10-19 17:02:52 +08:00
Kefu Chai
22aff9e87e osd: update mClock.*Queue to incorporate the changes on dmclock
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-10-19 15:39:46 +08:00