Commit Graph

98687 Commits

Author SHA1 Message Date
Sage Weil
3d519bd96a Merge PR #28242 into master
* refs/pull/28242/head:
	osdc/Objecter: fix OSDMap leak in handle_osd_map

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-06-06 12:15:06 -05:00
Sage Weil
5e569e1cf6 Merge PR #28363 into master
* refs/pull/28363/head:
	osd/ClassHandler: use std::variant for storing func
	osd/ClassHandler: no need to add `struct` before struct type
	osd/ClassHandler: use ceph::mutex instead of Mutex
	osd/ClassHandler: remove unused function
	osd/ClassHandler: s/string/std::string/ in header

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-06-06 12:14:49 -05:00
Jason Dillaman
2964297b9a
Merge pull request #28313 from Songweibin/wip-fix-get-args
rbd/action: fix error getting positional argument

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-06 11:19:35 -04:00
Jason Dillaman
df691487d5
Merge pull request #28253 from Songweibin/wip-39973-rbd
osd/OSDCap: rbd profile permits use of rbd_info

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-06 11:18:43 -04:00
Jason Dillaman
6cd6f5e607
Merge pull request #27902 from mikechristie/rbd-nbd-netlink-support
rbd-nbd: add netlink map/unmap support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-06 10:36:48 -04:00
Matt Benjamin
7a38429a08
Merge pull request #28370 from linuxbox2/wip-rgw-tntbucket
rgw_file: include tenant when hashing bucket names
2019-06-06 08:58:29 -04:00
Lenz Grimmer
e7b605ae5b
mgr/dashboard: show degraded/misplaced/unfound objects. (#28104)
mgr/dashboard: show degraded/misplaced/unfound objects.

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-06-06 11:46:36 +02:00
Kefu Chai
e7a682ca54
Merge pull request #28039 from majianpeng/make-tcmalloc/jemalloc-work-well
make perf_async_msgr link jemalloc/tcmalloc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-06 17:38:58 +08:00
Xie Xingguo
2cc9324c05
Merge pull request #28373 from xiexingguo/wip-40104
mon, osd: parallel clean_pg_upmaps

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-06 17:37:15 +08:00
Jianpeng Ma
72c2926bae test/crimson: make perf_async_msgr link tcmalloc/jemalloc.
For 1M performance, perf_async_msgr w/ tcmalloc/jemalloc is better than
libc.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2019-06-06 11:26:58 +08:00
xie xingguo
a45112a976 test: add parallel clean_pg_upmaps test
With parallel clean_pg_upmaps feature on, the total time cost
of the performance test which now can utilize up to 8 threads for
parallel upmap validating decreased from:

maybe_remove_pg_upmaps (~10000 pg_upmap_items) latency:104s

to:

clean_pg_upmaps (~10000 pg_upmap_items) latency:7s

Note that by default the mon uses only 4 worker threads for
CPU intensive background work, you could further increase
the "mon_cpu_threads" option value if you decided the
time-consuming of clean_pg_upmaps still matters.

Fixes: http://tracker.ceph.com/issues/40104
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-06 10:10:46 +08:00
xie xingguo
c395f45f1f mon/OSDMonitor: do clean_pg_upmaps the parallel way if necessary
There could definitely be some certain cases we could reliably
skip this kind of checking, but there is no easy way to separate
those out.
However, this is clearly the general way to do the massive pg
upmap clean-up job more efficiently and hence should make sense
in all cases.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-06 10:10:45 +08:00
xie xingguo
4d5cf1e417 osd/OSDMap: split clean_pg_upmaps into smaller helpers
- it's good to read.
- the updating pending_inc part should be made independent
  since it is going to be racy while running in parallel.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-06 10:10:45 +08:00
Kefu Chai
848c40a0aa
Merge pull request #28402 from tchaikov/wip-crimson-cleanup
crimson/osd: some cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-06-06 09:52:34 +08:00
David Zafman
590b4138ae
Merge pull request #28302 from dzafman/wip-40078
test: Make sure that extra scheduled scrubs don't confuse test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-05 14:43:30 -07:00
Matt Benjamin
84a2c22108 rgw_file: pretty-print fh_key
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 11:25:11 -04:00
Matt Benjamin
8e0fd5fbfa rgw_file: include tenant in hashes of object
Because bucket names are taken as object names in the top
of an export.  Make hashing by tenant general to avoid disjoint
hashing of bucket.

Fixes: http://tracker.ceph.com/issues/40118

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 11:25:06 -04:00
Matt Benjamin
b800a9de83 rgw_file: include tenant when hashing bucket names
Prevent identical paths from distinct tenants from colliding in
RGW NFS handle cache.

Fixes: http://tracker.ceph.com/issues/40118

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 11:24:59 -04:00
Sage Weil
65379e3165 Merge PR #28353 into master
* refs/pull/28353/head:
	qa/suites/rados/multimon: skew clocks 2s (< paxos lease)
	qa/suites/rados/multimon: create_rbd_pool: false
	qa/suites/rados/multimon: no osds when skewing clock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-05 10:22:26 -05:00
Sage Weil
0a22a5572b Merge PR #28365 into master
* refs/pull/28365/head:
	global/global_context: always add '\0' after strncpy()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-06-05 10:21:19 -05:00
Casey Bodley
93c056c09c
Merge pull request #28391 from cbodley/wip-39984
cls/rgw: test before accessing pkeys->rbegin()

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 09:06:06 -04:00
Kefu Chai
5e1fad62fc crimson/osd/osd.cc: add editor variables/options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:23:50 +08:00
Kefu Chai
343fb96ddd crimson/osd: s/ostream/std::ostream/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:23:49 +08:00
Kefu Chai
1a09f7d21a crimson/osd: mark one of PG::do_peering_event()s private
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:23:49 +08:00
Kefu Chai
6e2087c524
Merge pull request #27995 from richael02/fix_indent
vstart.sh: unify the indent

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:11:43 +08:00
Kefu Chai
439ed8ba48
Merge pull request #28358 from liu-chunmei/ceph_seastar_futurized_store
crimson: add FuturizedStore to encapsulate CyanStore

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 14:33:57 +08:00
Xie Xingguo
302d7bcdd8
Merge pull request #27735 from xiexingguo/wip-device-class-noout
osd: revamp {noup,nodown,noin,noout} related commands

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-05 14:17:06 +08:00
Mike Christie
fd7ef58356 rbd nbd: always try to load module on map
nl and ioctl both need the module loaded so just always load it during
mapping.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
3abc42536f rbd nbd: add support for netlink interface
This adds support for mapping and unmapping using the netlink
interface. If the user passes in --try-netlink during mapping then we
will try to use the netlink interface and if that is not supported use
the ioctl interface. For unmap we will always try netlink and if that is
not supported drop down to the ioctl.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
8815671553 cmake and build: add netlink lib requirement detection
The next patch adds netlink support to rbd-nbd. This patch just adds the
build related changes to bring in the netlink lib and test and build
against it.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
3fe16add7d cmake: Move WITH_RBD option earlier
The next patch will add a WITH_RBD check to the LINUX check block so we
only check for the netlink libs when those 2 conditions are met, so this
moves the WIT_RBD option setup to before the LINUX check.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
b4ba176593 rbd nbd: Add cond to signal nbd disconnection
With the ioctl interface we know when a disconnect has been executed
because the NBD_DO_IO ioctl will return. The netlink interface returns
after the nl connect cmd call has completed, so this patch adds a cond
that signals when a nl disconnect/NBD_DISCONNECT has been executed and
the server thread has handled the resulting NBD_CMD_DISC.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
129a348eda rbd nbd: just move signal handler
This just moves the signal handler to make it easier to call the netlink
disconnect helpers in the next patches.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
b9ca0c1f28 rbd nbd: move nbd index parsing to function
Make the nbd index parsing into a functio so we can use in the ioctl and
netlink paths.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
8c44f8532a rbd nbd: move server startup code to function
The netlink connect cmd interface requires the server to be running when
the connect call completes or commands will timeout/fail. This breaks the
server code to 2 functions so we can control when it starts and runs.

Right now it looks odd in that start_server will allocate and return
the server only for it be freed after run, and it is not touched. The
netlink enablement patch at the end will use the returned server.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
11c6c61549 rbd nbd: move nbd map ioctls to new function
The next patches will add nbd map netlink support so this moves the ioctl
code to a new function to keep things cleaner and not overloading do_map.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Kefu Chai
afe10e01d7
Merge pull request #28366 from albert-github/feature/bug_docu_formula_invocation
common/hobject: Error invocation of formula in documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 12:52:03 +08:00
Kefu Chai
2e57e76e51
Merge pull request #27994 from xiexingguo/wip-balancer-fudge
mgr/balancer: fix fudge

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-05 12:13:44 +08:00
Kefu Chai
f5529ad0b0
Merge pull request #28342 from rzarzynski/wip-messages-std-for-pgcreate2
messages: MOSDPGCreate2 doesn't assume using namespace std.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 12:07:36 +08:00
Kefu Chai
318f12c38e
Merge pull request #28340 from elliswuuuuuuuu/remove_unused_variable
mgr: remove unused variable pool_name

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 12:06:04 +08:00
Kefu Chai
e531b6aa87
Merge pull request #28268 from liewegas/wip-progress-bar
mon: fix off-by-one rendering progress bar

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2019-06-05 12:03:08 +08:00
Kefu Chai
95f31c4e0f vstart.sh: add editor variables/options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 11:46:15 +08:00
Kefu Chai
c176faee4a vstart.sh: add an extra param when "run 'mds'"
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 11:30:16 +08:00
Richael Zhuang
a80aad81ad vstart.sh: unify the indent
Change-Id: Ie1dbe25f81609e5b2f6751da9e87e191a2f78bc2
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
2019-06-05 11:27:05 +08:00
xie xingguo
f6fd4a312e osd/OSDMapMapping: make ParallelPGMapper can accept input pgs
The existing "prime temp" machinism is a good inspiration
for cluster with a large number of pgs that need to do various
calculations quickly.
I am planning to do the upmap tidy-up work the same way, hence
the need for an alternate way of specifying pgs to process other
than taking directly from the map.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-05 10:52:34 +08:00
Patrick Donnelly
2cdcefbf1c
Merge PR #28324 into master
* refs/pull/28324/head:
	client: clean up error checking and return of _lookup_parent
	client: drop Client::lookup_parent

Reviewed-by: huanwen ren <ren.huanwen@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-06-04 17:37:26 -07:00
Patrick Donnelly
a32db16908
Merge PR #28116 into master
* refs/pull/28116/head:
	client: fix "ceph.snap.btime" vxattr value
	client: fix "ceph.dir.rctime" vxattr value

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-04 17:36:04 -07:00
Patrick Donnelly
da9116cbd8
Merge PR #28371 into master
* refs/pull/28371/head:
	qa: fix kclient symlink to use latest rhel7
	qa: fix latest distros and add helper symlinks

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-04 16:22:26 -07:00
Patrick Donnelly
f256c848d1
Merge PR #28148 into master
* refs/pull/28148/head:
	common/secret.c: fix key parsing when doing a remount

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-06-04 16:21:22 -07:00
chunmei Liu
82fedbd008 crimson: add FuturizedStore to encapsulate cyanstore
only add encapsulation. alien store will be implement later.
exception handling will be implement later.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>
2019-06-04 15:07:07 -07:00