Commit Graph

98703 Commits

Author SHA1 Message Date
Yuri Weinstein
dd1ef1081a
Merge pull request #28435 from yuriw/wip-yuriw-smoke-master
qa/tests: reduced distro to run to be random
2019-06-07 09:13:49 -07:00
Kefu Chai
5cc1a62fda
Merge pull request #28428 from tchaikov/wip-package-crimson-osd
rpm,debian,install-deps: package crimson-osd

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2019-06-08 00:03:22 +08:00
Kefu Chai
f03d8e2548 ceph.spec.in: align "Requires" of ceph-osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-07 22:17:19 +08:00
Kefu Chai
34b02f00d0 ceph.spec.in: package crimson-osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-07 22:17:19 +08:00
Kefu Chai
9bdf2dfa0d debian/: add ceph-crimson-osd package
this package is commented out using `# Crimson`. and it will be enabled
only if we are building "crimson" flavor builds.

this package depends on ceph-osd, because it'll basically be a drop-in
replacement of ceph-osd. and it shares all the dependencies of ceph-osd.
so let's just reuse ceph-osd package for now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-07 22:17:19 +08:00
Kefu Chai
05899c1b86 cmake: install crimson-osd if WITH_SEASTAR
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-07 22:17:19 +08:00
Kefu Chai
7f19bf39d6 install-deps.sh: populate WITH_SEASTAR in the same way as for_make_check
because `install-deps.sh` is executed using `source`, we have to pass
these options using env variables. but before this change, `WITH_SEASTAR` is used directly,
while `FOR_MAKE_CHECK` is checked and translated to a local variable
`for_make_check`. which, in my opinion, has better readability.

so, in this change, `WITH_SEASTAR` is translated to `with_seastar`
variable in `install-deps.sh`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-07 22:17:19 +08:00
Yuri Weinstein
4c335b31fe
Merge pull request #28439 from yuriw/wip-yuriw-crontab-master
qa/tests:  filtered in only "trusty"
2019-06-06 16:24:19 -07:00
Patrick Donnelly
2b7d75a219
Merge PR #27731 into master
* refs/pull/27731/head:
	qa: use mimic-O upgrade process

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-06 14:48:44 -07:00
Yuri Weinstein
7939b1a10f qa/tests: filtered in only "trusty"
Fixes: http://tracker.ceph.com/issues/40195
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-06-06 14:36:57 -07:00
Patrick Donnelly
66f18ecd09
qa: use mimic-O upgrade process
Fixes: https://tracker.ceph.com/issues/39436

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-06-06 13:06:56 -07:00
Sage Weil
78122147dc Merge PR #27015 into master
* refs/pull/27015/head:
	OSD: rollforward may need to mark pglog dirty

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-06 12:15:50 -05:00
Sage Weil
3d519bd96a Merge PR #28242 into master
* refs/pull/28242/head:
	osdc/Objecter: fix OSDMap leak in handle_osd_map

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-06-06 12:15:06 -05:00
Sage Weil
5e569e1cf6 Merge PR #28363 into master
* refs/pull/28363/head:
	osd/ClassHandler: use std::variant for storing func
	osd/ClassHandler: no need to add `struct` before struct type
	osd/ClassHandler: use ceph::mutex instead of Mutex
	osd/ClassHandler: remove unused function
	osd/ClassHandler: s/string/std::string/ in header

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-06-06 12:14:49 -05:00
Yuri Weinstein
c6c90663c3 qa/tests: reduced distro to run to be random
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-06-06 10:08:55 -07:00
Jason Dillaman
2964297b9a
Merge pull request #28313 from Songweibin/wip-fix-get-args
rbd/action: fix error getting positional argument

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-06 11:19:35 -04:00
Jason Dillaman
df691487d5
Merge pull request #28253 from Songweibin/wip-39973-rbd
osd/OSDCap: rbd profile permits use of rbd_info

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-06 11:18:43 -04:00
Jason Dillaman
6cd6f5e607
Merge pull request #27902 from mikechristie/rbd-nbd-netlink-support
rbd-nbd: add netlink map/unmap support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-06-06 10:36:48 -04:00
Matt Benjamin
7a38429a08
Merge pull request #28370 from linuxbox2/wip-rgw-tntbucket
rgw_file: include tenant when hashing bucket names
2019-06-06 08:58:29 -04:00
Kefu Chai
4d363edc59 d/control,install-deps: use d/control for installing seastar deps
prepare for building ceph-crimson-osd deb package

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-06 18:06:43 +08:00
Lenz Grimmer
e7b605ae5b
mgr/dashboard: show degraded/misplaced/unfound objects. (#28104)
mgr/dashboard: show degraded/misplaced/unfound objects.

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-06-06 11:46:36 +02:00
Kefu Chai
e7a682ca54
Merge pull request #28039 from majianpeng/make-tcmalloc/jemalloc-work-well
make perf_async_msgr link jemalloc/tcmalloc

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-06 17:38:58 +08:00
Xie Xingguo
2cc9324c05
Merge pull request #28373 from xiexingguo/wip-40104
mon, osd: parallel clean_pg_upmaps

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-06 17:37:15 +08:00
Jianpeng Ma
72c2926bae test/crimson: make perf_async_msgr link tcmalloc/jemalloc.
For 1M performance, perf_async_msgr w/ tcmalloc/jemalloc is better than
libc.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2019-06-06 11:26:58 +08:00
Kefu Chai
3e7741aa2d cmake: add crimson-osd as a dependency of "tests" target
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-06 10:54:54 +08:00
xie xingguo
a45112a976 test: add parallel clean_pg_upmaps test
With parallel clean_pg_upmaps feature on, the total time cost
of the performance test which now can utilize up to 8 threads for
parallel upmap validating decreased from:

maybe_remove_pg_upmaps (~10000 pg_upmap_items) latency:104s

to:

clean_pg_upmaps (~10000 pg_upmap_items) latency:7s

Note that by default the mon uses only 4 worker threads for
CPU intensive background work, you could further increase
the "mon_cpu_threads" option value if you decided the
time-consuming of clean_pg_upmaps still matters.

Fixes: http://tracker.ceph.com/issues/40104
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-06 10:10:46 +08:00
xie xingguo
c395f45f1f mon/OSDMonitor: do clean_pg_upmaps the parallel way if necessary
There could definitely be some certain cases we could reliably
skip this kind of checking, but there is no easy way to separate
those out.
However, this is clearly the general way to do the massive pg
upmap clean-up job more efficiently and hence should make sense
in all cases.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-06 10:10:45 +08:00
xie xingguo
4d5cf1e417 osd/OSDMap: split clean_pg_upmaps into smaller helpers
- it's good to read.
- the updating pending_inc part should be made independent
  since it is going to be racy while running in parallel.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-06 10:10:45 +08:00
Kefu Chai
848c40a0aa
Merge pull request #28402 from tchaikov/wip-crimson-cleanup
crimson/osd: some cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-06-06 09:52:34 +08:00
David Zafman
590b4138ae
Merge pull request #28302 from dzafman/wip-40078
test: Make sure that extra scheduled scrubs don't confuse test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-05 14:43:30 -07:00
Matt Benjamin
84a2c22108 rgw_file: pretty-print fh_key
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 11:25:11 -04:00
Matt Benjamin
8e0fd5fbfa rgw_file: include tenant in hashes of object
Because bucket names are taken as object names in the top
of an export.  Make hashing by tenant general to avoid disjoint
hashing of bucket.

Fixes: http://tracker.ceph.com/issues/40118

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 11:25:06 -04:00
Matt Benjamin
b800a9de83 rgw_file: include tenant when hashing bucket names
Prevent identical paths from distinct tenants from colliding in
RGW NFS handle cache.

Fixes: http://tracker.ceph.com/issues/40118

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 11:24:59 -04:00
Sage Weil
65379e3165 Merge PR #28353 into master
* refs/pull/28353/head:
	qa/suites/rados/multimon: skew clocks 2s (< paxos lease)
	qa/suites/rados/multimon: create_rbd_pool: false
	qa/suites/rados/multimon: no osds when skewing clock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-06-05 10:22:26 -05:00
Sage Weil
0a22a5572b Merge PR #28365 into master
* refs/pull/28365/head:
	global/global_context: always add '\0' after strncpy()

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-06-05 10:21:19 -05:00
Casey Bodley
93c056c09c
Merge pull request #28391 from cbodley/wip-39984
cls/rgw: test before accessing pkeys->rbegin()

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-06-05 09:06:06 -04:00
Kefu Chai
5e1fad62fc crimson/osd/osd.cc: add editor variables/options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:23:50 +08:00
Kefu Chai
343fb96ddd crimson/osd: s/ostream/std::ostream/
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:23:49 +08:00
Kefu Chai
1a09f7d21a crimson/osd: mark one of PG::do_peering_event()s private
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:23:49 +08:00
Kefu Chai
6e2087c524
Merge pull request #27995 from richael02/fix_indent
vstart.sh: unify the indent

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 15:11:43 +08:00
Kefu Chai
439ed8ba48
Merge pull request #28358 from liu-chunmei/ceph_seastar_futurized_store
crimson: add FuturizedStore to encapsulate CyanStore

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-05 14:33:57 +08:00
Xie Xingguo
302d7bcdd8
Merge pull request #27735 from xiexingguo/wip-device-class-noout
osd: revamp {noup,nodown,noin,noout} related commands

Reviewed-by: Sage Weil <sage@redhat.com>
2019-06-05 14:17:06 +08:00
Mike Christie
fd7ef58356 rbd nbd: always try to load module on map
nl and ioctl both need the module loaded so just always load it during
mapping.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
3abc42536f rbd nbd: add support for netlink interface
This adds support for mapping and unmapping using the netlink
interface. If the user passes in --try-netlink during mapping then we
will try to use the netlink interface and if that is not supported use
the ioctl interface. For unmap we will always try netlink and if that is
not supported drop down to the ioctl.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
8815671553 cmake and build: add netlink lib requirement detection
The next patch adds netlink support to rbd-nbd. This patch just adds the
build related changes to bring in the netlink lib and test and build
against it.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
3fe16add7d cmake: Move WITH_RBD option earlier
The next patch will add a WITH_RBD check to the LINUX check block so we
only check for the netlink libs when those 2 conditions are met, so this
moves the WIT_RBD option setup to before the LINUX check.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
b4ba176593 rbd nbd: Add cond to signal nbd disconnection
With the ioctl interface we know when a disconnect has been executed
because the NBD_DO_IO ioctl will return. The netlink interface returns
after the nl connect cmd call has completed, so this patch adds a cond
that signals when a nl disconnect/NBD_DISCONNECT has been executed and
the server thread has handled the resulting NBD_CMD_DISC.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
129a348eda rbd nbd: just move signal handler
This just moves the signal handler to make it easier to call the netlink
disconnect helpers in the next patches.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
b9ca0c1f28 rbd nbd: move nbd index parsing to function
Make the nbd index parsing into a functio so we can use in the ioctl and
netlink paths.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00
Mike Christie
8c44f8532a rbd nbd: move server startup code to function
The netlink connect cmd interface requires the server to be running when
the connect call completes or commands will timeout/fail. This breaks the
server code to 2 functions so we can control when it starts and runs.

Right now it looks odd in that start_server will allocate and return
the server only for it be freed after run, and it is not touched. The
netlink enablement patch at the end will use the returned server.

Signed-off-by: Mike Christie <mchristi@redhat.com>
2019-06-04 23:56:53 -05:00