Commit Graph

99497 Commits

Author SHA1 Message Date
Matt Benjamin
0c505dcab4
Merge pull request #28192 from tianshan/fix_39989
rgw: fix list bucket with start maker and delimiter '/' will miss next object…
2019-07-05 16:21:46 -04:00
Kefu Chai
73746f3807
Merge pull request #28902 from tchaikov/wip-fix-crimson-test-build
test/crimson: fix FTBFS

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-06 01:52:01 +08:00
Kefu Chai
c242d3d636 test/crimson: fix FTBFS
Connection::send_message(Message*)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-06 00:15:27 +08:00
Kefu Chai
057a352be8
Merge pull request #28890 from cyx1231st/wip-seastar-msgr-msgreuse
crimson/net: prevent reusing the sent messages

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-05 21:11:46 +08:00
Yingxin Cheng
fcafb089d2 crimson: change to use make_message()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-07-05 22:07:59 +08:00
Yingxin Cheng
f763d49603 crimson/net: check front_msg correctly during sweep
In order to check whether the front_msg is unchanged, we need to make sure:
* The sent message is not reused;
* The message to be checked is not freed;

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-07-05 21:53:39 +08:00
Yingxin Cheng
2d6e770492 crimson/net: prevent reusing the sent messages
We are tracking sent messages by seq, so reusing messages will cause
undefined behavior because seq should not be shared between different
messages.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2019-07-05 21:51:34 +08:00
Ricardo Marques
9b967b42f0
Merge pull request #28858 from callithea/wip-issue-40395
mgr/dashboard: Extend E2E test section

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-07-05 12:19:21 +01:00
Lenz Grimmer
8bc9c8eed4
mgr/dashboard: use mds_mem.dn for fs dentries (#28505)
mgr/dashboard: use mds_mem.dn for fs dentries

Reviewed-by: Volker Theile <vtheile@suse.com>
2019-07-05 11:42:01 +02:00
Kefu Chai
f22f12d54a
Merge pull request #28876 from tchaikov/wip-cmake-vstart-with-cython_cephfs
cmake: add cython_cephfs to vstart target

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-05 14:00:21 +08:00
Kefu Chai
d6c2ef6a11
Merge pull request #28845 from ifed01/wip-ifed-fix-alloc-dump
os/bluestore: more smart allocator dump when lacking space for bluefs.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2019-07-05 11:28:30 +08:00
Kefu Chai
9e435474ca
Merge pull request #28651 from ifed01/wip-ifed-remove-assert-bs-tool
os/bluestore/bluestore-tool: minor fixes around migrate

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2019-07-05 11:12:53 +08:00
Kefu Chai
3f6f85819f
Merge pull request #28732 from tchaikov/wip-32bit-size_t
osdc/Striper: specialize std::min<>

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-05 11:07:05 +08:00
Kefu Chai
c2905fb3eb
Merge pull request #28829 from tchaikov/wip-dmclock-uint
dmclock: pick up fix to replace uint

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-07-05 11:06:09 +08:00
Kefu Chai
a4ddc4bd85
Merge pull request #27834 from rzarzynski/wip-nss-drop-cms
rgw, common, build: drop NSS support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-05 11:03:33 +08:00
Igor Fedotov
5ea89a4e0c os/bluestore/bluestore-tool: omit device from both source and taget
lists when migrating.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-07-04 20:04:15 +03:00
Igor Fedotov
9297d0420d os/bluestore/bluestore-tool: fix error output
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-07-04 20:04:15 +03:00
Igor Fedotov
47edbfdce8 os/bluestore/bluestore-tool: do not assert when migrate command fails.
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2019-07-04 20:04:13 +03:00
Kefu Chai
ad56cb8aad
Merge pull request #28835 from croit/fix-24420
debian/control: add python-routes dependency for dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-05 01:02:32 +08:00
Sage Weil
1bdc707a90 Merge PR #28877 into master
* refs/pull/28877/head:
	doc/dev/corpus.rst: minor tweaks

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-04 09:17:11 -05:00
Kefu Chai
6f605dfcb1
Merge pull request #28856 from tchaikov/wip-rbd-nbd-basename
tools/rbd_nbd: use POSIX basename()

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-07-04 22:09:51 +08:00
Ramana Raja
becc3a16fd
Merge pull request #28875 from tchaikov/wip-mgr-volumes-cleanup
mgr/volumes: do not import unused module
2019-07-04 16:15:47 +05:30
Kefu Chai
a1048086b9 doc/dev/corpus.rst: minor tweaks
the executables are located in build/bin after we switched to cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-04 16:13:50 +08:00
Kefu Chai
13da624766 cmake: add cython_cephfs to vstart target
in vstart.sh, if MDS is enabled, `ceph fs volume create` is used to
create cephfs volume. and `fs volume create` command is implemented by
`src/pybind/mgr/volumes/module.py`, which in turn uses `cephfs` python
binding indirectly. so we need to add `cephfs` to `vstart` target to
facilidate the cephfs development using vstart.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-04 15:13:06 +08:00
Kefu Chai
bc13b7941d mgr/volumes: do not import unused module
cephfs is not used in this module, so drop its `import`

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-04 15:01:31 +08:00
Kefu Chai
00a67b07b1
Merge pull request #28753 from tchaikov/wip-doc-conf
doc/rados/configuration: update to be in sync with ConfUtils changes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-07-04 13:23:24 +08:00
Brad Hubbard
1c74e0ae42
Merge pull request #28803 from badone/wip-ceph_ansible_upgrade-ansible-2.8
qa/ceph-ansible: Move to ansible 2.8

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-04 14:51:18 +10:00
Brad Hubbard
b673584778
Merge pull request #28807 from badone/wip-ceph_ansible-cephfs_pools-pg_num-fix
qa/ceph-ansible: Replace pgs with pg_num

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-04 14:50:12 +10:00
Sage Weil
71e5cba00b Merge PR #28867 into master
* refs/pull/28867/head:
	qa/standalone/ceph-helpers: more osd debug

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-07-03 21:27:20 -05:00
Sage Weil
948cd0db43 Merge PR #28865 into master
* refs/pull/28865/head:
	mon/OSDMonitor: fix _lookup_snap to verify the pool matches
	ceph_test_rados_api_*: make failing to clean up namespace non-fatal
	osd: store purged_snaps history under separate object

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-07-03 21:26:57 -05:00
Sage Weil
e2a0ea78c5 Merge PR #28864 into master
* refs/pull/28864/head:
	doc/releases/releases.yaml: set target EOL to Jun 1 for L and M
	doc/releases: update schedule to describe a 12-month cycle

Reviewed-by: Noah Watkins <nwatkins@redhat.com>
2019-07-03 18:33:06 -05:00
David Zafman
fe3b693d0f
Merge pull request #28334 from dzafman/wip-40073
osd: Fix the way that auto repair triggers after regular scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-07-03 15:27:27 -07:00
Sage Weil
0d0759531a qa/standalone/ceph-helpers: more osd debug
debug_ms=1
debug_monc=20

Hunting down http://tracker.ceph.com/issues/40666

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 16:53:00 -05:00
Sage Weil
0a48392ce0 mon/OSDMonitor: fix _lookup_snap to verify the pool matches
We don't want to get false positives from keys for other pools.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 15:59:44 -05:00
Andrew Schoen
0144c55336
Merge pull request #28866 from alfredodeza/wip-rm40665
ceph-volume broken assertion errors after pytest changes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-07-03 15:16:23 -05:00
Alfredo Deza
bfff70d07a ceph-volume api.lvm catch IndexError when parsing dmmapper output
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-03 14:59:54 -04:00
Alfredo Deza
484be7ffb1 ceph-volume tests update to use error.value instead of str(error)
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-03 14:59:53 -04:00
Sage Weil
51c50d0001 ceph_test_rados_api_*: make failing to clean up namespace non-fatal
If we leak snaps, failing to clean up a namespace can happen and won't
get fixed until we scrub purged_snaps.  Make this a non-fatal condition.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 13:30:55 -05:00
Sage Weil
65b7143cd4 osd: store purged_snaps history under separate object
We can't put this in the snapmapper object because filestore does not
allow multiple concurrent omap iterators on the same object.  (This is a
limitation that could be fixed with some read/write locking, but not
without some significant changes to DBObjectMap; since that is old crufty
legacy code let's avoid touching it!)

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 13:29:15 -05:00
Kefu Chai
4fed8e6111
Merge pull request #28788 from hjwsm1989/cleanup-debian
debian: remove dup ceph-fuse line

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-04 00:33:14 +08:00
Sage Weil
7d71489b2e doc/releases/releases.yaml: set target EOL to Jun 1 for L and M
Don't list an actual Luminous EOL yet.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 10:55:03 -05:00
Sage Weil
0bb6556dd3 doc/releases: update schedule to describe a 12-month cycle
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 10:54:39 -05:00
Paul Emmerich
05a59b6d0d debian/control: add python-routes dependency
the dashboard requires python-routes via cherrypy/_cpdispatch.py during runtime
but the cherrypy debian package only recommends it and doesn't depend on it

Fixes: https://tracker.ceph.com/issues/24420
Signed-off-by: Paul Emmerich <paul.emmerich@croit.io>
2019-07-03 17:36:49 +02:00
Sage Weil
cbccab9df5 Merge PR #28785 into master
* refs/pull/28785/head:
	osd/PrimaryLogPG: do_op - do not create head object twice

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-03 08:30:51 -05:00
Sage Weil
9fa27a252c Merge PR #28782 into master
* refs/pull/28782/head:
	common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-03 08:24:29 -05:00
Sage Weil
5e2d69b8d8 Merge PR #28821 into master
* refs/pull/28821/head:
	os/bluestore/bluefs_types: consolidate contiguous extents

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-07-03 08:24:02 -05:00
Sage Weil
1cc7617c8a Merge PR #28330 into master
* refs/pull/28330/head:
	osd: drop osd_lock during scrub
	ceph_test_rados_api_tier_pp: tolerate ENOENT or success from deleted snap
	osd: automatically scrub purged_snaps every deep scrub interval
	osd: move scrub_purged_snaps to helper
	osd/OSDMap: SERVER_OCTOPUS feature bit is now significant
	ceph_test_rados_api_snapshots_pp: drop unnecessary assert
	mon/OSDMonitor: record last_purged_snaps_scrub from beacon to osdmap
	osd: report last_purged_snaps_scrub as part of beacon
	osd: log purged_snaps scrub to cluster log
	osd: record last_purged_snaps_scrub in superblock
	osd/OSDMap: add last_purged_snaps_stamp to osd_xinfo_t
	mon/OSDMonitor: fix bug in try_prune_purged_snaps
	mon/OSDMonitor: record snap removal seq as purged
	mon/OSDMonitor: do not bother reporting gaps in removed_snaps
	osdc/Objecter: don't worry about gap_removed_snaps from map gaps
	mds/SnapServer: make not about pre-octopus compat code
	osd: implement scrub_purged_snaps command
	osd/PrimaryLogPG: always remove the snap we are trimming
	ceph_test_rados_api_snapshots_pp: (partial) test to reproduce stray clones
	osd: sync old purged_snaps on startup after upgrade or osd creation
	osd: record purged_snaps when we store new maps
	mon/OSDMonitor: add messages to get past purged_snaps
	mon/OSDMonitor: record pre-octopus purged snaps with first octopus map
	mon/OSDMonitor: record purged_snaps for each epoch
	mon/OSDMonitor: make_snap_epoch_key -> make_removed_snap_epoch_key
	osd/osd_types: add purged_snaps_last to OSDSuperblock
	osd/osd_types: clean up initial values for OSDSuperblock
	mon/OSDMonitor: make {removed,purged}_snap storage more efficient
	mon/OSDMonitor: move (removed, purged) snap update into a helper
	mon/OSDMonitor: generalize/refactor lookup_*_snap
	mon/OSDMonitor: refactor snap key and value helpers
	mon/OSDMonitor: make_snap_key -> make_removed_snap_key, make_purged_snap_key
	mon/OSDMonitor: fix lookup_purged_snap implementation
	mon/OSDMonitor: lookup_pruned_snap -> lookup_purged_snap
	osd: adjust snapmapper keys on first start as octopus
	osd/SnapMapper: include poolid in snap index
	mon/OSDMonitor: document osd snap metadata format
	osd/SnapMapper: document stored keys and values
	mon/OSDMonitor: use structured binding for prepare_remove_snaps
	mon/OSDMonitor: send MRemoveSnaps back to octopus MDS
	mds/SnapServer: handle MRemoveSnaps acks from mon
	CMakeLists: include 'cephfs' (which includes libcephfs) in 'vstart' target
	mon/PaxosService: add C_ReplyOp
	vnewosd.sh: add script to add a new osd to an existing vstart
	vstart.sh: remove useless auth add for osds
	vstart.sh: wait for mgr volume module to start up
	mon/OSDMonitor: make snap removal handle dups safely
	mon/OSDMonitor: only update removed_snaps when pre-octopus
	ceph_test_rados: stop doing long object names
	ceph_test_rados_api_tier_pp: fix osd version checks
	osd/PrimaryLogPG: use get_ssc_as_of for snapc for flushing clones
	osd/PrimaryLogPG: only maintain SnapSet::snaps for pre-octopus compat
	mon/OSDMonitor: only maintain pg_pool_t::removed_snaps for pre-octopus
	osd/osd_types: mark SnapSet::snaps as legacy
	osd/osd_types: SnapSet::get_ssc_as_of: use clone_snaps
	osd/PrimaryLogPG: change fabrication of promoted clone snaps
	osd/PrimaryLogPG: only filter SnapSet::snaps for flush for pre-octopus compat
	osd/PrimaryLogPG: trim_objects: only filter SnapSet::snaps for pre-octopus
	osd/PrimaryLogPG: make best effort to sanitize clones on copy-from
	mds/SnapServer: int -> int32_t for encoded type
	messages/MRemoveSnaps: int -> int32_t on encoded type
	osd/PrimaryLogPG: find_object_context: trust SnapSet's clone_snaps
	osd/PrimaryLogPG: use osdmap removed_snaps_queue for snap trimming
	mon/OSDMonitor: avoid is_removed_snap()
	osd/PeeringState: drop some mimic conditionals
	osd/PG: drop pre-mimic snap_trimq code
	osd/PeeringState: removed pre-mimic removed snap tracking
	osd: move snap_interval_set_t to osd_types
	mon: drop mon_debug_no_require_mimic
	mon/OSDMonitor: remove pre-mimic snap behavior support
	mon/OSDMonitor: remove support for pre-mimic conversion
	osd/osd_types: remove build_removed_snaps(), maybe_update_removed_snaps()
	osd: remove luminous compat code for removed_snaps

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-03 08:22:34 -05:00
Volker Theile
0951269be6
mgr/dashboard: RGW rest client instances cache eviction (#28480)
mgr/dashboard: RGW rest client instances cache eviction

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-07-03 11:45:29 +02:00
Lenz Grimmer
8c76bd1da9
Merge pull request #28265 from ricardoasmarques/wip-iscsi-logged-in
mgr/dashboard: Display iSCSI "logged in" info

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-07-03 11:34:37 +02:00
Lenz Grimmer
797690c8b4
mgr/dashboard: Pool list shows current r/w byte usage in graph (#28153)
mgr/dashboard: Pool list shows current r/w byte usage in graph

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-07-03 11:26:30 +02:00