Commit Graph

99453 Commits

Author SHA1 Message Date
Dominique Leuenberger
b48ea142de
Packaging: Drop systemd BuildRequires in case of building for SUSE
There is already pkgconfig(systemd) present, which, in SUSE's packaging,
translates to the same package (systemd.pc is shipped as part of the main
systemd package). Not explicitly mentioning 'systemd' as package name allows
the openSUSE Build Service though to find shortcuts by using the
bootstrap packages, i.e. systemd-mini (ABI/API complete).

Signed-off-by: Dominique Leuenberger <dimstar@opensuse.org>
2019-07-05 09:15:20 +02:00
Sage Weil
1bdc707a90 Merge PR #28877 into master
* refs/pull/28877/head:
	doc/dev/corpus.rst: minor tweaks

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-04 09:17:11 -05:00
Kefu Chai
6f605dfcb1
Merge pull request #28856 from tchaikov/wip-rbd-nbd-basename
tools/rbd_nbd: use POSIX basename()

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-07-04 22:09:51 +08:00
Ramana Raja
becc3a16fd
Merge pull request #28875 from tchaikov/wip-mgr-volumes-cleanup
mgr/volumes: do not import unused module
2019-07-04 16:15:47 +05:30
Kefu Chai
a1048086b9 doc/dev/corpus.rst: minor tweaks
the executables are located in build/bin after we switched to cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-04 16:13:50 +08:00
Kefu Chai
bc13b7941d mgr/volumes: do not import unused module
cephfs is not used in this module, so drop its `import`

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-04 15:01:31 +08:00
Kefu Chai
00a67b07b1
Merge pull request #28753 from tchaikov/wip-doc-conf
doc/rados/configuration: update to be in sync with ConfUtils changes

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-07-04 13:23:24 +08:00
Brad Hubbard
1c74e0ae42
Merge pull request #28803 from badone/wip-ceph_ansible_upgrade-ansible-2.8
qa/ceph-ansible: Move to ansible 2.8

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-04 14:51:18 +10:00
Brad Hubbard
b673584778
Merge pull request #28807 from badone/wip-ceph_ansible-cephfs_pools-pg_num-fix
qa/ceph-ansible: Replace pgs with pg_num

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-04 14:50:12 +10:00
Sage Weil
71e5cba00b Merge PR #28867 into master
* refs/pull/28867/head:
	qa/standalone/ceph-helpers: more osd debug

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-07-03 21:27:20 -05:00
Sage Weil
948cd0db43 Merge PR #28865 into master
* refs/pull/28865/head:
	mon/OSDMonitor: fix _lookup_snap to verify the pool matches
	ceph_test_rados_api_*: make failing to clean up namespace non-fatal
	osd: store purged_snaps history under separate object

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-07-03 21:26:57 -05:00
Sage Weil
e2a0ea78c5 Merge PR #28864 into master
* refs/pull/28864/head:
	doc/releases/releases.yaml: set target EOL to Jun 1 for L and M
	doc/releases: update schedule to describe a 12-month cycle

Reviewed-by: Noah Watkins <nwatkins@redhat.com>
2019-07-03 18:33:06 -05:00
David Zafman
fe3b693d0f
Merge pull request #28334 from dzafman/wip-40073
osd: Fix the way that auto repair triggers after regular scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-07-03 15:27:27 -07:00
Sage Weil
0d0759531a qa/standalone/ceph-helpers: more osd debug
debug_ms=1
debug_monc=20

Hunting down http://tracker.ceph.com/issues/40666

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 16:53:00 -05:00
Sage Weil
0a48392ce0 mon/OSDMonitor: fix _lookup_snap to verify the pool matches
We don't want to get false positives from keys for other pools.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 15:59:44 -05:00
Andrew Schoen
0144c55336
Merge pull request #28866 from alfredodeza/wip-rm40665
ceph-volume broken assertion errors after pytest changes

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-07-03 15:16:23 -05:00
Alfredo Deza
bfff70d07a ceph-volume api.lvm catch IndexError when parsing dmmapper output
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-03 14:59:54 -04:00
Alfredo Deza
484be7ffb1 ceph-volume tests update to use error.value instead of str(error)
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-07-03 14:59:53 -04:00
Sage Weil
51c50d0001 ceph_test_rados_api_*: make failing to clean up namespace non-fatal
If we leak snaps, failing to clean up a namespace can happen and won't
get fixed until we scrub purged_snaps.  Make this a non-fatal condition.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 13:30:55 -05:00
Sage Weil
65b7143cd4 osd: store purged_snaps history under separate object
We can't put this in the snapmapper object because filestore does not
allow multiple concurrent omap iterators on the same object.  (This is a
limitation that could be fixed with some read/write locking, but not
without some significant changes to DBObjectMap; since that is old crufty
legacy code let's avoid touching it!)

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 13:29:15 -05:00
Kefu Chai
4fed8e6111
Merge pull request #28788 from hjwsm1989/cleanup-debian
debian: remove dup ceph-fuse line

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-04 00:33:14 +08:00
Sage Weil
7d71489b2e doc/releases/releases.yaml: set target EOL to Jun 1 for L and M
Don't list an actual Luminous EOL yet.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 10:55:03 -05:00
Sage Weil
0bb6556dd3 doc/releases: update schedule to describe a 12-month cycle
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-03 10:54:39 -05:00
Sage Weil
cbccab9df5 Merge PR #28785 into master
* refs/pull/28785/head:
	osd/PrimaryLogPG: do_op - do not create head object twice

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-03 08:30:51 -05:00
Sage Weil
9fa27a252c Merge PR #28782 into master
* refs/pull/28782/head:
	common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-03 08:24:29 -05:00
Sage Weil
5e2d69b8d8 Merge PR #28821 into master
* refs/pull/28821/head:
	os/bluestore/bluefs_types: consolidate contiguous extents

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-07-03 08:24:02 -05:00
Sage Weil
1cc7617c8a Merge PR #28330 into master
* refs/pull/28330/head:
	osd: drop osd_lock during scrub
	ceph_test_rados_api_tier_pp: tolerate ENOENT or success from deleted snap
	osd: automatically scrub purged_snaps every deep scrub interval
	osd: move scrub_purged_snaps to helper
	osd/OSDMap: SERVER_OCTOPUS feature bit is now significant
	ceph_test_rados_api_snapshots_pp: drop unnecessary assert
	mon/OSDMonitor: record last_purged_snaps_scrub from beacon to osdmap
	osd: report last_purged_snaps_scrub as part of beacon
	osd: log purged_snaps scrub to cluster log
	osd: record last_purged_snaps_scrub in superblock
	osd/OSDMap: add last_purged_snaps_stamp to osd_xinfo_t
	mon/OSDMonitor: fix bug in try_prune_purged_snaps
	mon/OSDMonitor: record snap removal seq as purged
	mon/OSDMonitor: do not bother reporting gaps in removed_snaps
	osdc/Objecter: don't worry about gap_removed_snaps from map gaps
	mds/SnapServer: make not about pre-octopus compat code
	osd: implement scrub_purged_snaps command
	osd/PrimaryLogPG: always remove the snap we are trimming
	ceph_test_rados_api_snapshots_pp: (partial) test to reproduce stray clones
	osd: sync old purged_snaps on startup after upgrade or osd creation
	osd: record purged_snaps when we store new maps
	mon/OSDMonitor: add messages to get past purged_snaps
	mon/OSDMonitor: record pre-octopus purged snaps with first octopus map
	mon/OSDMonitor: record purged_snaps for each epoch
	mon/OSDMonitor: make_snap_epoch_key -> make_removed_snap_epoch_key
	osd/osd_types: add purged_snaps_last to OSDSuperblock
	osd/osd_types: clean up initial values for OSDSuperblock
	mon/OSDMonitor: make {removed,purged}_snap storage more efficient
	mon/OSDMonitor: move (removed, purged) snap update into a helper
	mon/OSDMonitor: generalize/refactor lookup_*_snap
	mon/OSDMonitor: refactor snap key and value helpers
	mon/OSDMonitor: make_snap_key -> make_removed_snap_key, make_purged_snap_key
	mon/OSDMonitor: fix lookup_purged_snap implementation
	mon/OSDMonitor: lookup_pruned_snap -> lookup_purged_snap
	osd: adjust snapmapper keys on first start as octopus
	osd/SnapMapper: include poolid in snap index
	mon/OSDMonitor: document osd snap metadata format
	osd/SnapMapper: document stored keys and values
	mon/OSDMonitor: use structured binding for prepare_remove_snaps
	mon/OSDMonitor: send MRemoveSnaps back to octopus MDS
	mds/SnapServer: handle MRemoveSnaps acks from mon
	CMakeLists: include 'cephfs' (which includes libcephfs) in 'vstart' target
	mon/PaxosService: add C_ReplyOp
	vnewosd.sh: add script to add a new osd to an existing vstart
	vstart.sh: remove useless auth add for osds
	vstart.sh: wait for mgr volume module to start up
	mon/OSDMonitor: make snap removal handle dups safely
	mon/OSDMonitor: only update removed_snaps when pre-octopus
	ceph_test_rados: stop doing long object names
	ceph_test_rados_api_tier_pp: fix osd version checks
	osd/PrimaryLogPG: use get_ssc_as_of for snapc for flushing clones
	osd/PrimaryLogPG: only maintain SnapSet::snaps for pre-octopus compat
	mon/OSDMonitor: only maintain pg_pool_t::removed_snaps for pre-octopus
	osd/osd_types: mark SnapSet::snaps as legacy
	osd/osd_types: SnapSet::get_ssc_as_of: use clone_snaps
	osd/PrimaryLogPG: change fabrication of promoted clone snaps
	osd/PrimaryLogPG: only filter SnapSet::snaps for flush for pre-octopus compat
	osd/PrimaryLogPG: trim_objects: only filter SnapSet::snaps for pre-octopus
	osd/PrimaryLogPG: make best effort to sanitize clones on copy-from
	mds/SnapServer: int -> int32_t for encoded type
	messages/MRemoveSnaps: int -> int32_t on encoded type
	osd/PrimaryLogPG: find_object_context: trust SnapSet's clone_snaps
	osd/PrimaryLogPG: use osdmap removed_snaps_queue for snap trimming
	mon/OSDMonitor: avoid is_removed_snap()
	osd/PeeringState: drop some mimic conditionals
	osd/PG: drop pre-mimic snap_trimq code
	osd/PeeringState: removed pre-mimic removed snap tracking
	osd: move snap_interval_set_t to osd_types
	mon: drop mon_debug_no_require_mimic
	mon/OSDMonitor: remove pre-mimic snap behavior support
	mon/OSDMonitor: remove support for pre-mimic conversion
	osd/osd_types: remove build_removed_snaps(), maybe_update_removed_snaps()
	osd: remove luminous compat code for removed_snaps

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-03 08:22:34 -05:00
Volker Theile
0951269be6
mgr/dashboard: RGW rest client instances cache eviction (#28480)
mgr/dashboard: RGW rest client instances cache eviction

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-07-03 11:45:29 +02:00
Lenz Grimmer
8c76bd1da9
Merge pull request #28265 from ricardoasmarques/wip-iscsi-logged-in
mgr/dashboard: Display iSCSI "logged in" info

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-07-03 11:34:37 +02:00
Lenz Grimmer
797690c8b4
mgr/dashboard: Pool list shows current r/w byte usage in graph (#28153)
mgr/dashboard: Pool list shows current r/w byte usage in graph

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-07-03 11:26:30 +02:00
Lenz Grimmer
8a6aec4617
mgr/dashboard: Allow viewing and setting pool quotas (#27945)
mgr/dashboard: Allow viewing and setting pool quotas

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-07-03 11:20:25 +02:00
Lenz Grimmer
eb0358dcbc
mgr/dashboard: fix MDS charts are stacked in Filesystems page (#28341)
mgr/dashboard: fix MDS charts are stacked in Filesystems page

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-07-03 11:17:50 +02:00
Lenz Grimmer
e92808989e
mgr/dashboard: Interlock fast-diff and object-map RBD imag… (#28076)
mgr/dashboard: Interlock `fast-diff` and `object-map` RBD image features

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-07-03 11:13:20 +02:00
Ricardo Marques
8176ad2c73
Merge pull request #27997 from LenzGr/master-documentation
doc: Added dashboard features, improved wording

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-03 09:23:52 +01:00
Volker Theile
7be30f8b2f
mgr/dashboard: fix HACKING.rst rendering (#28841)
mgr/dashboard: fix HACKING.rst rendering

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-07-03 10:10:40 +02:00
Kefu Chai
2066c88ddb
Merge pull request #28542 from cyx1231st/wip-seastar-msgr-perf
test/crimson: improved perf_crimson_msgr with timer and sampled lat

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-03 15:41:44 +08:00
Kefu Chai
f33119e699 tools/rbd_nbd: use POSIX basename()
* glibc offers two variants of basename(). one modifies the content of
  `path`, the other does not. to be standard compliant, and to fix
  the FTBFS with musl-libc, we need to use the POSIX variant.
* #include <libgen.h> for basename(3), the POSIX compliant one.

see
http://pubs.opengroup.org/onlinepubs/009695399/functions/basename.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-03 15:13:17 +08:00
Andrew Schoen
e9900a11ec
Merge pull request #28836 from alfredodeza/wip-toxsleep
ceph-volume tests add a sleep in tox for slow OSDs after booting

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2019-07-02 13:51:56 -05:00
Sage Weil
b17850a665 osd: drop osd_lock during scrub
We do not want to hold osd_lock while calling apply_transaction or else
we may cause a deadlock due to a completion event that needs osd_lock.

Also, we don't need to hold the lock here, so don't.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 10:51:57 -05:00
Patrick Nawracay
d0fe5ce52a mgr/dashboard: Add backwards compatibility to interlock of fast-diff and object-map
Fixes: http://tracker.ceph.com/issues/39451

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-07-02 16:56:17 +02:00
Patrick Nawracay
dfeba0b139 mgr/dashboard: Interlock fast-diff and object-map
Fixes: http://tracker.ceph.com/issues/39451

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-07-02 16:56:17 +02:00
Tatjana Dehler
3ec401960b mgr/dashboard: fix document rendering
Fixes: https://tracker.ceph.com/issues/40523
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-07-02 16:48:56 +02:00
Sage Weil
adacc20046 ceph_test_rados_api_tier_pp: tolerate ENOENT or success from deleted snap
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
5b0ed6ff9e osd: automatically scrub purged_snaps every deep scrub interval
With randomization.

We do this from tick() for simplicity.  It is a rare event, will take 10s
of seconds at most, and nothing else particularly time-sensitive is
happening from tick().

Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
647cfb4603 osd: move scrub_purged_snaps to helper
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
fc2a96638d osd/OSDMap: SERVER_OCTOPUS feature bit is now significant
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
0d10e63d8d ceph_test_rados_api_snapshots_pp: drop unnecessary assert
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
85ddc1a034 mon/OSDMonitor: record last_purged_snaps_scrub from beacon to osdmap
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
4f4dedb8d6 osd: report last_purged_snaps_scrub as part of beacon
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00
Sage Weil
ab475cda08 osd: log purged_snaps scrub to cluster log
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-02 08:37:51 -05:00