Commit Graph

2779 Commits

Author SHA1 Message Date
Patrick Donnelly
d929dae49b
Merge PR #17657 into master
* refs/remotes/upstream/pull/17657/head:
	mds: optimize MDCache::rejoin_scour_survivor_replicas()
	mds: fix MDSCacheObject::clear_replica_map
	mds: support limiting cache by memory
	common: refactor of lru
	mds: resolve unsigned coercion compiler warning
	common: use safer uint64_t for list size
	common: add bytes2str pretty print function
	mds: check if waiting is allocated before use
	mds: go back to compact_map for replicas
	mds: use mempool for cache objects
	mds: cleanup replica_map access
	common: add alloc_ptr smart pointer
	common: add warning on base class use of mempool
	common: use atomic uin64_t for counter

Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-09-13 20:08:51 -07:00
Casey Bodley
72b08e8f96 qa/smoke: add rgw crypto config for s3tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-09-13 11:22:28 -04:00
Xie Xingguo
0e604b112e Merge pull request #17515 from xiexingguo/wip-data-digest
osd/PrimaryLogPG: do not set data digest for bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-13 18:31:10 +08:00
xie xingguo
afcb617dc9 osd/PrimaryLogPG: do not generate data digest for BlueStore by default
BlueStore enables CRC by default, so this is a dup and gains
no more benefits.

Turn this off by default, which is good for performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2017-09-13 12:17:16 +08:00
Patrick Donnelly
06c94de584
mds: support limiting cache by memory
This introduces two config parameters:

    mds_cache_memory_limit: Sets the soft maximum of the cache to the given
    byte count. (Like mds_cache_size, this doesn't actually limit the maximum
    size of the cache. It just dictates the steady-state size.)

    mds_cache_reservation: This replaces mds_health_cache_threshold everywhere
    except the Beacon heartbeat sent to the mons. The idea here is to specify a
    reservation of memory (5% by default) for operations and the MDS tries to
    always maintain that reservation. So, the MDS will recall caps from clients
    when it begins dipping into its reservation of memory.

mds_cache_size still limits the cache by Inode count but is now by-default 0
(i.e. unlimited). The new preferred way of specifying cache limits is by memory
size. The default is 1GB.

Fixes: http://tracker.ceph.com/issues/20594
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1464976

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-12 20:02:41 -07:00
yonghengdexin735
fc5ac9ea69 common:fix error word
Signed-off-by: yonghengdexin735 <zhang.zezhu@zte.com.cn>
2017-09-13 10:22:08 +08:00
Patrick Donnelly
f48113f50f
Merge PR #17679 into master
* refs/remotes/upstream/pull/17679/head:
	qa: get asok path from ceph.conf
	qa: use config_path property instead of literal

Reviewed-by: John Spray <john.spray@redhat.com>
2017-09-12 15:56:32 -07:00
Patrick Donnelly
bc3a00af0f
qa: get asok path from ceph.conf
The asok path was recently moved to /tmp to resolve unix path too long errors.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-12 15:52:54 -07:00
Patrick Donnelly
f95798b3ad
qa: use config_path property instead of literal
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-12 15:50:33 -07:00
Tamilarasi Muthamizhan
4d8971e1c8 Merge pull request #17650 from ceph/wip-ceph-deploy-fix
qa/tests: Fix get_system_type failure due to invalid remote name
2017-09-12 15:07:24 -07:00
David Zafman
44f51024cc Merge pull request #17538 from dzafman/wip-21272
Add export and remove ceph-objectstore-tool command option

Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-11 20:12:27 -07:00
Mykola Golub
b196d38783 Merge pull request #17621 from idryomov/wip-krbd-xfstests-blockdev
qa/suites/krbd: enable xfstests blockdev tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-09-11 14:09:32 +03:00
Vasu Kulkarni
9fe58d5ed6 Fix get_system_type failure due to invalid remote name
recent changes caused the remote name to be invalid, fix the
arg passed to get_system_type

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-10 12:42:16 -07:00
Sage Weil
14e783c0b9 Merge pull request #17571 from jcsp/wip-21260
mon: fix dropping mgr metadata for active mgr (#21260)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-09-10 14:20:30 -05:00
Mykola Golub
0bafa7e8d4 Merge pull request #17559 from dillaman/wip-21181
qa/workunits/rbd: relax greps to support upgrade formatting change

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-09-09 11:23:57 +03:00
David Zafman
3bb20f6d75 ceph-objectstore-tool: Make pg removal require --force
Add new export-remove to combine the 2 operations

Fixes: http://tracker.ceph.com/issues/21272

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-08 17:56:05 -07:00
David Zafman
49ca1fff7f ceph-objectstore-tool: Better messages for bad --journal-path
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-08 17:50:46 -07:00
David Zafman
3ac219df2d test: Fix ceph-objectstore-tool test for standalone and latest code
vstart.sh now defaults to bluestore, so specify filestore
Set environment for run-standalone.sh and cmake build
Create td/cot_dir as test directory
Crush output format change
Change dir into test directory
Give a little time after pool creation
Check for core files as ceph-helpers.sh does

Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-08 16:53:53 -07:00
David Zafman
495c32fd31 test: Move ceph-objectstore-tool test to standalone
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-09-08 16:53:30 -07:00
Tamilarasi Muthamizhan
ec423100dc Merge pull request #17096 from ceph/wip-ansbile-fixes
[qa/tests]: misc ceph-ansible fixes and udpate
2017-09-08 14:47:12 -07:00
Yuri Weinstein
0c2a139ee6 Merge pull request #17513 from Liuchang0812/wip-max-avail-in-df
mon: incorrect MAX AVAIL in "ceph df"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-09-08 13:41:07 -07:00
Patrick Donnelly
1b7aae6cdc
Merge PR #17533 into master
* refs/remotes/upstream/pull/17533/head:
	qa: log ceph-fuse kill/cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-08 11:23:23 -07:00
Patrick Donnelly
52c79aecdd
Merge PR #17466 into master
* refs/remotes/upstream/pull/17466/head:
	qa: whitelist expected MDS_READ_ONLY for kcephfs test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-08 11:23:19 -07:00
Vasu Kulkarni
b1fd615e9f Stop the mgr on node which is not client, this will ensure
the client.0's mgr is active mgr for workunit to work.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-08 10:00:49 -07:00
John Spray
28cb4bb2e6 qa/tasks/mgr: regression test for 21260
(aka http://tracker.ceph.com/issues/21260)

Fixes: http://tracker.ceph.com/issues/21260
Signed-off-by: John Spray <john.spray@redhat.com>
2017-09-08 15:45:40 +02:00
Ilya Dryomov
131c1189d6 qa/suites/krbd: enable xfstests blockdev tests
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2017-09-08 15:45:34 +02:00
Vasu Kulkarni
e5b5a1b8c8 Test ceph-mgr RESTful api
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 17:10:04 -07:00
Vasu Kulkarni
e963bfd6ca Add additional variables as required by ceph-ansible for upstream
installation.

possible revert if those are made default
issue: https://github.com/ceph/ceph-ansible/issues/1834

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 16:54:51 -07:00
Tamilarasi Muthamizhan
594b736228 Merge pull request #17536 from ceph/wip-cd-rbd-pool
qa/tests: fix rbd pool creation for systemd tests
2017-09-07 15:13:32 -07:00
Vasu Kulkarni
5c43c5972d Add workaround for http://tracker.ceph.com/issues/20950
mgr bootstrap key differs on disk, rewrite the new key
using auth get.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:57 -07:00
Vasu Kulkarni
71ea4987e0 Add kraken to luminous upgrade case
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:57 -07:00
Vasu Kulkarni
526d4aef1c Add kernel tar/untar and systemd task
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:57 -07:00
Vasu Kulkarni
2116f76a1f Restart services after upgrade
This seems to be an issue and should be fixed in our systemd scripts,
need to discuss more on this.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:57 -07:00
Vasu Kulkarni
8a2b9a0809 since the roles are mapped inside ceph-deploy, store the roles that
are mapped and use the new mapped role for upgrades during later
stage.

eg: mon.a is mapped to mon.mira002 during install, store this mapping
and durig upgrade map it back to appropriate name to find the hostname
with that role

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:57 -07:00
Vasu Kulkarni
e7a0a4f87f Add doc string for missing ceph-deploy-branch
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:57 -07:00
Vasu Kulkarni
407ce2613d use mon.a as ceph-admin node, the previous get_first_mon returns
a sorted(mons)[0] as first mon.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:56 -07:00
Vasu Kulkarni
f3ec7cabf6 use elif instead of else to handle cases for jewel install
jewel needs neither filestore or bluestore as an option, so provide none
when running with jewel branch.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:56 -07:00
Vasu Kulkarni
9c85c76a90 Jewel to luminous upgrade scenario using ceph-deploy
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:56 -07:00
Vasu Kulkarni
9a73127c16 Add upgrade functionality using ceph-deploy
This is to test for customer like upgrade scenarios and to find
any issues that may be related to systemd, packaging etc

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:56 -07:00
Vasu Kulkarni
b409c3d163 Fix old flake8 issues
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:56 -07:00
Vasu Kulkarni
a9be0c387e Add option to skip mgr install for old release
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-07 12:22:56 -07:00
Sage Weil
1006b62fe6 Merge pull request #17322 from liewegas/wip-post-luminous-mon
mon: remove pre-luminous compat cruft (2 of many)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2017-09-07 13:47:43 -05:00
Sage Weil
e715533feb qa/workunits/*: require-osd-release mimic, not luminous
Signed-off-by: Sage Weil <sage@redhat.com>
2017-09-07 09:18:56 -04:00
Jason Dillaman
273c84578b qa/workunits/rbd: relax greps to support upgrade formatting change
Fixes: http://tracker.ceph.com/issues/21181
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-09-07 08:56:16 -04:00
Jason Dillaman
b0f1545d3b Merge pull request #16998 from trociny/wip-20860
rbd-mirror: update asok hook name on image rename

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-09-07 08:34:42 -04:00
Mykola Golub
f618508624 Merge pull request #17504 from dillaman/wip-21251
qa/suites/rbd: miscellaneous test fixes

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-09-07 11:34:01 +03:00
Vasu Kulkarni
f12127751d Add mgr create to fix single node test to reach healthy state
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-06 18:13:39 -07:00
Vasu Kulkarni
49f95b3d8b create the rbd pool right after install
rbd pool should exist for many rbd tests to work properly, create
the pool right after install is successful.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-09-06 18:07:54 -07:00
Sage Weil
1e272575ad Merge pull request #17505 from liewegas/wip-20910
qa/objectstore/bluestore*: less debug output

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-09-06 16:02:01 -05:00
Patrick Donnelly
b4f962a486
qa: log ceph-fuse kill/cleanup
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-09-06 13:40:11 -07:00