Commit Graph

85188 Commits

Author SHA1 Message Date
Kefu Chai
df9cd23605
Merge pull request #21112 from guzhongyan/fix-vlog
src: fix various log messages

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-10 18:51:12 +08:00
Kefu Chai
fa8a435e88
Merge pull request #21028 from rzarzynski/wip-osd-get_rw_locks-norefcnt
osd: avoid unnecessary ref-counting across PrimaryLogPG::get_rw_locks.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-10 18:49:01 +08:00
Kefu Chai
2c3cbc170e
Merge pull request #21235 from xiexingguo/wip-fix-obj-start
osd/osd_types: fix ideal lower bound object-id of pg

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-10 18:47:58 +08:00
Yan, Zheng
a7db43b5f7 tools/cephfs: instantiate MDSUtility after global_init()
cephfs-foo-tool from current master crashes immediately it starts.

Introduced by commit 6972273d53 "global: output usage on -h, --help,
or no args before contacting mons"

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/23624
2018-04-10 16:15:16 +08:00
Kefu Chai
1d8c043809 ceph: print output of "status" as string not as bytes
in python3, the stuff read from wire is represented as "bytes", but
the output of "ceph -w" is supposed to be consumed by human. so decode
it as utf-8.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-04-10 13:31:13 +08:00
Josh Durgin
ee03866b9f
Merge pull request #21313 from ceph/wip-pg-clean-timeout
qa/tasks: Add default timeout for wait for pg clean task

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-09 22:27:16 -07:00
Kefu Chai
3abe420365
Merge pull request #21284 from tchaikov/wip-cmake-libstdcxx
cmake: define HAVE_STDLIB_MAP_SPLICING for both libstdc++ and libc++

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-04-10 13:21:31 +08:00
Kefu Chai
7fdfa3c465
Merge pull request #21290 from rjfd/wip-dashboard-fix-rest-controller
mgr/dashboard: restcontroller: fix detection of id args in element requests

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-10 13:19:08 +08:00
Patrick Donnelly
359f5cccd6
Merge PR #21039 into master
* refs/pull/21039/head:
	qa/workunits/fs: test for cephfs rstats
	mds: make rstat.rctime follow inodes' ctime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-09 21:33:15 -07:00
Patrick Donnelly
098c6d370f
Merge PR #21040 into master
* refs/pull/21040/head:
	mds: mark damaged if sessions' preallocated inos don't match inotable

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-09 21:33:10 -07:00
Patrick Donnelly
acc6f2eddb
Merge PR #21132 into master
* refs/pull/21132/head:
	client: remove getgroups_cb
	client: remove _getgrouplist
	client: have init_gids just set alloced_gids to true
	client: remove init_groups
	fuse: handle errors appropriately when getting group list

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-04-09 21:33:06 -07:00
Patrick Donnelly
a3718d5978
Merge PR #21144 into master
* refs/pull/21144/head:
	mds: fix unhealth heartbeat during rejoin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-09 21:33:00 -07:00
Patrick Donnelly
dfca9664d4
Merge PR #21167 into master
* refs/pull/21167/head:
	mds: make stopping mds export strays and subtree at the same time
	mds: handle client requests while mds is stopping
	mds: check projected linkage before migrating stray dentry

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-09 21:32:56 -07:00
Patrick Donnelly
f818599836
Merge PR #21191 into master
* refs/pull/21191/head:
	client: fix request send_to_auth was never really used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-09 21:32:51 -07:00
Patrick Donnelly
ed657dee8c
Merge PR #21255 into master
* refs/pull/21255/head:
	mds: automatically create snaprealm for old format quota
	client: check if mds create snaprealm when setting quota
	mds: remove unused MDCache::snaprealm_create()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-09 21:32:47 -07:00
Brad Hubbard
c0dac8ecd2 qa/standalone/mon/misc.sh: Add osdmap-prune tests
Fixes: http://tracker.ceph.com/issues/23621

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-04-10 14:26:53 +10:00
Sage Weil
b0fe734c21 qa/tasks/mgr/test_failover: fix for id -> name rename in mgr metadata
This was changed by 428236de8b

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-09 21:48:40 -05:00
Brad Hubbard
3152797eaf msg/async/AsyncConnection: Fix FPE in process_connection
Check ms_inject_socket_failures is valid before we use it.

Fixes: http://tracker.ceph.com/issues/23618

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-04-10 12:05:11 +10:00
Yan, Zheng
fc0d340876 mds: limit run time of load balancer
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:48 +08:00
Yan, Zheng
65a35e2a76 mds: add list to track recently used sub-directories
MDBalancer::find_exports() can be more efficient with LRU list.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:48 +08:00
Yan, Zheng
376145d919 mds: calculate other mds' last_epoch_under locally
No need to get this information from MHeartbeat

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:48 +08:00
Yan, Zheng
40759407bf mds: cleanup mds_load map access/update
using map::insert to access map items looks strang

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:48 +08:00
Yan, Zheng
eb095453b9 mds: check export pin when choosing dirfrags for exporting
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:48 +08:00
Yan, Zheng
04831d463c mds: optimize MDBalancer::find_exports()
stop at subtree bounds; make freezing/frozen check more efficient.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
b9fb3a4cff mds: avoid creating unnecessary subtrees during load balance
When choosing dirfrags for exporting, consider subtrees imported
from other MDS first. It's because that exporting dirfrags chosen
by MDBalancer::find_export() will create new subtrees.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
cd6f77c6a0 mds: mds: optimize MDBalancer::try_rebalance()
1. change import_pop_map to multimap because subtrees may have the same
   popularity.
2. avoid calculating subtrees' popularity multiple times

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
15810b4360 mds: don't try exporting subdir if dirfrag is already being exported
It doesn't make sense to call MDBalancer::find_exports() for a dirfrag
that is already being exported.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
bb6d381f30 mds: don't try exporting dirfrags under mds's own mdsdir
Migrator will refuse to export them. Besides, stray directory can
never be subtree root.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
08116b71f6 mds: cleanup MDBalancer::try_rebalance()
no logic changes

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
896491268f mds: always pass current time to MDBalancer::{hit_inode,hit_dir}
There were codes that pass mdr->get_mds_stamp() to {hit_inode,hit_dir}.
The stamp can be older than 'last_decay' in decay counters,  which can
cause DecayCounter::decay() to malfunction.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:47 +08:00
Yan, Zheng
29e19c1d7d mds: remove unused MDBalancer::export_empties()
MDCache::trim() already does the same job

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:46 +08:00
Yan, Zheng
831c5d91cc mds: don't decay load counters when printing load
Changing load counters while printing load results confusing output.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:46 +08:00
Yan, Zheng
6538464bdb mds: adjust pop_nested when importing subtree
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:46 +08:00
Yan, Zheng
6b126a6919 mds: adjust subtree popularity when import fails
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:46 +08:00
Yan, Zheng
bedd98f878 mds: adjust subtree popularity after rename
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-04-10 09:19:44 +08:00
Yan, Zheng
e9689c1ff7 mds: fix request rate calculation
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: http://tracker.ceph.com/issues/21745
2018-04-10 09:19:29 +08:00
Josh Durgin
447a107d5f
Merge pull request #21295 from badone/wip-large-omap-scrub-big-random-loser-2
qa/suites/rados: Disable scrub backoff

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-09 17:44:33 -07:00
Vasu Kulkarni
7881a19d92 qa/tasks: wait_for_clean is called after ceph task as well after osd's are up,
the default timeout is none in that case, there are cases where it can hang  forever
due to error cases, since this dumps quite a lot of info the logs grow in GB's, with
default timeout of 1200 we can avoid such huge logs and fail sooner. Any tests needing
higher timeout can pass the required value.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-04-09 17:24:42 -07:00
Alfredo Deza
b1920d6e73 ceph-volume lvm.activate error if no bluestore OSDs are found
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 20:03:37 -04:00
Alfredo Deza
d6524297ac ceph-volume tests/functional add 'activate all' for xenial/filestore/dmcrypt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 20:03:37 -04:00
Alfredo Deza
452081e57e ceph-volume tests/functional add 'activate all' for xenial/bluestore/dmcrypt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 20:03:37 -04:00
Alfredo Deza
5513f4de4e ceph-volume tests/functional add 'activate all' for centos7/filestore/dmcrypt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 20:03:36 -04:00
Alfredo Deza
16421c9058 ceph-volume tests/functional add 'activate all' for centos7/bluestore/dmcrypt
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 20:03:36 -04:00
Alfredo Deza
ef09f3996b ceph-volume tests/functional add 'activate all' in filestore playbook
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 20:03:36 -04:00
Jianpeng Ma
59f69643c7 doc: remove unused config: "osd op threads"
This option already removed.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-04-10 05:39:59 +08:00
Alfredo Deza
797518a546 ceph-volume tests/functional add 'activate all' in bluestore playbook
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2018-04-09 17:25:07 -04:00
Yuri Weinstein
94c162a62a
Merge pull request #21309 from ceph/wip-yuriw-crontab
qa/tests - minor clean ups and made perf-suite run 3 times, so we can…

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-04-09 12:37:57 -07:00
Sage Weil
bfff8ec839
Merge pull request #21282 from liewegas/wip-pgpool-enc
osd/osd_types: fix pg_pool_t encoding for hammer

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Drymov <idryomov@redhat.com>
2018-04-09 14:16:49 -05:00
Yuri Weinstein
9a2005d664 qa/tests - minor clean ups and made perf-suite run 3 times, so we can have more data points
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-04-09 09:17:10 -07:00
Patrick Donnelly
3ab655b55d
Merge PR #21263 into master
* refs/pull/21263/head:
	doc: outline upgrade procedure for mds cluster

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2018-04-09 08:59:11 -07:00