Commit Graph

89559 Commits

Author SHA1 Message Date
Patrick Donnelly
21c56b7402
Merge PR #23575 into master
* refs/pull/23575/head:
	tools/cephfs-shell:pep8 code clean
	tools/cephfs-shell:resolved removing files using pattern inside sub-directories
	tools/cephfs-shell:changed output representation for the commands cwd and ls
	tools/cephfs-shell:resolved removing directories using pattern inside sub-directories

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-08-17 12:04:42 -07:00
Sage Weil
2b2f0ec6a5 Merge PR #18096 into master
* refs/pull/18096/head:
	os/bluestore: factor VDO into bluestore's 'allocated' value
	os/filestore: estimate omap_allocated
	os/ObjectMap: add get_db() accessor
	osd/OSDMap: include data, omap in 'osd df' output
	os/bluestore: report omap and internal metadata usage
	os/bluestore/BlueFS: drop get_fs_usage(); add get_used()
	kv: add per-prefix space estimation
	osd: decouple statfs update from hb peers, pg count
	osd: drop useless OSDService::update_osd_stat()
	osd/osd_types: separate accounting for data, omap, and metadata
	osd: re-add get_num_pgs()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-08-17 13:56:22 -05:00
Pavani Rajula
ed1b0c9c95 tools/cephfs-shell:pep8 code clean
Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>
2018-08-17 23:37:37 +05:30
Pavani Rajula
05428f4962 tools/cephfs-shell:resolved removing files using pattern inside sub-directories
Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>
2018-08-17 23:37:24 +05:30
Pavani Rajula
d5efa9caae tools/cephfs-shell:changed output representation for the commands cwd and ls
Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>
2018-08-17 23:37:05 +05:30
Pavani Rajula
384700acdf tools/cephfs-shell:resolved removing directories using pattern inside sub-directories
Signed-off-by: Pavani Rajula <rpavani1998@gmail.com>
2018-08-17 23:36:52 +05:30
Patrick Donnelly
b7448f8fe0
mds: simplify beacon init
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-08-17 09:27:02 -07:00
Kefu Chai
00b867fae2
Merge pull request #23623 from tchaikov/wip-inline-constants
messages: define HEAD_VERSION and COMPAT_VERSION inlined

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
2018-08-17 23:07:04 +08:00
Kefu Chai
09d388e97f messages: define HEAD_VERSION and COMPAT_VERSION inlined
otherwise we could have ../lib/libceph-common.so.0: undefined reference
to `MOSDPGNotify::HEAD_VERSION

when the compiler tries to be smart and pass them by reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-17 18:59:29 +08:00
Zhi Zhang
f2e4c9f06a mgr/status: Add standby-replay MDS ceph version
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
2018-08-17 17:59:22 +08:00
Lenz Grimmer
2d7d0d2069
Merge pull request #23419 from tspmelo/fix-26856
mgr/dashboard: Add decorator to skip parameter encoding

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-08-17 11:08:05 +02:00
Lenz Grimmer
39ef941966
Merge pull request #23446 from tspmelo/wip-25075
mgr/dashboard: Use human readable units on the sparkline graphs

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-08-17 11:07:48 +02:00
Lenz Grimmer
ec1c0f89a5
Merge pull request #23388 from ricardoasmarques/wip-fix-auto-logout
mgr/dashboard: Fix redirect to login page on session lost

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-08-17 11:04:31 +02:00
Erwan Velu
d1a807a277 run-make-check: Showing configuration before the build
The actual code is resetting the statistics before doing the actual compilation and prints them after the build.

That is nice to understand how much the cache was used but doesn't help understanding how much it _could_ have been used.

This patch is adding a reporting (-s) when cleaning the statistics so we can estimate :
- the actual number of files in cache
- the actual size of the cache

With this two missing information, its now possible estimate if there is some miss-usage of the cache.

Signed-off-by: Erwan Velu <erwan@redhat.com>
2018-08-17 10:16:58 +02:00
Kefu Chai
6f231ee864 cmake: link against gtest in a better way
* add FindGMock.cmake which allows user to use the libgtest-dev
  shipped by distro
* add GMock::{GMock,Main}, GTest::{GTest,Main} targets to be
  compatible with FindGTest.cmake and FindGMock.cmake, which
  expose the built libraries with properties adhered to
  them. so the consumer of them can import them in a better way.
* update tests to drop the commands like
  set_target_properties(foo PROPERTIES COMPILE_FLAGS ${UNITTEST_CXX_FLAGS}),
  as they are already linked against gmock and gtest.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-17 15:21:27 +08:00
Kefu Chai
303e3c4a6c cmake: fix Finddpdk.cmake
* add dpdk::dpdk if dpdk is built or found, as seastar checks for
  it before adding its component libraries. if user installs libdpdk
  and builds WITH_SEASTAR=ON or WITH_DPDK=ON, cmake fails to configure
  the building system without this fix.
* add dpdk::cflags target for populating the -march=<arch> compile
  option.
* also use pkg-config for finding dpdk.
* link common_async_dpdk against dpdk::dpdk instead

please note, the reason why we can remove the "-march=native" compile
option from Finddpdk.cmake, is that the distro shipped header files
are "generic" in the sense of "-march=<ARCH>", they do not enable
the arch specific intrinsic by default, hence the source files
including them do not need specific compiler options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-17 15:16:22 +08:00
Venky Shankar
f9b8ad5793 mds: access last std::vector element with back()
... rather than accessing it as an array. also convert
to range loops whereever necessary. plus some style
fixes.

Fixes: https://tracker.ceph.com/issues/25113
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2018-08-17 02:45:09 -04:00
Venky Shankar
bae6c9db65 mds: disallow certain file operations to "." and ".." dirents
Also, fixup return value for file operations such as rmdir()
and rename() on these directories.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2018-08-17 02:45:09 -04:00
Nathan Cutler
18dc98e3f0 cmake: do not build DPDK with -march=native
-march=native causes gcc to use opcodes according to whichever CPU happens to
be installed in the build host, which can be different for every build. This
makes it impossible to achieve a reproducible build.

Also, if the build host has a very new CPU, running the resulting binaries on
older CPUs (of the same family, i.e. x86_64) could result in segmentation
fault.

References: http://tracker.ceph.com/issues/24948
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-08-17 14:09:08 +08:00
Kefu Chai
65f6a3c564 cmake: honour CMAKE_REQUIRED_LIBRARIES in CheckIncludeFiles
to silence cmake warning regarding to CMP0075

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-08-17 14:08:57 +08:00
James McClune
9ee2d6b04c
doc: updated infernalis release reference link
Updated Infernalis reference link with reST label.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-08-16 22:37:54 -04:00
James McClune
a3a5036b10
doc: added ref label for infernalis release notes
Added reference label for Infernalis release notes.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-08-16 22:30:52 -04:00
James McClune
288d9713c2
doc: used reST label for ceph-deploy reference
Updated ceph-deploy man page link with ref label. Fixes
404 error. 

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-08-16 22:17:47 -04:00
James McClune
9fd5dc9cc2
doc: added reference label for ceph-deploy page
Added reST label for ceph-deploy page.

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-08-16 22:05:20 -04:00
Yan, Zheng
a7076fae36 mds: adjust export size after renaming directory into freezing subtree
restart exporting the subtree if rename makes export size too large

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:59:54 +08:00
James McClune
d772bd3b5f
doc: updated cross referencing locations link
Updated cross referencing locations link for doc/start/documenting-ceph.rst

Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-08-16 21:58:34 -04:00
Yan, Zheng
2788b132a8 mds: clear scatterlock's flushed when removing dirty
If mds imports scatterlock's parent inode later, the leftover flushed
flag may prevent Locker::eval_gather from calling scatter_writebehind
and result incorrect dirstat/neststat.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:44:23 +08:00
Yan, Zheng
da7fc1ad2c mds: export subtree part by part
When exporting a large subtree, migrator may only exports some portions
of the subtree. This patch makes migrator continue to export the rest
partions when previous operations finish.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:44:23 +08:00
Yan, Zheng
07ef150d3c mds: change default mds_max_export_size to 20M
1G seems too large. mds may speed long time on processing
imported/expored subtree.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:44:23 +08:00
Yan, Zheng
dd9ad34416 mds: limit total size of exporting subtrees
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:44:23 +08:00
Yan, Zheng
c163777ffe mds: optimize the way how max export size is enforced
The old way is checking export size after subtree gets frozen. It may
freeze a large subtree, but only exports small portion of the subtree.

The new way is choosing some subdirs according to the max export size,
then freeze these subdirs.

http://tracker.ceph.com/issues/25131
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:42:58 +08:00
Yan, Zheng
6538dae2c1 mds: make CInode::get_dirfrags container agnostic
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:40:16 +08:00
Yan, Zheng
efff5c69f1 mds: cleanup MutationImpl::More initialization
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:40:16 +08:00
Yan, Zheng
0f057feac3 mds: don't modify filepath when printing
filepath::depth() may call parse_bits(), which modifies mutable member
'bits'. dump_ops_in_flight asok command prints filepath without holding
mds_lock. So multiple threads may call parse_bits() at the same time.

Fixes: http://tracker.ceph.com/issues/26894
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:26:55 +08:00
Yan, Zheng
1013374863 mds: hold slave request refernce when dumping MDRequestImpl
dump_ops_in_flight asok command dumps MDRequestImpl without holding
mds_lock. MDS may free MDRequestImpl::slave_request in the middle of
dumping MDRequestImpl.

Fixes: http://tracker.ceph.com/issues/26894
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-08-17 09:26:45 +08:00
xie xingguo
12af0ed71b osd/PG: calc_min_last_complete_ondisk should return nothing
As we now no more use it to calc pg_trim_to.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-08-17 08:23:53 +08:00
xie xingguo
a9c4b7f425 osd/PrimaryLogPG: optimize recover order
By definition, async_recovery_targets should have a lower
priority than those coming from the acting set.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-08-17 08:23:53 +08:00
Patrick Donnelly
c25ef2a7ee
Merge PR #23468 into master
* refs/pull/23468/head:
	Allow to unittest shell using transcript
2018-08-16 16:34:23 -07:00
Patrick Donnelly
769a93f4ec
Merge PR #22555 into master
* refs/pull/22555/head:
	msg: define MFoo::create helper
	msg: add msgref cast method
	msg: cleanup factory/ref definition in messages
	mds: use message factory to avoid leaks
	msg: add factory method to correctly construct smart_ptr
	mds: remove dead MDS-MDS forwarding code
	mds: manage Message lifetime with intrusive_ptr
	common: add templated Context factory
	msg: add const version of get_payload
	msg: use queue of messages for dispatch
	msg: dispatch intrusive_ptr Messages
	msg: use deque for dispatcher pointers

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-08-16 16:17:42 -07:00
Nathan Cutler
cb1531dbd7 doc/releases/schedule: note Jewel EOL date
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-08-16 23:32:06 +02:00
Dan Mick
b221877462 debian/rules: fix ceph-mgr .pyc files left behind
Add second dh_python2 call with the "private" dir /usr/lib/ceph/mgr

Fixes: http://tracker.ceph.com/issues/26883
Signed-off-by: Dan Mick <dan.mick@redhat.com>
2018-08-16 14:29:15 -07:00
Patrick Donnelly
acd02343f4
Merge PR #23551 into master
* refs/pull/23551/head:
	qa: use recent kernel to kernel build testing

Reviewed-by: Sage Weil <sage@redhat.com>
2018-08-16 09:57:27 -07:00
Jason Dillaman
a5302e18ff
Merge pull request #23608 from tchaikov/wip-librbd-test
test/librbd: fix -Wsign-compare warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-08-16 12:43:04 -04:00
Patrick Donnelly
f0fe0936e6
qa: use recent kernel to kernel build testing
Fixes: http://tracker.ceph.com/issues/24679

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-08-16 09:16:39 -07:00
Patrick Donnelly
f0800a49ed
Merge PR #23597 into master
* refs/pull/23597/head:
	qa: add cephfs-shell skeleton test case

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-08-16 09:14:11 -07:00
Patrick Donnelly
0b5df6e00a
qa: add cephfs-shell skeleton test case
Right now just tests that "help" works.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-08-16 09:13:36 -07:00
Matt Benjamin
d12c72cf4c
Merge pull request #21286 from joke-lee/wip-auth-uri-encode-decode
rgw:  should recode  canonical_uri when caculate s3 v4 auth
2018-08-16 11:58:14 -04:00
Jesse Williamson
1a44357f5c package: modify ceph.spec.in to support libradosstriper conditional compilation
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2018-08-16 08:21:42 -07:00
Jesse Williamson
6e4db7b0a4 tools: make rados.cc compatible with conditionally-compiled striper
Signed-off-by: Jesse Williamson <jwilliamson@suse.de>
2018-08-16 08:21:37 -07:00
Yuri Weinstein
d3385fc6bb
Merge pull request #23533 from wjwithagen/wjw-fix-random_shuffle
osd: make random shuffle comply with C++17

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-08-16 08:09:21 -07:00