Commit Graph

66841 Commits

Author SHA1 Message Date
David Zafman
cde687ba50 test: Don't write to a poolid that this test might not have created
Signed-off-by: David Zafman <dzafman@redhat.com>
2016-12-07 17:27:01 -08:00
Sage Weil
96cf8a1b28 Merge pull request #12354 from umullan/correction
doc: correcting the object name
2016-12-07 18:31:54 -06:00
Sage Weil
6c3d015c68 msg/simple/Pipe: avoid returning 0 on poll timeout
If poll times out it will return 0 (no data to read on socket).  In
165e5abdbf we changed tcp_read_wait from
returning -1 to returning -errno, which means we return 0 instead of -1
in this case.

This makes tcp_read() get into an infinite loop by repeatedly trying to
read from the socket and getting EAGAIN.

Fix by explicitly checking for a 0 return from poll(2) and returning
EAGAIN in that case.

Fixes: http://tracker.ceph.com/issues/18184
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 18:25:55 -06:00
Samuel Just
fe456b80f4 ReplicatedPG::record_write_error: don't leak orig_reply on cancel
The submit_log_entries machinery depends on the destructor for the
functor cleaning up after itself to handle cancelation.  I could have
introduced a local intrusive_ptr and captured that instead, but this is
slightly less magic.

Fixes: http://tracker.ceph.com/issues/18180
Signed-off-by: Samuel Just <sjust@redhat.com>
2016-12-07 14:29:49 -08:00
Sage Weil
cd56396a8a Merge pull request #12347 from gregsfortytwo/wip-18157-fix
ceph-fuse: start up log on parent process before shutdown

Reviewed-by: Sage Weil <sage@redhat.com>
2016-12-07 15:59:57 -05:00
Patrick Donnelly
0a6d76f29b
fsx-mpi: fsx-mpi expects a file name not a dir
The test still fails even after being enabled:

	2016-12-07T18:00:44.337 INFO:teuthology.orchestra.run.mira105:Running: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts -wdir /home/ubuntu/cephtest/gmnt sudo /home/ubuntu/cephtest/fsx-mpi -o 1MB -N 50000 -p 10000 -l 1048576 /home/ubuntu/cephtest/gmnt/test'
	2016-12-07T18:00:44.486 INFO:teuthology.orchestra.run.mira105.stderr:Warning: Permanently added '172.21.8.122' (ECDSA) to the list of known hosts.
	2016-12-07T18:00:44.571 INFO:teuthology.orchestra.run.mira105.stdout:skipping zero size read
	2016-12-07T18:00:44.591 INFO:teuthology.orchestra.run.mira105.stdout:truncating to largest ever: 0x7cccb
	2016-12-07T18:00:44.606 INFO:teuthology.orchestra.run.mira083:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
	2016-12-07T18:00:44.611 INFO:teuthology.orchestra.run.mira100:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
	2016-12-07T18:00:44.614 INFO:teuthology.orchestra.run.mira105:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
	2016-12-07T18:00:44.887 INFO:teuthology.orchestra.run.mira105.stdout:skipping zero size read
	2016-12-07T18:00:44.954 INFO:teuthology.orchestra.run.mira105.stdout:Size error: expected 0xa6f7c stat 0xd4000 seek 0xd5000
	2016-12-07T18:00:44.954 INFO:teuthology.orchestra.run.mira105.stdout:LOG DUMP (2 total operations):
	2016-12-07T18:00:44.954 INFO:teuthology.orchestra.run.mira105.stdout:1(1 mod 256): SKIPPED (no operation)
	2016-12-07T18:00:44.954 INFO:teuthology.orchestra.run.mira105.stdout:2(2 mod 256): WRITE        0x1c748 thru 0xa6f7b    (0x8a834 bytes) HOLE
	2016-12-07T18:00:44.990 INFO:teuthology.orchestra.run.mira105.stdout:Correct content saved for comparison
	2016-12-07T18:00:44.990 INFO:teuthology.orchestra.run.mira105.stdout:(maybe hexdump "/home/ubuntu/cephtest/gmnt/test" vs "/home/ubuntu/cephtest/gmnt/test.fsxgood")
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:===================================================================================
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:=   EXIT CODE: 120
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:=   CLEANING UP REMAINING PROCESSES
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stdout:===================================================================================
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stderr:[proxy:0:0@mira105] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed
	2016-12-07T18:00:45.000 INFO:teuthology.orchestra.run.mira105.stderr:[proxy:0:0@mira105] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status
	2016-12-07T18:00:45.001 INFO:teuthology.orchestra.run.mira105.stderr:[proxy:0:0@mira105] main (./pm/pmiserv/pmip.c:206): demux engine error waiting for event
	2016-12-07T18:00:45.002 INFO:teuthology.orchestra.run.mira105.stderr:[mpiexec@mira105] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting
	2016-12-07T18:00:45.002 INFO:teuthology.orchestra.run.mira105.stderr:[mpiexec@mira105] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion
	2016-12-07T18:00:45.002 INFO:teuthology.orchestra.run.mira105.stderr:[mpiexec@mira105] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:217): launcher returned error waiting for completion
	2016-12-07T18:00:45.002 INFO:teuthology.orchestra.run.mira105.stderr:[mpiexec@mira105] main (./ui/mpich/mpiexec.c:331): process manager error waiting for completion

I am not sure what the cause is. I'm leaving the test disabled for now and merging this PR.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-12-07 14:08:57 -05:00
Patrick Donnelly
c70ec42db1
fsx-mpi: remove chef task
Per Sage's comment: https://github.com/ceph/ceph-qa-suite/pull/284#issuecomment-141463034

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-12-07 14:07:30 -05:00
Yehuda Sadeh
02e7e8395b Merge pull request #977 from ceph/wip-rgw-compression
rgw/verify: use rgw_compression_type = "random"

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-12-07 09:18:42 -08:00
Jeff Layton
18d2499d6c client: drop setuid/setgid bits on ownership change
When we hold exclusive auth caps, then the client is responsible for
handling changes to the mode. Make sure we remove any setuid/setgid
bits on an ownership change.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2016-12-07 12:10:56 -05:00
Jeff Layton
6da7250088 mds: clear setuid/setgid bits on ownership changes
If we get a ownership change, POSIX mandates that you clear the
setuid and setgid bits unless you are "appropriately privileged", in
which case the OS is allowed to leave them intact.

Linux however always clears those bits, regardless of the process
privileges, as that makes it simpler to close some potential races.
Have ceph do the same.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2016-12-07 12:10:56 -05:00
Jeff Layton
3d7fa89803 test: add new testcase for clearing setuid/setgid bits on chown/chgrp
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2016-12-07 12:10:45 -05:00
Jeff Layton
58345df35a test: remove improper casts from SetSize test
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2016-12-07 12:09:09 -05:00
Mykola Golub
9b965f9d8c Merge pull request #12259 from liupan1111/wip-fix-nbd-max-part
rbd-nbd: support partition for rbd-nbd mapped raw block device

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2016-12-07 18:07:05 +02:00
Sage Weil
508e81eac9 os/bluestore: bluestore_sync_submit_transaction = false
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 10:37:28 -05:00
Sage Weil
8fa08e96de upgrade/hammer-jewel-x/stress-split: no xenial
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 10:27:19 -05:00
Kefu Chai
899e650fe4 Merge pull request #12359 from tchaikov/wip-cmake
cmake: detect keyutils if WITH_LIBCEPHFS OR WITH_RBD

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2016-12-07 23:09:33 +08:00
Jason Dillaman
2c70df978d qa/workunits/rbd: removed qemu-iotest case 077
The test case is not stable due to racing console output. This
results in spurious failures.

Fixes: http://tracker.ceph.com/issues/10773
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-12-07 09:59:39 -05:00
Sage Weil
577cf95d59 Merge pull request #1211 from ceph/wip-15753-master
upgrade/hammer-jewel-x suite
2016-12-07 09:54:24 -05:00
Sage Weil
8ee41c6fa2 upgrade/hammer-jewel-x: redo stress-split
Make this easy: write a singel yaml that does the hammer install,
some limited work, then upgardes to jewel.  Copy it from the
parallel suite.  Then, symlink all of the rest from the jewel-x
stress-split suite.

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 09:53:39 -05:00
Sage Weil
836921e07d upgrade/hammer-jewel-x: misc fixes 2016-12-07 09:53:20 -05:00
Sage Weil
c6a0564aea upgrade/hammer-jewel-x: fix upgrade order 2016-12-07 09:53:20 -05:00
Sage Weil
e9b066672d upgrade/hammer-jewel-x: no xenial
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 09:53:20 -05:00
Sage Weil
3ea7610dc2 upgrade/hammer-jewel-x: no mds, no fs tests 2016-12-07 09:53:20 -05:00
Sage Weil
f774097a65 exclude ceph-mgr 2016-12-07 09:53:20 -05:00
Sage Weil
e3c3c67bcb upgrade/hammer-jewel-x/parallel: update
Merger hammer-x (jewel branch) and jewel-x (master branch).

Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 09:53:20 -05:00
Yuri Weinstein
03427961b9 Added new hammer-jewel-x suite
Removed rbd/test_librbd_python.sh for now
Removed all `sudo` from exec lines

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2016-12-07 09:53:20 -05:00
Sage Weil
26220f0608 mon/OSDMonitor: only show interesting flags in health warning
Also add PAUSEREC flag into the list.

Fixes: http://tracker.ceph.com/issues/18175
Signed-off-by: Sage Weil <sage@redhat.com>
2016-12-07 09:50:26 -05:00
Kefu Chai
329846a30b Merge pull request #12361 from dachary/wip-18030-pglog
tests: remove TestPGLog EXPECT_DEATH tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-12-07 22:42:58 +08:00
John Spray
aa0a0e2271 Merge pull request #12208 from ukernel/wip-18047
mds: don't access mdsmap from log submit thread

Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-07 14:15:25 +00:00
John Spray
d6b991a396 Merge pull request #12077 from jcsp/wip-17954
mds: fix dropping events in standby replay

Reviewed-by: Yan, Zheng <zyan@redhat.com>
2016-12-07 14:13:52 +00:00
John Spray
4ed4224391 Merge pull request #12133 from jcsp/wip-16924
mds: don't maintain bloom filters in standby replay

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2016-12-07 14:12:53 +00:00
John Spray
5a5a2b3f4a tasks/cephfs: use kill in test_reconnect_eviction
Previously relied on client being able to unmount
while the MDS was offline, which is not necessarily
so.  Use kill instead.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-12-07 13:40:09 +00:00
John Spray
4006cdebd8 Merge pull request #12360 from wido/docs-cephfs-datascan-workers
doc: Fix for worker arguments to cephfs-data-scan tool

Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-07 12:50:53 +00:00
John Spray
063d61d851 Merge pull request #1300 from batrick/multimds-merge
suites/multimds: sync with fs suites

Reviewed-by: John Spray <john.spray@redhat.com>
2016-12-07 12:19:54 +00:00
Loic Dachary
f1b457ff98 tests: remove TestPGLog EXPECT_DEATH tests
As per Sam Just advice, remove the EXPECT_DEATH tests to avoid
intermittent hang because they do not play well with threads.

Fixes: http://tracker.ceph.com/issues/18030

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-12-07 10:50:43 +01:00
huanwen ren
6a601eba5e test/cephfs: add readlink test
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2016-12-07 17:19:48 +08:00
huanwen ren
1217ef5530 libcephfs: add readlink function in cephfs.pyx
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2016-12-07 17:11:55 +08:00
Kefu Chai
8f4d3ce147 Merge pull request #12327 from iain-buclaw-sociomantic/pybind-validate-args
pybind/rados: Add @requires for all aio methods

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-12-07 16:55:31 +08:00
Wido den Hollander
82c2f840b6
doc: Fix for worker arguments to cephfs-data-scan tool
The docs were incorrect and these arguments should work.

Signed-off-by: Wido den Hollander <wido@42on.com>
2016-12-07 09:49:09 +01:00
Kefu Chai
321691d89a cmake: turn krbd into a static lib
* simplify the link dependencies.
* s/keyutils/${KEYUTILS_LIBRARIES}/

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-12-07 16:09:08 +08:00
Kefu Chai
b805505d25 cmake: find keyutils if rbd or libcephfs is enabled
* find_package(keyutils REQUIRED) if (WITH_LIBCEPHFS OR WITH_RBD)
  prior to this change, we detect keyutils if the building platform is not
  FreeBSD, we should instead check the WITH_* options, and let the
  maintainer to decided what is the best for his/her platform, and error
  out if the building host cannot fulfill the requirement to build the
  asserts.
* build krbd.cc if (WITH_RBD)

Signed-off-by: Kefu Chai <kchai@redhat.com>
2016-12-07 16:09:08 +08:00
xie xingguo
bf3f57afb7 os/bluestore: fix global commit latency
"start" is used to calculate the global bluestore commit latency
and hence shall not be updated at each internal state enter/exit.

Otherwise the l_bluestore_commit_lat counter won't reflect the
real commit latency precisely.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-12-07 15:37:10 +08:00
Uday Mullangi
5af460091c doc: correcting the object name
Signed-off-by: umullangi@walmartlabs.com
2016-12-07 11:47:20 +05:30
Patrick Donnelly
dc648a798a
suites/multimds: add kclient disable override
The libcephfs tests are negatively affected by other mounts. This commit
adds a kclient disable in addition to the ceph-fuse one.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-12-06 22:38:38 -05:00
Patrick Donnelly
9be89e58cf
suites/multimds: sync with fs suites
This commit synchronizes the multimds suite with the fs suite. The
basic/verify sub-suites now do the same tests except with different
cluster layouts (i.e. multiple actives). This is mostly accomplished by
symlinking parts of each sub-suite to its counterpart in the fs suite.

This commit also does a few things of note to the prior multimds suite:

o Turn on directory fragmentation.
o Add several tests from fs/basic/tasks to multimds/basic.
o Remove libcephfs as fs/basic/tasks already contain
  multimds/basic/tasks.

Prior implementation and discussion are in PR#1114: https://github.com/ceph/ceph-qa-suite/pull/1114

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2016-12-06 22:38:31 -05:00
Xie Xingguo
66953e3224 Merge pull request #12350 from chanyoung/struct-OSDOp-typos
osd: fix typos in "struct OSDOp" comments

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2016-12-07 10:04:23 +08:00
John Spray
5f1abf9c31 suites/kcephfs: disable broken python tests
Disabling strays because of:
http://tracker.ceph.com/issues/17193

Disabling test_failover because of:
http://tracker.ceph.com/issues/18161

Signed-off-by: John Spray <john.spray@redhat.com>
2016-12-07 01:03:41 +00:00
Yunchuan Wen
f9fc3548ae osd: mark queued flag for op
queue_op will check the op epoch with current osdmap epoch,
and then, the op will be push into waiting_for_map or mark
queued flag.
but when the op pop from waiting_for_map, take_op_map_waiters
forget to mark queued flag before handle it.

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>
2016-12-07 08:56:14 +08:00
John Spray
d86b67ef9d tasks/cephfs: make test_journal_repair work with more daemons
Previously this assumed it was running with exactly two MDS
daemons.  When there were more, it would fail to execute
"fs reset" because the extra daemons were active in
the map.

Signed-off-by: John Spray <john.spray@redhat.com>
2016-12-07 00:46:12 +00:00
John Spray
98842b7c96 suites/kcephfs: run python tests with 4 MDSs
For the benefit of test_failover

Signed-off-by: John Spray <john.spray@redhat.com>
2016-12-07 00:46:12 +00:00