Commit Graph

100249 Commits

Author SHA1 Message Date
Jason Dillaman
2e85381665 pybind/rbd: new OperationCanceled exception
This allows the error to be directly caught instead of attempting
to parse the OSError.errno

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-07-25 08:58:36 -04:00
Ricardo Dias
f57cdf46ef
monmaptool: added --addv option to usage description
Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-07-25 13:48:16 +01:00
Jason Dillaman
6dc8249c7b pybind/rbd: flatten, remove, trash_remove, migration progress callback
This callback can be used to track progress and also to attempt to cancel
the operation while it's in-progress by returning a negative error code
from the callback.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-07-25 08:25:03 -04:00
Rafael Quintero
30def4146e mgr/dashboard: Write E2E test to verify that Configuration has editing functionality
Fixes: https://tracker.ceph.com/issues/40803

Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>
2019-07-25 07:25:32 -04:00
alfonsomthd
55744ef51a mgr/dashboard: RGW bucket creation when no placement target received
Fixes: https://tracker.ceph.com/issues/40567
Signed-off-by: alfonsomthd <almartin@redhat.com>
2019-07-25 11:42:57 +02:00
Xiao Guodong
648730c602 cephfs: fix a memory leak
use a smart pointer instead of using 'new' to resolve a memory leak .

Fixes: https://tracker.ceph.com/issues/40936
Signed-off-by: XiaoGuoDong2019 <xiaogd@inspur.com>
2019-07-25 17:41:02 +08:00
songweibin
0c706687e5 src/test: s/strcpy/strncpy
fix klocwork issues:
function 'strcpy' does not check buffer boundaries but outputs to buffer 'watcher.addr' of fixed size (256)

Signed-off-by: songweibin <song.weibin@zte.com.cn>
2019-07-25 15:21:43 +08:00
alfonsomthd
6a9095f2c6 mgr/dashboard: run e2e tests against prod build (jenkins job)
Signed-off-by: alfonsomthd <almartin@redhat.com>
2019-07-25 08:40:49 +02:00
Rishabh Dave
a83ea849d7 qa/cephfs: make run_shell() accept args as str too
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:50 +05:30
Rishabh Dave
07688b96e1 qa/vstart_runner.py: ignores when source and destination are same
put_file() in vstart_runner.py should ignore the error when source and
destination paths supplied are same, since source and destination for
put_file() method in teuthology are on different machines, they
represent different locations.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:50 +05:30
Rishabh Dave
d94a42dc1b test_cephfs_shell: add a test for cd with arguments
Add a method to test that the cd command changes the current working
directory to the path passed as an argument to it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:50 +05:30
Rishabh Dave
dee9ec1650 test_cephfs_shell: add a test for cd with no arguments
Add a method to test that the cd command changes the current working
directory to root when no argument is passed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:50 +05:30
Rishabh Dave
70f22f69b5 test_cephfs_shell: group test methods
Move tests into different classes created on the basis of a shell
functionality within the testsuite. This arrangement allows running
tests related to only one functionality at a time.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:50 +05:30
Rishabh Dave
2a98e9cb31 test_cephfs_shell: add a new method to return script output
Add a new method that runs the given CephFS shell script on the given
mount and returns the output with whitespace removed.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:28 +05:30
Rishabh Dave
fef0180c93 test_cephfs_shell: allow running CephFS shell script
Add a method to test_cephfs_shell.py that allows running CephFS shell
script using option "-b" of cephfs-shell.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:51:11 +05:30
Rishabh Dave
6e3f5c4412 cephfs-shell: cd with no args shouldn't print an error message
Instead the current working directory should change to root when no
argument is passed to cd.

Fixes: http://tracker.ceph.com/issues/40476
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-25 10:36:38 +05:30
Jos Collin
35208ebbb5
doc: fix youtube channel url
Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-07-25 10:04:47 +05:30
Jos Collin
c5c5f031f7
doc: fix urls in posix.rst
Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-07-25 10:03:12 +05:30
Jos Collin
9db12b5fe8
doc: Update mount CephFS index
Updated based on the discussion:
https://github.com/ceph/ceph/pull/28900#discussion_r301812933

Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-07-25 07:09:26 +05:30
Kefu Chai
a29700e5e7
Merge pull request #29296 from tchaikov/wip-remove-script/dep-report.sh
script: remove dep-report.sh

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-25 09:36:48 +08:00
Kefu Chai
104c88cfea mgr/PyFormatter.h: add necessary #include
this header uses `std::string`, `std::string_view` and
`std::stringstream`, so it should include the corresponding headers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-25 09:00:28 +08:00
Kefu Chai
3409937003 mgr: check for unicode passed to "set_health_checks()"
in python2, in addition to `str`, a string could also be unicode
string. there is chance that user passes a unicode string to
`set_health_checks()`, so let's check for unicode string also.

in python3, all strings are `unicode` string, and `PyString_Check()` is
defined as an alias of `PyUnicode_Check()`, so we are fine with python3.

the wrapper method of `PyString_ToString()` returns a pair of converted
string and a bool. if it fails to detect a string, an empty string and
false are returned.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-25 09:00:28 +08:00
Kefu Chai
49ae4cf3b3 script: remove dep-report.sh
dep-report.sh depends on .Po and .Plo files under .deps/, which is
created by automake. but we've switched to CMake. so we cannot use
these depfile or temporary depfiles anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-25 08:12:37 +08:00
Sage Weil
0c0093e450 Merge PR #29034 into master
* refs/pull/29034/head:
	doc/mgr/crash: document missing commands, options
	qa/suites/rados/singleton/all/test-crash: whitelist RECENT_CRASH
	qa/suites/rados/mgr/tasks/insights: whitelist RECENT_CRASH
	qa/tasks/mgr/test_insights: crash module now rejects bad crash reports
	mgr/telemetry: fix remote into crash do_ls()
	mgr/crash: don't make these methods static
	mgr/BaseMgrModule: handle unicode health detail strings
	mgr/crash: verify timestamp is valid
	qa/suites/mgr: whitelist RECENT_CRASH
	mgr/crash: remove unused var
	mgr/crash: remove unused import 'six'
	qa/workunits/rados/test_crash: health check
	mgr/crash: improve validation on post
	mgr/crash: automatically prune old crashes after a year
	mgr/crash: raise RECENT_CRASH warning for recent (new) crashes
	mgr/crash: add 'crash ls-new'
	mgr/crash: add option and serve infra
	mgr/crash: keep copy of crashes in memory
	mgr/pg_autoscaler: adjust style to match built-in tables
	mgr/crash: make 'crash ls' a nice table with a NEW column
	mgr/crash: nicely format 'crash info' output
	mgr/crash: add 'crash archive <id>', 'crash archive-all' commands

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-07-24 17:20:21 -05:00
Matt Benjamin
9271226341
Merge pull request #26160 from linuxbox2/wip-rgw-lc-header
rgw: project and return lc expiration from GET/HEAD and PUT ops
2019-07-24 16:49:38 -04:00
Yuri Weinstein
c3df207003 qa/tests: added rgw into upgrade sequence to improve coverage - splits
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2019-07-24 13:25:32 -07:00
Sage Weil
1ea7570bb1 doc/mgr/crash: document missing commands, options
Signed-off-by: Sage Weil <sage@redhat.com>
2019-07-24 12:57:18 -05:00
Yuri Weinstein
2a733cd4ea
Merge pull request #29234 from yuriw/wip-yuriw-upgrade-rwg-master
qa/tests: added rgw into upgrade sequence to improve coverage
2019-07-24 09:00:49 -07:00
Jason Dillaman
ed6a047c71
Merge pull request #29263 from trociny/wip-rbd-external_callback
librbd: don't use complete_external_callback if ImageCtx destroyed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-07-24 11:11:22 -04:00
Sage Weil
066f93b188 Merge PR #29180 into master
* refs/pull/29180/head:
	mon/FSCommand: set pg_num_min via 'fs new', not in mgr/volumes
	mon/FSCommands: set pg_autoscale_factor on 'fs new', not via mgr/volumes
	mon: set recovery_priority=5 on fs metadata pool

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-07-24 09:31:49 -05:00
Sage Weil
fe45db9007 Merge PR #27575 into master
* refs/pull/27575/head:
	osd: avoid prep_object_replica_pushes() on clone object when head missing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-07-24 08:54:59 -05:00
Nathan Cutler
610e9f750e
Merge pull request #29261 from jan--f/add-auth-to-ghub-api-calls
script/ceph-backport-sh: add access_token parameter to all ghub api c…

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-07-24 15:44:29 +02:00
Sage Weil
99d699a3d2 Merge PR #29124 into master
* refs/pull/29124/head:
	test/old/test*buffers.cc: remove tests against bufferlist
	test/old/test_*seek_read: remove this test
	test/old/test{crush,bucket}: remove stale tests
	test/old/testmpi.cc: remove this test
	test/old/testxattr.cc: remove this test
	test/old/testtree.cc: remove test for BinaryTree

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-24 08:37:22 -05:00
Jason Dillaman
361e51f3fb librbd: abort an image removal if block objects cannot be removed
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-07-24 09:31:30 -04:00
Jason Dillaman
026df758ae librbd: allow ProgressCtx::update_progress to cancel maintenance ops
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-07-24 09:31:30 -04:00
Ricardo Dias
cf31e24066
mgr/dashboard: qa: fix RBD test when matching error strings
This commit fixes a problem caused by changes to the error
descriptions in the RBD python bindings introduced in:
a66b06cc764ba66cdabfe2856e9f1bcd738f0772

Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-07-24 14:24:18 +01:00
Jan Fajerski
c31be430f8 script/ceph-backport-sh: add access_token parameter to all ghub api calls
Otherwise one can get a "API rate limit exceeded for $YOUR_IP." message
when working from company offices and such.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-07-24 14:35:37 +02:00
Mykola Golub
634634fd8f librbd: don't use complete_external_callback if ImageCtx destroyed
complete_external_callback (which requires ImageCtx) is used to
prevent concurrent callbacks, and not needed in this particular case.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-07-24 13:35:01 +01:00
Kefu Chai
cfb69b81e3
Merge pull request #29148 from tchaikov/wip-cmake-python2
cmake: use python2 by default

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-07-24 20:14:01 +08:00
Mark Nelson
1d1edb1943 os/BlueStore: Split Cache into Onode/Buffer caches
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2019-07-24 08:10:00 -04:00
Varsha Rao
368a4e481d mds: Reorganize class members in Capability header
Fixes: https://tracker.ceph.com/issues/40866
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-07-24 07:42:02 -04:00
Jan Fajerski
14f9b8a133
Merge pull request #28448 from shyukri/wip-25030
ceph-volume: util: Use proper param substition
2019-07-24 13:36:27 +02:00
Kefu Chai
af6c487758
Merge pull request #29073 from dillaman/wip-40790
mon/mgr: add 'rbd_support' to list of always-on mgr modules

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-07-24 14:18:50 +08:00
Kefu Chai
310ccd9f9f qa/tasks/ceph_manager.py: always use self.logger
in fbd4836d, a regression is introduced:

self.log("failed to read erasure_code_profile. %s was likely removed",
pool)

because `self.log` is actually a lambda which just do

self.logger.info(x)

in this change

* `Thrasher.log()` is added for three reasons:
 - in PEP-8,

> Always use a def statement instead of an assignment statement that
> binds a lambda expression directly to an identifier
so a better way is to define a method using `def`
 - and i think it helps with the readability
* `logger` parameter is now mandatory now in the constructor of
  `Thrasher` class. because the instance of this class is only created
by `qa/tasks/thrashosds.py`, like:

thrash_proc = ceph_manager.Thrasher(
        cluster_manager,
        config,
        logger=log.getChild('thrasher')
        )

and `log.getChild()` does not return `None`, so there is no need to
handle that case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-07-24 10:28:00 +08:00
Kefu Chai
1a379bc202
Merge pull request #28765 from myoungwon/fix-chunk-scrub-dedup-tool
src/tools/ceph-dedup-tool: Fix chunk scrub

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-24 09:59:58 +08:00
Kefu Chai
511db86246
Merge pull request #28832 from majianpeng/osd-only-empty-do-wakeup
osd/OSD: only wake up empty pqueue.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-24 09:58:32 +08:00
Kefu Chai
10af76ab1b
Merge pull request #28778 from myoungwon/wip-chunk-flush-op
src/osd: add tier-flush op

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-24 09:56:47 +08:00
Kefu Chai
361b7febcc
Merge pull request #28983 from myoungwon/wip-dedup-io-test
src/test, qa/suites/rados/thrash: add dedup test

Reviewed-by: Sage Weil <sage@redhat.com>
2019-07-24 09:54:54 +08:00
Kefu Chai
4c2ffc8f19
Merge pull request #29053 from dmick/master
ceph-crash: use open(..,'rb') to read bytes for Python3

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-24 09:53:51 +08:00
Kefu Chai
0869d95000
Merge pull request #28054 from iotcg/starlingx
pybind: add verbose error message

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-07-24 09:46:44 +08:00