Commit Graph

72294 Commits

Author SHA1 Message Date
Casey Bodley
16ab3a08a3 qa/rgw: add multisite suite
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-18 13:37:35 -04:00
Casey Bodley
4722d1d920 qa/rgw: add rgw_multisite_tests task to run tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
Casey Bodley
b6d86be2c5 qa/rgw: add rgw_multisite task based on rgw_multi
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
Casey Bodley
a86ce77155 qa/rgw: add symlink to qa/tasks/rgw_multi
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
Casey Bodley
746c630999 qa/rgw: move startup polling logic to util/rgw.py
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
Casey Bodley
76e147614f qa/rgw: fixes for cluster name on cleanup
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
Casey Bodley
4c59d343c3 qa/rgw: move compression type out of ceph.conf
this makes the 'compression type' setting global to all gateways, and
makes the setting visible to other tasks in ctx.rgw.compression_type

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
Casey Bodley
d67018cb31 test/rgw: add import for StringIO
also removes unnecessary tuple parens

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-05-17 14:48:55 -04:00
John Spray
618b9b3f3f Merge pull request #15127 from tchaikov/wip-vstart-mgr-debug-ms
vstart: "debug_ms=1" for mgr by default

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-17 12:18:15 +01:00
Kefu Chai
fab6fc15e1 Merge pull request #15103 from drunkard/master
doc: minor fixes in radosgw/

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-17 17:00:56 +08:00
Kefu Chai
ae8eee2fde Merge pull request #15080 from hikycool/master
doc: update the usage of 'ceph-deploy purge'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-17 16:54:42 +08:00
Kefu Chai
a6daa84d1a vstart: "debug_ms=1" for mgr by default
Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-17 10:31:46 +08:00
Yuri Weinstein
26df056e34 Merge pull request #15087 from liewegas/wip-thrash-ec
qa/suites/rados: fix ec thrashing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-16 13:20:34 -07:00
Yuri Weinstein
07ac91d4b9 Merge pull request #15072 from liewegas/wip-crush-swap-bucket
mon,crush: add 'osd crush swap-bucket' command

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-16 13:19:59 -07:00
Yuri Weinstein
9187156b55 Merge pull request #15069 from tchaikov/wip-inttypes-cleanup
int_types.h: remove hacks to workaround old systems

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2017-05-16 13:18:23 -07:00
Yuri Weinstein
98827f4490 Merge pull request #15057 from liewegas/wip-osd-choose-cleanup
osd/PG: drop pre-firefly compat_mode for choose_*_acting

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
2017-05-16 13:17:37 -07:00
Yuri Weinstein
a49d9c5637 Merge pull request #15042 from badone/wip-redundant-headers-3
common: Remove redundant includes

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-16 13:16:43 -07:00
Yuri Weinstein
0e03291cf6 Merge pull request #14989 from liewegas/wip-19877
osd: fix past_intervals base case by adding epoch_pool_created to pg_history_t

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-16 13:13:36 -07:00
Jason Dillaman
38ba53d076 Merge pull request #15113 from trociny/wip-unittest_librbd-cleanup
test/librbd: unit tests cleanup

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-05-16 14:05:38 -04:00
Mykola Golub
dbde7af146 Merge pull request #15093 from dillaman/wip-19929
librbd: reacquire lock should update lock owner client id

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
2017-05-16 19:40:38 +03:00
Casey Bodley
978f1d3abb Merge pull request #10690 from ceph/wip-rgw-polymorphic-errors
rgw: polymorphic error codes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-16 11:54:27 -04:00
John Spray
60f904615f Merge pull request #15096 from jcsp/wip-journalrepair-test
qa: simplify TestJournalRepair

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-05-16 16:11:57 +01:00
Sage Weil
225bc60092 Merge pull request #15110 from tchaikov/wip-build-using-cmake
doc: use do_cmake.sh instead of `cmake ..`

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-16 09:20:50 -05:00
Kefu Chai
e7d1d877dd Merge pull request #15023 from tchaikov/wip-cmdparse-constness
cmdparse: more constness

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-05-16 22:19:40 +08:00
Sage Weil
035c398487 Merge pull request #15112 from jcsp/wip-19940
ceph-create-keys: update client.admin if it already exists

Reviewed-by: Sage Weil <sage@redhat.com>
2017-05-16 08:43:18 -05:00
Matt Benjamin
fc286d6115 Merge pull request #15097 from linuxbox2/wip-rgwfile-write3
rgw_file: v3: fix write-timer action
2017-05-16 09:07:09 -04:00
Mykola Golub
c0c644333b test/librbd: make unit tests pass when cache is disabled
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2017-05-16 14:39:41 +02:00
Mykola Golub
c065cbdb20 test/librbd: TestMockImageRemoveRequest gmock warnings
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
2017-05-16 14:39:37 +02:00
Drunkard Zhang
63855e381e doc: add hyperlink on ACL settings
The link listed at bottom but never used, I think it's missed by author.

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>
2017-05-16 20:32:29 +08:00
John Spray
82332b7a3b ceph-create-keys: update client.admin if it already exists
This means that anyone who de-privileges their client.admin
user will find that it gets resets with full privileges again.
This is necessary because we cannot distinguish between an old
key and a deliberately de-privileged key, and it's okay because
this is the admin key, it's meant to have access to everything.

Fixes: http://tracker.ceph.com/issues/19940
Signed-off-by: John Spray <john.spray@redhat.com>
2017-05-16 13:28:48 +01:00
Drunkard Zhang
0f811b3d5d doc: typo fixes in doc/radosgw/multisite
Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>
2017-05-16 20:20:49 +08:00
Kefu Chai
ba6e5e8d68 doc: use do_cmake.sh instead of cmake ..
user could be confused at seeing errors like:

src/CMakeLists.txt:644 (add_subdirectory): The source directory /src/lua
does not contain a CMakeLists.txt file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-05-16 18:54:41 +08:00
John Spray
3155335c4c Merge pull request #15094 from fullerdj/wip-djf-19893
qa/cephfs: Fix for test_data_scan

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-16 10:51:55 +01:00
John Spray
71e3a85368 Merge pull request #14938 from michaelsevilla/f19362
mds: add perf counters for file system operations

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-16 10:45:34 +01:00
John Spray
5efad43ec8 Merge pull request #15018 from ukernel/wip-19890
pybind: fix open flags calculation

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-16 10:15:03 +01:00
John Spray
ea4a165f17 Merge pull request #14769 from ukernel/wip-19755
osdc/Filer: truncate large file party by party

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-16 10:13:58 +01:00
Drunkard Zhang
e6af68c32d doc: change reference to internal link
Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>
2017-05-16 16:14:36 +08:00
Pritha Srivastava
da4c026f4a rgw: Searching for error codes and dumping output based on protocol flags.
The previous commit uses dynamic allocation to search for s3/ swift specific
error codes and dump the results. In order to avoid dynamic allocation,
we are making use of protocol flags to achieve the above.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2017-05-16 11:03:48 +05:30
Marcus Watts
b640ded886 polymorphic error codes.
make set_req_state_err a method of req_state

get rid of params.perr; it's never used

rgw - polymorphic error object.

polymorphic error object: this way sts, which has additional error fields,
and its own xml format for how errors should be produced, can overload
the base rgw_err class.

STL fixup: use real mappings for error mappings, not a linear search.
insubstantiate error mappings exactly once in src/rgw/rgw_common.cc, instead
of multiple copies from every source file that includes src/rgw/rgw_http_errors.h.

remove dump_errno(s, err) - it was never used, and the code looked bogus.

move rgw_http_error_to_errno from header to rgw_rest_client.cc,
the only place it's used.
move rgw_http_status_code and http_codes to rgw_rest.cc,
the only place it's used.

base class (rgw_err) implements s3 errors; swift & sts errors use
the base class and overlay their own additional error codes to
the ones recognized by the base class.

For a while, I was trying to just have a "generic" abort_early() that
used dump_more.  Now, I have an op aware form that works the old way.
I would like the http routines to known less about "op" structures,
which is why I'm making this distinction.

Splitting up generic and s3 flavors of some things:
rgw_rest_init() - split into s3 and generic forms.
end_header: s3 vs generic forms: generic form uses dump_more.
abort_early: s3 vs generic forms: generic form uses dump_more, error_content.

And fixing some mistakes from the merge:
Fixup: should not have deleted dump_errno() local storage forms.
Fixup:RGWRESTFlusher  - take a dump_more to make it generic.
Fixup: need RGWRESTOp::send_response()

Move swift_err up; use it in bulkdelete_respond
I Think I don't need s->prot_flags when using swift_err.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
2017-05-16 10:05:56 +05:30
Matt Benjamin
ce6ecb553b rgw_file: v3: fix write-timer action
For now, unify with v4 write-on-close path, by calling
RGWFileHandle::close() on write-timer expire, since it will
call write_finish() as a side-effect.

Fixes: http://tracker.ceph.com/issues/19932

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2017-05-15 17:37:26 -04:00
John Spray
6f1bf0dd9b Merge pull request #15052 from batrick/i19734
mds: use debug_mds for most subsys

Reviewed-by: John Spray <john.spray@redhat.com>
2017-05-15 22:31:27 +01:00
John Spray
2350555fe5 qa: simplify TestJournalRepair
This was sending lots of metadata ops to MDSs to persuade
them to migrate some subtrees, but that was flaky.  Use
the shiny new rank pinning functionality instead.

Signed-off-by: John Spray <john.spray@redhat.com>
2017-05-15 17:27:07 -04:00
Douglas Fuller
7f659e104d qa/cephfs: Fix for test_data_scan
Don't assume that test_data_scan will be run on exactly 2 MDS nodes.

Fixes: http://tracker.ceph.com/issues/19893
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-05-15 16:01:02 -04:00
Sage Weil
880dde8e47 Merge pull request #15091 from ceph/luminous
merge v12.0.2 release tag
2017-05-15 12:47:42 -05:00
Michael Sevilla
6de3e18414 mds: add perf counters for file system operations
Shows the types of operations that clients are doing by adding counters to the
spot where the MDS responds to clients. This avoids counting operations that
were restarted while waiting for locks or for metadata to load.  Does not
support per-client activity. This is the first step towards comprehensive
performance tracking in the file system.

Signed-off-by: Michael Sevilla <mikesevilla3@gmail.com>
2017-05-15 10:43:21 -07:00
Kefu Chai
6db74cf102 Merge pull request #15079 from ztczll/master
mailmap: ztczll affiliation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-05-16 00:25:01 +08:00
Jason Dillaman
21ce5e1670 librbd: reacquire lock should update lock owner client id
Fixes: http://tracker.ceph.com/issues/19929
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-05-15 12:12:12 -04:00
Sage Weil
80038f50b2 qa/suites/rados/threas-erasure-code-*: fix luminous facet
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-15 10:22:25 -04:00
Sage Weil
8fdbeb5bb9 qa/suites/rados/thrash-erasure-code-overwrites: fix
Signed-off-by: Sage Weil <sage@redhat.com>
2017-05-15 10:21:58 -04:00
Casey Bodley
bd58423b97 Merge pull request #15081 from tchaikov/wip-fio-readme
fio: update README.md so only the fio ceph engine is built

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-05-15 10:03:24 -04:00