Commit Graph

49940 Commits

Author SHA1 Message Date
Javier M. Mellid
809189b4c8 vstart.sh: avoid race condition starting rgw via vstart.sh
In slow environments running vstart.sh hits a race condition setting up
the rgw server. Creating the S3 and Swift users before booting rgw
avoids the issue.

Fixes: #14829

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
2016-02-20 17:33:21 +00:00
Josh Durgin
b89ba9986d Merge pull request #7627 from dillaman/wip-librbd-read-replay
librbd: block read requests until journal replayed

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-19 16:51:07 -08:00
Josh Durgin
ff82fd165a Merge pull request #7718 from dillaman/wip-librbd-mirror-peer
librbd: update of mirror pool mode and mirror peer handling

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-19 16:50:44 -08:00
Josh Durgin
44d7819cba Merge pull request #7698 from dillaman/wip-librbd-replay-flush
librbd: differentiate journal replay flush vs shut down

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-19 16:50:19 -08:00
Yehuda Sadeh
c8858019be Merge pull request #7720 from ceph/wip-rgw-aws4
rgw support for aws authentication v4 (Javier M. Mellid)
2016-02-19 16:14:06 -08:00
Yehuda Sadeh
552715995c Merge remote-tracking branch 'jmunhoz/wip-aws4' into wip-aws4
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Conflicts:
	src/rgw/rgw_auth_s3.h
	src/rgw/rgw_client_io.cc
	src/rgw/rgw_client_io.h
	src/rgw/rgw_common.h
	src/rgw/rgw_main.cc
	src/rgw/rgw_op.cc
	src/rgw/rgw_op.h
	src/rgw/rgw_rest.cc
	src/rgw/rgw_rest_metadata.cc
	src/rgw/rgw_rest_s3.cc
2016-02-19 14:34:13 -08:00
Yehuda Sadeh
a646103ede Merge pull request #7719 from ceph/wip-rgw-keystone-v3
rgw keystone v3 (Mark Barnes, Radoslaw Zarzynski)

support for keystone v3

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-19 13:42:30 -08:00
Jason Dillaman
700c3dc24b librbd: update mirror peer data structure
Replace cluster uuid with mirror peer uuid and support an
optional pool id within a peer (when using internal APIs
for testing).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-19 16:12:12 -05:00
Jason Dillaman
0ec038abca librbd: support image and pool-level mirroring modes
If pool-level mirroring is enabled, all journaled images within
the pool are eligible for mirroring.  If image-level mirroring
is enabled, mirroring needs to be explicitly enabled on each
image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-19 16:12:12 -05:00
Radoslaw Zarzynski
55cbc60d46 rgw: Keystone token parsing doesn't need to know API version.
This patch targets an issue with S3Extension in Keystone:
requested tokens always conform to Keystone API v2 - regardless
of the version used to make the request.

Previous implementation of KeystoneToken::parse() as well
as KeystoneToken::decode_json() had to know the API version
explicitly. Thus, they might be affected in the future by
changes in S3-compatibility middleware of Keystone.

Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:36 -08:00
Radoslaw Zarzynski
abbd91293d rgw: S3 always must use token format of Keystone v2.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:36 -08:00
Radoslaw Zarzynski
2f190fdcb8 rgw: Keystone token parsing should fail on misformed JSONs.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:36 -08:00
Radoslaw Zarzynski
90e1f39469 rgw: improve debugs in RGWPostHTTPData.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:36 -08:00
Radoslaw Zarzynski
bc6d7c304f rgw: ONLY formatting improvements for Keystone-related code.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:36 -08:00
Radoslaw Zarzynski
92764d423a rgw: handle Keystone API version with dedicated enum.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:36 -08:00
Radoslaw Zarzynski
8e90686805 rgw: fix wrong format of admin token request in Keystone v3.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:35 -08:00
Radoslaw Zarzynski
a622e3f9c9 rgw: rework serialization in token retrieval for Keystone's admin.
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:35 -08:00
Radoslaw Zarzynski
84bcb50600 rgw: fix Keystone v2 regression in role's JSON parsing
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:34 -08:00
Radoslaw Zarzynski
d3299b0fcb rgw: accommodate Keystone v3 in S3-related auth mechanisms.
This patch is based on changes proposed by Mark Barnes
<mark.barnes@ocado.com> (lperiquito) in commit
af71f6a518.

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:28 -08:00
Radoslaw Zarzynski
c5b114bba4 rgw: accommodate Keystone v3 in RGWSwift class.
This patch is based on changes proposed by Mark Barnes
<mark.barnes@ocado.com> (lperiquito) in commit
af71f6a518.

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:21 -08:00
Radoslaw Zarzynski
38ee66143e rgw: basic data structures for Keystone v3.
The change has been extracted from changeset proposed
in commit af71f6a518 by
Mark Barnes <mark.barnes@ocado.com> (lperiquito).

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:15 -08:00
Luis Periquito
46a488137b rgw: unify RGWValidateKeystoneToken with RGWPostHTTPData.
The change has been extracted from changeset proposed
in commit af71f6a518 by
Mark Barnes <mark.barnes@ocado.com> (lperiquito).

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:15 -08:00
Radoslaw Zarzynski
a868e598bc rgw: make RGWPostHTTPData able to extract X-Subject-Token.
The change has been extracted from changeset proposed
in commit af71f6a518 by
Mark Barnes <mark.barnes@ocado.com> (lperiquito).

Signed-off-by: Mark Barnes <mark.barnes@ocado.com>
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>
2016-02-19 12:16:05 -08:00
Jason Dillaman
7249a5cf70 librbd: read ops require journal replay
When journaling is enabled, a read op should acquire the exclusive
lock and replay the journal in case uncommitted writes are stored
in the journal.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-19 11:34:22 -05:00
Orit Wasserman
6582b7d509 Merge pull request #7717 from cbodley/wip-cmake-rbd-image-watcher
cmake: add missing librbd image_watcher sources

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
2016-02-19 17:33:47 +01:00
Casey Bodley
b61f467651 cmake: add missing librbd image_watcher sources
Fixes: #14823

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-02-19 11:18:32 -05:00
Yehuda Sadeh
0f45d58903 Merge pull request #7709 from ceph/wip-rgw-new-multisite-merge
rgw multisite v2 (Yehuda Sadeh, Orit Wasserman, Casey Bodley)
 - rename region to zonegroup
 - new zone and zonegroup configuration
 - new realm and period structures
 - rgw handles synchronization (no need for sync agent)
 - active-active data

Signed-off-by: Yehuda Sadeh
2016-02-19 07:22:03 -08:00
Kefu Chai
f6692a42df Merge pull request #7371 from theanalyst/mon/quota-msg-cleanup
mon: cleanup set-quota error msg

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-02-19 17:56:20 +08:00
Josh Durgin
49a7ef5001 Merge pull request #7708 from jdurgin/wip-cython-cleanup
pybind: replace __del__ with __dealloc__ for rbd

Reviewed-by: Hector Martin <marcan@marcan.st>
Reviewed-by: Mehdi Abaakouk <sileht@sileht.net>
2016-02-19 00:34:50 -08:00
Abhishek Lekshmanan
a917ccfce5 mon: cleanup set-quota error msg
When we send in an unrecognized field for set-quota, specify that the
options should be `max_bytes` or `max_objects`. This doesn't affect the
ceph-cli as the choices are caught at an earlier stage, however the
error message is seen while reaching the cluster via rados api for eg.

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2016-02-19 09:29:04 +01:00
Josh Durgin
b556b24a31 Merge pull request #7668 from dillaman/wip-librbd-replay-locks
librbd: use async librados notifications

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-19 00:15:36 -08:00
Yehuda Sadeh
9695a0b4d1 rgw: avoid empty object names
Don't try to sync an object with empty name (shouldn't happen, but
bugs may happen). Also, don't try to write_meta() of an empty object
name, in case we get there through a different path somehow.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 23:50:06 -08:00
Josh Durgin
3d391546f5 .gitignore: include recently added rgw programs
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-02-18 23:49:09 -08:00
Yehuda Sadeh
3f525f4043 rgw: fix s3 list bucket (affects format=json)
another merge casualty. The section wasn't opened properly.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 23:49:00 -08:00
Josh Durgin
0a988f95bc pybind: replace __del__ with __dealloc__ for rbd
Cython extension types like these don't call __del__, but use
__dealloc__ instead:

http://docs.cython.org/src/userguide/special_methods.html#finalization-method-dealloc

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2016-02-18 23:47:24 -08:00
Josh Durgin
5489ad3a21 Merge pull request #7706 from sileht/sileht/rados-cython
pybind: remove next() on iterators

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-18 23:46:21 -08:00
Yehuda Sadeh
1e47268557 rgw: only validate bucket name if not empty
This got broken due to code movement

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 23:09:48 -08:00
Yehuda Sadeh
804e6c8f94 rgw: resurrect lost code
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 23:07:07 -08:00
Mehdi Abaakouk
f98b4855e4 pybind: remove next() on iterator
Cython recommends to implement only __next__() methods.

http://docs.cython.org/src/userguide/special_methods.html#the-next-method

Signed-off-by: Mehdi Abaakouk <sileht@redhat.com>
2016-02-19 07:59:08 +01:00
Loic Dachary
78b3c40cbc Merge pull request #7641 from batrick/vstart-usage
Clarify usage on starting single osd/mds/mon.

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-02-19 13:25:46 +07:00
Yehuda Sadeh
72b0333017 test/Makefile-client.am: adjustment following merge
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 20:49:35 -08:00
Jason Dillaman
eee7d9e019 librbd: helper method for flushing journal commit positions
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-18 22:53:11 -05:00
Jason Dillaman
999a319dd9 journal: added interface to flush pending commit position updates
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-18 22:53:10 -05:00
Josh Durgin
398c493fa9 Merge pull request #7692 from sileht/sileht/rados-cython
pybind/rados: use __dealloc__ since __del__ is ignored by cython

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2016-02-18 17:42:26 -08:00
Yehuda Sadeh
4d494d5b05 Merge remote-tracking branch 'origin/master' into wip-rgw-new-multisite
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Conflicts:
	src/CMakeLists.txt
	src/rgw/Makefile.am
	src/rgw/rgw_admin.cc
	src/rgw/rgw_common.h
	src/rgw/rgw_main.cc
	src/rgw/rgw_op.cc
	src/rgw/rgw_rados.h
	src/rgw/rgw_rest_s3.cc
	src/test/Makefile-client.am
2016-02-18 17:14:07 -08:00
Jason Dillaman
1cfd965815 librbd: differentiate journal replay flush vs shut down
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-02-18 19:58:48 -05:00
Yehuda Sadeh
ac168e3f00 rgw: add drain_all() before exiting a cr
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 15:25:52 -08:00
Yehuda Sadeh
d95fcfa544 rgw: silence compilation #warnings
these are now FIXMEs

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 15:20:20 -08:00
Yehuda Sadeh
996e6e9a8a rgw: more error reporting
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 15:18:08 -08:00
Yehuda Sadeh
ac726c6dd4 rgw: data sync error handling (for sync init)
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2016-02-18 15:18:03 -08:00