Commit Graph

48957 Commits

Author SHA1 Message Date
Matt Benjamin
b9137e47e1 librgw: really make hexdump optional, set resid in readv unit test
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:29 -05:00
Matt Benjamin
5e0b6e4df0 librgw: fix 2 dout problems
* rgw_file.cc needed to include include/assert.h AFTER all rgw
  includes, for some reason

* use *_dout syntax as target of bl.hexdump in lbrgw_file_gp.cc

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:28 -05:00
Matt Benjamin
5139c312f9 librgw: try-implement rgw_readv/rgw_writev
The model in the rgw_readv call is the caller owns the struct rgw_uio,
and uses it to pass uio_offset.

The caller returns the rgw_uio filled out, and provides indirectly
rgw_uio->uio_vio (the full private data is at uio->uio_p1 on return,
the caller (p==private) must not touch it.  The caller returns
private data in rgw_readv_release() when finished.

Meanwhile rgw_writev is atomic.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:27 -05:00
Matt Benjamin
540ce769ce librgw: improve rgw_write and add WRITE_READ_VERIFY
Update rgw_write to return bytes written as an OUT argument,
use and verify in PUT test.

A new WRITE_READ_VERIFY test writes 16 checksummed, 64K data pages
to the object when --bulk specified.  The returned write value is
checked, but pages not re-read (yet).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:26 -05:00
Matt Benjamin
421f5eccfe librgw: remove junk prints
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:25 -05:00
Matt Benjamin
50abb8cbc5 librgw: set RGWGetObj::get_data = true, for great justice
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:24 -05:00
Matt Benjamin
290b06a91c librgw: rework library CephContext* and init
Remove duplicated init step.  Just reference the global
Ceph context for the library instance(s).

The immediate effect of this is to bring the (correctly initialized)
global context into all call paths, which (e.g.) fixes debug
log prints within subsystems.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:23 -05:00
Matt Benjamin
72fb1ad695 librgw: dispatch RGWDeleteRequest in object branch in rgw_unlink().
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:22 -05:00
Matt Benjamin
88c2d0283a librgw: implement RGWGetObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:21 -05:00
Matt Benjamin
b652126640 librgw: declare RGWDeleteObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:20 -05:00
Matt Benjamin
0ec522bc81 librgw: declare RGWGetObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:19 -05:00
Matt Benjamin
21e6ff3760 librgw: fix RGWPutObj s->content_length
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:18 -05:00
Matt Benjamin
24da131da3 RGWPutObjRequest: fix get_data()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:16 -05:00
Matt Benjamin
b3150b0f0d rgw log: disambiguate 'Read AccessControlPolicy' (logs)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:15 -05:00
Matt Benjamin
c352d4b2d5 librgw: add --prelist to gp to prime cache
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:14 -05:00
Matt Benjamin
7f981b10a2 librgw: fix rgw_open param names and invocation
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:13 -05:00
Matt Benjamin
9429cb13f9 librgw: avoid illegal rele()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:12 -05:00
Matt Benjamin
ec89aafd2f librgw: don't create/delete in librgw_file.cc (unit test)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:11 -05:00
Matt Benjamin
adcba208a6 librgw: fs->root_fh is a pointer (and assign it)
Assign and expose the expanded root_fh indirectly from the mounted
root filesystem structure.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:10 -05:00
Matt Benjamin
522c95884b librgw: bucket names can be const
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:09 -05:00
Matt Benjamin
e562357bba librgw: reify root handles
Reify root handle, remove traces of legacy (counter) handles.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:08 -05:00
Matt Benjamin
1e7539bed8 librgw: incremental RGWPutObj work and almost-complete RGWFH refactor.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:07 -05:00
Matt Benjamin
a50bc13d80 xxHash: build internally
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:06 -05:00
Matt Benjamin
1254f34f5c librgw: add ZPage/ZPageSet abstraction (gp unit tests)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:05 -05:00
Matt Benjamin
020aee9363 xxHash: add as submodule
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:03 -05:00
Matt Benjamin
fc0a895af2 librgw: rgw_file_handle refactor, plus get/put
Implement encapsulation of RGWFileHandle in rgw_file_handle,
so return these from rgw_lookup(), add new rgw_fh_rele() to release
internal state.

Currently the rgw_fh_rele just deletes, in future might be extended
with refcnt support.

Implement the first (atomic) stage of i/o using GET and PUT, add a
new librgw_file_gp test driver for this.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:02 -05:00
Matt Benjamin
03b661595c librgw: add options for bucket name, multi create, delete
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:01 -05:00
Matt Benjamin
abcec54002 librgw: fix acl assignment in RGWCreateBucket_OS_Lib
The logic to create a default S3 policy in
RGWCreateBucket_OS_Lib::get_params() was essentially correct, but
failed to store the resulting policy on the current request.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:00 -05:00
Matt Benjamin
29942221bd librgw: don't fake hostname (real hostname isn't used yet)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:59 -05:00
Matt Benjamin
cca325e11a librgw: don't allow running rgwfile tests as anon
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:58 -05:00
Matt Benjamin
3fcdd946b6 librgw: partial delete/unlink support
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:57 -05:00
Matt Benjamin
9fb454ac61 librgw: return result code from execute_request
Update rgw_mkdir and rgw_readdir to use it.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:56 -05:00
Matt Benjamin
c43384c09d librgw: fix uri, read_permissions virtual for create-bucket
* set uri in rgw_mkdir (duh)
* RGWCreateBucketRequest should override RGWHandler::read_permissions
  to return 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:55 -05:00
Matt Benjamin
460794ef1a librgw: block in create bucket, rgw_mkdir
Unit test runs, doesn't...seem to create a bucket.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:54 -05:00
Matt Benjamin
ceb2b0e97d librgw: re-derive rgw_os_lib family
* shorten names to match files
* remove derivation through rgw_rest intermediates

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:53 -05:00
Matt Benjamin
83bed47f4c librgw: s/rgw_rest_lib.{h,cc}/rgw_os_lib.{h,cc}
There is no longer an explicit (e.g., inherited) dependency on REST
implementations.

The remaining coupling (via RGWLibEnv, and struct req_state), can
be factored out later.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:52 -05:00
Matt Benjamin
cbcb641ef8 lirgw: call RGWHanlder_Lib::init_from_header
Call this from the general RGWLibRequest init() method, after
descendant header_init() has run.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:51 -05:00
Matt Benjamin
6dae209ceb librgw: cleanups, incremental ListBuckets fixes
1. remove unused RGWLibRequestEnv code marked to delete
2. restore call to RGWHandler::authorize
3. provide RGWLibRequest::authorize implementing it
   ** the implementation is incomplete, and shows need to expand
      the initial mount auth to deal with keystone, etc
4. since acl check is still not quite right in verify_permissions()
   disable this again (temporarily)
5. fix dout_subsys of rgw_request.cc

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:50 -05:00
Matt Benjamin
e29562954b librgw: forward cmdline args to librgw_create
Verified w/-d.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:48 -05:00
Matt Benjamin
55bb8d347a librgw: remove disabled RGWLib RGWREST::get_handler()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:47 -05:00
Matt Benjamin
ed7dc88e79 librgw: cleanups in process_request
* remove disabled S3 authorize step (user authorized on mount)
* re-enable op->verify_permission()

Skipping the latter makes RGW unhappy (1s stalls on each request),
but no objects are listing yet.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:46 -05:00
Matt Benjamin
ec96809cd0 librgw: pull RGWUserinfo user into RGWLibRequest
Also, return result from process_request(req, io) to caller.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:45 -05:00
Matt Benjamin
7e21e2386d librgw: cache RGWUserInfo in rgw_fs private handle
The private RGWLibFS type holds a RGWUserInfo, and
calls rgw_get_user_info_by_access_key and verifies the returned
secret, checks for suspended user, on rgw_mount.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:44 -05:00
Matt Benjamin
6688bc1cce librgw: split RGWUserInfo from struct req_state
The immediate purpose is to permit auth caching in RGWLib, but
it seems likely that this will be useful for general req_state
de-bloatification.

For now, all request processing paths put a RGWUserInfo on the
stack, so the lifetime of the object is unchanged from when it was
an expanded member.  A later change will introduce caching of
the objects.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:43 -05:00
Matt Benjamin
290e78812f librgw: RGWFileHandle and RGWLibFS
Experimentally reframe the C interface objects as subobjects in
private internal types.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:42 -05:00
Matt Benjamin
f9382815a3 librgw: fixes for RGWListBucketRequest callback
The cached req_state is still not correct at RGWOp::execute().

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:41 -05:00
Matt Benjamin
9c4de57800 librgw: untested RGWLibListBucketsRequest and caller.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:40 -05:00
Matt Benjamin
80a56f2004 librgw: fix 2 bugs in RGWListBuckets, remove scaffolding
1. the prior commit had authorize cognate with S3
1.1 which may be right
1.2 but maybe not for each request
1.3 and lacked priming/forged auth data

The forging of auth data is now easy, but the interesting part is
caching and efficiently attaching it so I paused, and

2. disabled auth, which was now overriding and anonymizing all
requests (good!)

3. fixed another instance of reinterpret_cast where (one of)
dynamic_cast or a saved pointer descendant request (avoids a runtime
check, so will add later) is needed

4. fixed bad offset argument to rgw_readdir

The list bucket facility now works, modulo non-auth.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:39 -05:00
Matt Benjamin
c44f19c438 librgw: process_request prologue fixups
* initialize RGWEnv with required HTTP_HOST (incorrectly!)
* assign req_state::cio (not really correctly)
* remove incorrect delete of RGWOp, which no longer alloc'd

Now runs, but don't see buckets.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:38 -05:00
Matt Benjamin
0b614a1b92 librgw: reprocess process_request
Refactor RGWLib variant of process_request(...) as a (actually 2,
one which puts an RGWLibIO on the stack) member function of
RGWLibProcess.

Also replace extern abort_early with static inline abort_req(),
since the rgw_rest extern assumes HTTP/REST.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:37 -05:00