Commit Graph

48972 Commits

Author SHA1 Message Date
Matt Benjamin
fd8f20e820 librgw: change validity check for rgw_lookup
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:46 -05:00
Matt Benjamin
71590b8b34 librgw: add RGWStatObjRequest and use it
* implement a stat operation over RGWGetObj (returning early)
* use stat to validate rgw_lookup
* use stat to implement rgw_getattr

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:44 -05:00
Matt Benjamin
0e5f5e2bcf librgw: add diagnostic dump_buckets() call.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:43 -05:00
Matt Benjamin
a5e6d559f0 librgw: fail RGWLibFS::authorize() on user-by-id lookup failure
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:42 -05:00
Matt Benjamin
0175a3d3a9 librgw: implement rgw_vfsstat
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:41 -05:00
Matt Benjamin
74e4c5ab38 librgw: rgw_mkdir and rgw_create return addresses
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:40 -05:00
Matt Benjamin
507ecab9ce librgw: pass struct rgw_fh_hk by reference
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:39 -05:00
Matt Benjamin
056c0cb812 librgw: always add struct before rgw_file_handle in rgw_file.h
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:38 -05:00
Matt Benjamin
8417a788fb librgw: bring back lookup_handle()
Now as a member of RGWLibFS, and exposed via rgw_lookup_handle()
for use by library clients.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:37 -05:00
Matt Benjamin
455d9d5c73 librgw_file_gp: check test state before closing null fh handle
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:36 -05:00
Matt Benjamin
9821d6fbf0 librgw: implement intrusive filehandle cache
The cache is keyed on a hash of bucket name, plus a hash of the
object name.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:35 -05:00
Matt Benjamin
d6e171a718 librgw: add common/cohort_lru.h
This a cache-line-friendly set of primitives for

a) shareded, intrusive maps based on boost::intrusive
b) an LRU system based on them

The system uses per-lane (shard) locks.  Lane count, front cache
size, and lock type are template params.

Initially, use this in preference to an unshareded map or hashtable,
in new RGW code.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:34 -05:00
Matt Benjamin
e1b4fc7e21 librgw_file: release file handles
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:33 -05:00
Matt Benjamin
73b19a9129 librgw_file_gp: don't leak a buffer::list
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:32 -05:00
Matt Benjamin
ab377b1865 librgw: checksum verify writev/readv read-after-write
The changes to GET_OBJECT and PUT_OBJECT test options are needed,
since they are (re) writing the same object.

Sadly, READV_AFTER_WRITEV is failing at page offset 7.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:30 -05:00
Matt Benjamin
b9137e47e1 librgw: really make hexdump optional, set resid in readv unit test
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:29 -05:00
Matt Benjamin
5e0b6e4df0 librgw: fix 2 dout problems
* rgw_file.cc needed to include include/assert.h AFTER all rgw
  includes, for some reason

* use *_dout syntax as target of bl.hexdump in lbrgw_file_gp.cc

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:28 -05:00
Matt Benjamin
5139c312f9 librgw: try-implement rgw_readv/rgw_writev
The model in the rgw_readv call is the caller owns the struct rgw_uio,
and uses it to pass uio_offset.

The caller returns the rgw_uio filled out, and provides indirectly
rgw_uio->uio_vio (the full private data is at uio->uio_p1 on return,
the caller (p==private) must not touch it.  The caller returns
private data in rgw_readv_release() when finished.

Meanwhile rgw_writev is atomic.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:27 -05:00
Matt Benjamin
540ce769ce librgw: improve rgw_write and add WRITE_READ_VERIFY
Update rgw_write to return bytes written as an OUT argument,
use and verify in PUT test.

A new WRITE_READ_VERIFY test writes 16 checksummed, 64K data pages
to the object when --bulk specified.  The returned write value is
checked, but pages not re-read (yet).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:26 -05:00
Matt Benjamin
421f5eccfe librgw: remove junk prints
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:25 -05:00
Matt Benjamin
50abb8cbc5 librgw: set RGWGetObj::get_data = true, for great justice
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:24 -05:00
Matt Benjamin
290b06a91c librgw: rework library CephContext* and init
Remove duplicated init step.  Just reference the global
Ceph context for the library instance(s).

The immediate effect of this is to bring the (correctly initialized)
global context into all call paths, which (e.g.) fixes debug
log prints within subsystems.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:23 -05:00
Matt Benjamin
72fb1ad695 librgw: dispatch RGWDeleteRequest in object branch in rgw_unlink().
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:22 -05:00
Matt Benjamin
88c2d0283a librgw: implement RGWGetObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:21 -05:00
Matt Benjamin
b652126640 librgw: declare RGWDeleteObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:20 -05:00
Matt Benjamin
0ec522bc81 librgw: declare RGWGetObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:19 -05:00
Matt Benjamin
21e6ff3760 librgw: fix RGWPutObj s->content_length
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:18 -05:00
Matt Benjamin
24da131da3 RGWPutObjRequest: fix get_data()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:16 -05:00
Matt Benjamin
b3150b0f0d rgw log: disambiguate 'Read AccessControlPolicy' (logs)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:15 -05:00
Matt Benjamin
c352d4b2d5 librgw: add --prelist to gp to prime cache
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:14 -05:00
Matt Benjamin
7f981b10a2 librgw: fix rgw_open param names and invocation
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:13 -05:00
Matt Benjamin
9429cb13f9 librgw: avoid illegal rele()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:12 -05:00
Matt Benjamin
ec89aafd2f librgw: don't create/delete in librgw_file.cc (unit test)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:11 -05:00
Matt Benjamin
adcba208a6 librgw: fs->root_fh is a pointer (and assign it)
Assign and expose the expanded root_fh indirectly from the mounted
root filesystem structure.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:10 -05:00
Matt Benjamin
522c95884b librgw: bucket names can be const
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:09 -05:00
Matt Benjamin
e562357bba librgw: reify root handles
Reify root handle, remove traces of legacy (counter) handles.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:08 -05:00
Matt Benjamin
1e7539bed8 librgw: incremental RGWPutObj work and almost-complete RGWFH refactor.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:07 -05:00
Matt Benjamin
a50bc13d80 xxHash: build internally
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:06 -05:00
Matt Benjamin
1254f34f5c librgw: add ZPage/ZPageSet abstraction (gp unit tests)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:05 -05:00
Matt Benjamin
020aee9363 xxHash: add as submodule
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:03 -05:00
Matt Benjamin
fc0a895af2 librgw: rgw_file_handle refactor, plus get/put
Implement encapsulation of RGWFileHandle in rgw_file_handle,
so return these from rgw_lookup(), add new rgw_fh_rele() to release
internal state.

Currently the rgw_fh_rele just deletes, in future might be extended
with refcnt support.

Implement the first (atomic) stage of i/o using GET and PUT, add a
new librgw_file_gp test driver for this.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:02 -05:00
Matt Benjamin
03b661595c librgw: add options for bucket name, multi create, delete
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:01 -05:00
Matt Benjamin
abcec54002 librgw: fix acl assignment in RGWCreateBucket_OS_Lib
The logic to create a default S3 policy in
RGWCreateBucket_OS_Lib::get_params() was essentially correct, but
failed to store the resulting policy on the current request.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:00 -05:00
Matt Benjamin
29942221bd librgw: don't fake hostname (real hostname isn't used yet)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:59 -05:00
Matt Benjamin
cca325e11a librgw: don't allow running rgwfile tests as anon
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:58 -05:00
Matt Benjamin
3fcdd946b6 librgw: partial delete/unlink support
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:57 -05:00
Matt Benjamin
9fb454ac61 librgw: return result code from execute_request
Update rgw_mkdir and rgw_readdir to use it.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:56 -05:00
Matt Benjamin
c43384c09d librgw: fix uri, read_permissions virtual for create-bucket
* set uri in rgw_mkdir (duh)
* RGWCreateBucketRequest should override RGWHandler::read_permissions
  to return 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:55 -05:00
Matt Benjamin
460794ef1a librgw: block in create bucket, rgw_mkdir
Unit test runs, doesn't...seem to create a bucket.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:54 -05:00
Matt Benjamin
ceb2b0e97d librgw: re-derive rgw_os_lib family
* shorten names to match files
* remove derivation through rgw_rest intermediates

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:04:53 -05:00