Commit Graph

48934 Commits

Author SHA1 Message Date
Matt Benjamin
80c85738d0 librgw: fix rgw_unlink
Fix handling of bucket namespace, remove bogus RGWLibFS::is_root()
logic.  Also remove a stray debug check.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:58 -05:00
Matt Benjamin
fbfafb3eee librgw: ok, make RGW vnodes cache Unix attrs
Ganesa ULP needs to see a sane progression of attrs in the CREATE
path.

With this change shell actions like

$ echo "hi" > /nfs41/mybucket/mom

work as expected.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:57 -05:00
Matt Benjamin
5f4ef7edff librgw: forge Unix attrs in rgw_create
Possibly RGW vnodes should cache Unix attrs, it would reduce the
latency of (frequent) getattr calls, and eventually lines up with
more materialized Unix attributes.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:56 -05:00
Matt Benjamin
c480472f00 librgw: fix Unix file modes
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:55 -05:00
Matt Benjamin
4fd6511c19 librgw: implement rgw_create + bugfixes
Incremental work to support CREATE,OPEN,WRITE*,CLOSE transactions.
Adds a new tuple-based signature for RGWFileHandle::lookup_fh, and
fixes an inconsisten initialization of fhk and fh.fh_hk.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:54 -05:00
Matt Benjamin
6db1f9dd79 librgw: fix size computation in RGWStatObjRequest
Overrides RGWGetObj::execute(), calls the parent method and then
saves obj_size from the manifest.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:53 -05:00
Matt Benjamin
bf6bee6117 librgw: fix many rgw_file return codes
This should all be -(POSIX), or at any rate negative.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:52 -05:00
Matt Benjamin
66823a06b6 librgw: don't abuse fs_private
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:51 -05:00
Matt Benjamin
857aa1e337 librgw: introduce RGW_LOOKUP_FLAG_CREATE
The current API relies on rgw_lookup and rgw_lookup_filehandle to
return file handle instances, and like POSIX, needs to deal in some
way with the NFS/POSIX open-for-create concept.

For now, allow rgw_lookup to return the canonical instance for an
object being created, i.e., if the option RGW_LOOKUP_FLAG_CREATE
is passed.  (I added an RGW_OPEN_FLAG_CREATE also, but it doesn't
do anything.)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:50 -05:00
Matt Benjamin
7f482c05b2 librgw: add getattr unit tests, restructure
The fids1, fids2 vectors were confusing, as fids2 mapped the
objects in the union of all buckets.

Now there's a matrix of <bucket,object>, and it's trivial to
scan the objects in any one bucket.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:49 -05:00
Matt Benjamin
3c94a95ba5 librgw: handle a 3 cases in rgw_getattr
Only the object case is mostly correct.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:48 -05:00
Matt Benjamin
885a603adb librgw: handle rgw_lookup in the root (minimally)
In addition, check real return code from RGWStatRequest.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:47 -05:00
Matt Benjamin
fd8f20e820 librgw: change validity check for rgw_lookup
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:46 -05:00
Matt Benjamin
71590b8b34 librgw: add RGWStatObjRequest and use it
* implement a stat operation over RGWGetObj (returning early)
* use stat to validate rgw_lookup
* use stat to implement rgw_getattr

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:44 -05:00
Matt Benjamin
0e5f5e2bcf librgw: add diagnostic dump_buckets() call.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:43 -05:00
Matt Benjamin
a5e6d559f0 librgw: fail RGWLibFS::authorize() on user-by-id lookup failure
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:42 -05:00
Matt Benjamin
0175a3d3a9 librgw: implement rgw_vfsstat
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:41 -05:00
Matt Benjamin
74e4c5ab38 librgw: rgw_mkdir and rgw_create return addresses
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:40 -05:00
Matt Benjamin
507ecab9ce librgw: pass struct rgw_fh_hk by reference
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:39 -05:00
Matt Benjamin
056c0cb812 librgw: always add struct before rgw_file_handle in rgw_file.h
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:38 -05:00
Matt Benjamin
8417a788fb librgw: bring back lookup_handle()
Now as a member of RGWLibFS, and exposed via rgw_lookup_handle()
for use by library clients.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:37 -05:00
Matt Benjamin
455d9d5c73 librgw_file_gp: check test state before closing null fh handle
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:36 -05:00
Matt Benjamin
9821d6fbf0 librgw: implement intrusive filehandle cache
The cache is keyed on a hash of bucket name, plus a hash of the
object name.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:35 -05:00
Matt Benjamin
d6e171a718 librgw: add common/cohort_lru.h
This a cache-line-friendly set of primitives for

a) shareded, intrusive maps based on boost::intrusive
b) an LRU system based on them

The system uses per-lane (shard) locks.  Lane count, front cache
size, and lock type are template params.

Initially, use this in preference to an unshareded map or hashtable,
in new RGW code.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:34 -05:00
Matt Benjamin
e1b4fc7e21 librgw_file: release file handles
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:33 -05:00
Matt Benjamin
73b19a9129 librgw_file_gp: don't leak a buffer::list
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:32 -05:00
Matt Benjamin
ab377b1865 librgw: checksum verify writev/readv read-after-write
The changes to GET_OBJECT and PUT_OBJECT test options are needed,
since they are (re) writing the same object.

Sadly, READV_AFTER_WRITEV is failing at page offset 7.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:30 -05:00
Matt Benjamin
b9137e47e1 librgw: really make hexdump optional, set resid in readv unit test
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:29 -05:00
Matt Benjamin
5e0b6e4df0 librgw: fix 2 dout problems
* rgw_file.cc needed to include include/assert.h AFTER all rgw
  includes, for some reason

* use *_dout syntax as target of bl.hexdump in lbrgw_file_gp.cc

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:28 -05:00
Matt Benjamin
5139c312f9 librgw: try-implement rgw_readv/rgw_writev
The model in the rgw_readv call is the caller owns the struct rgw_uio,
and uses it to pass uio_offset.

The caller returns the rgw_uio filled out, and provides indirectly
rgw_uio->uio_vio (the full private data is at uio->uio_p1 on return,
the caller (p==private) must not touch it.  The caller returns
private data in rgw_readv_release() when finished.

Meanwhile rgw_writev is atomic.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:27 -05:00
Matt Benjamin
540ce769ce librgw: improve rgw_write and add WRITE_READ_VERIFY
Update rgw_write to return bytes written as an OUT argument,
use and verify in PUT test.

A new WRITE_READ_VERIFY test writes 16 checksummed, 64K data pages
to the object when --bulk specified.  The returned write value is
checked, but pages not re-read (yet).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:26 -05:00
Matt Benjamin
421f5eccfe librgw: remove junk prints
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:25 -05:00
Matt Benjamin
50abb8cbc5 librgw: set RGWGetObj::get_data = true, for great justice
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:24 -05:00
Matt Benjamin
290b06a91c librgw: rework library CephContext* and init
Remove duplicated init step.  Just reference the global
Ceph context for the library instance(s).

The immediate effect of this is to bring the (correctly initialized)
global context into all call paths, which (e.g.) fixes debug
log prints within subsystems.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:23 -05:00
Matt Benjamin
72fb1ad695 librgw: dispatch RGWDeleteRequest in object branch in rgw_unlink().
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:22 -05:00
Matt Benjamin
88c2d0283a librgw: implement RGWGetObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:21 -05:00
Matt Benjamin
b652126640 librgw: declare RGWDeleteObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:20 -05:00
Matt Benjamin
0ec522bc81 librgw: declare RGWGetObjRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:19 -05:00
Matt Benjamin
21e6ff3760 librgw: fix RGWPutObj s->content_length
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:18 -05:00
Matt Benjamin
24da131da3 RGWPutObjRequest: fix get_data()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:16 -05:00
Matt Benjamin
b3150b0f0d rgw log: disambiguate 'Read AccessControlPolicy' (logs)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:15 -05:00
Matt Benjamin
c352d4b2d5 librgw: add --prelist to gp to prime cache
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:14 -05:00
Matt Benjamin
7f981b10a2 librgw: fix rgw_open param names and invocation
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:13 -05:00
Matt Benjamin
9429cb13f9 librgw: avoid illegal rele()
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:12 -05:00
Matt Benjamin
ec89aafd2f librgw: don't create/delete in librgw_file.cc (unit test)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:11 -05:00
Matt Benjamin
adcba208a6 librgw: fs->root_fh is a pointer (and assign it)
Assign and expose the expanded root_fh indirectly from the mounted
root filesystem structure.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:10 -05:00
Matt Benjamin
522c95884b librgw: bucket names can be const
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:09 -05:00
Matt Benjamin
e562357bba librgw: reify root handles
Reify root handle, remove traces of legacy (counter) handles.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:08 -05:00
Matt Benjamin
1e7539bed8 librgw: incremental RGWPutObj work and almost-complete RGWFH refactor.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:07 -05:00
Matt Benjamin
a50bc13d80 xxHash: build internally
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:05:06 -05:00