Commit Graph

49978 Commits

Author SHA1 Message Date
Matt Benjamin
2a860f422b librgw: fix timestamp on create (another case)
In the rgw_create regular file case, the timestamp and size update
logic (though present) was being bypassed because the "FLAG_CREATE"
indicator which formerly propagated up from RGWLibFS::lookup_fh
was lost in the LRU change, which refactored the code for creating
new RGWFileHandle objects.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:03 -05:00
Matt Benjamin
e68600bef5 librgw: cleanup, move 3 prints to log channel
Save a pointer to CephContext in RGWProcess for use by logger--
using the existing pointer in ThreadPool would be an intrusive
change.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:01 -05:00
Matt Benjamin
7ef64851b1 librgw: add missing RGWFileHandle::stat in rgw_mkdir
The lack of stat here induced a serious memory corruption, due to
the organization of Ganesha file attributes (and their reliance on
the FSAL to initialize the memory).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:00 -05:00
Matt Benjamin
35445cf7df librgw: fix leading-slash special case, again
Fix boilerplate to sanitize relative_object_name() in rgw_create
as well as the RGWStatObjRequest case of RGWLibFS::stat_leaf.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:59 -05:00
Matt Benjamin
4f8e765263 librgw: disable name-cache lookups w/CREATE
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:58 -05:00
Matt Benjamin
b57f51bf1d librgw: fix a prefix bug and bucket timestamp bug
The fix for missing bucket.creation_time in RGWStatBucket is not
complete--we're not carrying all the info over.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:57 -05:00
Matt Benjamin
998ed65d50 librgw: ignore "/" as a common_prefix
This happens commonly, apparently when a parent object name ends
with '/' (as all explicit directories do).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:56 -05:00
Matt Benjamin
441bff73c2 librgw: rgw_getattr refactor, RGWLibFS::stat_leaf (object)
Rewrite rgw_getattr to just dispatch into RGWLibFS::getattr, where
we can better use RGWFileHandle and RGWLibFS internals.

Initially, this change intended to dispatch a new RGWStatObjRequest
in the file case, but during implementation, discovered that in
(apparently) all file cases, it has already been done--but the path
had stopped working with the naming change.

Current GETATTR_DIRS1 unit test, when repeated so relying on lookup
(rather than saving size after rgw_write/rgw_close), passes.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:55 -05:00
Matt Benjamin
ce62d80595 librgw: fix rgw_read, add unit test
rgw_read had been trivially broken by the relative path change,
fixed.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:54 -05:00
Matt Benjamin
1b797a72b5 librgw: add rgw_getattr tests
Re-implement rgw_getattr to just dispatch into RGWFileHandle::stat,
we expect all objects to stat correctly for their type.

The size of file objects is currently reporting as 0, where we expect
a small, positive size (reviewing).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:53 -05:00
Matt Benjamin
117208c7d4 librgw: re-implement rgw_create
rgw_create hadn't been re-implemented for NFS namespace behavior,
where we expect it to create empty file objects.

It needs to do a full name conflict check with prefix-matching,
to catch conflicts within object names.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:52 -05:00
Matt Benjamin
ec924b6656 librgw: update rgw_* api calls so that all take a flags arg
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:51 -05:00
Matt Benjamin
4f2dd4670f librgw: normalize obj_rec::sync calls, fix signedness of refcnt asserts
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:50 -05:00
Matt Benjamin
a50416bcd6 librgw: partial getattrs/gc checks
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:49 -05:00
Matt Benjamin
402a76f209 librgw: fix handle type in rgw_mkdir, finish dirs1 create tests
Fixes handle type after rgw_mkdir.

Add handle type checks to nfsns unit tests.  Adds tests to create
files and directories with non-bucket parent.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:47 -05:00
Matt Benjamin
d18fe86b54 librgw: set directory ctime and mtime
Adds method to RGWStatBucketRequest to access the creation_time of
the returned bucket.

(For now, that's the only timestamp we have.)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:46 -05:00
Matt Benjamin
d8175918d3 librgw: refactor nfsns dirs1 and fix a "bad unlink" case
Fix the error returns from rgw_unlink.  This is sufficient to
catch the errors that arise from operation failure.

We won't notice attempts to remove interior directories--and for
various reasons, that may be problematic (skip for now).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:45 -05:00
Matt Benjamin
558c211571 librgw: incremental bucket-object 2-level tests
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:44 -05:00
Matt Benjamin
12fd3faaec librgw: enforce S3 object name restrictions
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:43 -05:00
Matt Benjamin
66b6b1fd3e librgw: enforce S3 bucket name restrictions
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:42 -05:00
Matt Benjamin
54149b06c1 librgw: FLAG_CREATE cleanup
Split the transiently-creating concept (CREATING) from the request
to create objects used in lookup_fh.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:41 -05:00
Matt Benjamin
c1d1e15dcb librgw: nfsns dirs1 delete, make hier1 conditional
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:40 -05:00
Matt Benjamin
41b2357a46 librgw: incr. fix various nfs ops (REBASEME)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:39 -05:00
Matt Benjamin
b008fe5204 librgw: reorg SETUP_ENUMERATE test
Stat nfsroot bucket, create it only if !exist && do_create.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:38 -05:00
Matt Benjamin
b7730c0a6b librgw: conditionally send '.' and '..' (rgw_readdir)
Ganesha traditionally does not expect them, but they may be needed
when bypassing the MDCACHE (2.4).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:37 -05:00
Matt Benjamin
5e75c1834a librgw: stop abusing parents
Also make sure that root file handle has directory as its variant
type.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:36 -05:00
Matt Benjamin
7cc6a6dbeb librgw: likely correct marker handling
New logic propagates is_truncated traveral-finished indicator up
from bucket and object list operations in all cases, and correctly
expands relative paths.

Unit test output looks promising.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:34 -05:00
Matt Benjamin
446d92c4c8 librgw: is_truncated experiments
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:33 -05:00
Matt Benjamin
bd07c2ad4f librgw nfsns: create/delete updates
Add ability to delete marker objs, and create (but not yet delete)
nfs_hier objects.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:32 -05:00
Matt Benjamin
adfb96a5c2 librgw: don't early terminate readdirs (in progress)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:31 -05:00
Matt Benjamin
9a6944c747 librgw: remove unused st
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:30 -05:00
Matt Benjamin
603f245ea3 librgw: split nfsns bucket and object create blocks
Helps to see the respective timings.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:29 -05:00
Matt Benjamin
82aababc46 librgw_file_nfsns: bugfix MARKER1_SETUP
Successfully creates 5*1024 objects at /nfsroot/nfs_marker/, verifies
externally.

Follow-up rgw_readdir shows 999 results, apparently the initial readdir
segment works correctly, but restart from offset fails.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:28 -05:00
Matt Benjamin
e6b3be3cbf librgw: consolidate dup'd librgw, rgwlib
There can be only one.  Fixes crash on shutdown.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:27 -05:00
Matt Benjamin
103b9dde9e librgw: remove search_prefix()
Since it turns out this is a synonym for relative_object_name().

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:26 -05:00
Matt Benjamin
5b6af8a1af incremental marker
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:25 -05:00
Matt Benjamin
80d5201ae2 librgw: omit always-false depth test.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:24 -05:00
Matt Benjamin
bd4b984472 librgw: rework full_object_name() and friends
Flex RGWFileHandle::full_object_name() to build either a search
prefix (omits leading "/" and bucket name) or else the full path
to the root (e.g., for pretty-printing), depending on the value
of a boolean omit_bucket argument.

New convenience methods search_prefix() and relative_object_name()
select these, respectively.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:23 -05:00
Matt Benjamin
4158721036 librgw: add continued readdir test
Verify that files created in a single, large directory are correctly
enumerated through a sequence of rgw_readdir() calls, at successive
offsets checking cookie/marker and bounds invariants.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:22 -05:00
Matt Benjamin
1097ed4cf3 marker
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:20 -05:00
Matt Benjamin
a557c898c0 Fix copyright statements on new test driver files (Red Hat, Inc.)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:19 -05:00
Matt Benjamin
597f69d953 librgw: dir traversal state GC added
To minimize overhead, traversal events are modeled as a linear
sequence of fh_key, timestamp tuples.  GC operates out-of-line in
LIFO order, and includes a ref cycle.  If the matching directory
object can't be found by handle, it's already gone, and nothing
happens.

To avoid lock contention, scanning of the event sequence is done
in groups of 24 by default, but in groups of sequence/4 if the
event sequence grows larger than 500 events.

Push READDIR events on successful component readdir (bucket or
object listing partial result, as appropriate).

Revise locking, timestamp and update link count in all cases.  Link
counts are approximate (metadata to represent it is not stored).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:18 -05:00
Matt Benjamin
b970fd005e librgw: restore readdir prints (move to dout)
They are useful for checking the exact strings being produced,
and in which sequence (object or common prefix).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:17 -05:00
Matt Benjamin
7acb20d877 librgw: move readdir operations into RGWFileHandle::readdir
The motivation here is to localize state updates to the class,
e.g., to hide consistency logic.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:16 -05:00
Matt Benjamin
453219cb12 librgw: implement small-directory name cache
Augment the existing marker cache with cache of name->type
mappings for up to 128 dirents (appx. 4x the common upper-bound
of 32).

The motivation here is to use the name and type information from
readdir, rather than discarding it.  Boosts perf. at least 60%
on unit test corpus.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:15 -05:00
Matt Benjamin
e8fc2b757e librgw: remote redundnat prints in RGWReaddirRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:14 -05:00
Matt Benjamin
49a034afea librgw: don't increment offset at rcb (it is no longer a counter)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:13 -05:00
Matt Benjamin
bd06808907 librgw: fill in dirent cache
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:12 -05:00
Matt Benjamin
33856c191b librgw: implement framework for fs periodic work
This can be used for, e.g., gc of stale handles and directory
traversals.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:11 -05:00
Matt Benjamin
3237f0b96a librgw: add intrusive refcnt to RGWLibFS
Ensures that file system instances cannot be disposed while (e.g.)
background processing is ongoing.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:10 -05:00