Commit Graph

49076 Commits

Author SHA1 Message Date
Matt Benjamin
de0f86691b librgw: add debian and rpm packaging
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 13:07:24 -05:00
Matt Benjamin
2f706ef084 librgw: consolidate S3 bucket name validation
There is more cleanup potential here, but it is intrusive, as the
validators for bucket and object names have been factored through
the handler inheritance hierarchy.

This change just deletes the S3 validate_bucket_name for now, as
that method was large.  The corresponding object name validator was
not defined in RGWHandler_REST_S3, so I've left it as is, for now.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 13:07:08 -05:00
Matt Benjamin
cfb8c81eee librgw: fix copyright in src/include/rados/rgw_file.h
This file originated at Red Hat.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:29 -05:00
Yehuda Sadeh
195185d403 librgw: initialize RGWLib pointer members
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:28 -05:00
Matt Benjamin
a293c529e6 librgw: remove slo, dlo, and supplied-md5 logic in RGWWriteRequest
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:27 -05:00
Matt Benjamin
3d7fa4f725 librgw: remove RGWLibRequest "magic" numbers added for debugging
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:26 -05:00
Matt Benjamin
303f542055 librgw: comment rgw_nfs config_ops.h tunables
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:25 -05:00
Matt Benjamin
48c0f5b438 cohort_lru: remove #if 0 section
The declaration being skipped was meant as a placeholder for future
optimization work, moved to a comment.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:24 -05:00
Yehuda Sadeh
165079de10 {rgw,common}/Makefile.am: add a few more missing .h files
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:23 -05:00
Yehuda Sadeh
acadd09b86 common: add xxhash.h to Makefile.am
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:22 -05:00
Yehuda Sadeh
47a5c0cf0f rgw: fix null dereference
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:21 -05:00
Matt Benjamin
1f6799df67 librgw/rgw: fix librgw.la linkage
Ensure that key symbols from global are satisfied in the library.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:20 -05:00
Matt Benjamin
c38e3cbb6f librgw/rgw: reorganize autotools linkage
Adds the RGW-NFS library to the radosgw build, with unit tests.
Adds -fPIC to compile flags to satisfy the linker, but should have
been evident to automake/libtool.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:19 -05:00
Matt Benjamin
269d90a46f librgw: call obj_rec::sync() on dirs1_b
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:18 -05:00
Matt Benjamin
77a403971d librgw_file_nfsns: create readf_out_name, if needed
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:17 -05:00
Matt Benjamin
fa9cd660c9 librgw: remove deprecated/unused librgw personality
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:15 -05:00
Matt Benjamin
4b42c57bde librgw: properly detect RGWRados init failure
Unit tests correctly see fail result from librgw_create.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:14 -05:00
Matt Benjamin
aa969585d6 librgw: remove deprecated librgw methods
Remove unused routines:

 * librgw_acl_bin2xml
 * librgw_free_xml
 * librgw_acl_xml2bin
 * librgw_free_bin(librgw_t rgw, char* bin)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:13 -05:00
Matt Benjamin
e43eb4d634 librgw: temporary fix RGWWrite
The current behavior of put_data_and_throttle leads to data
curruption, because the supplied buffer is apparently still being
written when the call returns--regardless of the value of
need_to_wait.

For now, do a buffer::copy of the supplied buffer, rather than the
original buffer::static, since that will at least be released
when no longer needed.

Probably this improves apparent rgw_write performance, but is not
what was originally intended.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:12 -05:00
Matt Benjamin
8aa4378106 librgw: cleanup rgw_write
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:11 -05:00
Matt Benjamin
6b9b3d8f98 librgw: add new rgw_write unit test
WRITEF1 writes back data wread in READF1.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:10 -05:00
Matt Benjamin
0eda995ca4 librgw: fix RGWReadRequest send_response_data
In hindsight, it seems clear how to consume the available bytes and
account for the bytes written into the caller's buffer.

Verified by unit test on data from S3, with a 6M file read in 1M
chunks, the preferred max read in modern Linux kernel NFS.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:09 -05:00
Matt Benjamin
06aff87308 librgw: add cross-bucket rgw_rename test
It passes.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:08 -05:00
Matt Benjamin
b35de4796c librgw: fix intra-bucket rename
Avoid hopeless fail due to binding const::std::string& to a
temporary.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:07 -05:00
Matt Benjamin
ca6e93bb35 librgw: add support for rgw_rename tests
Implements intra-bucket rgw_rename test, which unfortunately fails
currently.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:06 -05:00
Matt Benjamin
2e109ec3d2 librgw: fix rgw_unlink and delete atomicity
Move internal unlink logic into new RGWLibFS::unlink method, fix
logic for files and directories.

Use RGWFileHandle::mtx and  RGWFileHandle::FLAG_DELETED to ensure
atomicity of handle creates and deletes.

Remove handles for unlinked objects from cache.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:05 -05:00
Matt Benjamin
ad3a22dacb librgw: implement object rename
The implementation uses currently does a copy, followed by a
delete operation (if successful).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:04 -05:00
Matt Benjamin
2a860f422b librgw: fix timestamp on create (another case)
In the rgw_create regular file case, the timestamp and size update
logic (though present) was being bypassed because the "FLAG_CREATE"
indicator which formerly propagated up from RGWLibFS::lookup_fh
was lost in the LRU change, which refactored the code for creating
new RGWFileHandle objects.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:03 -05:00
Matt Benjamin
e68600bef5 librgw: cleanup, move 3 prints to log channel
Save a pointer to CephContext in RGWProcess for use by logger--
using the existing pointer in ThreadPool would be an intrusive
change.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:01 -05:00
Matt Benjamin
7ef64851b1 librgw: add missing RGWFileHandle::stat in rgw_mkdir
The lack of stat here induced a serious memory corruption, due to
the organization of Ganesha file attributes (and their reliance on
the FSAL to initialize the memory).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:08:00 -05:00
Matt Benjamin
35445cf7df librgw: fix leading-slash special case, again
Fix boilerplate to sanitize relative_object_name() in rgw_create
as well as the RGWStatObjRequest case of RGWLibFS::stat_leaf.
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:59 -05:00
Matt Benjamin
4f8e765263 librgw: disable name-cache lookups w/CREATE
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:58 -05:00
Matt Benjamin
b57f51bf1d librgw: fix a prefix bug and bucket timestamp bug
The fix for missing bucket.creation_time in RGWStatBucket is not
complete--we're not carrying all the info over.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:57 -05:00
Matt Benjamin
998ed65d50 librgw: ignore "/" as a common_prefix
This happens commonly, apparently when a parent object name ends
with '/' (as all explicit directories do).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:56 -05:00
Matt Benjamin
441bff73c2 librgw: rgw_getattr refactor, RGWLibFS::stat_leaf (object)
Rewrite rgw_getattr to just dispatch into RGWLibFS::getattr, where
we can better use RGWFileHandle and RGWLibFS internals.

Initially, this change intended to dispatch a new RGWStatObjRequest
in the file case, but during implementation, discovered that in
(apparently) all file cases, it has already been done--but the path
had stopped working with the naming change.

Current GETATTR_DIRS1 unit test, when repeated so relying on lookup
(rather than saving size after rgw_write/rgw_close), passes.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:55 -05:00
Matt Benjamin
ce62d80595 librgw: fix rgw_read, add unit test
rgw_read had been trivially broken by the relative path change,
fixed.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:54 -05:00
Matt Benjamin
1b797a72b5 librgw: add rgw_getattr tests
Re-implement rgw_getattr to just dispatch into RGWFileHandle::stat,
we expect all objects to stat correctly for their type.

The size of file objects is currently reporting as 0, where we expect
a small, positive size (reviewing).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:53 -05:00
Matt Benjamin
117208c7d4 librgw: re-implement rgw_create
rgw_create hadn't been re-implemented for NFS namespace behavior,
where we expect it to create empty file objects.

It needs to do a full name conflict check with prefix-matching,
to catch conflicts within object names.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:52 -05:00
Matt Benjamin
ec924b6656 librgw: update rgw_* api calls so that all take a flags arg
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:51 -05:00
Matt Benjamin
4f2dd4670f librgw: normalize obj_rec::sync calls, fix signedness of refcnt asserts
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:50 -05:00
Matt Benjamin
a50416bcd6 librgw: partial getattrs/gc checks
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:49 -05:00
Matt Benjamin
402a76f209 librgw: fix handle type in rgw_mkdir, finish dirs1 create tests
Fixes handle type after rgw_mkdir.

Add handle type checks to nfsns unit tests.  Adds tests to create
files and directories with non-bucket parent.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:47 -05:00
Matt Benjamin
d18fe86b54 librgw: set directory ctime and mtime
Adds method to RGWStatBucketRequest to access the creation_time of
the returned bucket.

(For now, that's the only timestamp we have.)

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:46 -05:00
Matt Benjamin
d8175918d3 librgw: refactor nfsns dirs1 and fix a "bad unlink" case
Fix the error returns from rgw_unlink.  This is sufficient to
catch the errors that arise from operation failure.

We won't notice attempts to remove interior directories--and for
various reasons, that may be problematic (skip for now).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:45 -05:00
Matt Benjamin
558c211571 librgw: incremental bucket-object 2-level tests
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:44 -05:00
Matt Benjamin
12fd3faaec librgw: enforce S3 object name restrictions
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:43 -05:00
Matt Benjamin
66b6b1fd3e librgw: enforce S3 bucket name restrictions
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:42 -05:00
Matt Benjamin
54149b06c1 librgw: FLAG_CREATE cleanup
Split the transiently-creating concept (CREATING) from the request
to create objects used in lookup_fh.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:41 -05:00
Matt Benjamin
c1d1e15dcb librgw: nfsns dirs1 delete, make hier1 conditional
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:40 -05:00
Matt Benjamin
41b2357a46 librgw: incr. fix various nfs ops (REBASEME)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2016-02-12 12:07:39 -05:00