Commit Graph

29422 Commits

Author SHA1 Message Date
Yehuda Sadeh
89d3f47f62 radosgw-admin: can set user's bucket quota
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-10 14:45:46 -07:00
John Wilkins
cb9ebd6f36 doc: Minor correction.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-10-10 14:31:37 -07:00
Yehuda Sadeh
14eabd4aa7 rgw: bucket quota threshold
Add bucket quota threshold so that when we're passed that value we
reread the bucket stats before every write and not rely on cached value.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-10 13:51:24 -07:00
Yehuda Sadeh
34d05262e5 rgw: configurable bucket quota size
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-10 13:51:23 -07:00
Yehuda Sadeh
fc35807768 rgw: protect against concurrent async quota updates
Leverage the cache lru_map locking for making sure that we don't end
up with more than a single concurrent async update on the same bucket
within the same update window.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-10 13:51:23 -07:00
Yehuda Sadeh
2e4ecc29de rgw: async quota update
Asynchronously update bucket stats when a period passed, but bucket
stats are within the ttl window.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-10 13:51:17 -07:00
John Wilkins
5bc6327666 doc: Merge cleanup.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-10-10 12:21:34 -07:00
John Wilkins
c0c332c120 doc: minor clean up.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-10-10 12:20:37 -07:00
John Wilkins
fa909ad6ed doc: Update from wip-doc-quickstart branch.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2013-10-10 11:47:56 -07:00
Yan, Zheng
007f06ec17 mds: fix infinite loop of MDCache::populate_mydir().
make MDCache::populate_mydir() only fetch bare-bone stray dirs.
After all stray dirs are populated, call MDCache::scan_stray_dir(),
it fetches incomplete stray dirs.

Fixes: #4405
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-10 10:53:24 +08:00
Samuel Just
1f50750d0f ReplicatedPG: remove the other backfill related flushes
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-09 13:35:42 -07:00
Samuel Just
db6623fa3e RadosModel: send racing read on write
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-09 13:35:42 -07:00
Samuel Just
2b216c3ae2 ReplicatedPG: block reads on an object until the write is committed
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-10-09 13:35:42 -07:00
Yehuda Sadeh
e8a2992d6c rgw: rearrange includes
Need to have ceph_json.h first due to some boost weirdness.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-10-08 14:34:53 -07:00
Sandon Van Ness
6ff9570726 Go back to $PWD in fsstress.sh if compiling from source.
Although fsstress was being called with a static path the directory
it was writing to was in the current directory so doing a cd to the
source directory that is made in /tmp and then removing it later
caused it to be unable to write the files in a non-existent dir.

This change gets the current path first and cd's back into it after
it is done compiling fsstress.

Issue #6479.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>
2013-10-08 12:05:11 -07:00
Sandon Van Ness
8e493ef23b Go back to $PWD in fsstress.sh if compiling from source.
Although fsstress was being called with a static path the directory
it was writing to was in the current directory so doing a cd to the
source directory that is made in /tmp and then removing it later
caused it to be unable to write the files in a non-existent dir.

This change gets the current path first and cd's back into it after
it is done compiling fsstress.

Issue #6479.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>
2013-10-08 12:05:02 -07:00
Sage Weil
1575895bdc Merge pull request #700 from dachary/master
doc: fix openstack rbd installation command
2013-10-08 07:52:42 -07:00
Loic Dachary
768fb0a2a2 doc: fix openstack rbd installation command
This cannot work because client.volumes.key is not a command

   ssh {your-compute-host} client.volumes.key

replace with a tee to allow for copy/paste as well as using the

   cat client.volumes.key

in the following lines.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-10-08 12:07:00 +02:00
David Zafman
86b5b42718 Merge pull request #699 from dachary/wip-ghobject-assert
os: stronger assert on FileStore::lfn_open

Reviewed-by: David Zafman <david.zafman@inktank.com>
2013-10-07 17:05:01 -07:00
Loic Dachary
e21e5736b3 os: stronger assert on FileStore::lfn_open
Signed-off-by: Loic Dachary <loic@dachary.org>
2013-10-08 01:15:41 +02:00
Greg Farnum
b9cc91dcef Merge branch 'next' 2013-10-07 14:40:42 -07:00
Gregory Farnum
9f75a42336 Merge pull request #697 from dachary/master
This lets "make check" pass, so I like it.
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-10-07 14:15:19 -07:00
Loic Dachary
0f323bc4d6 common: unintended use of the wrong bloom_filter prototype
For encoding/decoding test purposes. The generate_test_instances should
be improved to cover more encoding/decoding cases.

Signed-off-by: Loic Dachary <loic@dachary.org>
2013-10-07 23:11:20 +02:00
Greg Farnum
4b911cf817 ReplicatedPG: copy: use aggregate return code instead of individual Op return
It appears that the OSD is not filling in the individual return codes, and they
should be equivalent for all purposes we care about here (the only Op we are
doing is the copy-get, and if it fails we are getting its failure code).

Reported-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 13:50:30 -07:00
Sage Weil
6da4b91c07 os/FileStore: fix ENOENT error code for getattrs()
In commit dc0dfb9e01 the omap xattrs code
moved up a block and r was no longer local to the block.  Translate
ENOENT -> 0 to compensate.

Fix the same error in _rmattrs().

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 13:50:08 -07:00
Gregory Farnum
13b80bb446 Merge pull request #610 from ceph/wip-optracker
Reviewed-by: Samuel Just <sam.just@inktank.com>
2013-10-07 08:56:46 -07:00
Sage Weil
f7c5d1ac94 Merge remote-tracking branch 'gh/next' 2013-10-07 04:37:20 -07:00
João Eduardo Luís
02da1a1d79 Merge pull request #695 from ceph/wip-mon-auth
fix mon double-free when dropping unhandled messages, and allow "get monmap" messages to go through without authenticating for MonCliente::get_monmap_privately()

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-10-07 04:31:36 -07:00
Sage Weil
901427c522 Merge pull request #693 from ceph/wip-bloom
bloom_filter improvements, cleanups

Reviewed-by: Loic Dachary <loic@dachary.org>
2013-10-07 00:28:35 -07:00
Sage Weil
71ee6d7c8d mon: allow MMonGetMap without authentication
This is used by the MonClient::get_monmap_privately() helper.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 15:17:50 -07:00
Sage Weil
f279641894 mon: do not put() unhandle message
If we return false because we aren't handling a message, we should not put
the ref.  This fixes a double-free.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 15:17:50 -07:00
Sage Weil
0cae3a13fa common/bloom_filter: add a few assertions and checks for bit_table_ == NULL
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:28:20 -07:00
Sage Weil
cf34adb99b common/bloom_filter: simplify compressible_bloom_filter encode/decode/dump
Re-use bloom_filter for this, as per Loic's suggestion.  Much nicer!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:22:41 -07:00
Sage Weil
f45c3b98d3 common/bloom_filter: fix compress; improve argument
Fix bug in compress() when compressing to less than .5 or original.

Make the argument have sane units (target size relative to current size;
not a precentage reduction).

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:18:35 -07:00
Sage Weil
055dc4c1f2 common/bloom_filter: fix estimated element count for compressed filters
We need to compensate for the fact that there are fewer bits than there
used to be.

This is a crappy adjustment; it is non-linear.  It's clone enough for now,
though.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-06 10:18:30 -07:00
Alan Grosskurth
16dec28c02 ceph.spec.in: Always depend on junit4 (fixes bug #6216)
The RPM spec file currently lists `junit` as a dependency on all
platforms except recent versions of OpenSUSE. This is problematic
because `junit` refers to version 3.x, which doesn't satisfy the check
in the configure script:

    configure: Cannot find junit4.jar (apt-get install junit4)

As a result, rpmbuild doesn't build `libcephfs-test.jar` and fails:

    http://tracker.ceph.com/issues/6216

So let's depend on `junit4` in all cases. This package is provided by
all supported platforms (el6, fc17, fc18, fc19, opensuse12.2,
opensuse12, sles11), and my testing on CentOS 6 shows that rpmbuild
now succeeds and produces the expected `libcephfs_jni1` RPM.
2013-10-06 00:42:43 -07:00
Sage Weil
82f6ec596c doc/release-notes: v0.70
Note that a bunch of stuff we thought would go in 0.70 is actually in 0.69,
so the update/release notes were adjusted accordingly.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-05 21:08:55 -07:00
Sage Weil
aaabc65739 common/bloom_filter: methods for density, approx unique element counts
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:35 -07:00
Sage Weil
04091f87dd common/bloom_filter: remember original target size
This isn't strictly needed for core functionality, but it is convenient to
know.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:35 -07:00
Sage Weil
d07f5f3588 common/bloom_filter: drop raw_table_size_ member
We were storing table_size_ and raw_table_size_, where one is the size in
bits and the other is the size in bytes.  This is silly.  Store only the
size in bytes.

Also, bytes are always 8 bits, so use bit shifts and drop some of that
silliness too.

Move the member declarations to the top of the class so you read them
before the methods.

Fix some annoying whitespace.

Avoid allocating a 0-length array.

Mark the encoding incompatible with v1.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:57:18 -07:00
Sage Weil
ea2378e539 common/bloom_filter: make compressible_bloom_filter encodable
Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:56:18 -07:00
Sage Weil
92a60a0584 mon/PGMap: make generated test instances obey new invariant
As of 091809b814 we keep an osd_map epoch
for any osd_stat update, and assert as much.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-10-04 22:06:04 -07:00
Yan, Zheng
e0be61e4ab qa/workunits: Add large number of files in a directory test script
Test the directory fragments feature

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
f82aefafd4 mds: reduce verbosity of dir fragment logging
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
488ddad7fe mds: fix bloom filter leaks
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
1d14d11a78 mds: stop propagating rstat if freezing/frozen dirfrag is encountered.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
576da8e507 mds: don't scatter gather update bare-bones dirfrags
avoid adding bare-bones dirfrag that has not yet been fetched from
the disk to the journal.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
c7db090b2a fragtree_t: fix get_leaves_under()
If fragtree is (*^1, 1*^1) and we want leaves under frag 000*.
get_leaves_under() return frag 0*, frag 10* and frag 11*. This is
obviously wrong.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
c40c5dc68d client: handle dirfrag mismatch when processing readdir reply
If client has outdated directory fragments information, it may request
readdir an non-existent directory fragment. In this case, the MDS finds
an approximate directory fragment and sends its contents back to the
client. When receiving a reply with fragment that is different than the
requested one, the client need to reset the 'readdir offset'.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00
Yan, Zheng
405465128a client: use dir_result_t::END as flag
So we don't lose the latest readdir frag and offset after marking
end of readdir.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
2013-10-05 11:31:12 +08:00