Commit Graph

29757 Commits

Author SHA1 Message Date
Yehuda Sadeh
a68b52afa4 cls_user: keep time info about stats update
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
0b7968cda9 cls_user: header cleanup
put stats under a new struct

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
45b222930d rgw: user quota should rely on cached info
user quota should rely on cached info if it says that we're at the
limit, since backend info might not be up to date.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:51 -08:00
Yehuda Sadeh
c59a98e0b1 rgw: more quota fixes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:51 -08:00
Yehuda Sadeh
12f5801bbd rgw: configure user quota also through region map
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:51 -08:00
Yehuda Sadeh
a6fae68900 rgw: more rgw_quota fixes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:51 -08:00
Yehuda Sadeh
04b5a71116 rgw: some quota fixes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:50 -08:00
Yehuda Sadeh
cfb9553c84 rgw: enable user quota
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:50 -08:00
Yehuda Sadeh
15c01895fb radosgw-admin: user quota interface
also add needed functionality in RGWUser

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:50 -08:00
Yehuda Sadeh
a854d8937d rgw: some fixes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:50 -08:00
Yehuda Sadeh
c1c40df12b rgw: implement quota handlers for user stats
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:50 -08:00
Yehuda Sadeh
c13634e888 rgw: extend infrastructure to handle user stats read
Some helper functions that return data in required format for quota, add
async functions.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:49 -08:00
Yehuda Sadeh
364f7c54fc rgw: move some code around, no actual changes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:49 -08:00
Yehuda Sadeh
92a87b3dfd rgw: refactor quota cache
bucket quota cache is going to be reused, add a new abstract layer.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:49 -08:00
Yehuda Sadeh
4aee3fa228 rgw: rename RGWBucketStats to RGWStorageStats
make it more generic, will be reused for user's stats.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:49 -08:00
Yehuda Sadeh
5612ded3b1 rgw: pass bucket owner all around
User quota operations require that we know who the actual user we do the
operation on is. Pass that info when creating new object and when
removing objects.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:48 -08:00
Yehuda Sadeh
cbe1da85c8 radosgw-admin: bucket stats --sync-stats [--bucket=...]
Sync user stats, either for a single buckets, or for all user's buckets.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:48 -08:00
Yehuda Sadeh
c8793fbad0 cls/user: some fixes + handle old buckets case
If a bucket was created prior to the new accounting, we can't substract
the old values, as the header stats didn't account for them.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:48 -08:00
Yehuda Sadeh
eef7ba48df rgw, radosgw-admin, cls/user: retrieve user stats
add a new op to retrieve user header, new radosgw-admin command to stat
user.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:48 -08:00
Yehuda Sadeh
edcf9fed02 cls/user: a new op to retrieve user header
user header holds user total stats

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:47 -08:00
Yehuda Sadeh
438aea7f7f cls/user: accounting functionality
Keep user total accounting in omap header, handle add/dec functionality.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:47 -08:00
Yehuda Sadeh
c7b4d00815 rgw: move bucket add / remove to new user objclass
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:47 -08:00
Yehuda Sadeh
23aa65f62a rgw: replace user bucket listing with objclass call
instead of directly accessing the omap operation

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:47 -08:00
Yehuda Sadeh
248c4aefff rgw: new user objclass
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:41 -08:00
Sage Weil
8cf2a84a03 Merge pull request #893 from jdurgin/wip-init-highlander
init, upstart: prevent daemons being started by both

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-03 16:39:38 -08:00
Josh Durgin
5e34beb61b init, upstart: prevent daemons being started by both
There can be only one init system starting a daemon. If there is a
host entry in ceph.conf for a daemon, sysvinit would try to start it
even if the daemon's directory did not include a sysvinit file. This
preserves backwards compatibility with older installs using sysvinit,
but if an upstart file is present in the daemon's directory, upstart
will try to start them, regardless of host entries in ceph.conf.

If there's an upstart file in a daemon's directory and a host entry
for that daemon in ceph.conf, both sysvinit and upstart would attempt
to manage it.

Fix this by only starting daemons if the marker file for the other
init system is not present. This maintains backwards compatibility
with older installs using neither sysvinit or upstart marker files,
and does not break any valid configurations. The only configuration
that would break is one with both sysvinit and upstart files present
for the same daemon.

Backport: emperor, dumpling
Reported-by: Tim Spriggs <tims@uahirise.org>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-12-03 15:28:05 -08:00
Sage Weil
031b9ed5da Merge pull request #877 from ceph/wip-6892
rgw: don't error out on empty owner when setting acls

Reviewed-by: Sage Weil <sage@inktank.com>
2013-12-02 21:55:28 -08:00
Greg Farnum
4fdc5d9e9d workunits: use integers instead of true/false for hashpspool command
Backport: emperor

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com
2013-12-02 13:26:37 -08:00
Sage Weil
e80ab94bf4 Partial revert "mon: osd pool set syntax relaxed, modify unit tests"
This reverts commit 08327fed82, except
for the hashpspool bit.  We switched back to an integer argument in
commit 337195f046.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-12-01 22:21:31 -08:00
Yehuda Sadeh
14cf4caff5 rgw: don't error out on empty owner when setting acls
Fixes: #6892
Backport: dumpling, emperor
s3cmd specifies empty owner field when trying to set acls on object
/ bucket. We errored out as it didn't match the current owner name, but
with this change we ignore it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-11-27 13:36:38 -08:00
Sage Weil
ff60c364d3 Merge pull request #859 from ceph/wip-pr781
fix the bug  ctypes.util.find_library to search for librados failed on...
2013-11-26 15:09:18 -08:00
Josh Durgin
890bd5b3a8 Merge pull request #862 from ceph/wip-6829
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-11-26 09:06:15 -08:00
Josh Durgin
8db8ed73da Merge pull request #871 from dmick/wip-6894
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-11-26 09:02:26 -08:00
Dan Mick
c409e362c8 mon: osd dump should dump pool snaps as array, not object
Fixes: #6894
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-11-25 22:37:59 -08:00
Josh Durgin
1804e136da Merge pull request #835 from ceph/wip-6705
qa: workunits: mon: ping.py: test 'ceph ping'

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-11-25 18:03:02 -08:00
Sage Weil
fd1ec0fa90 Merge pull request #863 from ceph/wip-6804
rgw: lower some debug message

Reviewed-by: Sage Weil <sage@inktank.com>
2013-11-25 07:53:03 -08:00
Yehuda Sadeh
b35fc1bc2e rgw: lower some debug message
Fixes: #6084
Backport: dumpling, emperor

Reported-by: Ron Allred <rallred@itrefined.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-11-22 07:08:29 -08:00
Yehuda Sadeh
561e7b0b28 rgw: initialize RGWUserAdminOpState::system_specified
Fixes: #6829
Backport: dumpling, emperor
We didn't init this member variable, which might cause that when
modifying user info that has this flag set the 'system' flag might
inadvertently reset.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2013-11-22 07:06:35 -08:00
Gregory Farnum
203065c076 Merge pull request #857 from ceph/wip-6796
mon: OSDMonitor: receive CephInt on 'osd pool set' instead on CephString

Reviewed-by: Greg Farnum <greg@inktank.com>
2013-11-21 18:56:49 -08:00
Joao Eduardo Luis
b2ee9352fc PendingReleaseNotes: mention 6796 and 'ceph osd pool set' behavior change
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-11-22 02:49:28 +00:00
Joao Eduardo Luis
7c6d43cf5b doc: rados: operations: pools: document 'osd pool set foo hashpspool'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-11-22 02:49:27 +00:00
Joao Eduardo Luis
49d2fb7142 mon: OSDMonitor: don't crash if formatter is invalid during osd crush dump
Code would assume a formatter would always be defined. If a 'plain'
formatter or even an invalid formatter were to be supplied, the monitor
would crash and burn in poor style.

Fixes: 6820
Backport: emperor

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-11-22 02:49:20 +00:00
Joao Eduardo Luis
337195f046 mon: OSDMonitor: receive CephInt on 'osd pool set' instead on CephString
This partially reverts 2fe0d0d9 in order to allow Emperor monitors to
forward mon command messages to Dumpling monitors without breaking a
cluster.

The need for this patch became obvious after issue #6796 was triggered.
Basically, in a mixed cluster of Emperor/Dumpling monitors, if a client
happens to obtain the command descriptions from an Emperor monitor and
then issue an 'osd pool set' this can turn out in one of two ways:

1. client msg gets forwarded to an Emperor leader and everything's a-okay;
2. client msg gets forwarded to a Dumpling leader and the string fails to
be interpreted without the monitor noticing, thus leaving the monitor with
an uninitialized variable leading to trouble.

If 2 is triggered, a multitude of bad things can happen, such as thousands
of pg splits, due to a simple 'osd set pool foo pg_num 128' turning out
to be interpreted as 109120394 or some other random number.

This patch is such that we make sure the client sends an integer instead
of a string. We also make sure to interpret anything the client sends as
possibly being a string, or an integer.

Fixes: 6796
Backport: emperor

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-11-22 02:49:11 +00:00
Joao Eduardo Luis
7191bb2b24 mon: OSDMonitor: drop cmdval_get() for unused variable
We don't ever use any value as a float, so just drop obtaining it.  This
makes it easier to partially revert 2fe0d0d9 in an upcoming patch.

Backport: emperor

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-11-22 02:48:54 +00:00
Joao Eduardo Luis
50868a528d qa: workunits: mon: ping.py: test 'ceph ping'
Basic testing by forcing each monitor out of quorum at a time and making
sure they still reply to ping requests.

Fixes: #6705

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-11-22 01:43:06 +00:00
Josh Durgin
f753d56a9e test: use older names for module setup/teardown
setUp and tearDown require nosetests 0.11, but 0.10.4 is the latest on
centos. Rename to use the older aliases, which still work with newer
versions of nosetests as well.

Fixes: #6368
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-11-20 18:37:32 -08:00
Dan Mick
a3f4148467 Merge pull request #854 from ceph/wip-osd-bench-size
osd: fix bench block size

Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-11-19 15:41:11 -08:00
Josh Durgin
40a76ef0d0 osd: fix bench block size
The command was declared to take 'size' in dumpling, but was trying to
read 'bsize' instead, so it always used the default of 4MiB. Change
the bench command to read 'size', so it matches what existing clients
are sending.

Fixes: #6795
Backport: emperor, dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-11-18 14:39:23 -08:00
Samuel Just
703f9a09e2 Revert "JounralingObjectStore: journal->committed_thru after replay"
committed_thru can race with the sync thread.

This reverts commit d8d27f13e1.
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-11-18 12:58:27 -08:00
Samuel Just
878f3540d1 ceph-filestore-tool: add tool for fixing lost objects
Used to repair: #6761
Backport: emperor
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-11-14 21:38:38 -08:00