Commit Graph

31357 Commits

Author SHA1 Message Date
Derek Yarnell
64a0b4fa56 packaging: apply udev hack rule to RHEL
In the RPM spec file there is a test to deploy the uuid hack udev rules
for older udev operating systems. This includes CentOS and RHEL, but the
check currently only is for CentOS, causing RHEL clients to get a bogus
osd rules file.

Adjust the conditional to apply to RHEL as well as CentOS. (The %{rhel}
macro is defined in both platforms' redhat-rpm-config package.)

Fixes http://tracker.ceph.com/issues/7245

Signed-off-by: Ken Dreyer <ken.dreyer@inktank.com>
2014-01-27 12:32:51 -07:00
Yehuda Sadeh
cacdfd9165 Merge branch 'wip-user-quota-2'
Conflicts:
	src/common/config_opts.h
	src/rgw/rgw_common.h
	src/rgw/rgw_json_enc.cc
	src/rgw/rgw_quota.cc
	src/rgw/rgw_rados.cc
	src/rgw/rgw_rados.h
	src/rgw/rgw_user.cc
	src/rgw/rgw_user.h

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-27 11:29:14 -08:00
Li Wang
9265d76a25 client: Avoid uninline empty inline data
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:25:00 +08:00
Li Wang
b99e9d3b51 mds: Handle client compatibility
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:25:00 +08:00
Li Wang
ddbaa5c22f mds: Shutdown old mds when inline enable
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:25:00 +08:00
Li Wang
e3c4c4cd65 mds: Add inline switch
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:25:00 +08:00
Li Wang
81c4f54e0a client: Make inline size tunable
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:25:00 +08:00
Li Wang
40a693918c client: Fallocate inline data path
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:25:00 +08:00
Li Wang
47e6a78207 client: Write inline data path
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
a0cb524251 client: Read inline data path
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
4a5c09909d client: Add inline data migration helper
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
165ddb9ad7 client: Push inline data to mds by send cap
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
3c05e80983 client: Receive inline data pushed from mds
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
be3db0b8f4 client: Add inline fields to Inode
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
45c2ab4580 mds: Receive updated inline data from client
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
1280d99eab mds: Push inline data to client in inodestat
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:59 +08:00
Li Wang
7af7a5e1d5 ceph: Add inline fields to InodeStat
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:58 +08:00
Li Wang
cf63f7e6ea mds: Push inline data to client in cap message
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:58 +08:00
Li Wang
3aa7f0bcab mds: Add inline fields to Capability
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:58 +08:00
Li Wang
0f00f07d64 osdc: Add write method with truncate parameters
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:58 +08:00
Li Wang
ac5c9dd3f7 ceph: Add inline fields to MClientCaps
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
2014-01-27 15:24:58 +08:00
Sage Weil
715586fed7 Merge pull request #1143 from dachary/wip-vstart-disk-usage
unittests: fail early when low on disk

Reviewed-by: Sage Weil <sage@inktank.com>
2014-01-26 12:14:29 -08:00
Loic Dachary
c1c1aa9157 Merge pull request #1003 from dachary/wip-handle-forward-unittest
mon: test for MForward messages

Reviewed-by: Joao Eduardo Luis <joao@inktank.com>
2014-01-26 05:25:36 -08:00
Loic Dachary
fffbfc942e mon: test for MForward messages
Creates two mons, send a mon command to the peon and check that the
command is forwarded to the leader.

Add a test to check that the connection features of the client are
preserved when the message reaches the leader.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-26 13:40:53 +01:00
Loic Dachary
d9a689d69a mon: shell test helpers to run MONs from sources
The intent is to make it more convenient to reproduce a specific mon
behavior and observe the result by grepping the logs. It can be handy
for bug diagnostic. The test could be included as a unit test to
be run on make check.

The setup function will prepare a directory and kill leftover from a
previous run. The teardown function cleans up on success. The run
function is expected to be provided by the calling script and can make
use of the run_mon function to mkfs + run a monitor.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-26 13:40:53 +01:00
Loic Dachary
c06eca2448 unittests: fail early when low on disk
Scripts from qa that are run as unittests via test/vstart_wrapper.sh may
fail because the partition on which it runs is low on space ( 95% full
). When it happens the cause of the problem may be unclear because it
is likely to show only as a client not being able to reach the mon.

A test is added in test/vstart_wrapper.sh to verify the disk space usage
using the same method as the mon would and fail with a detailed error if
it is the case.

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-01-26 13:32:57 +01:00
Yehuda Sadeh
9413a5196e rgw: simplify a code path
remove some duplicate code.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:33:57 -08:00
Yehuda Sadeh
1b0567b0af radosgw-admin: add usage for user stats [--sync-stats]
add missing usage

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:25:51 -08:00
John Wilkins
97edd2fcad doc: Switched logging to true. Now required for sync agent.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-01-24 15:11:32 -08:00
Yehuda Sadeh
9a55fa1cac cls_user: add generate_test_instances() implementation
to all the encoded/decoded stuff

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:07:36 -08:00
Yehuda Sadeh
e5dc8d6583 rgw, cls_user: handle error cases related to response decoding
Certain operations weren't handling errors in decode, add some missing
logic.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:07:36 -08:00
Yehuda Sadeh
8a69ac89e2 rgw: implement restful set user quota request
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:07:36 -08:00
Yehuda Sadeh
92cabd411c rgw: new restful api for retrieving user quota info
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:07:36 -08:00
Yehuda Sadeh
0f7b3c722f rgw: quota thread for full user stats sync
Get user stats up to date periodically. Add configurables for different
periods, whether we update idle users.
Make sure radosgw-admin does not start the quota threads.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 15:07:35 -08:00
Greg Farnum
51ffede3e3 mon: do not use CEPH_FEATURES_ALL for things that touch the disk
We want to encode with our quorum_features instead. Remaining uses of
CEPH_FEATURES_ALL are:
1) when the Elector is sharing its supported features
2) in a MonMap function which is used by monmaptool
3) In the Monitor for winning a standalone election, for ephemeral data,
  and for doing mkfs (when we necessarily don't have quorum_features).
4) When doing ceph-mon --inject-monmap (we don't persist the quorum_features
  to disk, so we can't use them here).
5) in MMonElection, for doing the default monmap encoding (which is
  re-encoded later if the final features don't match CEPH_FEATURES_ALL).
6) As the default encode features for the OSDMap (the monitor always
  supplies quorum_features instead).

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-24 12:33:56 -08:00
Greg Farnum
3d4a6739f2 Elector: send an OP_NAK MMonElection to old peers who support it
Only new monitors support receiving OP_NAK from a peer without crashing, but
when we add new required features in the future, our monitors can accept
an OP_NAK message which tells them what features they're missing. Then they
will print out an error message and shut down.
(Unfortunately, doing a clean shutdown from here would require a lot of
infrastructure, so we just call exit(0).)

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-24 12:33:56 -08:00
Greg Farnum
687b570b71 Elector: ignore messages from mons without required feature capabilities
We maintain a list of required_features which the other monitor's features
must supply. This starts out at 0 and is initialized from the monitor's
list of features whenever we start electing.
Despite the scary sound of "just ignore it", this is safe: the monitor
will only record features as required once a quorum has formed in which
every monitor supports them. After that happens, monitors which do not
support those features will be unable to read the whole mon store/understand
the pg reports/whatever else, so letting them into the quorum would be buggy
behavior.
So if we ignore a monitor, it will not be able to start nor join
an election round with anybody who was in our quorum -- that is, the
ignored monitor cannot form a separate quorum. By ignoring it here, we
also prevent it from endlessly calling elections against the real
quorum.

Unfortunately there is no way to communicate to old monitors that they
cannot join the quorum -- there are no existing messages for that purpose,
and eg adding a new op to the MMonElection message will just cause it
to crash, which we don't want to do either.

Fixes: #7215

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-24 12:33:56 -08:00
Greg Farnum
41796c07ac Monitor: add a function to get the required quorum features from the local compatset
We will use this to avoid deferring to people who can't read our store.

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-24 12:33:56 -08:00
Greg Farnum
f8e4af2ddb OSDMonitor: do not allow creation of EC pools if the quorum can't handle it
Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-24 12:33:56 -08:00
Greg Farnum
e79e1d28a6 Monitor: introduce a function that translates quorum features into disk features
After an election, we call apply_quorum_to_compatset_features(). It translates
from the quorum features into monitor disk state features we care about and
adds them to the disk store. This prevents an older daemon from starting up
on a store it doesn't understand.

While strictly speaking we don't need to add the EC feature until we create
an EC pool, all monitors which speak the new OSDMap encoding also support EC
pools, and having more than one feature makes the pattern clearer.

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-01-24 12:33:55 -08:00
Yehuda Sadeh
2f9a93d385 rgw: round bytes to the next 4k for quota use
Be more consistent as we round it like this in the objclass, so we need
the accounting to match.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:53 -08:00
Yehuda Sadeh
6f34df45ed rgw: sync bucket stats thread
a new thread that periodically sync stats of recently modified buckets.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:53 -08:00
Yehuda Sadeh
7545177ab7 rgw, cls_user: fix bucket creation
There's a single op to create and update the user bucket info, however,
the cases differ a bit, as we only need to guard against ENOENT if we're
updating the info.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:53 -08:00
Yehuda Sadeh
d9920439e2 radosgw-admin: dump header when requested for user stats
header also contains sync timestamps, so we should dump it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
5976bfcc58 radosgw-admin: update stats sync completion
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
ba39512892 cls_user: op to keep track of sync completion time
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
a68b52afa4 cls_user: keep time info about stats update
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
0b7968cda9 cls_user: header cleanup
put stats under a new struct

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:52 -08:00
Yehuda Sadeh
45b222930d rgw: user quota should rely on cached info
user quota should rely on cached info if it says that we're at the
limit, since backend info might not be up to date.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:51 -08:00
Yehuda Sadeh
c59a98e0b1 rgw: more quota fixes
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2014-01-24 10:28:51 -08:00