Commit Graph

20199 Commits

Author SHA1 Message Date
Sage Weil
8eb5c9b839 mon: allow keys to be cleared in ceph osd crush set ... loc key/values
If we encounter a 'key=' with no value, clear any previous value for that
key, so that

 'a=foo b=bar a='

is equivalent to

 'b=bar'

Fixes: #2540
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-11 15:22:35 -07:00
Yehuda Sadeh
6d8d0592d6 rgw: fix cli test
Get it up to date with latest cli changes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 15:13:18 -07:00
Yehuda Sadeh
9a1ea4baf8 rgw: usage log cleanups
Also, recalc_round_timestamp should be done under lock.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 15:09:00 -07:00
Yehuda Sadeh
5959798b25 Merge pull request #13 from fghaas/rgw-manpage
doc: update examples in radosgw-admin man page
2012-06-11 14:59:38 -07:00
Florian Haas
753207c3f2 doc: update examples in radosgw-admin man page
* "user gen" is gone and is now "user create"
* --uid option is mandatory for "user create"

Mailing list thread: http://www.spinics.net/lists/ceph-devel/msg06744.html

Signed-off-by: Florian Haas <florian@hastexo.com>
2012-06-11 23:49:11 +02:00
Yehuda Sadeh
d2b6e49c81 rgw: rgw_rados related fixes for usage logging
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 14:25:01 -07:00
Yehuda Sadeh
046395cb6a rgw/rgw_cls_api.h cleanup
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 14:00:20 -07:00
Yehuda Sadeh
9a56a5b8a7 rgw_admin: usage admin commands
- radosgw-admin usage show
 - radosgw-admin usage trim

Both commands can get --start-date, --end-date, --uid

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
744a1b31d8 rgw: usage logger
Accumulate usage info and flush it periodically.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
baa3aff437 rgw: access methods for new usage ops
Wrappers for new rados class methods, and new RGWRados
methods to handle usage functionality.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
ea2f955204 rgw: new config options
New config options for usage logging:
 - rgw_enable_usage_log: enable usage logging
 - rgw_usage_log_flush_threshold - limit on number of pending updates
                                   before synchronously flushing update
 - rgw_usage_log_tick_interval - asynchronous flush interval
 - rgw_usage_max_shards - split info across that many objects
 - rgw_usage_max_user_shards - split single user info across that many
                               objects

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
9a70ec94c9 rgw: new class methods for handling usage information
The new methods are:
 - user_usage_log_add: add new usage information
 - user_usage_log_read: get usage information
 - user_usage_log_trim: remove usage information

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:47 -07:00
Yehuda Sadeh
d265bb69fc rgw: don't fail initialization if socket path returns ENXIO
ENXIO is expected when trying to read the unix domain socket

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 12:56:04 -07:00
Sage Weil
78c35a444a Merge remote-tracking branch 'gh/wip-crush-tunables' 2012-06-11 10:36:01 -07:00
Josh Durgin
985168348f cls_rbd: add get_all_features method
This is useful for reporting which features an osd supports, and for
testing rados_exec. Update the rados api tests to use this method
instead of test_exec, which was removed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-06-11 09:58:15 -07:00
Yan, Zheng
ee7a027b9d mon: fix pg state logging
PGMap->num_pg_by_state is a PG state to number of PG in the state
mapping. PGMonitor::update_logger wrongly interprets the mapping.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-11 09:55:36 -07:00
Josh Durgin
5cd33cdd23 workunits/rbd: add workunit for running cls_rbd tests
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 22:01:19 -07:00
Josh Durgin
031d42ad1f workunits/rbd: disable remove_with_watcher test
This will fail until #2533 is fixed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 21:59:07 -07:00
Sage Weil
720aa4663f Merge remote-tracking branch 'gh/wip-rbd-format' 2012-06-10 14:12:16 -07:00
Josh Durgin
3d22546101 librbd: remove unnecessary notify from add_snap()
The only caller, snapshot_add(), already does a notify when add_snap()
succeeds.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 00:14:22 -07:00
Josh Durgin
44e5f3dc9b librbd: ignore RBD_MAX_BLOCK_NAME_SIZE when generating object ids
The actual data object ids don't need to be artificially restricted in
length.  RBD_MAX_BLOCK_NAME_SIZE just limits the size of the object
prefix, since it's used in rbd_info_t.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 00:14:03 -07:00
Josh Durgin
3d7925b1ae workunits/rbd: allow creating images in different formats
This will allow adding more arguments for testing combinations
of features in the future.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
a5026303f4 rados: add commands to interact with object maps
The input values are stored as-is, and any values read are dumped in
hex. Rename listomap to listomapkeys to distinguish from
listomapvalues. Also add it to the man page.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
123250b49d test_librbd, test_rbd.py: optionally use the new format
Read the desired features from an environment variable, RBD_FEATURES.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
074ffc5940 rbd.py: add support for creating images in the new format
The new arguments are optional, so they are backwards compatible.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
857bbccc59 rbd: update for the new format
No features exist right now, so there are no extra options for them.
The old format is still used by default, and since the default will
change with layering, --new-format will be removed at that point and is
intentionally left undocumented.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
5b40b4b94c librbd: add create2 to create an image with the new format
This will fail if features are requested that the client or server
does not support. Currently there are no features defined, so
zero is the only valid value.

copy() preserves the format and features of the source image.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
John Wilkins
b25f27705b doc: Segregated set up git from clone.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2012-06-09 14:45:05 -07:00
Sage Weil
83377755d1 debian: ceph depends on python
Due to the new ceph-disk-* scripts.  Newer dists are smart enough to figure
this out, but maverick is not.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-08 22:10:43 -07:00
Sage Weil
c526bdb5be CrushTester: drop range marking
This is subsumed by mark-down-ratio 1 with mark-down-bucket-ratio N.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-08 21:37:02 -07:00
Sage Weil
2436457924 crushtool: drop useless clitest
This is an ancient test for an old 'bug' in functionality we're removing.
Also, it is sensitive to tester output, which will be changing a lot in
the coming weeks/months.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-08 22:30:22 -07:00
Sage Weil
f2fa31b354 CrushTester: simplify, clean up mark down
- put it in a separate function
- operate on temporary weight vector, not user-modified input
- guard the whole thing with an #ifdef
- permute candidates and use first N, to ensure we end up picking the right
  number of buckets/items.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-08 22:30:22 -07:00
Sage Weil
c5042fecf3 CrushTester: some whitespace
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-08 22:30:22 -07:00
caleb miles
9b0195bf93 CrushTester: randomize mark_down behavior
When marking a batch of devices down, select which buckets to visit sort
of randomly.

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-06-08 22:30:16 -07:00
caleb miles
4921bbb9cb CrushTester: remove dead code
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-06-08 21:28:24 -07:00
caleb miles
72580769f8 CrushTester: some cleanup
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-06-08 21:28:24 -07:00
caleb miles
2deac081b7 CrushTester: mark_down_ratio and mark_down_bucket_ratio
Move functionality to allow user to control bucket vs device mark-down
probabilities independently.

Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-06-08 21:28:24 -07:00
Sage Weil
4551808fa0 mon: add 'auth get-or-create' variant that outputs a keyring
This is more convenient when you're creating daemons, since you want to
write out a keyring file for them.  This lets you do it in a single
command.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-08 17:22:10 -07:00
Josh Durgin
bfbed50886 librbd: use ImageCtx members instead of the old header in resize()
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:23 -07:00
Josh Durgin
5bc34f4a9a librbd: validate order before creating an image
The value must be passed, and it shouldn't be below 4k
(enforced by the command line tool already) or above the
range expressible in the header.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
127b425901 librbd: rename md_oid parameters to header_oid
This is more consistent with the rest of the code now,
and is a bit more clear.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
ef38394f21 librbd: make rename work with any header format
Instead of interpreting the header, just copy all the data and
omap values from the original header to the newly name one.
This will continue working with future header changes.

We can create the new header and write all data and omap values
to it atomically to avoid some races.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
f1d69639c5 librbd: use cls_client functions for calling class methods
Use the old or new methods make resize, snapshot add and snapsnhot
remove work with both old and new formats.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
8e51e06094 librbd: remove on-disk header argument from helper functions
Make most of them take the parameters they actually use.
trim_image() now takes an ImageCtx, which means remove() must
open the image. This has the nice side effect of not duplicating
the snapshot listing code for the old format.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
00f4e84153 librbd: check that the current snapid for a snap name matches
Checking that it exists doesn't prevent you from having the snapshot
change out from under you in the following situation:

You have the image open at snapshot "foo".
Someone removes snapshot "foo", writes some data to the image, and
creates a new snapshot called "foo".

This second snapshot will have a different id, but nothing prevents it
from having the name of a previously deleted snapshot.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
4eb2138e63 librbd: update ictx_refresh to work with both formats
It now sets the member variables of ImageCtx so other functions
don't have to use the on-disk header. If the features use by
the new format are incompatible with this client, an error is returned.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
7376a77364 librbd: Update ImageCtx for new format
Detect the format when an image is opened by the presence of the
original format header object. Use member variables of ImageCtx to
store image metadata instead of the on-disk header format
ImageCtx::header.

This lays the foundation for changing the rest of librbd to work with
old and new formats.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:22 -07:00
Josh Durgin
dcc7c96339 cls_rbd: add methods for interacting with the new header format
Add the client side (cls_rbd_client) for testing as well.
librbd will use the functions in cls_rbd_client to interact with cls_rbd.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 16:38:19 -07:00
Josh Durgin
70686c599b librbd: remove useless ENOMEM checks
There will be an exception if memory can't be allocated.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 14:31:49 -07:00
Josh Durgin
13aa578d63 DBObjectMap: remove extra semicolon
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-08 14:31:49 -07:00