Commit Graph

19922 Commits

Author SHA1 Message Date
Samuel Just
f822c0257e PG: track purged pgs during active
See bug #2462.

The following sequence could cause a log assuming a non-empty pg
to an empty replica:

1. primary sends query to stray
2. stray sends notify to primary
3. primary sends purge to stray removing stray from peer_info
4. stray recieves query and sends a notify
5. stray recieves purge and purges its pg
6. primary recieves notify from stray and adds it to peer_info
	note: peer_info[stray] is now wrong
7. acting set changes, primary is still primary, stray is replica
8. primary sends log to replica based on incorrect info from 6.

This patch adds a purged_peer set which is populated during purge_strays
and cleared during start_peering_interval.  The primary will ignore
notifies from the peer once the peer is in this set.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2012-06-12 13:01:57 -07:00
Sage Weil
e9390bb495 Merge branch 'wip-keyrings'
Reviewed-by: Greg Farnum <greg@inktank.com>
2012-06-12 12:48:37 -07:00
Sage Weil
4ec4b4774d ceph_argparse: make entityname parsing error more helpful
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:55 -07:00
Sage Weil
6f9ecef7ce rgw: default keyring to $rgw_data/keyring
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
ca2010ee95 vstart.sh: set data dirs instead of keyring locations
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
e958efa29a new default keyring location for daemons
$osd_data/keyring
$mds_data/keyring

Fixes: #2498
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
eeeb5993b4 mds: add mds_data config
This is where the keyring will live.  Eventually more may move in too.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
ce79f9e62e config: improve variable substitution
- allow ${foo_bar}
- prevent $foogarbage from substitution variable 'foo'
- use std::string throughout
- improve tests

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
d422bf159d config: expand any config variable in config
This is currently broken wrt 'foo' vs 'foo_bar', but otherwise works.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
500c72e8ee config: make meta expansion in set_val optional
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
72edd353f4 osdcaps: drop dead peer_type code
Not sure what this wasn intended for.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
8b5336a1f8 osd: simplify session peer type check
Check the authenticated peer name directly; don't store a copy in the
OSDCaps struct.  Also set the entity_name member of the session!

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:54 -07:00
Sage Weil
4cc122cc93 entity_name: is_*() methods
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:50:53 -07:00
caleb miles
b88a5e5344 crush: fix retry histogram array overflow due to indexing error
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2012-06-12 11:29:13 -07:00
Sage Weil
37dea2d8db cls_rgw: use updated internal omap api
These were changed by 23ebc093a6.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:28:04 -07:00
Sage Weil
6fd80d3e56 cls_rgw: fix CLS_LOG macro usage
Now takes a level.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 11:27:36 -07:00
Sage Weil
c31b4e3100 remove ceph-kdump-copy
Moved to its own repo.

Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 10:10:58 -07:00
Sage Weil
d64600dfae debian: limit build archs
We depend on leveldb, which only builds on

 amd64 armel armhf i386 ia64 mipsel

Reported-by: Laszlo Boszormenyi (GCS) <gcs@debian.hu>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-12 10:08:20 -07:00
Sage Weil
6a3ec40138 Merge remote-tracking branch 'gh/wip-2516-2'
Reviewed-by: Sage Weil <sage@inktank.com>
2012-06-11 18:36:59 -07:00
Yehuda Sadeh
a3f86b80a6 rgw: usage statistics also count num of ops/successful_ops
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 16:39:15 -07:00
Laszlo Boszormenyi (GCS)
d023cf5c35 debian: move ./configure to correct rules section
Signed-off-by: Laszlo Boszormenyi (GCS) <gcs@debian.hu>
2012-06-11 15:48:50 -07:00
Sage Weil
d63aca33f0 debian: remove dup python build-dep
Reported-by: Laszlo Boszormenyi (GCS) <gcs@debian.hu>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-11 15:48:38 -07:00
Tommi Virtanen
af16f4593b upstart: Read crush location and weight from ceph.conf.
This introduces two new config variables, osd_crush_location
and osd_crush_weight. Not currently included in config_opts.h,
as these are not used in the C++ code.

Signed-off-by: Tommi Virtanen <tv@inktank.com>
2012-06-11 15:27:02 -07:00
Sage Weil
8eb5c9b839 mon: allow keys to be cleared in ceph osd crush set ... loc key/values
If we encounter a 'key=' with no value, clear any previous value for that
key, so that

 'a=foo b=bar a='

is equivalent to

 'b=bar'

Fixes: #2540
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-11 15:22:35 -07:00
Yehuda Sadeh
6d8d0592d6 rgw: fix cli test
Get it up to date with latest cli changes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 15:13:18 -07:00
Yehuda Sadeh
9a1ea4baf8 rgw: usage log cleanups
Also, recalc_round_timestamp should be done under lock.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 15:09:00 -07:00
Yehuda Sadeh
5959798b25 Merge pull request #13 from fghaas/rgw-manpage
doc: update examples in radosgw-admin man page
2012-06-11 14:59:38 -07:00
Florian Haas
753207c3f2 doc: update examples in radosgw-admin man page
* "user gen" is gone and is now "user create"
* --uid option is mandatory for "user create"

Mailing list thread: http://www.spinics.net/lists/ceph-devel/msg06744.html

Signed-off-by: Florian Haas <florian@hastexo.com>
2012-06-11 23:49:11 +02:00
Yehuda Sadeh
d2b6e49c81 rgw: rgw_rados related fixes for usage logging
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 14:25:01 -07:00
Yehuda Sadeh
046395cb6a rgw/rgw_cls_api.h cleanup
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 14:00:20 -07:00
Yehuda Sadeh
9a56a5b8a7 rgw_admin: usage admin commands
- radosgw-admin usage show
 - radosgw-admin usage trim

Both commands can get --start-date, --end-date, --uid

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
744a1b31d8 rgw: usage logger
Accumulate usage info and flush it periodically.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
baa3aff437 rgw: access methods for new usage ops
Wrappers for new rados class methods, and new RGWRados
methods to handle usage functionality.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
ea2f955204 rgw: new config options
New config options for usage logging:
 - rgw_enable_usage_log: enable usage logging
 - rgw_usage_log_flush_threshold - limit on number of pending updates
                                   before synchronously flushing update
 - rgw_usage_log_tick_interval - asynchronous flush interval
 - rgw_usage_max_shards - split info across that many objects
 - rgw_usage_max_user_shards - split single user info across that many
                               objects

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:51 -07:00
Yehuda Sadeh
9a70ec94c9 rgw: new class methods for handling usage information
The new methods are:
 - user_usage_log_add: add new usage information
 - user_usage_log_read: get usage information
 - user_usage_log_trim: remove usage information

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 13:25:47 -07:00
Yehuda Sadeh
d265bb69fc rgw: don't fail initialization if socket path returns ENXIO
ENXIO is expected when trying to read the unix domain socket

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
2012-06-11 12:56:04 -07:00
Sage Weil
78c35a444a Merge remote-tracking branch 'gh/wip-crush-tunables' 2012-06-11 10:36:01 -07:00
Josh Durgin
985168348f cls_rbd: add get_all_features method
This is useful for reporting which features an osd supports, and for
testing rados_exec. Update the rados api tests to use this method
instead of test_exec, which was removed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
2012-06-11 09:58:15 -07:00
Yan, Zheng
ee7a027b9d mon: fix pg state logging
PGMap->num_pg_by_state is a PG state to number of PG in the state
mapping. PGMonitor::update_logger wrongly interprets the mapping.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
2012-06-11 09:55:36 -07:00
Josh Durgin
5cd33cdd23 workunits/rbd: add workunit for running cls_rbd tests
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 22:01:19 -07:00
Josh Durgin
031d42ad1f workunits/rbd: disable remove_with_watcher test
This will fail until #2533 is fixed.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 21:59:07 -07:00
Sage Weil
720aa4663f Merge remote-tracking branch 'gh/wip-rbd-format' 2012-06-10 14:12:16 -07:00
Josh Durgin
3d22546101 librbd: remove unnecessary notify from add_snap()
The only caller, snapshot_add(), already does a notify when add_snap()
succeeds.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 00:14:22 -07:00
Josh Durgin
44e5f3dc9b librbd: ignore RBD_MAX_BLOCK_NAME_SIZE when generating object ids
The actual data object ids don't need to be artificially restricted in
length.  RBD_MAX_BLOCK_NAME_SIZE just limits the size of the object
prefix, since it's used in rbd_info_t.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-10 00:14:03 -07:00
Josh Durgin
3d7925b1ae workunits/rbd: allow creating images in different formats
This will allow adding more arguments for testing combinations
of features in the future.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
a5026303f4 rados: add commands to interact with object maps
The input values are stored as-is, and any values read are dumped in
hex. Rename listomap to listomapkeys to distinguish from
listomapvalues. Also add it to the man page.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
123250b49d test_librbd, test_rbd.py: optionally use the new format
Read the desired features from an environment variable, RBD_FEATURES.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
074ffc5940 rbd.py: add support for creating images in the new format
The new arguments are optional, so they are backwards compatible.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
857bbccc59 rbd: update for the new format
No features exist right now, so there are no extra options for them.
The old format is still used by default, and since the default will
change with layering, --new-format will be removed at that point and is
intentionally left undocumented.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00
Josh Durgin
5b40b4b94c librbd: add create2 to create an image with the new format
This will fail if features are requested that the client or server
does not support. Currently there are no features defined, so
zero is the only valid value.

copy() preserves the format and features of the source image.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2012-06-09 17:24:50 -07:00