Commit Graph

14577 Commits

Author SHA1 Message Date
Tommi Virtanen
9b37f4fa5f Allow embedded '\0' in bufferlists when copying to std::string.
Signed-off-by: Tommi Virtanen <tommi.virtanen@dreamhost.com>
2011-06-01 09:44:55 -07:00
Sage Weil
1528d2c42b debian: depend on libboost-dev >= 1.34
for statechart.  Partially fixes #1124.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 12:58:35 -07:00
Sage Weil
0cfa911ff4 osd: don't leak Connection reference
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 09:37:34 -07:00
Sage Weil
8aa67aa402 osd: ignore old/stale heartbeat messages
If we get heartbeat messages from old epochs from peers that are not
current, drop them and mark the connection down.  Even if they are peers
we _should_ have (because we haven't gotten a notify yet to learn about
a pg we should have but don't yet) we have a newer map epoch and will learn
about them shortly, reopening the connection.

Fixes: #1107
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 09:37:31 -07:00
Sage Weil
e5c9100bca osd: fix map sharing due to heartbeats
- share the map with the cluster addr
- use the new {note,get}_peer_epoch helpers to do it sanely
- don't share if we're booting; see 818fa33a66

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 09:37:28 -07:00
Sage Weil
b3fb58ea4e crushtool: add -v verbose for --test mode
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-30 12:37:31 -07:00
Colin Patrick McCabe
7e1de380eb hadoop: track Hadoop API changes
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-27 14:46:37 -07:00
Colin Patrick McCabe
4ddf8df804 SimpleMessenger: allow multiple calls to shutdown
Fixes a case where radostool crashed on an error shutdown.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-27 14:05:27 -07:00
Colin Patrick McCabe
8490b784cf common/Thread.h: const cleanup
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-27 14:01:45 -07:00
Colin Patrick McCabe
7cfb3b6afe Merge branch 'wip-obsync' 2011-05-27 10:59:21 -07:00
Sage Weil
574b58f389 mkcephfs: pass config to osdmaptool
This lets OSDMap::create_simple() see g_conf.osd_pool_default_size when
creating the initial data, metadata, and rbd pools.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-26 21:37:03 -07:00
Sage Weil
d2ab764b34 drop useless cm.txt
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-26 21:31:18 -07:00
Sage Weil
1292436b8f osdmap: take default pool size from config
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-26 21:20:55 -07:00
Sage Weil
9b8daa92cd crushtool: update help
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 15:07:37 -07:00
Colin Patrick McCabe
6f704e3398 obysnc: rgw target: validate all users
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 14:11:04 -07:00
Colin Patrick McCabe
ae5bbc7b08 Merge branch 'wip-obsync' 2011-05-26 13:17:03 -07:00
Sage Weil
22082c4f7c mon: remove pg_temp mappings when we delete pools
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 13:17:12 -07:00
Colin Patrick McCabe
e0cbb131e1 test-obsync: test sync directly from s3->rgw
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 13:15:50 -07:00
Sage Weil
a93c86e525 crushtool: fix --add-item weight being zero when parent bucket(s) created
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 13:12:04 -07:00
Colin Patrick McCabe
56d5d95921 obsync: fix bucket creation through rgw target
The rgw: target can now create buckets. Add a test.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 11:21:23 -07:00
Sage Weil
9cefb56be8 Merge branch 'stable' 2011-05-26 11:04:03 -07:00
Colin Patrick McCabe
b2c1bff87c test-obsync: test big objects, user-defined xattr
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 10:25:40 -07:00
Sage Weil
e9eeb1610c mkcephfs: set rdir for local mon setup
Fixes: #1113
Reported-by: Bernard Grymonpon <bernard@openminds.be>
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 10:19:04 -07:00
Sage Weil
5d51b8fdb0 init-ceph: ssh
Another bell/whistle.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 09:55:37 -07:00
Colin Patrick McCabe
05cfb4d5ca obysnc: fix content-type on RGWStore
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 17:48:02 -07:00
Colin Patrick McCabe
6cf67a26ad test-obsync: compare_directory now compares xattrs
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 17:36:36 -07:00
Colin Patrick McCabe
4cae0ea8d0 ceph-pybind-test: test embedded NULLs in data
Test embedded nulls in rados data. Fix a bug in rados.Object.__str__

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 15:55:22 -07:00
Colin Patrick McCabe
a2d3529542 obsync: more fixes for RgwStore
* Fix content-type handling

* add vvprint and use it in Object::equals.

* support RgwStore::prefix

* more tests

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 15:49:10 -07:00
Colin Patrick McCabe
b76874f681 pybind/rados: correctly return data with NULLs
Correctly handle returning data with embedded NULLs in it.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 15:48:31 -07:00
Colin Patrick McCabe
970897ce84 pybind/rados.py: throw NoData on ENODATA
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 15:21:37 -07:00
Sage Weil
28c3026543 mds: fix canceled lock attempt
If client tries to lock a file, has to wait, and then cancels the attempt,
the client will send an unlock request to unwind its state.

 - the unlock now removes the waiting lock attempt from the wait list
 - when the lock request retries and finds it is no longer on the wait
   list it will fail.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-25 14:56:28 -07:00
Colin Patrick McCabe
d38001c798 pybind/rados.py: rados.Object.key should be string
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 14:22:51 -07:00
Colin Patrick McCabe
b255482354 obysnc: RgwStore: make sure destination users exist
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 12:58:39 -07:00
Colin Patrick McCabe
5d865fb6e7 obsync: fix DST_OWNER
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 12:36:37 -07:00
Yehuda Sadeh
73e28f2e8b rgw: return EACCES if acl xattr doesn't exist 2011-05-25 12:33:12 -07:00
Colin Patrick McCabe
ea76712ae3 obsync: Add boto_retries, remove rgw_store.prefix
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 12:05:15 -07:00
Colin Patrick McCabe
0adaa6b6ac rados python bindings: handle xattrs with NULL
Handle extended attributes that contain NULL bytes correctly, rather
than treating everything as zero-terminated C strings.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 11:01:16 -07:00
Samuel Just
d4bfd964fb PG: fix race in _activate_committed
Previously, _activate_committed would access the osdmap epoch racing
with handle_osd_map's osdmap update.  This would allow a message to be
sent from a replica to the primary tagged with the same epoch as
last_warm_restart, though the event actually occured before
last_warm_restart.  Thus the primary would fail to ignore the event and
transition to crashed.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-05-25 10:54:27 -07:00
Colin Patrick McCabe
7de7ba0012 RgwStore: fix some ACL issues
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 10:50:15 -07:00
Colin Patrick McCabe
3f5f562073 test-obysnc.py: support librgw testing
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 10:21:47 -07:00
Colin Patrick McCabe
e4e098ba94 Rename RadosStore to RgwStore
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 10:21:47 -07:00
Colin Patrick McCabe
17053739b4 test-obsync: refactor a little bit
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 10:21:47 -07:00
Colin Patrick McCabe
42f873e692 Proper ACL support for rados targets
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-25 10:21:47 -07:00
Sage Weil
0aa18f32e8 mds: do not shift to EXCL or MIX while rdlocked
There was an old change in file_eval() that was allowing us to switch from
SYNC to MIX or EXCL while there were rdlocks, which either caused lots of
lock thrashing or could (I think) hang things up completely.  This was
from ea10a672, an ancient fix for something related that appears to have
taken out the rdlocked check by accident.

In my tests (one writer, one stat-er), this took things from long stalls
(up to 20 seconds) to very responsive stats.  Yay!

Fixes: #791
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-25 09:38:06 -07:00
Sage Weil
fe955881a8 crushtool: clean up add-item a bit; don't add item to same bucket twice
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-24 21:14:59 -07:00
Sage Weil
dd89ff4420 crushtool: fix remove-item
Scan all buckets instead of doing a tree traverse.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-24 21:05:47 -07:00
Sage Weil
1c334d1aa7 radosgw_admin: update clitest
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-24 20:30:38 -07:00
Colin Patrick McCabe
ab01d74e91 mkcephfs.in: print out usage if no actions given
If the user didn't specify any actions, print out a usage message rather
than silently exiting.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-24 18:16:08 -07:00
Colin Patrick McCabe
f7ea7c989b rgw: Fix RGWAccess::init_storage_provider
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-24 17:53:49 -07:00
Sage Weil
c67dd164a4 mkcephfs: error out on bad usage
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-24 17:06:13 -07:00