Commit Graph

14604 Commits

Author SHA1 Message Date
Yehuda Sadeh
a97451f6b1 librados: support clone_range 2011-06-03 11:22:11 -07:00
Yehuda Sadeh
44fe80abf7 Merge branch 'osd_clonerange' into rgw-multipart 2011-06-03 09:14:39 -07:00
Yehuda Sadeh
d4edd17cc4 rgw: multipart: use locator on created parts 2011-06-02 16:58:27 -07:00
Yehuda Sadeh
37666185a8 rgw: multipart additions and fixes 2011-06-02 15:24:11 -07:00
Yehuda Sadeh
4d03e1595b rgw: some more multipard upload list 2011-06-02 13:45:20 -07:00
Yehuda Sadeh
52bf3fce26 rgw: extend multipart list parts response 2011-06-02 13:33:49 -07:00
Sage Weil
a670b4b3c5 osd: implement clonerange
Clone ranges of bytes between objects, provided

 - src object locators match dest object
 - src objects are not missing
 - src objects are all head (CEPH_NOSNAP)
 -

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-02 13:24:09 -07:00
Sage Weil
fc4cc399f8 osd: give obc refs to RepGather
Just give the ref to RepGather instead of doing a get and put.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-02 13:24:03 -07:00
Yehuda Sadeh
b152a93c72 rgw: more cleanup 2011-06-02 10:39:18 -07:00
Yehuda Sadeh
3546cfdd54 rgw: some cleanup 2011-06-02 10:28:24 -07:00
Yehuda Sadeh
50731646a7 rgw: multipart upload parser test util 2011-06-02 09:49:30 -07:00
Yehuda Sadeh
2f3f36abe8 rgw: fix multipart upload complete parser 2011-06-02 09:39:30 -07:00
Yehuda Sadeh
711a77cfa0 rgw: multipart complete fix 2011-06-02 09:30:48 -07:00
Sage Weil
0cce0a5e99 filestore: allow clone_range to different offsets
The old OP_CLONERANGE would only clone a range of bytes at the same offset
in both objects.  Add an OP_CLONERANGE2 op code that adds a dst offset.
Continue to support the old op code so that we can decode old transactions.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-06-02 09:19:39 -07:00
Sage Weil
502baeab98 filestore: fix fallback/slow do_clone_range
We need to seek to the appropriate offsets on the src and destination
fd's for this to do the right thing.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-06-02 09:17:22 -07:00
Yehuda Sadeh
65dc8411a0 rgw: implement list multipart
still partially implemented
2011-06-01 15:13:49 -07:00
Yehuda Sadeh
5dd0e12225 rgw: handle multipart completion
still wip
2011-05-31 16:05:45 -07:00
Yehuda Sadeh
d29b3b77e6 rgw: parser for multi upload completion 2011-05-31 15:32:47 -07:00
Yehuda Sadeh
0806e65bd5 rgw: some more xml reshuffling 2011-05-31 14:59:40 -07:00
Yehuda Sadeh
f5d6be6ec9 rgw: move generic xml parsing code to some shared location 2011-05-31 14:31:57 -07:00
Sage Weil
efee7466c5 objecter, osd: clonerange operation
Add a src_oids field to MOSDOp, referenced by a new CLONERANGE osd op type
that will clone data from one object to another.

- The src_oids will need to have the same locator as the destination object
  type to ensure this operation can succeed.
- The OSD is going to have to do extra work to ensure the src object(s)
  are readable, etc., at the time of the clone.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 14:28:52 -07:00
Sage Weil
07c1989a18 librados: implement aio_flush
Implement a per-ioctx flush that blocks until all previously submitted
aio operations on the ioctx are safe.  Each aio gets a sequence number and
is put on a linked list attached to the ioctx.  The flush operation waits
for it to drain to the watermark set when flush is first called.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-31 13:58:19 -07:00
Sage Weil
6db2a4e213 crushtool: error out if uniform weights vary
Fixes: #1075
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 13:46:13 -07:00
Josh Durgin
35b19a4167 osd: fix ScrubFinalizeWQ::_clear condition
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2011-05-31 13:28:13 -07:00
Sage Weil
1528d2c42b debian: depend on libboost-dev >= 1.34
for statechart.  Partially fixes #1124.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 12:58:35 -07:00
Sage Weil
0cfa911ff4 osd: don't leak Connection reference
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 09:37:34 -07:00
Sage Weil
8aa67aa402 osd: ignore old/stale heartbeat messages
If we get heartbeat messages from old epochs from peers that are not
current, drop them and mark the connection down.  Even if they are peers
we _should_ have (because we haven't gotten a notify yet to learn about
a pg we should have but don't yet) we have a newer map epoch and will learn
about them shortly, reopening the connection.

Fixes: #1107
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 09:37:31 -07:00
Sage Weil
e5c9100bca osd: fix map sharing due to heartbeats
- share the map with the cluster addr
- use the new {note,get}_peer_epoch helpers to do it sanely
- don't share if we're booting; see 818fa33a66

Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-31 09:37:28 -07:00
Sage Weil
b3fb58ea4e crushtool: add -v verbose for --test mode
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-30 12:37:31 -07:00
Colin Patrick McCabe
7e1de380eb hadoop: track Hadoop API changes
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-27 14:46:37 -07:00
Yehuda Sadeh
232cd6b3be rgw: generate random upload id 2011-05-27 14:22:55 -07:00
Colin Patrick McCabe
4ddf8df804 SimpleMessenger: allow multiple calls to shutdown
Fixes a case where radostool crashed on an error shutdown.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-27 14:05:27 -07:00
Colin Patrick McCabe
8490b784cf common/Thread.h: const cleanup
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-27 14:01:45 -07:00
Yehuda Sadeh
a0d521b2ee rgw: fix signing for some requests 2011-05-27 13:35:47 -07:00
Yehuda Sadeh
818bfd15ce rgw: serve multipard init upload
still needs to generate a random hidden object, and use rados locator
for accessing it.
2011-05-27 12:50:27 -07:00
Colin Patrick McCabe
7cfb3b6afe Merge branch 'wip-obsync' 2011-05-27 10:59:21 -07:00
Sage Weil
574b58f389 mkcephfs: pass config to osdmaptool
This lets OSDMap::create_simple() see g_conf.osd_pool_default_size when
creating the initial data, metadata, and rbd pools.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-26 21:37:03 -07:00
Sage Weil
d2ab764b34 drop useless cm.txt
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-26 21:31:18 -07:00
Sage Weil
1292436b8f osdmap: take default pool size from config
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-05-26 21:20:55 -07:00
Yehuda Sadeh
9e8484e8ab rgw: handle POST requests for s3 2011-05-26 15:18:48 -07:00
Sage Weil
9b8daa92cd crushtool: update help
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 15:07:37 -07:00
Colin Patrick McCabe
6f704e3398 obysnc: rgw target: validate all users
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 14:11:04 -07:00
Colin Patrick McCabe
ae5bbc7b08 Merge branch 'wip-obsync' 2011-05-26 13:17:03 -07:00
Sage Weil
22082c4f7c mon: remove pg_temp mappings when we delete pools
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 13:17:12 -07:00
Colin Patrick McCabe
e0cbb131e1 test-obsync: test sync directly from s3->rgw
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 13:15:50 -07:00
Sage Weil
a93c86e525 crushtool: fix --add-item weight being zero when parent bucket(s) created
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 13:12:04 -07:00
Colin Patrick McCabe
56d5d95921 obsync: fix bucket creation through rgw target
The rgw: target can now create buckets. Add a test.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 11:21:23 -07:00
Sage Weil
9cefb56be8 Merge branch 'stable' 2011-05-26 11:04:03 -07:00
Colin Patrick McCabe
b2c1bff87c test-obsync: test big objects, user-defined xattr
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-05-26 10:25:40 -07:00
Sage Weil
e9eeb1610c mkcephfs: set rdir for local mon setup
Fixes: #1113
Reported-by: Bernard Grymonpon <bernard@openminds.be>
Signed-off-by: Sage Weil <sage@newdream.net>
2011-05-26 10:19:04 -07:00