Commit Graph

15315 Commits

Author SHA1 Message Date
Yehuda Sadeh
43575c7ac7 rgw: fix concurrency issue 2011-07-19 14:02:11 -07:00
Yehuda Sadeh
51039da1a6 rgw: use thread pool for rgw work 2011-07-18 17:14:25 -07:00
Yehuda Sadeh
a7d791de55 rgw: multithreaded handling 2011-07-15 16:46:01 -07:00
Yehuda Sadeh
02abdca1f6 rgw: multiple fixes, prepare main loop for multithreading 2011-07-15 12:54:32 -07:00
Yehuda Sadeh
5d606c22bd rgw: fix log 2011-07-14 15:55:05 -07:00
Yehuda Sadeh
0593001d83 rgw: cleanup, remove globals and static craft 2011-07-14 15:29:36 -07:00
Yehuda Sadeh
0cfaee69ba Merge branch 'rgw-sync-cache'
Conflicts:
	src/rgw/rgw_op.cc
2011-07-14 12:45:07 -07:00
Yehuda Sadeh
ba31462a2c rgw: configurable way to enable/disable cache 2011-07-14 12:43:25 -07:00
Samuel Just
8a8c1e815c ReplicatedPG: project changes to clone_overlap
Previously, changes to clone_overlap were incorrect since make_writeable
is called after do_osd_ops.  Now, ctx->modified_ranges will hold the set
of offsets modified during the transaction to be applied to
clone_overlap during make_writeable.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-14 11:22:31 -07:00
Sage Weil
279a6ff672 Merge branch 'wip-codingstyle' 2011-07-14 10:58:58 -07:00
Sage Weil
a9f815b554 CodingStyle: whitespace
Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-14 10:50:08 -07:00
Sage Weil
bc6eb10586 CodingStyle: final decisions?
- _d suffix for naked struct/class types (not _t!)

- m_ prefix for class members

- prefer braces for single line ifs.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-14 10:49:33 -07:00
Samuel Just
357db1d0c5 ReplicatedPG: sub_op_push fix
We need to iterate over the intersection of data_subset and data_needed,
not the original data_needed.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-13 17:07:37 -07:00
Sage Weil
3d1457908d Merge remote branch 'origin/wip-1257' 2011-07-13 16:47:07 -07:00
Colin Patrick McCabe
143e2dd5fe Don't build build tests unless requested
Build tests (that check if there are unresolved symbols in libraries)
can slow down the build a lot. We should only enable them when
developers need them.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 16:12:40 -07:00
Colin Patrick McCabe
2de0156c5d rados parallel tests: prepend "test"
Just to avoid any confusion, all these test apps should begin with the
word "test".

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 16:01:29 -07:00
Colin Patrick McCabe
f4ea3312fa rados_list_parallel: add test of list-while-modify
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 15:58:19 -07:00
Yehuda Sadeh
2d757ac056 rgw: unwatch cache sync object before exiting 2011-07-13 15:56:05 -07:00
Colin Patrick McCabe
8e8bd67cdd testrados: fix
Change testrados back to a simple single-threaded test. The
multi-threaded tests need to be properly designed to run multi-threaded.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 15:43:09 -07:00
Colin Patrick McCabe
b7d12e5ea4 systest_runnable: print line when joining runnables
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 15:42:53 -07:00
Colin Patrick McCabe
a32bc89436 Create rados_delete_pools_parallel test
* add rados_delete_pools_parallel test

* change RETURN_IF_NOT_VAL -> RETURN1_IF_NOT_VAL.
We want to return a non-zero error code when the value is something we
don't expect, even if that unexpected value is 0.

* st_rados_list_objects: add option to ignore list errors (for the
deletion test)

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 15:25:54 -07:00
Samuel Just
585fab7c5b ReplicatedPG: data_subset may not contain data_needed
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-13 15:12:31 -07:00
Samuel Just
44ea232a7b encoding.h: need to pass by reference
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-13 15:12:31 -07:00
Samuel Just
5869591b82 ReplicatedPG: fix snaps decode in recover_primary
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-13 15:12:31 -07:00
Samuel Just
a8dc9a01c2 test/osd: TestSnaps now writes large, random objects with holes
In order to test clone recovery pathways, TestSnaps now writes to random
subsets of an object between snapshots.  Objects may now also be very
large.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-07-13 15:12:31 -07:00
Sage Weil
aec77962f2 mds: fix handle_simple_lock on LOCK replica
We were ignoring caps.  To avoid this kind of mistake, use eval_gather(),
like handle_file_lock() does.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-13 14:57:57 -07:00
Sage Weil
1e31961277 mds: drop useless from field from MInodeFileCaps
While we're updating the protocol, clean this up too.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-13 14:57:57 -07:00
Sage Weil
2539d16a19 mds: print cur+nested auth_pins on inode
And clean up a bit.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-07-13 14:57:57 -07:00
Sage Weil
a078aee139 mds: move to MIX state if writer wanted and no wanted loner
We can just look at the target loner here, which also takes any caps wanted
by other replicas on other MDSs into account.  Otherwise we need to
to duplicate the CInode::calc_ideal_loner() logic.

This assumes the loner field is accurate.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-07-13 14:57:57 -07:00
Sage Weil
7fe5bb53d4 mds: print mds_caps_wanted on inode
along with the client caps info

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-07-13 14:57:57 -07:00
Sage Weil
314be34081 mds: improve print() for MInodeFileCaps
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-07-13 14:57:57 -07:00
Sage Weil
a928f767b9 mds: migrate loner_cap state
It is tedious to infer what the old loner_cap was pre-migration. Just send
it over the wire and set it explicitly. Usually when we eval() we would
have come to the same conclusion, but when we didn't, we got into
inconsistent/impossible states where the issue caps don't match the loner
state (Asx issued but no loner_cap set).  That meant the next issue was
a revocation with no lock state change, which led to yet more problems
down the line.

This is an internal MDS protocol change.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-13 14:57:57 -07:00
Sage Weil
f0409c45a4 mds: clear [want_]loner_cap on cap export
These fields are meaningless on the replica.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-07-13 14:57:57 -07:00
Colin Patrick McCabe
16bedd2f85 Move list objects logic to st_rados_list_objects
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:50 -07:00
Colin Patrick McCabe
9b83072f07 rados_list_parallel: fix rados_ioctx_destroy uses
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:50 -07:00
Colin Patrick McCabe
0c4f937e88 rados_list_parallel: add test4
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:50 -07:00
Colin Patrick McCabe
6d7c86a499 rados_list_parallel: test adding objects + listing
Test one runnable adding objects while another runnable lists them.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:50 -07:00
Colin Patrick McCabe
a360b7dd12 Add rerun.sh
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:50 -07:00
Colin Patrick McCabe
b9c0a728de rados_list_parallel: make num_objects selectable
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:50 -07:00
Colin Patrick McCabe
2b95a614f9 add CrossProcessSem::reinit, fix rados_list_parallel
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:49 -07:00
Colin Patrick McCabe
27fbe8be78 Now we can put runnable logs into separate files
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:49 -07:00
Colin Patrick McCabe
cccdd43b38 Add rados_open_pools_parallel test
Also add rados_open_pools_parallel to .gitignore

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 14:40:40 -07:00
Yehuda Sadeh
aaa1a02275 rgw: clean up temp object if failed to PUT
Conflicts:

	src/rgw/rgw_op.cc
2011-07-13 14:16:21 -07:00
Yehuda Sadeh
4089e4d11e rgw: clean up temp object if failed to PUT 2011-07-13 14:08:38 -07:00
Yehuda Sadeh
5983604453 rgw: remove include of curl/types.h 2011-07-13 12:23:47 -07:00
Colin Patrick McCabe
cb869782d2 test/rgw_escape: add JSON + utf-8 test
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 11:47:24 -07:00
Colin Patrick McCabe
e8b612571e test/rgw_escape.cc: add utf-8 test
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-13 11:46:02 -07:00
Tommi Virtanen
f29b9bd7de CodingStyle: Typos, whitespace. 2011-07-13 11:25:22 -07:00
Yehuda Sadeh
6bca619f4b rgw: fix escaping clobbering utf8 2011-07-13 10:46:39 -07:00
Sage Weil
8af8ec850e Revert "mds: handle importer failure on exporter"
This reverts commit bf768cb146.

This won't actually happen with f44cffdb8c
2011-07-13 09:26:54 -07:00