Commit Graph

19073 Commits

Author SHA1 Message Date
Josh Durgin
7197825d59 Remove unused Completion.h header
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:35 -07:00
Josh Durgin
d0b1f72cf8 ObjectCacher: add some asserts about when the lock is held
Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:35 -07:00
Josh Durgin
68e2e00410 librbd: flush cache before creating a snapshot
This is a temporary workaround until the ObjectCacher
is smarter about snapshots.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:35 -07:00
Josh Durgin
bb35ecf0f5 librbd: fix bytes read accounting in read_iterate
ObjectCacher will never do short reads, and always returns 0.
librados may do short reads at the end of an object.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Josh Durgin
579d30b617 librbd: check for writes to snapshots
librados does this for us normally, but caching does not check for this.
We might as well check early to avoid scheduling a bunch of aios anyway.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Josh Durgin
c0073770b4 librbd: allow writeback caching
This uses the existing infrastructure of ObjectCacher for
buffer management and expiry.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Josh Durgin
9546c15898 librbd: remove writeback window
This is superseded by a full-fledged writeback cache.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Josh Durgin
d6dbff01a3 ObjectCacher: remove dependency on Objecter
Abstract out how writeback is done with a WritebackHandler object.
For RBD caching, this will be done by librados, but the Client uses
the Objecter directly.

This also requires different locks, since librbd does not have access
to the lock the underlying Objecter uses. Thus, both lock and the
writeback handler are parameters of the ObjectCacher constructor.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Josh Durgin
abbfd93782 Filer: make file_to_extents static
Now the ObjectCacher has no need for a Filer instance (not that
it had a good reason for one before).

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Josh Durgin
cbbd5f97ca ObjectCacher: check for read/write errors
The objecter doesn't fix every error for us.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2012-04-13 20:46:34 -07:00
Sage Weil
34a10fae86 librados: do aio callbacks in async thread
Call user completions in an async thread.  This allows callers to call back
into librados from the callback, and allows them to take locks in their
callbacks that they hold when queuing requests (making their life much
easier).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:46:34 -07:00
Sage Weil
a0fc3336fa librados: unconditonally associate IoCtxImpl with AioCompletionImpl
We were only setting the pointer if we were on the aio_write_list.  Set
this pointer for any aio.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:46:34 -07:00
Sage Weil
cc1ae672f5 filestore: flush log on failure injection
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:41:35 -07:00
Sage Weil
ad2eba4370 Merge branch 'wip-2226b'
Conflicts:
	src/common/config_opts.h
	src/os/FileStore.cc
	src/os/FileStore.h
2012-04-13 20:41:17 -07:00
Sage Weil
f052c82b5d run_seed_to: assume tool is in path, not cwd
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:38 -07:00
Sage Weil
4fc48e5b97 run_seed_to_range.sh: run filestore idempotent sequence tests over a range
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:38 -07:00
Sage Weil
a8b9b98c36 run_seed_to.sh: set -e
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:25 -07:00
Sage Weil
95bb4239e2 filestore: use our assert
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:25 -07:00
Sage Weil
c77298471f test_idempotent_sequence: simple test script
Takes seed and kill point, then verifies the result.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:25 -07:00
Joao Eduardo Luis
2cb0032e13 DeterministicOpSequence: writing to object being cloned in the same tx.
We write to the object being cloned prior to the clone to ensure we are
cloning a valid range of bytes.

The write and the clone were being done in two distinct transactions,
which would trigger a diff mismatch if a failure happened to occur within
the write tx. By not differentiating the transactions when building a
pristine copy, we were executing one more transaction (the clone_range one)
than the transactions that were executed in the failed filestore, thus
triggering the mismatch (one more object in the pristine filestore than on
the failed filestore).

Now we issue a single transaction, containing both the clone_range()
preceded by a write().

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:25 -07:00
Joao Eduardo Luis
bb0499a413 test_idempotent_sequence: outputting collection being diff'ed.
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:24 -07:00
Joao Eduardo Luis
21be4ad92a test_idempotent_sequence: Add more info on diff's dout's.
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:24 -07:00
Samuel Just
2d24eb33f9 FileStore: don't return ENOENT from object_map getters
ENOENT in those contexts means that the object_map does not know
about the object, not that the object does not exist.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-04-13 20:38:24 -07:00
Sage Weil
56fb027f2a test_idempotent_sequence: make get-last-op return 0 if no txns applied
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:24 -07:00
Sage Weil
2d20b4c21a test_idempotent_sequence: return true from diff if different
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:23 -07:00
Sage Weil
08cf4d2825 test_idempotent_sequence: include object name in diff output
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:23 -07:00
Sage Weil
a5f533a7ed test_idempotent_sequence: fix diff return value, mount behavior
No mkfs!

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:23 -07:00
Sage Weil
1b451294e8 test_idempotent_sequence: fix/enable do_coll_add
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:23 -07:00
Sage Weil
27cb82e507 test_idempotent_sequence: use get_obj_at() for clone source
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:23 -07:00
Sage Weil
794fc0b69a test_idempotent_sequence: fix collection_move arg order
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:22 -07:00
Joao Eduardo Luis
84175c7d88 VerifyFileStore: delete from tree
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:22 -07:00
Joao Eduardo Luis
4289911006 Makefile: remove VerifyFileStore from test_idempotent_sequence build
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:22 -07:00
Joao Eduardo Luis
2c940a4f99 test_idempotent_sequence: Remove references to VerifyFileStore
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:22 -07:00
Joao Eduardo Luis
e270e0b533 test_idempotent_sequence: diff two stores.
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:22 -07:00
Sage Weil
5d30581e01 test_idempotent_sequence: add remove op
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:22 -07:00
Sage Weil
bec1e50e75 test_idempotent_sequence: remove existing target for collection_add
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:21 -07:00
Sage Weil
db3d14b944 test_idempotent_sequence: fewer objects per collection by default
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:21 -07:00
Sage Weil
51bc6dfd8e test_idempotent_sequence: make object names overlap
This way when we move objects between collections they will replace
each other in interesting ways.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:21 -07:00
Sage Weil
c056f60380 test_idempotent_sequence: simplify clone
- pick existing source
- any target is ok, as long as it != source

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:21 -07:00
Sage Weil
bc5dffe9c0 test_idempotent_sequence: replace on coll_move
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:21 -07:00
Sage Weil
a8c320571b test_idempotent_sequence: whitesepace
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:21 -07:00
Sage Weil
5786694662 test_idempotent_sequence: implement get-last-op
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:20 -07:00
Sage Weil
795274bc0f test_idempotent_sequence: include tx number in each transaction
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:20 -07:00
Sage Weil
9dba3172b3 test_idempotent_sequence: require nonexistent filestore dir
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:20 -07:00
Sage Weil
5ced86becb filestore: init filestore_kill_at in ctor
Otherwise we don't get the option for FileStore instances created after
common_init_finish() (which does md_config_t::call_all_observers()).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:20 -07:00
Joao Eduardo Luis
975e59ad0d test_idempotent_sequence: Add commands and lose a couple of optional args.
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:20 -07:00
Sage Weil
a5a8395069 filestore: name internally
We need to allow the perfcounter name to be controlled so that we can have
two instances of FileStore in the same process that don't step on each
other.  Default to 'filestore'.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:19 -07:00
Sage Weil
01daa88d8b test_idempotent_sequence: no need to reinject value that is already there
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-13 20:38:19 -07:00
Joao Eduardo Luis
4a6cfcb31c deterministicopseq: add collection_rename() support 2012-04-13 20:38:19 -07:00
Joao Eduardo Luis
dee47bb203 test_idempotent_sequence: Generate a reproducible sequence of txs.
With this test we aim at reproducing the same sequence of transactions
as long as we are provided with the same seed between runs.

We also allow failures to be injected onto the filestore if the
--filestore-kill-at <VAL> argument is passed, and we provide verification
when --test-verify-at <VAL> is provided.

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-13 20:38:19 -07:00