This mirrors the c++ version, connect_cluster_pp(), and removes the
same code from create_one_pool().
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Using a test case allows us to remove a boatload of boilerplate code
in all the tests, and focus them more on what they're actually
testing.
Create one for C tests, and one for C++ tests, with the same
functionality:
- create a pool when the test case starts
- between individual tests, create an ioctx and set its namespace uniquely
- delete objects from the default namespace during individual test teardown
- delete the pool only when the whole test case is finished
In gtest, a test case is the whole set of tests declared as part of
the same class using TEST_F(). Many tests create and delete individual
pools, but this is unnecessary for independent operation in most
cases, since we can use namespaces for that now. Since pool creation
and deletion dominates test run time, using these test cases makes
running many of the tests much faster.
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
This way it doesn't have to be compiled many times, and it's easier
to add new general functionality to new files without adding those
.cc files to every test program.
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Just before sending an op, prepare_mutate_op() is called, creating a
new Op. prepare_read_op() already copied over all the out-params
correctly, but for write operations the individual op return value
pointers were not copied, so they would not be filled in. With this
fixed, librados users can get the per-op return codes again.
Partially fixes: #6483
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Define a new manifest structure. The idea is that the manifest defines a
set of rules for structuring the object parts. There are a few terms to
note:
- head: the head part of the object, which is the part that contains
the first chunk of data. An object might not have a head (as in the
case of multipart-part objects.
- stripe: data portion of a single rgw object that resides on a single
rados object.
- part: a collection of stripes that make a contiguous part of an
object. A regular object will only have one part (although might have
many stripes), a multipart object might have many parts. Each part
has a fixed stripe size, although the last stripe of a part might
be smaller than that. Consecutive parts may be merged if their stripe
value is the same.
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Prior to EC pools, unfound => missing. Now, unfound (unreadable,
really) is dependent on the PGBackend requirements for reconstituting
an object. This also means recovering an object missing on a replica
but not the primary requires tracking the missing_loc set.
Thus, rather than maintaining missing_loc only for objects missing
on the primary, the MissingLoc structure will track all missing
objects actingbackfill-wide until each object is recovered.
For simplicity, since we don't really know what objects need recovery
until activation (and since we can't do anything with that information
prior to activation anyway), we defer populating the missing_loc
information until activation.
We need peers to rollback divergent log entries before we attempt to
read the relevant objects. The simplest way to accomplish this seems to
be the simply choose to always activate peers if search_for_missing
turns up missing objects.
Due to EC pools, missing is necessary, but not sufficient for readability.
Thus, we instead check is_unreadable for cases where we need to read the object
and reserve is_missing for cases where we need the object context.
wait_for_missing_object becomes waiting_for_unreadable_object in order to avoid
having another layer of waiting_for_* maps. These ops may be requeued
either when the primary is recovered or when the object is no longer degraded,
depending on when the object becomes readable.
Signed-off-by: Samuel Just <sam.just@inktank.com>
ECBackend needs to be able to require that a readable
set of the most recent interval to write be available
in order to ensure that it rolls back the log far
enough.
Signed-off-by: Samuel Just <sam.just@inktank.com>
ECBackend's primary specific logic mostly won't treat the
primary shard specially, so it'll be handy to have primary
agnostic helpers for get_shard_info and get_shard_missing.
Signed-off-by: Samuel Just <sam.just@inktank.com>
Otherwise, we cannot correctly determine up_from/up_thru for
old intervals. Also, we need this information to determine
when a new interval starts due to a new primary without a
change in the acting set.
Signed-off-by: Samuel Just <sam.just@inktank.com>
