Rather than putting all of RGW into librgw, only put rgw_acl.cc. Have
RGW use librgw instead of re-including the same sources.
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
The DST_CONSISTENCY variable allows us to specify that the destination
is expected to use read-after-write consistency. If that is the case, we
don't have to do slow retries if certain operations fail.
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
boto_tool now grabs the configuration variables it needs from the
s3-tests config file, similar to s3-tests and test-obsync.
Also, changed argument parsing to be more straightforward.
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
Use dout for logging so that the librgw library user can turn off or
redirect the logs if necessary.
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
Unlike most of the cap options, max_size is an inode member. This meant
that if we got a shared cap grant from a replica MDS, we would set
the max_size to 0!
This caused hangs because when the client would request a new, larger
max_size from the auth MDS then the auth MDS would see the new size
as being smaller than the current max, and drop the message as
being spurious.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
We missed this case: we can be auth for the inode being moved without
being auth for the srcdn (first case) or owning a clobbered inode
(last case).
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
Previously we wanted to journal if we had caps on something. Now
that we're being strict about only journaling stuff we're auth for,
that's a bad choice to make.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
This ensures that we will send a response to the MDS letting it know
that we've revoked our caps.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
This should prevent us from "losing" caps off the dirty list. See
#1063. If we have dirty caps we don't want to short-circuit out
of sending caps just because what we're issued matches what we want.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
For each pg, start out with start_epoch = 0 in the first request. For
subsequent requests, set it to the first reply's epoch. This forces the
OSD to ignore our cookie and "restart" if the pg mapping changes and there
is a possibility of incomplete results.
The price we pay is the possibility of duplicate results.
Fixes: #1030
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
If the pgls.start_epoch is set, the cookie is only considered valid if the
osd pg interval has not changed since then. If it has, then the cookie
is no longer valid and is ignored, effectively restarting the pgls process.
Old clients never set this and are unaffected.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
This will prepend any missing objects to the set of objects returned by
a sequence of PGLS operations. Because recovery can progress in parallel,
we may get some objects returned twice (first as a missing item, later
because it is on disk). This is better than not getting it at all. The
client will need to uniq the results as needed.
Because the missing set is guaranteed not to grow over a given mapping
interval only, the client should restart the whole PGLS sequence if the
pg primary changes.
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
This switches the key to the uint64_t (version_t) only, which is still
unique given a particular timeline (which is all we care about given a
particular Missing::missing). The last_requested pointer is updated
accordingly.
This will facilitate a hack to make PGLS work for degraded PGs (by using
the rmissing version offset as a cookie).
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
There were some odd pieces that are artifacts of a very old and
different use. Remove them to simplify the interface and be
less confusing.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
Fix the obsync unit tests to take into account the new ACL changes.
ACLs must be either translated or ignored when copying between the
main and alt test buckets.
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
This reverts commit a7f87965f5.
This commit breaks at least how Filer does zeroing. We
should redo the commit with some audits for what else
will break.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
Maintain an explicit Connection handle to send messages and mark_down old
monitor connections. Ignore any incoming message that is not part of that
session. This fixes problems with incoming messages that race with
session restarts.
Fixes: #1033
Reported-by: Wido den Hollander <wido@widodh.nl>
Signed-off-by: Sage Weil <sage@newdream.net>
* integrate rados_sync with rados_tool
* Improve rados tool usage a bit
* Rename test_rados_sync.sh to test_rados_tool.sh
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
