Commit Graph

819 Commits

Author SHA1 Message Date
John Mulligan
1f02b52df2 cephfs admin: update test helper functions to work on ceph "pacific"
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-30 15:23:40 +02:00
John Mulligan
52ef909e2b workflows: enable running ci against ceph pacific
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-30 15:23:40 +02:00
John Mulligan
e76404c6aa readme: document v0.9.0 and the ceph versions it supports
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-23 13:34:44 +00:00
John Mulligan
e5accf5dfc readme: no more need to document cli args needed to run fuse
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-23 13:07:21 +00:00
John Mulligan
90e5af86ef containers: drop the unused rbd-fuse from the install packages
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-23 13:07:21 +00:00
John Mulligan
7c19066bed makefile: remove the use of fuse from multi container tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-23 13:07:21 +00:00
John Mulligan
cf29cdbb27 entrypoint.sh: replace uses of rbd-fuse with rbd command line tool
Replace uses of the rbd-fuse virtual file system with command line
invocations of `rbd import` and `rbd export`. This allows us to drop
the dependency on rbd-fuse (and fuse entirely, again). Fuse requires
additional privs and special stuff so not using it makes running the
container simpler and more secure.

The only downside that I see to this change is that `rbd import` refuses
to overwrite existing images. This means the logic of the setup
function needed to be slightly changed but I don't consider it all
that different.

I did also slightly decrease the size of the image to ~500K. I didn't
think we needed to be copying so much zeros over the wire nor did
mirroring a lot of zeros demonstrate much IMO.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-23 13:07:21 +00:00
John Mulligan
312e4cf6b5 Makefile: allow passing arguments to the test container
Allow passing additional arguments for the entrypoint.sh script to
the test container. This is useful to limit the test run to
a single package with `--test-pkg=<X>` for example.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-19 14:01:58 -04:00
John Mulligan
fc9bd71eb3 Makefile: workaround for running dual containers with podman
When running the new test-multi-container make rule, I found that
podman generated "random" hex based hostnames for the containers.
These names work internally and do resolve but I found that putting
these in the ceph config files does not work as ceph sees them as
addresses, not hostnames, presumably due to the hex.
This change simply specifies the hostnames manually on the cli
so that both podman and docker should match docker's default behavior.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-19 14:01:58 -04:00
John Mulligan
40e1224871 rbd: add tests for GetGlobalMirrorStatus based on two ceph clusters
With the infrastructure for running two ceph clusters now available,
we add tests to ensure the functionality of GetGlobalMirrorStatus in
the more typical case, when one or more mirroring clusters exist,

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-13 09:47:43 +00:00
John Mulligan
de9505f862 rbd: add a test helper to get a conn for a given cluster by config
This function can be used to connect to a non-default ceph cluster,
in a simple, convenient way similar to the function we have for
connecting to the default cluster.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-13 09:47:43 +00:00
John Mulligan
f84140356a rbd: add CreateMirrorSnapshot implementing rbd_mirror_image_create_snapshot
This function is used to "manually" trigger a snapshot for snapshot
based mirroring. It is needed as a pre-req to test actual cross-cluster
mirroring and the full functionality of GetGlobalMirrorStatus.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-13 09:47:43 +00:00
John Mulligan
0946b9e0e2 entrypoint.sh: export MIRROR_CONF for use by go test code
Later changes to the test Go code will make use of the MIRROR_CONF
variable to access the secondary ceph cluster.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-13 09:47:43 +00:00
John Mulligan
5c537e20ad rbd: add GetGlobalMirrorStatus implementing rbd_mirror_image_get_global_status
This function is added, along with required return types and some helper
functions, to support fetching the global mirroring status component
that makes up a major part of `rbd mirror image status` command's
output.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-13 09:47:43 +00:00
John Mulligan
dfa74510aa cephfs admin: replace mon and mgr command helpers with common versions
Remove the logic previously ported over to internal/commands and just
leave the functions in place as to not have to excessively touch the
existing code.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan
55a5732b3f cephfs admin: replace "command tracer" with common version from commands
Previously, this functionality was taken into commands package. Use
that.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan
20b3c151b3 cephfs admin: replace response with commands.Response
Now that we have shared support working with "command json" we attempt
a minimal conversion, using aliases where appropriate, to the
new commands.Response type.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan
0b1e3f5d7e commands: port tracing helper type from cephfs/admin
This debugging code can be made common.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan
6b3b0fdc46 commands: add a test file for new shared Response type
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan
93676ab094 commands: port response type and related to commands pkg
Convert the response type from cephfs/admin to the new package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
John Mulligan
42e9904d0b commands: start a new internal sub-package for working with json commands
This code was part of cephfs/admin but will be shared with a soon-to-be
rbd/admin package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-04-07 00:25:44 +00:00
Sven Anderson
0cc1dcb90d ci: add optional mirroring setup entrypoint.sh
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
Sven Anderson
b72758be55 ci: add multi-container test to workflows
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
Sven Anderson
9bd143e06e ci: add test-multi-container Makefile target
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
Sven Anderson
362929f8d1 ci: add wait option to entrypoint script
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
Sven Anderson
3fb003d886 ci: prepare micro-osd.sh for cross-container connections
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-04-06 08:59:55 -04:00
Sven Anderson
014a3f6308 ptrguard: assert pointer and uintptr have same size
Uintptr are guaranteed to be at least of the size of a pointer.  However, in
theory it could be larger.  Since the code of PtrGuard assumes both have the
same size and to be on the safe side, this change adds a compile-time check,
that the sizes of uintptr and pointers are the same.

Signed-off-by: Sven Anderson <sven@anderson.de>
2021-03-26 11:41:14 +00:00
John Mulligan
e9f8b47f71 containers: use the current Go 1.15 release
The current version we are testing with is no longer supported.
This updates Go to a supported released version.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-24 14:00:05 +00:00
John Mulligan
78ee562863 makefile: remove /dev/fuse and SYS_ADMIN cap from test
When invoking podman/docker/etc. it is no longer needed to add fuse
support, or (AFAICT) keeping cap SYS_ADMIN.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
2d85e6f235 entrypoint: remove GO_CEPH_TEST_REQUIRE_MOUNT environment var
This env var is no longer used, we can drop it from the entrypoint
script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
7031cabd4a micro osd: remove the use of cephfs fuse mount
Now that none of the tests need an external (fuse) mount of the fs under
test we can eliminate fuse from our ceph setup script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
f88a2b13f9 cephfs: remove useMount test helper function
Now that there are no tests using the fuse mount point and the useMount
function that detects an external mount point is needed, we can remove
the useMount function itself and the env var used to control it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
57a37c6c74 cephfs: remove uses of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
f0142920b5 cephfs: remove uses of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
df9834fc82 cephfs: remove uses of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
John Mulligan
3b9317b608 cephfs: remove a use of external mount of test fs
The external mount point was mainly used for stat prior to a Stat
function being part of go-ceph's cephfs package. Now that we have
Statx, we can replace the need for the external mount and os.Stat.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-21 06:19:40 +00:00
Sven Anderson
1c72979ce0 ptrguard: assert that //go:uintptrescapes actually works
This adds a test that assures that the special //go:uintptrescapes comment
before the storeUntilRelease() function works as intended, that is the
garbage collector doesn't touch the object referenced by the uintptr until
the function returns after Release() is called.  The test will fail if the
//go:uintptrescapes comment is disabled (removed) or stops working in future
versions of go.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-03-19 09:47:19 -04:00
John Mulligan
51885f0875 rbd: add test case for Stringer supporting mirroring constants
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-19 11:23:37 +00:00
John Mulligan
b614e93b9d rbd: add String method for MirrorImageState
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-19 11:23:37 +00:00
John Mulligan
741e7d43bd rbd: add String method for ImageMirrorMode
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-19 11:23:37 +00:00
John Mulligan
406868763c rbd: add String method for MirrorMode
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-03-19 11:23:37 +00:00
Sven Anderson
d75b75fb75 ci: add branch pattern 'pr/**' to PR test workflow
In order to run CI jobs also on PRs that are based on other PRs, this
change adds the pattern 'pr/**' to the tested base branches.  That
means, if PRs are pushed that are based on branches that start with
pr/... these PRs are also get tested. So, if we - by convention - push
PRs to branches like for example pr/ansiwen/myfix42, then all PRs
that are not based on master but another PR get tested as well.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-03-12 10:07:58 -05:00
Sven Anderson
527553718c cephfs: add a missing userMount() to tests
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-03-05 04:30:30 +00:00
John Mulligan
97bccd7a90 readme: document v0.8.0 and the ceph versions it supports
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-10 12:46:41 +01:00
Sven Anderson
01c9818729 ci: add GODEBUG=cgocheck=2 to the container environment
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 13:36:28 -05:00
Sven Anderson
5768911be7 ci: add a test run for Iovec based on PtrGuard
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Sven Anderson
b8a803ccbb cutil: use SyncBuffer for iovec type
The main motivation for PtrGuard was read and write buffers as they
are used in iovec.  This change uses SyncBuffer for the iovec
implementation, so that the no-copy PtrGuard implementation can be
enabled with the with_ptrguard build tag.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Sven Anderson
430dea5b7f cutil: add SyncBuffer abstraction layer to PtrGuard
In order to be able to swich on and off PtrGuard usage, SyncBuffer
implements a typical usage pattern of PtrGuard, that is a data buffer
referenced from inside a C allocated struct, in two ways: 1) using a
PtrGuard to dierctly store a pointer to the Go buffer in C memory, or
2) allocating a C buffer of same size and syncing data back with
C.memcpy().  The implementation can be chosen with the with_ptrguard
build tag.

Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
Sven Anderson
ac57250a5b cutil: add some useful function aliases
Signed-off-by: Sven Anderson <sven@redhat.com>
2021-02-08 11:09:25 -05:00
John Mulligan
1286e0d4d0 rbd: add GroupSnapRollbackWithProgress function
* Add GroupSnapRollbackWithProgress implementing rbd_group_snap_rollback_with_progress

This has supporting code basically the same as our other existing
callback functions.
Tests too.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2021-02-08 15:29:11 +00:00