Commit Graph

584 Commits

Author SHA1 Message Date
John Mulligan 665611d054 rados: add MonCommandTarget & MonCommandTargetWithInputBuffer functions
These functions wrap rados_mon_command_target and allow the user to send
"command JSON" to specific MONs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-27 13:38:07 -04:00
John Mulligan 2202a2c32d rados: add test functions for OsdCommand
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-27 13:38:07 -04:00
John Mulligan 40f80cc846 rados: add OsdCommand & OsdCommandWithInputBuffer functions
These functions wrap rados_osd_command and allow the user to send
"command JSON" to OSDs.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-27 13:38:07 -04:00
John Mulligan 4b24a7849e rados: remove redundant private xxxCommand functions
These private functions with names like monCommand and pgCommand were no
different from the public xxxCommandWithInputBuffer commands. Thus the
redundant private functions are removed to simplify the file. As private
functions these can be added back later w/o fuss if the behavior of
the xxxCommand and xxxCommandWithInputBuffer calls need to differ more
in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-27 13:38:07 -04:00
Mudit Agarwal 092c0b7832 cephfs: add ceph_flock() function
Added ceph_flock() function which applies or removes an advisory lock on an open file.

Fixes: https://github.com/ceph/go-ceph/issues/248

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-27 10:40:52 -04:00
Mudit Agarwal 1d30b636ff cephfs: add ceph_fallocate() function
Added ceph_fallocate() which  preallocates or releases disk space
for the file for the given byte range.

Fixes: https://github.com/ceph/go-ceph/issues/246

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-20 13:12:55 -04:00
John Mulligan 80833a6789 cephfs: add Fstatx function to file type
Use the CephStatx type previously added to the library to implement a
wrapper for the ceph_fstatx function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-19 17:06:44 -04:00
John Mulligan 807eafb9d5 cephfs: add ReadDirPlus function and associated types
Add the ReadDirPlus function call, wrapping ceph_readdirplus_r. Add
DirEntryPlus which is a DirEntry plus a getter for the statx field.
Add a test similar to the ReadDir test and (since it is possible to
induce an error with ceph_readdirplus_r) a test for the error handling
path to improve code coverage.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-19 16:35:58 -04:00
John Mulligan fa21b454ab implements: add brief description to build and use the tool
This is a dev tool mainly for just those of us who work on the project,
so this is mainly just a quick-start type description.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan fb7c0f6561 implements: renamed "covered" to "found"
To be clearer about what the tool detects, rename "covered" to "found"
such as not to indicate an undocumented but used function is considered
"covered". Future arguments about what "covered" really means to be
scheduled at a later date. :-)

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 3ad6ad0473 implements: drop --annotate option
Currently, annotating the functions (are they documented, deprecated,
etc) in the function listing is not very costly and there are very
few situations where that extra information would be unwanted. Just
annotate the listing every time when running the tool.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan b450208827 makefile: add rule to run implements tool
Right now the tool doesn't do much checking but it does check what is
implemented, so we call the rule 'check-implements'. All current
sub-packages are checked. Use IMPLEMENTS_OPTS to pass additional cli
options to the tool.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 2d188de3f3 gitignore: add implements (binary)
The implements tool, when compiled, is named implements [1]. We never
want git to track it.

1 - Shocking, I know!

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan c3b3290fb4 makefile: add rule to build implements tool
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 141cd73223 implements: add implements cli interface
Add a main.go file to complete the `implements` cli tool.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 2d41cc93ca implements: add a JSON based report output
Allowing the tool to generate a JSON report makes it easier to add
additional tools later to make use of this data, without hand parsing
the text output.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 58745462e5 implements: add a textual report
Add simple reporting functionality to the implements package that is
intended to mimic the output of the current contrib/apicompare.py
script.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 49ac91b5b9 implements: process C sources use castxml
Add a file for processing C sources using the `castxml` tool. This tool
produces XML describing a C AST. This file adds go code for invoking the
tool and minimal processing of the XML it generates.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 18d8083de3 implements: add initial code for process Go sources
This code makes use of Go's native AST processing packages and allows us
to analyze our source code using the go-ceph conventions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan ce7911ea35 implements: stub in the initial support for functions pulled from C
These types will be used to track what C functions are expected to be
implemented.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan 80300e4e5a implements: initialize the internal package with minimal logging support
This log.go file establishes the ability for the package's user to hand
it any logger that meets the very minimal interface.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan f082075f26 implmements: introduce the implements tool
Mostly just a stub at the moment. Details to follow.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-17 11:26:46 -04:00
John Mulligan e2b1139310 makefile: include formatting as part of "make check"
Previously, "make check" only ran revive and its static checks.  The
entrypoint.sh script ran a formatting check *after* setting up ceph.

Now, "make check" runs both revive and gofmt. New make rules
check-revive and check-format run the individual tools respectively.
This is done to quicken the feedback in both the CI and locally for
formatting mistakes. Users can now easily run `make check
test-container` and avoid a long wait to set up ceph just to find out
there was a silly formatting error present.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-15 10:20:35 -04:00
John Mulligan ce8e45541b cephfs: add test coverage for invalid file state in Fchown
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-14 09:27:39 -04:00
John Mulligan 92a85b4385 cephfs: add test coverage for invalid file state in Fchmod
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-14 09:27:39 -04:00
John Mulligan 9a621942f3 gitignore: ignore .test and .json files
The .test files are generated by some makefile rules and should
never be checked in.
There's also no reason to track json files which may be generated
by running tools.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-14 14:35:01 +02:00
John Mulligan e2a78eec02 cutil: allow passing free functions to command output type
The *_command functions in librados and libcephfs document the use
of specific free functions for data allocated. These functions are
currently just wrappers around C's free() function. However, to be
more strictly compliant this change adds a free-function callback
to the CommandOutput type and the specific free functions are now
used outside the unit tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan f2d0bb4692 cephfs: replace copy-n-paste *_command functions argument handling
Similar to the functions in the rados pkg, cephfs package has a function
MdsCommand that is mostly boilerplate that to set up the C function's
arguments. Replace it with the types from the new cutil internal
package.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan 817b6b68c2 rados: replace copy-n-paste *_command functions argument handling
Replace the argument handling (wrangling?) around *_command functions,
such as MonCommand, with the new command input and output helper types
from the internal/cutil package.
No external change should be visible.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan 7256dbb27b entrypoint: enable building and testing of internal/cutil package
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan e40744fdf6 cutil: add a new internal package for c+go utility functions
Now we have sufficient boilerplate in our code for interacting with
various types and ceph calls with similar needs we establish a new
internal package, "cutil" (C utilities).

Note that many of the return types are wrapped. This is due to the
limits placed on us by cgo.  Despite the irritating limitations Go
places on "exporting" C types it still ought to help in the long run for
patterns that are very common or patterns that are subtle and we want to
write specific tests for.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:18:08 -04:00
John Mulligan d4079e3949 cephfs: add path based Statx function implmenting ceph_statx
Add a Statx wrapper for ceph_statx.
Add a type wrapping the statx status info that exposes the various
fields from the C-struct.
Add a type wrapping struct timespec, based on golang's x/sys, for the
time fields in the struct.
Note that the ceph struct is not the same as the linux statx struct.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-07 13:44:19 -04:00
muagarwa 27a1824ca0 cephfs: implement wrappers for fchmod and fchown
Fchmod() implemenets ceph_fchmod()
Fchown() implements ceph_fchown()
To fix: https://github.com/ceph/go-ceph/issues/240

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-07 09:43:05 +02:00
muagarwa fa7300fff8 cephfs: Validate mount.mount usage in cephfs/path.go
ceph MountInfo pointer is being passed in the functions without a validation check, if it is nil there may be a crash. Fixed the functions to first validate mount.mount

Signed-off-by: Mudit Agarwal muagarwa@redhat.com
2020-05-06 10:04:43 -04:00
root e89344cae1 fix typos 2020-05-01 09:37:09 -04:00
muagarwa 9e43e5119d cephfs: adding Link, Symlink and Readlink functions
Link function implements ceph_link().
Symlink function implements ceph_symlink().
Readlink function implements ceph_readlink().
To fix https://github.com/ceph/go-ceph/issues/218

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-04-30 13:28:51 -04:00
John Mulligan fde15e439e cephfs: add DType type and getter function for directory entries
CephFS returns Linux-style struct dirent that has a d_type field.
It appears to properly make use of the field. This change exposes
the field using a DType type and associated constants as well
as a DType() function for the DirEntry.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-30 12:54:37 -04:00
John Mulligan 7b2e18b5e1 cephfs: reject negative offsets for ReadAt/WriteAt
Negative offsets that are passed through ReadAt/WriteAt to the ceph
calls cause them to behave like Read/Write and this is undesirable
at the go layer. Additionally, the os packages' file functions
reject negative offsets as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan f7936e4ac4 cephfs: add a test to ensure behavior of mixed use of Read + ReadAt
Add a test case to ensure the behavior of a mixed used [1] of Read
and ReadAt calls is consistent and ReadAt doesn't side effect
the file position used by Read.

1 - Not recommended, but we should not break now or in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan 938506e1d3 cephfs: have Read/ReadAt return io.EOF error when nothing is read
Have Read and ReadAt functions return io.EOF when nothing is read
from the file so that it matches the current behavior of file types
in Go standard library.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan 1899072b27 cephfs: add additional error handling and state checks to file funcs
Now, the close function is idempotent wrt being called multiple times
on a valid file. Additionally, check the state of the file in various
functions to produce sensible errors, matching those cephfs itself
returns, if the object was not constructed properly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan c37ec2104a cephfs: add an Unlink func implmenting ceph_unlink
The unlink function can now be used to clean up files from the other
existing test cases.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan 854a834b1c cephfs: implement file IO functions for open/close/read/write/seek
Implement core file I/O functions based on a file handle wrapper.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-28 17:49:49 +02:00
John Mulligan fb2736dd0a rados: add MgrCommand and MgrCommandWithInputBuffer functions
Add functions for issuing commands to ceph mgrs with
MgrCommand and MgrCommandWithInputBuffer, implemented simliarly
to the other *Command style functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-27 14:38:16 -04:00
John Mulligan ff805ee1e1 rados: remove named return values in command.go
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-27 14:38:16 -04:00
John Mulligan 79dfe8e631 rados: move *Command functions into a separate file
Organize the rados package a bit more by moving functions that wrap
rados_xyz_command functions into a dedicated file. Move related test to
a separate file as well.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-27 14:38:16 -04:00
muagarwa 301d8e427f Makefile: Fix CONTAINER_OPTS for Fedora
make test-container is failing on Fedora based distribution because of bad label option "disabled".
Change label option to "disable"

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-04-23 08:22:07 -04:00
John Mulligan 1eb81971a6 workflows: get coverage result from ci container and archive it
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-21 15:56:40 -04:00
John Mulligan 204a7558d4 makefile: support getting results locally from container test
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-21 15:56:40 -04:00
Sven Anderson 6072f41bd6 update more code to use the retry lib.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-04-20 15:08:36 -04:00