For the forthcoming Curator, we don't record timezone information in
the samples, nor do we in the curation remarks. All times are
recorded UTC. That said, for the test environment to better match
production, the special instant should be in UTC.
The curator work can be done easier if dto.SampleKey is no longer
directly accessed but rather has a higher level type around it that
captures a certain modicum of business logic. This doesn't look
terribly interesting today, but it will get more so.
After SampleValue was refactored into SampleValueSeries, which
involves plural values under a common super key, the stochastic
test was never refreshed to reflect this reality. We had other
tests that validated the functionality, but this one was
insufficently forward-ported.
This makes the memory persistence the backing store for views and
adjusts the MetricPersistence interface accordingly. It also removes
unused Get* method implementations from the LevelDB persistence so they
don't need to be adapted to the new interface. In the future, we should
rethink these interfaces.
All staleness and interpolation handling is now removed from the storage
layer and will be handled only by the query layer in the future.
The original append queue telemetry never worked, because it was
updated only upon the exit of the select statement, which would
usually liberate the queues of contents. This has been fixed to
be reported arbitrarily.
The queue sizes are now parameterizable via flags.
It is the case with the benchmark tool that we thought that we
generated multiple series and saved them to the disk as such, when
in reality, we overwrote the fields of the outgoing metrics via
Go map reference behavior. This was accidental. In the course of
diagnosing this, a few errors were found:
1. ``newSeriesFrontier`` should check to see if the candidate fingerprint is within the given domain of the ``diskFrontier``. If not, as the contract in the docstring stipulates, a ``nil`` ``seriesFrontier`` should be emitted.
2. In the interests of aiding debugging, the raw LevelDB ``levigoIterator`` type now includes a helpful forensics ``String()`` method.
This work produced additional cleanups:
1. ``Close() error`` with the storage stack is technically incorrect, since nowhere in the bowels of it does an error actually occur. The interface has been simplified to remove this for now.
After this commit, we'll need to add validations that it does the
desired work, which we presently know that it doesn't. Given the
changes I made with a plethora of renamings, I want to commit this
now before it gets even larger.
The curator doesn't do anything yet; rather, this is the type
definition including the anciliary testing scaffold.
Improve Makefile and Git developer experience.
The top-level Makefile was a bit overloaded in terms of generation of
assets and their management. This has been offloaded into separate
Makefiles.
The Git developer experience sucked due to lack of .gitignore
policies.
Also: Fix faulty skiplist naming from old merge.
The LevelDB storage types return an interface type now that wraps
around the underlying iterator. This both enhances testability but
improves upon, in my opinion, the interface design for the LevelDB
iterator.
Secondarily, the resource reaping behaviors for the LevelDB iterators
have been improved by dropping the externalized io.Closer object.
Finally, the iterator provisioning methods provide the option for
indicating whether one wants a snapshotted iterator or not.
EachFunc is deprecated.
Remove deprecated ``Pair`` and ``GetAll``.
These were originally used for forensic and the old gorest impl.
Nothing today in the user-facing path nor the tests uses them,
especially since the advent of the ForEach protocol in the
interface.
- Kill Close in Persistent and document interface.
- Extract batching behavior into interface.
- Kill IteratorManager, which was used for unknown reasons.
juliusv
Matt T. Proud
Matt T. Proud