df1b2a2422
Fix indexing latency instrumentation.
...
Change-Id: I532c170121cd2996d1a378adbb1fd551cd5a4e38
2014-11-25 17:07:44 +01:00
a746fbb8bc
Instrument indexing: queue length, batch sizes and latencies.
...
Change-Id: I60bcbd24b160e47d418a485d8cffa39344a257c6
2014-11-25 17:07:44 +01:00
e9ff29c547
Comment/code cleanup.
...
Change-Id: I38736e3d0fec79759a2bafa35aecf914480ff810
2014-11-25 17:07:44 +01:00
0031a448e2
Add WaitForIndexing.
...
Change-Id: I5a5c975c4246632f937413322c855bbe63d00802
2014-11-25 17:07:44 +01:00
c7aad110fb
Add an indexing queue and batch the ops.
...
Some other improvements on the way, in particular codec -> codable
renaming and addition of LookupSet methods.
Change-Id: I978f8f3f84ca8e4d39a9d9f152ae0ad274bbf4e2
2014-11-25 17:07:44 +01:00
71206dbc06
More code cleanups.
...
Add license text everywhere.
And others....
Change-Id: I11ccde267a2ef7eb366c4788ba7aeae14ba7545c
2014-11-25 17:07:44 +01:00
630b5a087a
Also consider on-disk fingerprints during purge.
...
This reintroduces LevelDB iterators so that we can iterate through all
the on-disk fingerprints.
Change-Id: I007ee4638d038d2a4461bbda27f30fcaad411474
2014-11-25 17:07:35 +01:00
f5f9f3514a
Major code cleanup.
...
- Make it go-vet and golint clean.
- Add comments, TODOs, etc.
Change-Id: If1392d96f3d5b4cdde597b10c8dff1769fcfabe2
2014-11-25 17:02:53 +01:00
bbf49200ab
Implement methods in persistence.go.
...
Change-Id: I804cdd0b30420e171825fd86fe1281eca0d5e638
2014-11-25 17:02:23 +01:00
5a128a04a9
Major reorganization of the storage.
...
Most important, the heads file will now persist all the chunk descs,
too. Implicitly, it will serve as the persisted form of the
fp-to-series map.
Change-Id: Ic867e78f2714d54c3b5733939cc5aef43f7bd08d
2014-11-25 17:02:01 +01:00
4770cf76a4
Make index package more self-contained.
...
Moved interna from diskPersistence into the indexer.
TotalIndexer now called diskIndexer.
Change-Id: I6c8c62cb171f12bbd8a5474773af7786d71ba388
2014-11-25 17:02:01 +01:00
89f10e8eb2
Move to using the standard library interfaces for encoding/decoding.
...
BinaryMarshaler instead of encodable.
BinaryUnmarshaler instead of decodable.
Left 'codable' in place for lack of a better word.
Change-Id: I8a104be7d6db916e8dbc47ff95e6ff73b845ac22
2014-11-25 17:02:01 +01:00
7e85711df0
Beginnings of a tiered index implementation.
...
This reintroduces a LevelDB-based metrics index.
Change-Id: I4111540301c52255a07b2f570761707a32f72c05
2014-11-25 17:02:00 +01:00
8dfaa5ecd2
Remove use of freelists for chunk bufs.
...
Change-Id: Ib887fdb61e1d96da0cd32545817b925ba88831c1
2014-11-25 17:02:00 +01:00
ecdf5ab14f
Index-persistence switched from gob to a hand-coded solution.
...
Change-Id: Ib4ec42535bd08df16d34d4774bb638e35c5a1841
2014-11-25 17:02:00 +01:00
e7ed39c9a6
Initial experimental snapshot of next-gen storage.
...
Change-Id: Ifb8709960dbedd1d9f5efd88cdd359ee9fa9d26d
2014-11-25 17:02:00 +01:00
Julius Volz
Bjoern Rabenstein