Merge pull request #250 from simonpasquier/update-doc

Update doc

README.md

@@ -3,3 +3,5 @@
 This repository contains the new Prometheus storage layer that will be used in its 2.0 release.
 
 A writeup of its design can be found [here](https://fabxc.org/blog/2017-04-10-writing-a-tsdb/).
+
+See also the [format documentation](docs/format/README.md).

docs/format/README.md

@@ -0,0 +1,5 @@
+## TSDB format
+
+* [Index](index.md)
+* [Chunks](chunks.md)
+* [Tombstones](tombstones.md)

docs/format/index.md

@@ -54,7 +54,7 @@ Strings are referenced by sequential indexing. The strings are sorted in lexicog
 │ ├──────────────────────┴───────────────┤ │
 │ │                . . .                 │ │
 │ ├──────────────────────┬───────────────┤ │
-│ │ len(str_n) <uvarint> │ str_1 <bytes> │ │
+│ │ len(str_n) <uvarint> │ str_n <bytes> │ │
 │ └──────────────────────┴───────────────┘ │
 ├──────────────────────────────────────────┤
 │ CRC32 <4b>                               │
@@ -119,8 +119,8 @@ After the labels, the number of indexed chunks is encoded, followed by a sequenc
 
 ### Label Index
 
-A label index section indexes the existing (combined) values for one or more label names.  
-The `#names` field determines the number indexed label names, followed by the total number of entries in the `#entries` field. The body holds `#entries` symbol table reference tuples of length of length `#names`. The value tuples are sorted in lexicographically increasing order.
+A label index section indexes the existing (combined) values for one or more label names.
+The `#names` field determines the number of indexed label names, followed by the total number of entries in the `#entries` field. The body holds #entries / #names tuples of symbol table references, each tuple being of #names length. The value tuples are sorted in lexicographically increasing order.
 
 ```
 ┌───────────────┬────────────────┬────────────────┐
@@ -139,11 +139,19 @@ The `#names` field determines the number indexed label names, followed by the to
 └─────────────────────────────────────────────────┘
 ```
 
-The sequence of label index sections is finalized by an offset table pointing to the beginning of each label index section for a given set of label names.
+For instance, a single label name with 4 different values will be encoded as:
+
+```
+┌────┬───┬───┬──────────────┬──────────────┬──────────────┬──────────────┬───────┐
+│ 24 │ 1 │ 4 │ ref(value_0) | ref(value_1) | ref(value_2) | ref(value_3) | CRC32 |
+└────┴───┴───┴──────────────┴──────────────┴──────────────┴──────────────┴───────┘
+```
+
+The sequence of label index sections is finalized by an [offset table](#offset-table) pointing to the beginning of each label index section for a given set of label names.
 
 ### Postings
 
-Postings sections store monotonically increasing lists of series references that contain a given label pair associated with the list.  
+Postings sections store monotonically increasing lists of series references that contain a given label pair associated with the list.
 
 ```
 ┌────────────────────┬────────────────────┐
@@ -161,7 +169,7 @@ Postings sections store monotonically increasing lists of series references that
 └─────────────────────────────────────────┘
 ```
 
-The sequence of postings sections is finalized by an offset table pointing to the beginning of each postings section for a given set of label names.
+The sequence of postings sections is finalized by an [offset table](#offset-table) pointing to the beginning of each postings section for a given set of label names.
 
 ### Offset Table
 

encoding_helpers.go

@@ -11,7 +11,7 @@ import (
 
 var errInvalidSize = errors.New("invalid size")
 
-// enbuf is a helper type to populate a byte slice with various types.
+// encbuf is a helper type to populate a byte slice with various types.
 type encbuf struct {
 	b []byte
 	c [binary.MaxVarintLen64]byte

querier.go

@@ -117,7 +117,7 @@ func (q *querier) Close() error {
 	return merr.Err()
 }
 
-// NewBlockQuerier returns a queries against the readers.
+// NewBlockQuerier returns a querier against the reader.
 func NewBlockQuerier(b BlockReader, mint, maxt int64) (Querier, error) {
 	indexr, err := b.Index()
 	if err != nil {