underlying client capabilities:

- read + cache
- read sync
- write sync
- write + buffer
  (...potentially eventually augmented by byte ranges)

whatever system of modes, tokens, etc. has to satisfy the basic
constraint that no conflicting capabilities are ever in the 
hands of clients.


questions:
- is there any use to clients writing to a replica?
  - reading, yes.. 100,000 open same file..


------

simplest approach:
- all readers, writers go through authority
- all open, close traffic at replicas forwarded to auth

- fh state migrates with exports.



--------

less simple:
- all writers go through authority
  - open, close traffic fw
- readers from any replica
  - need token from auth
- weird auth <-> replica <-> client interactions ensue!


--------

even more complex (and totally FLAWED, ignore this!)

- clients can open a file with any replica (for read or write).
- replica gets a read or write token from the primary
  - primary thus knows if it's all read, all write, mixed, or none.
- once replica has a token it can service as many clients (of given type(s)) as it wants.
- on export, tokens are moved too.
  - primary give _itself_ a token too!  much simpler.

- clients maintain a mode for each open file: rdonly, wronly, rdwr, lock
- globally, the mode is controlled by the primary, based on the mixture of 
  read and write tokens issued



- [optional] if a client has a file open rdwr and the mode is rdonly or wronly, it can
  request to read or write from the mds (which might twiddle the mode for performance
  reasons.. e.g. lots of ppl rdwr but no actual reading)




--------