Lower memory requirements

[tokee]

In order to make stored String values into DocValues, all the Strings are kept in a TreeSet<BytesRef>. The overhead is very large, as 1MB of heap can hold only 6500 values. Having 1M unique stored values in a field requires 153MB of heap.

Most of the memory is used for Java Objects and references, not the characters themselves. Storing the BytesRef in a BytesRef-array and keeping track of ordering and duplicates with a heap of native longs would be one memory-saving solution.