Adds new options for LuceneQueryOptions to work around limitations imposed by version 3.4.0 where the maximum data set size that can be paged is 10k when using Skip/Take.
If paging over data sets larger than 10k items is required and is not possible without using the more efficient SearchAfter feature, than these new options will allow Skip/Take to work but will have some performance implications.
- SkipTakeMaxResults - this will override the default value of 10k.
- AutoCalculateSkipTakeMaxResults - if set to true, than the total number of documents in the index will be calculated and used as the value for SkipTakeMaxResults. This means an additional index query is required for each query.
The reason the default 10K limit is imposed is because if this number is exceeded than additional memory and performance overheads will be incurred due to what Lucene allocates under the hood. Even Elastic Search has this limitation. For paging over large data sets, it is recommended to use the SearchAfter feature.
Examples have been added to the FluentApiTests.Paging_With_Skip_Take test for a reference on how to use these new options.
Full Changelog: v3.5.0...v3.6.0