options: Add the ability to generate and validate options per use case (#385)

[AmnonHanuhov]

No description provided.

[AmnonHanuhov]

@mrambacher
A few things to take notice of:

1. In C++ there isn't really a convenient way with enough abstractions to generate random numbers like there is in python.
2. UseCase is an abstract class but should we declare GetInstance() there since this function will be implemented in each class that inherits from UseCase in order for it to be singleton?
3. GetInstance() isn't thread-safe.
4. The implementation of Validate is still not clear to me. The specific use case should know the valid range but where do we hold this range? Ideally I would want this range-per-option to appear in some sort of single point of truth so that db_crashtest.py, for example, can use it too. It can be a json or a yaml file where each option has the min and max value it can get and then we parse it in the python script and in the c++ class. Just a thought, let me know what you think.

[mrambacher]

@mrambacher A few things to take notice of:

1. In C++ there isn't really a convenient way with enough abstractions to generate random numbers like there is in python.

RocksDB has a Random number generator class in util/Random. If this is not sufficient for these purposes, that class should probably be improved.

2. UseCase is an abstract class but should we declare GetInstance() there since this function will be implemented in each class that inherits from UseCase in order for it to be singleton?

There is no need for a GetInstance in this manner. Something similar to what is done for in util/Comparator should suffice.

3. GetInstance() isn't thread-safe.

One option here might be std::once

4. The implementation of Validate is still not clear to me. The specific use case should know the valid range but where do we hold this range? Ideally I would want this range-per-option to appear in some sort of single point of truth so that db_crashtest.py, for example, can use it too. It can be a json or a yaml file where each option has the min and max value it can get and then we parse it in the python script and in the c++ class. Just a thought, let me know what you think.

Are the parameters validated in the crash test scripts? I think you can just code the ranges into the C++ class and do away with the code in the crash test scripts but am not positive.

[AmnonHanuhov]

@mrambacher A few things to take notice of:

1. In C++ there isn't really a convenient way with enough abstractions to generate random numbers like there is in python.

RocksDB has a Random number generator class in util/Random. If this is not sufficient for these purposes, that class should probably be improved.

It isn't sufficient, for example it doesn't support choosing a random number from a given set of numbers rather than a range. I should probably start from adding this feature.

2. UseCase is an abstract class but should we declare GetInstance() there since this function will be implemented in each class that inherits from UseCase in order for it to be singleton?

There is no need for a GetInstance in this manner. Something similar to what is done for in util/Comparator should suffice.

In Comparator class I see that there is an assignment operator implemented, so it isn't a singleton, to my understanding.

4. The implementation of Validate is still not clear to me. The specific use case should know the valid range but where do we hold this range? Ideally I would want this range-per-option to appear in some sort of single point of truth so that db_crashtest.py, for example, can use it too. It can be a json or a yaml file where each option has the min and max value it can get and then we parse it in the python script and in the c++ class. Just a thought, let me know what you think.

Are the parameters validated in the crash test scripts? I think you can just code the ranges into the C++ class and do away with the code in the crash test scripts but am not positive.

Doesn't seem like the script validates the parameters. What if someone decides to change the range or the set of numbers from which we sample, for a given option in the script, he will then have to remember to change it in the c++ code as well. I feel like implementing it that way is kind of error-prone.

[mrambacher]

@mrambacher A few things to take notice of:

1. In C++ there isn't really a convenient way with enough abstractions to generate random numbers like there is in python.

RocksDB has a Random number generator class in util/Random. If this is not sufficient for these purposes, that class should probably be improved.

It isn't sufficient, for example it doesn't support choosing a random number from a given set of numbers rather than a range. I should probably start from adding this feature.

Can you give me an example of a range that you are talking about? There is already a "Random::Uniform" method. Is that not sufficient?

BTW, there is also a "Random64" class, which seems very similar to what you were proposing.

2. UseCase is an abstract class but should we declare GetInstance() there since this function will be implemented in each class that inherits from UseCase in order for it to be singleton?

There is no need for a GetInstance in this manner. Something similar to what is done for in util/Comparator should suffice.

In Comparator class I see that there is an assignment operator implemented, so it isn't a singleton, to my understanding.

A comparator is an abstract class. Most likely, those assignments are for an InternalComparator and not a Comparator per-se.

4. The implementation of Validate is still not clear to me. The specific use case should know the valid range but where do we hold this range? Ideally I would want this range-per-option to appear in some sort of single point of truth so that db_crashtest.py, for example, can use it too. It can be a json or a yaml file where each option has the min and max value it can get and then we parse it in the python script and in the c++ class. Just a thought, let me know what you think.

Are the parameters validated in the crash test scripts? I think you can just code the ranges into the C++ class and do away with the code in the crash test scripts but am not positive.

Doesn't seem like the script validates the parameters. What if someone decides to change the range or the set of numbers from which we sample, for a given option in the script, he will then have to remember to change it in the c++ code as well. I feel like implementing it that way is kind of error-prone.

I think the point is that the script would not generate these random parameters any longer. Instead, the script would say generate a "crash usage" set of options and that would be what is used. If "option validation" validation is enabled, the DB::Open (or something else) would validate that the options are in the range which are acceptable to the installed use cases, such as "crash usage". So crash usage would potentially have two places (in validation and generation) where it knows what the range of acceptable parameters for it are.