Hello Milan. About one out of 25 times, when running test_dropout_layer this occurs:

If you can't fix it today, I'll work on it tomorrow. Have a good afternoon

Edit: I adjusted it in #195.

There is still a case, in which all the units turn off. This occurs with a probability of (1 / dropout_rate) ^ n_units
In this case, output value is equal to NAN. In my opinion, we should add a control in which output is set to 0. Let me know what you think.

You're right, currently the the effective dropout rate is not exactly the specified dropout_rate because in this (naive) implementation we simply initialize the mask with a uniform random distribution and set it to 0 if it falls below dropout_rate and 1 otherwise. One way to handle it is to rescale the sample from random_number such that the dropout_rate becomes the percentile of the sample.

Regarding the test failing at 1/25, notice that each test run makes 10000 dropout runs, so the effective failure rate is (10^-5). I think it's the precision issue (i.e. tolerance too small in the test), rather than the scaling of the outputs. However, you may still be correct that there's an issue with scaling the outputs, as you suggested in #195, I just don't think it's related to the test failure.

I think we should handle the calculation of output values and even redefining the dropout, if needed. If you want to do a meeting in order to discuss this, i'm here. just answer me within 3 hours please

Hi, I just looked at my code (haven't looked earlier when I wrote here), and I see the very serious and obvious bug in calculating the scale, which your approach fixes. No proof needed :). I'll write in the other PR on the process on opening a PR against this one, rather than a new one.