-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathresults.txt
55 lines (50 loc) · 3.79 KB
/
results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
AA experiments:
naive_lower naive_upper delta_lower delta_upper logdelta_lower logdelta_upper fieller_lower fieller_upper
0 -0.065199 0.343398 -0.078473 0.356672 -0.058957 0.378839 -0.059782 0.381686
1 -0.180533 0.226153 -0.183239 0.228859 -0.163811 0.251080 -0.162117 0.258039
2 -0.219469 0.160040 -0.216859 0.157431 -0.199918 0.176698 -0.202598 0.176331
3 -0.205480 0.185026 -0.204483 0.184030 -0.186609 0.204403 -0.188058 0.206099
4 -0.323898 0.037483 -0.310024 0.023609 -0.294790 0.040956 -0.296479 0.041704
... ... ... ... ... ... ... ... ...
9995 -0.110160 0.311871 -0.120558 0.322269 -0.099713 0.346107 -0.100344 0.349798
9996 -0.174542 0.253423 -0.177729 0.256611 -0.156543 0.280963 -0.162276 0.277314
9997 -0.137893 0.264658 -0.144200 0.270965 -0.125195 0.292611 -0.125978 0.295517
9998 -0.096479 0.321903 -0.108804 0.334228 -0.088148 0.357817 -0.087380 0.363537
9999 -0.202805 0.203391 -0.202837 0.203423 -0.183540 0.225518 -0.183711 0.229617
Type I error:
mean conf_int
naive 0.049 (0.0449, 0.0535)
delta 0.0535 (0.0492, 0.0581)
logdelta 0.0508 (0.0466, 0.0553)
fieller 0.049 (0.0449, 0.0535)
True confidence levels:
conf_level conf_level_ci quantiles
naive 0.951 (0.9465, 0.9551) (0.0228, 0.9738)
delta 0.9465 (0.9419, 0.9508) (0.0133, 0.9598)
logdelta 0.9492 (0.9447, 0.9534) (0.0236, 0.9728)
fieller 0.951 (0.9465, 0.9551) (0.0228, 0.9738)
AB experiments:
naive_lower naive_upper delta_lower delta_upper logdelta_lower logdelta_upper fieller_lower fieller_upper
0 0.103445 0.622491 0.065706 0.660230 0.095889 0.695137 0.094293 0.700121
1 0.089392 0.554488 0.060092 0.583788 0.084394 0.611522 0.082362 0.613786
2 0.124493 0.588437 0.091626 0.621304 0.115877 0.648926 0.114563 0.652165
3 -0.073888 0.347140 -0.086893 0.360145 -0.066288 0.383637 -0.067781 0.385864
4 0.255967 0.780476 0.206474 0.829970 0.236398 0.864284 0.235014 0.869026
... ... ... ... ... ... ... ... ...
9995 -0.074644 0.358424 -0.088870 0.372649 -0.067047 0.397619 -0.068176 0.400803
9996 0.034739 0.488819 0.012182 0.511376 0.035318 0.537774 0.032151 0.538102
9997 -0.191417 0.230553 -0.193667 0.232804 -0.172846 0.256741 -0.172522 0.262137
9998 0.181933 0.660245 0.141084 0.701094 0.166944 0.730584 0.166714 0.735855
9999 0.271330 0.783885 0.224555 0.830660 0.252722 0.862811 0.250595 0.865786
Statistical power:
mean conf_int
naive 0.7247 (0.7158, 0.7334)
delta 0.6505 (0.641, 0.6598)
logdelta 0.7284 (0.7195, 0.7371)
fieller 0.7247 (0.7158, 0.7334)
True confidence levels:
conf_level conf_level_ci quantiles
naive 0.9172 (0.9116, 0.9225) (0.0369, 0.9541)
delta 0.9455 (0.9408, 0.9498) (0.015, 0.9605)
logdelta 0.9461 (0.9414, 0.9504) (0.0253, 0.9714)
fieller 0.9479 (0.9433, 0.9521) (0.0246, 0.9725)