High precision numbers seem to be rounded

[ramsay-t]

Processing source files via the API (in smother) seems to round very high precision numbers when they are written back to the file. E.g.:

tt() ->
test([101]),
test([-1]),
test([278]),
test([97]),
test(10.500000000000000000000000000001).

becomes:

tt() ->
smother_server:log(string_paramtest, {{9, 1}, {14, 43}}, []),
test([101]),
test([-1]),
test([278]),
test([97]),
test(10.5).

I'm using the unix pretty printer, so that could be related:

wrangler_prettypr:print_ast('unix',AST2),

[simonjohnthompson]

Hi Ramsay - thanks for this. In fact this reflects the fact that those two numbers are the same in Erlang
Eshell V7.0 (abort with ^G)
1> 10.500000000000000000000000000001==10.5.
true
2> 10.500000000000000000000000000001=:=10.5.
true
3>

[ramsay-t]

This came up because it was causing a problem for someone's tests. I'll try and track down the original.

None the less, a refactoring tool probably shouldn't be altering the text without being asked. Presumably this is because the Wrangler AST has them stored by value rather than by "image"? Will this then break things like literals in other bases? I.e. if I typed 16#F because I want to work in hex (for some good presentation reason), will wrangler then rewrite this to base ten?

[ramsay-t]

Interestingly, I have just tried a hex representation and it was unchanged, but now (OTP17) it doesn't seem to damage the high precision number either. And OTP version issue maybe?

-module(hex).

-export([f/1,test/0]).

f(A) ->
io:format("pn",[A]).

test() ->
f(16#F),
f(0.0000000000000000000000000000000000000000000000000001),
f(10.500000000000000000000000000000000000000000000000001).

[simonjohnthompson]

Hi Ramsay - thanks for this. The point I was making was that though syntactially the two numbers are different, their semantics are identical in Erlang. Nonetheless we’ll look at this.

S.

On 7 Aug 2015, at 10:14, Ramsay Taylor [email protected] wrote:

Interestingly, I have just tried a hex representation and it was unchanged, but now (OTP17) it doesn't seem to damage the high precision number either. And OTP version issue maybe?

-module(hex).

-export([f/1,test/0]).

f(A) ->
io:format("pn",[A]).

test() ->
f(16#F),
f(0.0000000000000000000000000000000000000000000000000001),
f(10.500000000000000000000000000000000000000000000000001).

—
Reply to this email directly or view it on GitHub #66 (comment).

Simon Thompson | Professor of Logic and Computation
School of Computing | University of Kent | Canterbury, CT2 7NF, UK
[email protected] | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt