Rule for exp mutates its input

[mcabbott]

Should these agree?

julia> using Zygote

julia> gradient(x -> sum(abs, exp(x)' + exp(x)'), [1 2; 3 4.0])
([169.91192742898497 320.5548284667717; 232.38736013199076 434.4143324629697],)

julia> gradient(x -> sum(abs, 2 * exp(x)'), [1 2; 3 4.0])
([226.54923657197995 427.40643795569565; 309.849813509321 579.2191099506263],)

I think the problem is that the rule for exp mutates the gradient it receives, which isn't safe as there's no guarantee this isn't shared with other rules -- here, the rule for + is doing Δ -> (Δ, Δ).

From here #381 (comment), but not sure when this was introduced. Haven't tried to check whether other rules do this too.

Xref FluxML/Zygote.jl#981 maybe --- it would be nice to change the convention to mutate Δ freely by default, and copy only when necessary.

[oxinabox]

Yeah, this should be fixed.

At it's root this is the same problem as the accumulation in presence of aliasing.
AD is not linearly typed, but rather linearly typed if you have a special dupe and elim operators that I need to understand.

[sethaxen]

I'm not certain why I chose to mutate the cotangents. Definitely seems to be problematic.