end-to-end early-finalizer optimization targets

[aviatesk]

From a conversation with @jpsamaroo , I think GPU-related computations could be end-to-end targets of our early-finalizer optimization.

Julian Samaroo 12:55 AM
For GPU-heavy code, like from Flux, it's my understanding that we're overusing GPU memory because the GC doesn't see the GPU memory pressure. So by having CuArrays be finalized as soon as possible, we can avoid this issue.

12:56
And Flux tends to allocate a lot of temporaries that are only needed for a single layer (since that layer will then allocate the next array, and so on), so we could free each layer-generated output at the end of executing the next layer.

12:57
Separately, for GPU kernel code, it might also let us avoid having to do device-side malloc in more situations.