Virtual nodes and constants inlining
CPU single thread with in-lining optimizations. Operators: arithmetic, power (non-differentiable exponent), ReLu. Shape inference: pointwise; transpose; compose; extended einsum (arbitrary permuting and summing-out of individual or matched axes, pointwise ellipsis, broadcasting); dynamic indexing with inner-product-like (pointwise) and outer-product-like variants. Backends: interpreter with tracing, compiled by ocamlopt
, compiled in-process by gccjit
. Optimizations: virtual nodes -- when cells of a tensor are not "recurrent" (accessed across steps) and not accessed too many times, in-lines the computation and does not materialize tensors; scalar constant subexpression elimination -- for 1D constant tensors, computes the subexpression at compile time and in-lines the value. Text-based visualization: tensors with up to 5 varying axes (other axes fixed), computation graphs with side-by-side subtree layout, plotting "line" graphs and decision boundaries, benchmark tables.