[enhancement] Make invoke side-effecting + make a "safe to remove side-effecting op" category

[grebe]

What's hard to do? (limit 100 words)

There are places where we need to decide if it's OK to remove or change an op, and the check often looks something like

if (OpIsSideEffecting(node->op()) && !node->Is<Gate>()) { ... }

or something like that. Relatedly, #1805 implies that perhaps invoke() ops should be considered side-effecting, but you probably should be able to remove ffi-invokes.

Current best alternative workaround (limit 100 words)

Ad-hoc categories scattered throughout code + hope that fuzzing catches when these categories are misaligned.

Your view of the "best case XLS enhancement" (limit 100 words)

Make a category like MustKeepSideEffectOp() that excludes gate() and FFI invokes(), but includes "regular" invokes. Alternatively, make a separate op for FFI stuff.