[Tool] - Add mechanism to save operators' tensors to file

[goliaro]

Description of changes:

Many times we have had to rewrite custom code to save an operator's tensors to file for debugging purposes. This operation is often complicated by the fact that tensors may be partitioned, and the partitions accessed simultaneously (if using tensor parallelism). In addition, tensors are usually overwritten at each decoding step. Further, in speculative inference mode, we want to be able to distinguish the tensors from each SSM and the LLM.

This PR introduces a simple way to automatically save the inference tensors to file. The weights are saved only once per model, whereas the inputs/outputs once per iteration. Currently, the tensors are saved in text format for simplicity, but we can later switch to using a binary format to save space on disk.

To use this debugging tool, set the parameter inference_debugging to True when initializing the Python runtime, or pass the --inference-debugging flag when launching a FlexFlow C++ program.

Related Issues:

Linked Issues:

• Issue #

Issues closed by this PR:

• Closes #

---

This change is