Fix some bugs in retrying of operations

[jnation3406]

This fixes a bug where retrying operations came up with a different cache key so they wouldn't actually update the original operations status. Also handles a potential timeout error if an operation takes more than the default timeout (10 minutes) - this way the error should be shown to the user rather than just failing silently and never updating the operation from in progress. There is still a periodic error I get when retrying operations many times locally:

[default]` WARNING: File may have been truncated: actual file length (5931008) is smaller than the expected size (18774720) [astropy.io.fits.file]
[default] [2025-02-18 22:51:11,951] [PID 52] [Thread-7] [astropy] [WARNING] File may have been truncated: actual file length (5931008) is smaller than the expected size (18774720)
[default] [2025-02-18 22:51:11,956] [PID 52] [Thread-7] [root] [ERROR] buffer is too small for requested array
[default] Traceback (most recent call last):
[default]   File "/datalab/./datalab/datalab_session/tasks.py", line 24, in execute_data_operation
[default]     operation_class(input_data).operate()
[default]   File "/datalab/./datalab/datalab_session/data_operations/rgb_stack.py", line 130, in operate
[default]     input_handlers, zmin_list, zmax_list = self._process_inputs(rgb_inputs)
[default]   File "/datalab/./datalab/datalab_session/data_operations/rgb_stack.py", line 91, in _process_inputs
[default]     input_fits_list.append(InputDataHandler(input['basename'], input['source']))
[default]   File "/datalab/./datalab/datalab_session/data_operations/input_data_handler.py", line 32, in __init__
[default]     self.sci_data = get_hdu(self.fits_file, 'SCI').data
[default]   File "/datalab/./datalab/datalab_session/utils/file_utils.py", line 23, in get_hdu
[default]     extension_copy = hdu[extension].copy()
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/io/fits/hdu/table.py", line 506, in copy
[default]     return self.__class__(data=self.data.copy(), header=self._header.copy())
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/utils/decorators.py", line 836, in __get__
[default]     val = self.fget(obj)
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/io/fits/hdu/compressed/compressed.py", line 606, in data
[default]     data = self.section[...]
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/io/fits/hdu/compressed/section.py", line 54, in __getitem__
[default]     data = decompress_image_data_section(
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/io/fits/hdu/compressed/_tiled_compression.py", line 371, in decompress_image_data_section
[default]     heap_cache = bintable._get_raw_data(
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/io/fits/hdu/base.py", line 559, in _get_raw_data
[default]     return self._file.readarray(offset=offset, dtype=code, shape=shape)
[default]   File "/usr/local/lib/python3.10/site-packages/astropy/io/fits/file.py", line 403, in readarray
[default]     return np.ndarray(
[default] TypeError: buffer is too small for requested array

This only happens sometimes, so I'm not sure what the root cause is but it looks like it only partially downloads or uses the file and then fails due to that?

[LTDakin]

Looks like its failing the RGB test, could be because of the input sorting. If it's still able to create an rgb file then it should be fine to save the temp file thats tested and replace the current test file