How to recover scale value from microscaling-formatted Array?

[wonjeon]

Hello, I'm testing microscaling format and have a question to recover scale value from microscaling-formatted Array.

input = [0.0, 0.5, 40.5, 106.25, -52.0, -8.0]
arr = Array(Dtype('e4m3mxfp', scale='auto', input)
arr_bytes = arr.tobytes()
np_arr = np.array(arr_bytes, dtype=np.void)
print(arr.dtype.scale) # prints 0.25
np.save('input.py')

arr_bytes2 = np.load("input.py").view(np.void).item()
arr2 = Array('e4m3mxfp', arr_bytes2.item())
print(arr2) # prints Array('e4m3mxfp', [0.0, 2.0, 160.0, 416.0, -208.0, -32.0]) - different from arr or input
scale = arr2.dtype.scale
print(scale) # prints None

I appreciate your help. Thanks!

[scott-griffiths]

Hi. Thanks for the question.

The values in the array are a combination of the byte data and the dtype used to interpret them. When you use the 'auto' scale it works out 0.25 as the scale for the dtype and that scale forms part of the dtype. If you later load in the same byte data with a different dtype it will have different values.

>>> arr.dtype
Dtype('e4m3mxfp', scale=2 ** -2)
>>> arr2.dtype
Dtype('e4m3mxfp')

It's not possible to recover the scale from just the byte data - you have to know the whole dtype to interpret it correctly. When using these formats the scale will generally be stored alongside the data (typically in a e8m0mxfp value) so it will just be read in with the block of e4m3mxfp data. If you look at the MXFP standards you'll see the scale and a block of data it scales stored together, although the exact format for this isn't specified.

So to recap, you can't recover the scale, you need to store it alongside the data.

Hope that helps.

[wonjeon]

Got it. Thanks for your comment and confirmation.