Faster xr.DataArray.__init__

[valentinsulzer]

Is there a way to create a data-array that bypasses the various checks / copy operations to make it faster?

import xarray as xr
import numpy as np

x = np.linspace(0, 1, 100)  # 5 us
xr.DataArray(x, coords={"x": x})  # 75 us

[Illviljan]

I've felt this issue as well. They're not really public options but you could use xr.DataArray(x, coords={"x": x}, fastpath=True):

xarray/xarray/core/dataarray.py

Lines 401 to 411 in c919739

	def __init__(
	self,
	data: Any = dtypes.NA,
	coords: Sequence[Sequence | pd.Index | DataArray] | Mapping | None = None,
	dims: str | Iterable[Hashable] | None = None,
	name: Hashable | None = None,
	attrs: Mapping | None = None,
	# internal parameters
	indexes: Mapping[Any, Index] | None = None,
	fastpath: bool = False,
	) -> None:

Depending on your real issue using xr.Variable instead speeds things up further.

[valentinsulzer]

Thanks for the tip. If using fastpath it expects indexes to be provided. How do I set indexes in that case? I am only using the resulting DataArray for interpolation.
Thanks for the tip on xr.Variable

[benbovy]

There are usually more than check and copy operations when creating a new DataArray, e.g., creating pandas indexes for dimension coordinates (default behavior).

Using public API you could effectively skip the creation of indexes (after #8711 is merged!) with something like xr.DataArray(x, coords=xr.Coordinates({"x": x}, indexes={}), and you would normally see some nice speedup already.

However you might still want to have an indexed coordinate if you use the DataArray for interpolation.

Note that if you want to use fastpath=True you must manually build and pass a mapping of xr.Variable objects as coords and a mapping of xr.indexes.Index objects as indexes, so I'm not sure that in the end it will be much faster.

[valentinsulzer]

Ok, thanks! Sounds like I just have to take the hit on the setup time. I've changed it so that the DataArray only gets created right before I use it for interpolation (instead of every time even if I don't interpolate), which already helps a lot .

[Illviljan]

Here's a simple example why I like using xr.Variables:

import numpy as np
import xarray as xr
from time import perf_counter


# %% Inputs
names = np.core.defchararray.add("long_variable_name", np.arange(0, 100).astype(str))
time = np.array([0, 1])
coords = dict(time=time)
value = np.array(["0", "b"], dtype=str)

# %% Insert to Dataset with DataArray:
time_start = perf_counter()

ds = xr.Dataset(coords=coords)
for v in names:
    ds[v] = xr.DataArray(data=value, coords=coords)

time_end = perf_counter()
time_elapsed = time_end - time_start
print("Insert to Dataset with DataArray:", time_elapsed)

# %% Insert to Dataset with Variable:
time_start = perf_counter()

ds = xr.Dataset(coords=coords)
for v in names:
    ds[v] = xr.Variable("time", value)

time_end = perf_counter()
time_elapsed = time_end - time_start
print("Insert to Dataset with Variable:", time_elapsed)

# %% Insert to Dataset with tuple:
time_start = perf_counter()

ds = xr.Dataset(coords=coords)
for v in names:
    ds[v] = ("time", value)

time_end = perf_counter()
time_elapsed = time_end - time_start
print("Insert to Dataset with tuple:", time_elapsed)


# %% Dict of DataArray then create Dataset:
time_start = perf_counter()

data_vars = dict()
for v in names:
    data_vars[v] = xr.DataArray(data=value, coords=coords)
ds = xr.Dataset(data_vars=data_vars, coords=coords)

time_end = perf_counter()
time_elapsed = time_end - time_start
print("Dict of DataArrays then create Dataset:", time_elapsed)

# %% Dict of Variables then create Dataset:
time_start = perf_counter()

data_vars = dict()
for v in names:
    data_vars[v] = xr.Variable("time", value)
ds = xr.Dataset(data_vars=data_vars, coords=coords)

time_end = perf_counter()
time_elapsed = time_end - time_start
print("Dict of Variables then create Dataset:", time_elapsed)

# %% Dict of tuples then create Dataset:
time_start = perf_counter()

data_vars = dict()
for v in names:
    data_vars[v] = ("time", value)
ds = xr.Dataset(data_vars=data_vars, coords=coords)

time_end = perf_counter()
time_elapsed = time_end - time_start
print("Dict of tuples then create Dataset:", time_elapsed)

Insert to Dataset with DataArray: 0.1873271000222303
Insert to Dataset with Variable: 0.12849729997105896
Insert to Dataset with tuple: 0.11775939998915419
Dict of DataArrays then create Dataset: 0.06659770000260323
Dict of Variables then create Dataset: 0.0030629000393673778
Dict of tuples then create Dataset: 0.00291129999095574  # Winner! :)

The results have gotten better, compare #7224 (comment)

I remember having a similar mindset can improve interpolation as well. (I can't find those examples anymore though...)