[Model: TFT] StaticCovariateEncoder fails with empty tensor concatenation error

[emhedlin]

The issue

When trying to use the TFT model with static features, the model fails with a RuntimeError: torch.cat(): expected a non-empty list of Tensors.

The model failed with real world data I was using, so I tried with generate_series() data and it failed in the same way.

Versions / Dependencies

neuralforecast version: 2.0.1
python version: 3.10.12
os: Pop Os

Reproduction script

from neuralforecast.utils import generate_series
from neuralforecast.auto import TFT

train, static = generate_series(n_series = 4, freq='D', n_static_features=4)

model = [
    TFT(
        h=5,
        loss=MAE(),
        futr_exog_list=[],
        hist_exog_list=[],
        stat_exog_list=['static_0', 'static_1', 'static_2', 'static_3'],
        input_size=10,
        hidden_size = 10,
        n_head=5,
        max_steps=1500,
        enable_progress_bar=True,
    )
]

nf = NeuralForecast(models=model, freq='D')
nf.fit(df=train, static_df=static)

Issue Severity

None

[JQGoh]

@emhedlin
What if you omit the lines

futr_exog_list=[],
hist_exog_list=[],

Alternatively, if you want to pass in these two and they are not really used, I suggest you try with None instead of empty list and see whether the issue still remain

[emhedlin]

Training still fails when I:

• Omit or change the futr_exog_list / hist_exog_list args to None
• Include known and unknown regressors:

from neuralforecast.utils import generate_series
from neuralforecast.auto import TFT

train, static = generate_series(
    n_series = 4, 
    freq='D', 
    n_static_features=4,
    n_temporal_features=4
)

unknown = ['temporal_0', 'temporal_1']
known = ['temporal_2', 'temporal_3']

model = [
    TFT(
        h=5,
        loss=MAE(),
        futr_exog_list=known,
        hist_exog_list=unknown,
        stat_exog_list=['static_0', 'static_1', 'static_2', 'static_3'],
        input_size=10,
        hidden_size = 10,
        n_head=5,
        max_steps=1500,
        enable_progress_bar=True,
    )
]

nf = NeuralForecast(models=model, freq='D')
nf.fit(df=train, static_df=static)

For transparency, I have known and unknown regressors with my real world data, and everything works perfectly as long as I specify no static regressors (stat_exog_list = [] or stat_exog_list = None). As soon as I supply static regressors, I get the RuntimeError: RuntimeError: torch.cat(): expected a non-empty list of Tensors.

[marcopeix]

Hello!

The error occurs because in the code, we expect rnn_type to be either "lstm" or "gru", but the default value is set to "LSTM" (capital letters). I'll fix it in a PR, it's a really small bug.

For now, you can set rnn_type="lstm" and it will work. You can also use "gru", just make sure it's lowercase.

Thanks for pointing it out!

[emhedlin]

Thanks @marcopeix , this works 👍