[Core] predict_insample with multivariate models gives NaN values on last input_size

[carusyte]

What happened + What you expected to happen

It's unclear what the expected result of predict_insample should look like. In previous version, it provided all predicted values based on the training data provided during fit. However using the latest commit from main branch, the last input_size sample prediction is NaN instead.

Versions / Dependencies

Main branch with commit 939056c

Reproduction script

import pandas as pd
import numpy as np
import logging
import torch
from neuralforecast.core import NeuralForecast
from neuralforecast.models import TSMixerx

# Prep dummy data
start_date = "2024-06-01"
end_date = "2025-02-19"
date_range = pd.date_range(start=start_date, end=end_date, freq="B")
np.random.seed(0)
df = pd.DataFrame(
    {
        "unique_id": "dummy",
        "ds": date_range,
        "y": np.random.randn(len(date_range)),
        "val1": np.random.randn(len(date_range)),
        "val2": np.random.rand(len(date_range)),
    }
)

logging.getLogger("pytorch_lightning").setLevel(logging.ERROR)
torch.set_float32_matmul_precision("medium")

horizon = 10
input_size = 30
val_size = 50
models = [
    TSMixerx(
        h=horizon,
        input_size=input_size,
        n_series=1,
        max_steps=100,
        random_seed=0,
        hist_exog_list=["val1", "val2"],
    ),
]

nf = NeuralForecast(
    models=models,
    freq="B",
    local_scaler_type="robust",
)
nf.fit(df=df, val_size=val_size)

Y_hat_insample = nf.predict_insample(step_size=horizon)
# Y_hat_insample = nf.predict_insample()

print(Y_hat_insample)

Issue Severity

High: It blocks me from completing my task.

[marcopeix]

The issue happens only with multivariate models. Univariate models work fine. I need to investigate further.

[marcopeix]

Note that this behaviour is also observed in previous versions for multivariate models. Our tests did not include multivariate models, so we missed it.

[carusyte]

Note that this behaviour is also observed in previous versions for multivariate models. Our tests did not include multivariate models, so we missed it.

You're right, sorry for my incorrect descriptions. In fact previously, predict_insample may encounter error as recorded in this issue #1056, and I nearly forgot that I applied a stopgap from this #1056 (comment) just to continue my work. I'm not quite sure if this is the correct fix, anyway.

[carusyte]

You may notice that the windows tensor shape began to lose some elements here in

neuralforecast/neuralforecast/common/_base_multivariate.py

Lines 199 to 204 in 939056c

	if step == "predict":
	predict_step_size = self.predict_step_size
	cutoff = -self.input_size - self.test_size
	temporal = batch["temporal"][:, :, cutoff:]

Whereas in the same function of _base_windows.py, it applied padding to the temporal:

neuralforecast/neuralforecast/common/_base_windows.py

Lines 212 to 225 in 939056c

	if step == "predict":
	initial_input = temporal.shape[-1] - self.test_size
	if (
	initial_input <= self.input_size
	): # There is not enough data to predict first timestamp
	padder_left = nn.ConstantPad1d(
	padding=(self.input_size - initial_input, 0), value=0.0
	)
	temporal = padder_left(temporal)
	predict_step_size = self.predict_step_size
	cutoff = -self.input_size - self.test_size
	temporal = temporal[:, :, cutoff:]

[marcopeix]

Yes, saw that too! Seems that it fixes the issue. I'm testing if it impacts multivariate models' performance at all before pushing a fix. Thanks for pointing it out!

[marcopeix]

This is now fixed by the merge of #1023