Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue: Confusion with optimize_beta and Quantile Prediction in mapieTimeseries #583

Open
olitei opened this issue Dec 27, 2024 · 1 comment
Labels
Backlog This is in the MAPIE team development backlog, yet to be prioritised. TS regression Related to time series regression

Comments

@olitei
Copy link

olitei commented Dec 27, 2024

I am using the mapieTimeseries method from the MAPIE library for time series forecasting and I'm encountering a confusion regarding the use of alpha and optimize_beta when predicting specific quantiles (0.1 and 0.9) alongside the point forecast (0.5).

Goal:

I want to predict a point forecast (quantile 0.5) and predict the 0.1 and 0.9 quantiles for a given dataset.
I have set alpha=0.2 for 80% coverage.
Current Approach:

I use model.predict(optimize_beta=True) to make predictions.
However, I realized that setting optimize_beta=True means the model is adjusting the lower and upper bounds for 80% coverage, and not necessarily at the 0.1 and 0.9 quantiles.
Question:

Am I correct in understanding that when using optimize_beta=True, the model is optimizing the bounds to achieve 80% coverage (i.e., at the alpha=0.2 level), but it may not directly correspond to the 0.1 and 0.9 quantiles I want?
To predict specific quantiles like 0.1 and 0.9, should I set optimize_beta=False? In this case, I expect the model to predict exactly the 0.1 and 0.9 quantiles.
Observation:

My results appear quite similar when using both optimize_beta=True and optimize_beta=False, which is making me unsure about the correct approach.

@Valentin-Laurent
Copy link
Collaborator

Hello @olitei.
We realized recently that the optimize_beta feature is not working as expected (see issue #588). This explains why your results appear similar with and without this feature. Your understanding of the theory is correct.
We're currently discussing the priority of this fix VS other ongoing developments. Contributions are welcome! :)

@Valentin-Laurent Valentin-Laurent added Bug Backlog This is in the MAPIE team development backlog, yet to be prioritised. TS regression Related to time series regression and removed Bug labels Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backlog This is in the MAPIE team development backlog, yet to be prioritised. TS regression Related to time series regression
Projects
None yet
Development

No branches or pull requests

2 participants