New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Active learning concept: On-line calibration of the UQ intervals #14

Open

1 of 3 tasks

WardLT opened this issue Apr 20, 2021 · 1 comment

Open

1 of 3 tasks

Active learning concept: On-line calibration of the UQ intervals #14

WardLT opened this issue Apr 20, 2021 · 1 comment

Collaborator

WardLT commented Apr 20, 2021 •

edited

Loading

We should dynamically change the calibration for our UQ intervals during a run.

We should talk to Shengjia Zhao (Stanford) about this.

Needs:

Recalibration algorithms
Being able to run the recalibration of the models over time.
Optional: Re-running the experimental design without re-running inference.

WardLT mentioned this issue

Understand why active learning gets worse with more data #15

Closed

8 tasks

Collaborator Author

WardLT commented Oct 11, 2021

We now fit a linear recalibration factor using https://github.com/uncertainty-toolbox/uncertainty-toolbox.

See https://github.com/exalearn/electrolyte-design/blob/master/notebooks/production-models/ip-vacuum_nob-adia-smb/schnet-delta_smb-adiabatic/bootstrap-ensemble/0_assess-error-bars.ipynb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment