Build an iterative phonon flow (#306)

* add some drafts * pre-commit auto-fixes * restructure and rely on flow instead * restructure and rely on flow instead * pre-commit auto-fixes * more restructuring of completeworkflow * pre-commit auto-fixes * more restructuring of completeworkflow * bring the workflow more in shape * pre-commit auto-fixes * fix the workflow stepwise * pre-commit auto-fixes * fix some more probles * pre-commit auto-fixes * hard code displacement to 0.01 in benchmark, fix other more issues in workflow * pre-commit auto-fixes * fix more logic problems * make outputs nicer * pre-commit auto-fixes * fix a bunch of tests in auto phonons after the new changes * fix more tests * fix benchmark tests * daza * fix more tests * fix pre database position for all workflows * add more documentaion and fix number of jobs, addition of get_output * pre-commit auto-fixes * add more documentation * fix list comprehension * pre-commit auto-fixes * mace stuff * add to data * add to data * add autoplex preprint to readme (#309) * Update README.md * pre-commit auto-fixes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update README.md (#310) * Update README.md * pre-commit auto-fixes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update README.md (#311) * fix ranom seed for structures that are too similar * pre-commit auto-fixes * fix ranom seed for structures that are too similar * pre-commit auto-fixes * default random seed * default random seed * fix write benchmark generation beyond runs in jobflow * pre-commit auto-fixes * fix random seed and add strict tests * fix lorbit problems * fix test * fix test * fix rms computation * pre-commit auto-fixes * add one more test * add documentation * add hint on the default displacement * add hint on the default displacement * fix phonon data generation * pre-commit auto-fixes * clean up * fix references * add test for stratisfied split * fix doc issues and data issues * pre-commit auto-fixes * fix doc issues and data issues * pre-commit auto-fixes * fix import issues * pre-commit auto-fixes * fix import issues * pre-commit auto-fixes * add test for pre-database * deactivate filter* on test data * pre-commit auto-fixes * Add tests for test file sizes * pre-commit auto-fixes * remove definition * fix output of do_ * pre-commit auto-fixes * extend test to more generations and fix dft reference passing * pre-commit auto-fixes * fix doc * fix doc * fix doc * fix doc * pre-commit auto-fixes * try increasing myst heading anchors * try adding back _all__ block * add try except block in rmse calculation * pre-commit auto-fixes * fix some more None issues * pre-commit auto-fixes * fix some more None issues * fix some more None issues * pre-commit auto-fixes * test iterative_phonon_flow.py * test iterative_phonon_flow.py fix * fix format * fix confusing list outputs * pre-commit auto-fixes * very small fixes * extend gluexml unit test * Update src/autoplex/data/common/jobs.py Co-authored-by: J. George <[email protected]> * pre-commit auto-fixes * fix linting * improve function name and decription, improve the related unit test --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Aakash Ashok Naik <[email protected]> Co-authored-by: QuantumChemist <[email protected]> Co-authored-by: Christina Ertural <[email protected]>
autoatml · Jan 9, 2025 · 50bccd1 · 50bccd1
1 parent bb342fb
commit 50bccd1
Show file tree

Hide file tree

Showing 205 changed files with 2,190 additions and 249 deletions.
diff --git a/docs/conf.py b/docs/conf.py
@@ -58,7 +58,7 @@
 mermaid_output_format = 'raw'
 mermaid_params = ['--theme', 'base']
 
-myst_heading_anchors = 2  # enable headings as link targets
+myst_heading_anchors = 4  # enable headings as link targets
 myst_enable_extensions = [
     "amsmath",
     "colon_fence",

diff --git a/docs/user/phonon/flows/benchmark/benchmark.md b/docs/user/phonon/flows/benchmark/benchmark.md
@@ -7,6 +7,7 @@ This tutorial will help you understand all the `autoplex` benchmark specificatio
 ## General settings
 
 For the benchmark, you do not have to worry about a lot of settings. The crucial part here is the number of benchmark structures you are interested in.
+All benchmark harmonic phonon runs will always be generated with a displacement of 0.01 even though the fitting procedure can also include different displacements.
 
 ```python
 from mp_api.client import MPRester

diff --git a/docs/user/phonon/flows/flows.md b/docs/user/phonon/flows/flows.md
@@ -12,7 +12,9 @@ This tutorial will demonstrate how to use `autoplex` with its default setup and
 
 The complete workflow of `autoplex` involves the data generation 
 (including the execution of VASP calculations), 
-the fitting of the machine-learned interatomic potential (MLIP) and the benchmark to the DFT results.
+the fitting of the machine-learned interatomic potential (MLIP) and the benchmark to the DFT results. 
+
+We also have an iterative version of this workflow that reruns the complete workflow until a certain quality of the phonons is reached. It is described below.
 
 ### Before running the workflow
 
@@ -89,7 +91,6 @@ Next, we are going to construct the workflow based on the rocksalt-type LiCl ([*
 Remember to replace `YOUR_MP_API_KEY` with your personal [Materials Project API key](https://next-gen.materialsproject.org/api#api-key).
 
 ```python
-from jobflow.core.flow import Flow
 from mp_api.client import MPRester
 from autoplex.auto.phonons.flows import CompleteDFTvsMLBenchmarkWorkflow
 
@@ -220,3 +221,40 @@ Potential  Structure  MPID        Displacement (Å)  RMSE (THz)  imagmodes(pot)
 GAP        LiCl       mp-22905    0.01              0.57608     False           False           full            atom-wise f=0.1: n_sparse = 6000, SOAP delta = 0.5
 ```
 
+## Iterative version of the default workflow
+
+To systematically converge the quality of the potentials, we have built an iterative version of the default workflow `CompleteDFTvsMLBenchmarkWorkflow`. It will run the `CompleteDFTvsMLBenchmarkWorkflow` until the worst RMSE value of the benchmark structures falls under a certain value or a maximum number of repetitions is reached.
+
+We allow users in the first generation to use a slightly different  workflow than in the subsequent generations. This can help to initially obtain enough structures for an MLIP fit and only slightly increase the number of structures in the next generations. Please don't forget to deactivate the phonon data generation after the first iteration.
+
+```python
+from mp_api.client import MPRester
+from autoplex.auto.phonons.flows import CompleteDFTvsMLBenchmarkWorkflow, IterativeCompleteDFTvsMLBenchmarkWorkflow
+
+mpr = MPRester(api_key='YOUR_MP_API_KEY')
+structure_list = []
+benchmark_structure_list = []
+mpids = ["mp-22905"]  
+# you can put as many mpids as needed e.g. mpids = ["mp-22905", "mp-1185319"] for all LiCl entries in the Materials Project
+mpbenchmark = ["mp-22905"]
+for mpid in mpids:
+    structure = mpr.get_structure_by_material_id(mpid)
+    structure_list.append(structure)
+for mpbm in mpbenchmark:
+    bm_structure = mpr.get_structure_by_material_id(mpbm)
+    benchmark_structure_list.append(bm_structure)
+
+complete_flow=IterativeCompleteDFTvsMLBenchmarkWorkflow(rms_max=0.2, max_iterations=4, 
+                                                        complete_dft_vs_ml_benchmark_workflow_0=CompleteDFTvsMLBenchmarkWorkflow(
+    apply_data_preprocessing=True, add_dft_phonon_struct=True,
+), 
+                                                        complete_dft_vs_ml_benchmark_workflow_1=CompleteDFTvsMLBenchmarkWorkflow(
+    apply_data_preprocessing=True, add_dft_phonon_struct=False,
+)
+                                                        ).make(
+    structure_list=structure_list, mp_ids=mpids, 
+    benchmark_structures=benchmark_structure_list, benchmark_mp_ids=mpbenchmark)
+
+complete_flow.name = "tutorial"
+autoplex_flow = complete_flow
+```
diff --git a/src/autoplex/auto/phonons/flows.py b/src/autoplex/auto/phonons/flows.py