Skip to content

Latest commit

 

History

History
25 lines (20 loc) · 1.33 KB

README.md

File metadata and controls

25 lines (20 loc) · 1.33 KB

Lachesis

This artifact accompanies the paper Lachesis: Predicting LLM Inference Accuracy using Structural Properties of Reasoning Paths accepted to DeepTest'25, a workshop at ICSE'25.

Guide to Reproduction

0. Raw Data Files

  • AutoFL/data/*: Contains failing tests and covered code snippet data from BugsInPy and Defects4J. All data is sourced from the AutoFL repository (https://github.com/coinse/autofl).
  • AutoFL/results/* and AutoFL/combined_fl_results/*: Results from AutoFL. Iteration 1 through 5 are obtained from the AutoFL repository, while iterations 6 through 10 are generated through direct execution.

1. Representation of Reasoning Paths

  • AutoFL/name_utils.py: Includes functions for processing arguments. This file was sourced from the AutoFL repository to ensure consistent preprocessing with AutoFL.
  • data/*: LIM and LIG data represented using various embedding methods.
  • represent_data.py: Code for generating LIM and LIG data. To obtain the dataset representing reasoning paths, please excute the following commend:
python represent_data.py

2. Reproduce Results in the Paper

  • final_gcn.ipynb, final_lstm.ipynb, get_baselines.ipynb: Experimental code for training models and calculating final results.