From de7bcf93666d91dc3fca6ccc4f78172e27149868 Mon Sep 17 00:00:00 2001 From: Irina Bigoulaeva Date: Tue, 14 Jan 2025 13:31:16 +0100 Subject: [PATCH] update readme --- README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index d3fba4d..c19dd3a 100644 --- a/README.md +++ b/README.md @@ -149,7 +149,7 @@ Once the dataset(s) have been created, call `train.py` with the desired paramete ```bash python3 train.py --model_path_idx 1 --model_size_idx 1 --num_epochs 1 --prompts_type 0 --num_train 20000 --batch_size 2 ``` -The model will be saved to the `saved_models` folder and will have a unique name generated according to the timestamp at the start of training. +The model will be saved to the `saved_models` folder and will have a unique name generated according to the timestamp at the start of training. Information about the training run (e.g. GPU architecture, hyperparameters) is saved to `train_logs.csv`. #### Important Parameters * `--prompts_type`: The type of prompt to train on, as listed in `train.py`. @@ -161,6 +161,10 @@ For model testing, call one of the two testing scripts: `test_regularprompt.sh` #### Important Parameters * `--run_name`: The unique timestamp of the model in `saved_models`. +The results are written to `eval_logs.csv` and `bertscore_evals.csv` + +* `eval_logs.csv` contains the raw accuracy scores for multiple-choice tasks, as well as scores for generative tasks. +* `bertscore_evals.csv` contains scores for tasks evaluated using BERTScore Accuracy (mostly multiple choice tasks). The BERTScore Accuracy takes priority over the raw scores for these tasks in `eval_logs.csv`. ## Cite