Skip to content

Commit

Permalink
Adding stateguards for dolomite and granite and model path check
Browse files Browse the repository at this point in the history
Signed-off-by: Mustafa Eyceoz <[email protected]>
  • Loading branch information
Maxusmusti committed Oct 14, 2024
1 parent 078ccad commit 4a90e6d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/instructlab/training/data_process.py
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ def main(args: DataProcessArgs):
)
data_with_labels = data_with_labels.select_columns(["labels", "input_ids", "len"])
# use path to get the stem of the file
data_with_labels.to_json(Path(args.data_output_path) / f"data.jsonl")
data_with_labels.to_json(Path(args.data_output_path) / "data.jsonl")


if __name__ == "__main__":
Expand Down

0 comments on commit 4a90e6d

Please sign in to comment.