Skip to content

Commit

Permalink
adds short section on lora params
Browse files Browse the repository at this point in the history
  • Loading branch information
djliden committed Apr 12, 2024
1 parent 7cabce3 commit a96ff47
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion notebooks/drafts/olmo_peft.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we set up the [`LoraConfig`](https://huggingface.co/docs/peft/en/package_reference/lora#peft.LoraConfig). This config specifies which layers we apply the LoRA adapters to"
"Next, we set up the [`LoraConfig`](https://huggingface.co/docs/peft/en/package_reference/lora#peft.LoraConfig). This config specifies which layers we apply the LoRA adapters to (`target_modules`); the rank and scaling factor for the adapters (`r` and `lora_alpha`); and the dropout rate (`lora_dropout`)."
]
},
{
Expand Down

0 comments on commit a96ff47

Please sign in to comment.