Skip to content

Commit

Permalink
docs: fix a mistake in the ggml-rope.md
Browse files Browse the repository at this point in the history
Signed-off-by: Daniel Bevenius <[email protected]>
  • Loading branch information
danbev committed Jul 28, 2024
1 parent 9839659 commit ad4e33a
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions notes/ggml-rope.md
Original file line number Diff line number Diff line change
Expand Up @@ -688,8 +688,8 @@ of dimensions used for RoPE:
dst_data[1] = src[1];
}
```
But does the same thing as the `!is_neox` block but for the remaining
dimensions.
Notice that this actually does not perform a rotation but is simply copying the
values from the src tensor to the dst tensor.
In "standard" RoPE take the embedding input dimensions in pairs when we process
them. So we will take the 128 dimensions and simple process them as pairs in
Expand Down

0 comments on commit ad4e33a

Please sign in to comment.