Code

support GroupQueryAttention#676

Code

Open

jslhcl wants to merge 5 commits intomainmicrosoft/onnxruntime-extensions:mainfrom leca/gqamicrosoft/onnxruntime-extensions:leca/gqaCopy head branch name to clipboard

+6,515-8

Commits

Commits on Mar 22, 2024

• support GroupQueryAttention

  jslhcl

  committed
  49aa6ec

  Copy full SHA for 49aa6ec

Commits on May 1, 2024

• fix build break

  

  Lei Cao

  committed
  a7eeca4

  Copy full SHA for a7eeca4

Commits on May 2, 2024

• add ReleaseMayInplace support and GQA is working under UT

  

  Lei Cao

  committed
  ce44a24

  Copy full SHA for ce44a24

Commits on May 21, 2024

• fix UniquePtr order to avoid crash, start UT to validate paged attention

  jslhcl

  committed
  9cc74a3

  Copy full SHA for 9cc74a3

Commits on May 22, 2024

• hack GQA to change the order of parameters to make GQA run without past_key nor past_value. Runtime CUDA error when copying result from GPU to CPU

  jslhcl

  committed
  81c33a4

  Copy full SHA for 81c33a4