Code

WIP: Multi query attention#3

Code

Open

RaymondLi0 wants to merge 9 commits intoload-iterbigcode-project/Megatron-LM-deprecated:load-iterfrom multi-query-attentionbigcode-project/Megatron-LM-deprecated:multi-query-attentionCopy head branch name to clipboard

+381-11

Commits

Commits on Aug 8, 2022

• add multi-query attention logic in attention module

  RaymondLi0

  committed
  4064969

  Copy full SHA for 4064969

Commits on Aug 9, 2022

• add kv weight gradient reduction in tensor-parallel group

  RaymondLi0

  committed
  190e328

  Copy full SHA for 190e328

Commits on Aug 10, 2022

• more efficient multiquery attention

  RaymondLi0

  committed
  6fd0c29

  Copy full SHA for 6fd0c29

Commits on Sep 2, 2022

• raise if trying to uyse multi-query cross-atteention

  RaymondLi0

  committed
  254ff4b

  Copy full SHA for 254ff4b

• remove expand_key_value parameter since CoreAttention for multi-query is now in a separate class

  RaymondLi0

  committed
  eaf6174

  Copy full SHA for eaf6174

• remove most timers

  RaymondLi0

  committed
  1513137

  Copy full SHA for 1513137

Commits on Sep 7, 2022

• Merge branch 'load-iter' into multi-query-attention

  RaymondLi0

  committed
  d63c4b6

  Copy full SHA for d63c4b6

• resolve conflict

  RaymondLi0

  committed
  5045d6f

  Copy full SHA for 5045d6f

• implement alibi in multiquery core-attention

  RaymondLi0

  committed
  2117058

  Copy full SHA for 2117058