Code

Mixtral batching support#108

Code

Merged

sunggg merged 10 commits intooctoml:batch-servingoctoml/mlc-llm:batch-servingfrom vinx13:moe-batchvinx13/mlc-llm:moe-batchCopy head branch name to clipboardDec 12, 2023

+477-17

Commits

Commits on Dec 11, 2023

• moe arch

  vinx13

  committed
  5789469

  Copy full SHA for 5789469

• param mapping

  vinx13

  committed
  2643016

  Copy full SHA for 2643016

• fix

  vinx13

  committed
  c072277

  Copy full SHA for c072277

• adopt hf weights

  vinx13

  committed
  8e25c56

  Copy full SHA for 8e25c56

• fix

  vinx13

  committed
  f279349

  Copy full SHA for f279349

• speedup weight preprocess

  vinx13

  committed
  a39ca01

  Copy full SHA for a39ca01

• fix

  vinx13

  committed
  1f267ea

  Copy full SHA for 1f267ea

• batch

  vinx13

  committed
  fca1b39

  Copy full SHA for fca1b39

Commits on Dec 12, 2023

• remove deadcode

  vinx13

  committed
  4618de1

  Copy full SHA for 4618de1

• cleanup

  vinx13

  committed
  113bd18

  Copy full SHA for 113bd18