Code

[Inference] Integrate vllm example#262

Code

Merged

xwu99 merged 25 commits intointel:mainintel/llm-on-ray:mainfrom KepingYan:integrate_vllm_exampleKepingYan/llm-on-ray:integrate_vllm_exampleCopy head branch name to clipboardAug 16, 2024

+364-248

Commits

Commits on Jun 25, 2024

• integrate vllm example

  KepingYan

  committed
  444a38f

  Copy full SHA for 444a38f

Commits on Jun 26, 2024

• Merge remote-tracking branch 'upstream/main' into integrate_vllm_example

  KepingYan

  committed
  bc8cd09

  Copy full SHA for bc8cd09

• use vllm by default, remove ipex & deepspeed releted configs

  KepingYan

  committed
  0eb215a

  Copy full SHA for 0eb215a

• fix ci

  KepingYan

  committed
  ca68fb2

  Copy full SHA for ca68fb2

Commits on Jun 27, 2024

• Modify CI, the default is vllm engine

  KepingYan

  committed
  73203cf

  Copy full SHA for 73203cf

Commits on Jun 28, 2024

• fix ci

  KepingYan

  committed
  5bcad56

  Copy full SHA for 5bcad56

• fix ci

  KepingYan

  committed
  6ac99ec

  Copy full SHA for 6ac99ec

• fix ci, remove google/gemma2 due to huggingface token

  KepingYan

  committed
  de052bf

  Copy full SHA for de052bf

• fix ci & add params, placement_bundles is no longer used when tp=1

  KepingYan

  committed
  11ae87f

  Copy full SHA for 11ae87f

Commits on Jul 16, 2024

• mv vllm engine into route actor

  KepingYan

  committed
  9bffc53

  Copy full SHA for 9bffc53

• Merge remote-tracking branch 'upstream/main' into integrate_vllm_example

  KepingYan

  committed
  cc4f5ec

  Copy full SHA for cc4f5ec

• fix ci

  KepingYan

  committed
  cdc91b2

  Copy full SHA for cdc91b2

Commits on Jul 17, 2024

• fix ci

  KepingYan

  committed
  87b0af9

  Copy full SHA for 87b0af9

• fix ci

  KepingYan

  committed
  3e47c51

  Copy full SHA for 3e47c51

Commits on Jul 18, 2024

• fix ci

  KepingYan

  committed
  457582f

  Copy full SHA for 457582f

Commits on Jul 19, 2024

• fix ci

  KepingYan

  committed
  4f0a17e

  Copy full SHA for 4f0a17e

• Merge remote-tracking branch 'upstream/main' into integrate_vllm_example

  KepingYan

  committed
  f626139

  Copy full SHA for f626139

• fix ci

  KepingYan

  committed
  21db092

  Copy full SHA for 21db092

Commits on Jul 22, 2024

• fix ci

  KepingYan

  committed
  8475805

  Copy full SHA for 8475805

• fix ci

  KepingYan

  committed
  b56519d

  Copy full SHA for b56519d

Commits on Jul 23, 2024

• fix ci

  KepingYan

  committed
  87ccf7d

  Copy full SHA for 87ccf7d

• revert precision

  KepingYan

  committed
  5d8c256

  Copy full SHA for 5d8c256

• use fp32 because ipexbf16 needs AVX512

  KepingYan

  committed
  129cbcc

  Copy full SHA for 129cbcc

Commits on Aug 1, 2024

• Merge remote-tracking branch 'upstream/main' into integrate_vllm_example

  KepingYan

  committed
  124196b

  Copy full SHA for 124196b

• address comments

  KepingYan

  committed
  6243b67

  Copy full SHA for 6243b67