merge ppl.nn.llm #920

ouonline · 2024-04-16T07:50:38Z

No description provided.

(key_value_cache, multi_head_attention). And they will be re-support after optimized

[fix] fix private to public

openppl-public and others added 30 commits September 1, 2023 15:13

initial import

4d9db83

[misc] remove comments

0cbaecb

[fix] pplnn_llm.cc: add missing header

7fadf3f

[misc] update build command

4a5e29c

[doc] update quick start for running model

e5bded0

[opt] alloc cublas workspace in device

9d82016

[opt] only use cublas workspace for sm90

77d8fee

[doc] add comment

e3f1560

[refactor] reafactor rotary position embedding

42a8224

[refactor] refactor parallel ops

506cec5

[refactor] remove deprecate code

41afb26

(key_value_cache, multi_head_attention). And they will be re-support after optimized

[doc] ppl.llm system overview

f84023c

[fix] pic link in system_overview.md

b9cec2a

[fix] fix PPLNN_BUILD_TOOLS

76fb017

[doc] update platform

853acef

[feature] i8i8 quantize rev0

cfe8c00

[feature] add llama benchmark

90361a0

[misc] add quant_method ffor benchmark

9096f30

[fix] init allocator in PlainDevice

d7ae2bb

[doc] add benchmark llama doc

58265a1

[feature] enbale fp6 gemm bias to support internlm

f8f3ec2

[fix] orin: use sync allocator for engine device

1e1f9ab

[feature] i8i8 gemm/dequant support bias for internlm

35ed734

[opt] use zero-copy on weight quantization

3b5c15d

[fix] fix compile when nccl disbaled

cbf9e56

[opt] do not save backup quantized weight and scale

656e1ea

Update build.sh

ca1714f

[fix] add kv cache shape set, add model config check

bf99cb3

[opt] use rowmajor int8 linear

bde4fcb

[opt] support plain mm-policy

9c11698

Alcanderian and others added 26 commits January 17, 2024 15:49

[feature] add host device to llm_cuda engine

4375e12

[fix] fix ~LlmHostDevice()

34a7e6a

[fix] fix llm_cuda kernel.h

e66c8fa

[feature] benchmark support pmx model

e8d9b64

[fix] fix PPLNN_ENABLE_PMX_MODEL off compile bug

45c1254

[feature] python support init

b823e4b

[kernel] add moe linear op

8b64bbb

[feature] add moe_select, moe_reduce, linear

aadfd66

[fix] fix private to public

[opt] change expert_offset to host tensor

236ec44

[fix] change benchmark input to random number

2277361

[refactor] refactor moe ops

48348b8

[misc] change mhca api

5a97a1a

[fix] fix rms norm

5387747

[misc] rename MM_BESTFIT => MM_BEST_FIT to be the same as other engines

7244a6d

[misc] add ppnnn_llm.py

079d4a2

[feature] add Swish op

69004cb

[opt] do not use algo heuristic for sm>=89

55423df

[feature] add Add op

5356412

[feature] resume mha op

1678262

[feature] add new op PositionIndex

74769a3

[fix] fix register op

529c2ce

[fix] fix debug log

8787647

[feature] export cmake config

fc7b41f

merge ppl.nn.llm

6e87bef

[fix] opencv 4 or later is required to build the classification sample

ed3c410

[fix] put generated pb files into a lib

73dbd8d

ouonline closed this Apr 16, 2024

ouonline reopened this Apr 16, 2024

ouonline merged commit fc4de99 into OpenPPL:master Apr 16, 2024
0 of 8 checks passed

ouonline deleted the merged branch April 16, 2024 07:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge ppl.nn.llm #920

merge ppl.nn.llm #920

ouonline commented Apr 16, 2024

merge ppl.nn.llm #920

merge ppl.nn.llm #920

Conversation

ouonline commented Apr 16, 2024