Skip to content

Commit

Permalink
update falcon metadata (#404)
Browse files Browse the repository at this point in the history
A couple of small changes for consistency.

Also in the lord of the rings it was eagles, not falcons.
  • Loading branch information
philipkiely-baseten authored Jan 28, 2025
1 parent fc6119d commit 768cbd1
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 18 deletions.
12 changes: 4 additions & 8 deletions falcon/falcon3-10B-trt-llm-spec-dec/config.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,16 @@
model_metadata:
tags:
- openai-compatible
example_model_input:
frequency_penalty: 1
max_tokens: 512
messages:
- content: You are a knowledgable, engaging, history teacher.
- content: You are a knowledgable, engaging, biology teacher.
role: system
- content: What was the role of falcons in Lord of the Rings?
- content: What makes falcons effective hunters?
role: user
stream: true
temperature: 0.3
top_k: 40
top_p: 1.0
temperature: 0.6
repo_id: tiiuae/Falcon3-10B-Instruct
model_name: Falcon3-10B-Instruct-with-1B-SpecDec
model_name: Falcon 3 10B Instruct
resources:
accelerator: H100
cpu: "1"
Expand Down
16 changes: 6 additions & 10 deletions falcon/falcon3-3B-trt-llm-engine-high-throughput/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,22 @@ build_commands: []
environment_variables: {}
external_package_dirs: []
model_metadata:
tags:
- openai-compatible
example_model_input:
frequency_penalty: 1
max_tokens: 512
messages:
- content: You are a knowledgable, engaging, history teacher.
- content: You are a knowledgable, engaging, biology teacher.
role: system
- content: What was the role of falcons in Lord of the Rings?
- content: What makes falcons effective hunters?
role: user
stream: true
temperature: 0.3
top_k: 40
top_p: 1.0
temperature: 0.6
repo_id: tiiuae/Falcon3-3B-Instruct
model_name: Falcon3-3B-Instruct-H100
model_name: Falcon 3 3B Instruct
python_version: py39
requirements: []
resources:
accelerator: H100
accelerator: A10G
cpu: "1"
memory: 24Gi
use_gpu: true
Expand All @@ -43,4 +39,4 @@ trt_llm:
batch_scheduler_policy: max_utilization
enable_chunked_context: true
kv_cache_free_gpu_mem_fraction: 0.85
request_default_max_tokens: 131072
request_default_max_tokens: 8192

0 comments on commit 768cbd1

Please sign in to comment.