Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spike at new CLI and SDK implementation #243

Merged
merged 5 commits into from
Nov 27, 2023
Merged

Spike at new CLI and SDK implementation #243

merged 5 commits into from
Nov 27, 2023

Conversation

gerred
Copy link
Contributor

@gerred gerred commented Nov 14, 2023

This reduces the code of the repeater into:

from typing import Any, Generator

from leapfrogai.llm import LLM, GenerationConfig


@LLM
class Repeater:
    def generate(
        self, prompt: str, config: GenerationConfig
    ) -> Generator[str, Any, Any]:
        yield prompt

That can be then run with:

leapfrogai main:Repeater

I might go down more of a FastAPI road where instead it's like:

from leapfrogai.backends import LLM, GenerationConfig

repeater = LLM()

@repeater.generate()
def generate(prompt: str, config: GenerationConfig) -> Generator[str, Any, Any]):
    yield prompt

That way it looks more like:

leapfrogai main:repeater

Would love thoughts on class vs. function style. This PR also completely separates the logic of serving the gRPC backend from the logic of generation.

@gerred gerred requested a review from YrrepNoj November 14, 2023 17:21
@gerred gerred merged commit 0cf8e37 into main Nov 27, 2023
2 checks passed
@gerred gerred deleted the refresh-sdk branch November 27, 2023 14:31
andrewrisse pushed a commit that referenced this pull request Apr 18, 2024
Spike at new CLI and SDK implementation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants