Add Outlines
#1364
Add Outlines
#1364
Conversation
yvan-sraka
force-pushed
the
1346-add-outlines
branch
from
8cb052d to
26b14b9
Compare
We should allow functions that are not decorated, such as def build_prompt(a: int) -> str:
return f"What is {a} squared?"Because we do not want to force the use of templates on users. |
|
Also you can deprecate the code in |
outlines/outline.py
Outdated
| prompt = self.template(*args) | ||
|
|
||
| # Generate the response using the model | ||
| response = self.model.generate(prompt) |
There was a problem hiding this comment.
This should use either outlines.generate.json, outlines.generate.regex, etc. The interface is necessarily going to be brittle for now. You can take a look at the v1 branch to see how this is going to be more robust with the undergoing refactor.
There was a problem hiding this comment.
Hmm, okay, but which one of those should I use by default? And how should I let users switch to the other one, by adding a fourth optional argument, e.g. generation_kind, WDYT?
There was a problem hiding this comment.
I just realized that maybe I misunderstood the issue, and this PR should infer the correct regex to match the user's required output type. E.g., if a user asks for an int, I should ask outlines.generate.regex something like ^[+-]?[0-9]+$. IIUC, I'm unsure how to scale this to arbitrary return types. Should we define a list of supported basic return types? Using ast.literal_eval was the most flexible solution I came up with, but I'm not sure how well it handles custom data structures. In such cases (when the return type isn't in the list of straightforward Python builtins), I think we should check that users provide types that implement deserialization from JSON (and then use outlines.generate.json), WDYT?
There was a problem hiding this comment.
Let's only accept Pydantic models / JSON Schema strings for now and use outlines.generate.json. We'll be able to easily generalise after we push the refactor in the v1 branch.
There was a problem hiding this comment.
I've updated the PR (and the usage example in the docstring). Does it still look like what we want?
yvan-sraka
force-pushed
the
1346-add-outlines
branch
from
26b14b9 to
e328144
Compare
That's already how this PR works, I just wanted to confirm that this was the intended design and not an omission!
Should I handle that in this PR or a separate one? By deprecate, do you mean adding a user-facing warning when it's used, rather than removing it entirely? |
We can do it in this PR. Let's add a user-facing warning, and announce that it will be removed in favour of the |
yvan-sraka
force-pushed
the
1346-add-outlines
branch
3 times, most recently
from
9a2a79a to
1faa76f
Compare
yvan-sraka
force-pushed
the
1346-add-outlines
branch
2 times, most recently
from
ddd5409 to
214a2f3
Compare
yvan-sraka
force-pushed
the
1346-add-outlines
branch
from
214a2f3 to
b2512e9
Compare
And the `output_type` should now be a Pydantic model or a JSON Schema str
yvan-sraka
force-pushed
the
1346-add-outlines
branch
from
b2512e9 to
9a9acba
Compare
| @@ -10,6 +10,9 @@ | |||
| from outlines.generate.api import SequenceGenerator | |||
| from outlines.prompts import Prompt | |||
|
|
|||
| # Raising a warning here caused all the tests to fail… | |||
| print("The 'function' module is deprecated and will be removed in a future release.") | |||
|
|
|||
There was a problem hiding this comment.
Maybe better place would be putting warning in __post_init__ of Function?
There was a problem hiding this comment.
Moving my previous comment here for visibility:
It would be nice to specify which release (1.0.0) so people can pin their version to < this release?
| """ | ||
|
|
||
| def __init__(self, model, template, output_type): | ||
| if not (isinstance(output_type, str) or issubclass(output_type, BaseModel)): |
There was a problem hiding this comment.
This seems like an oversimplified check to confirm str to be a json schema string: isinstance(output_type, str). Even though v1 will handle this much more gracefully, perhaps, even for a temp solution it worth to verify str being an actual json schema?
There was a problem hiding this comment.
It's surprisingly difficult to find a tool that validates a Json Schema. Would be very happy if you found one!
| ) | ||
| result = outline_instance(3) | ||
|
|
||
| assert result["result"] == 6 |
There was a problem hiding this comment.
Understanding that this is a draft yet, but still to mention it: tests for misc errors flows are missing
| """ | ||
|
|
||
| def __init__(self, model, template, output_type): | ||
| if not (isinstance(output_type, str) or issubclass(output_type, BaseModel)): |
There was a problem hiding this comment.
It's surprisingly difficult to find a tool that validates a Json Schema. Would be very happy if you found one!
| self.generator = generate.json(model, output_type) | ||
|
|
||
| def __call__(self, *args): | ||
| prompt = self.template(*args) |
There was a problem hiding this comment.
Maybe worth to support **kwargs as well?
Fix #1346, WDYT about the integration with
outlines.prompt? (Here, the template argument is just a function that returns astr, so it could be either an@outlines.prompt-decorated function or a regular one)