Skip to content

Commit 71ffe0a

Browse files
committed
[V1] Update structured output offline inference example
Update this example to work with V1. The grammar specified was not comaptible with xgrammar. This worked with V0 as we would automatically fall back to outlines. In V1 we have fallback support, but you have to opt-in to it by setting your backend to `auto`. Grammar type support varies among the backends. Signed-off-by: Russell Bryant <[email protected]>
1 parent 0115ccd commit 71ffe0a

File tree

1 file changed

+7
-12
lines changed

1 file changed

+7
-12
lines changed

examples/offline_inference/structured_outputs.py

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
print(outputs[0].outputs[0].text)
2020

2121
# Guided decoding by Regex
22-
guided_decoding_params = GuidedDecodingParams(regex="\w+@\w+\.com\n")
22+
guided_decoding_params = GuidedDecodingParams(regex=r"\w+@\w+\.com\n")
2323
sampling_params = SamplingParams(guided_decoding=guided_decoding_params,
2424
stop=["\n"])
2525
prompt = ("Generate an email address for Alan Turing, who works in Enigma."
@@ -57,17 +57,12 @@ class CarDescription(BaseModel):
5757

5858
# Guided decoding by Grammar
5959
simplified_sql_grammar = """
60-
?start: select_statement
61-
62-
?select_statement: "SELECT " column_list " FROM " table_name
63-
64-
?column_list: column_name ("," column_name)*
65-
66-
?table_name: identifier
67-
68-
?column_name: identifier
69-
70-
?identifier: /[a-zA-Z_][a-zA-Z0-9_]*/
60+
root ::= select_statement
61+
select_statement ::= "SELECT " column " from " table " where " condition
62+
column ::= "col_1 " | "col_2 "
63+
table ::= "table_1 " | "table_2 "
64+
condition ::= column "= " number
65+
number ::= "1 " | "2 "
7166
"""
7267
guided_decoding_params = GuidedDecodingParams(grammar=simplified_sql_grammar)
7368
sampling_params = SamplingParams(guided_decoding=guided_decoding_params)

0 commit comments

Comments
 (0)