[Bug]: Close feature gaps when using xgrammar for structured output

[russellb]

🐛 Describe the bug

As of v0.6.5, we use xgrammar as the default backend for structured output. However, not all ways of expressing output requirements are supported. This issue is for tracking the list of known cases needed to be resolved for making xgrammar the default in all cases.

Fallback cases can be found here:

vllm/vllm/model_executor/guided_decoding/__init__.py

Lines 40 to 76 in d06e824

	if guided_params.backend == "xgrammar":
	# xgrammar only has x86 wheels for linux, fallback to outlines
	from vllm.platforms import current_platform
	if current_platform.get_cpu_architecture() is not CpuArchEnum.X86:
	logger.warning("xgrammar is only supported on x86 CPUs. "
	"Falling back to use outlines instead.")
	guided_params.backend = "outlines"
	# xgrammar doesn't support regex or choice, fallback to outlines
	if guided_params.regex is not None or guided_params.choice is not None:
	logger.warning(
	"xgrammar only supports json or grammar guided decoding. "
	"Falling back to use outlines instead.")
	guided_params.backend = "outlines"
	# xgrammar doesn't support some JSON schema features
	elif (guided_params.json is not None
	and has_xgrammar_unsupported_json_features(guided_params.json)):
	logger.warning(
	"xgrammar does not support advanced JSON schema features like "
	"patterns or numeric ranges. "
	"Falling back to use outlines instead.")
	guided_params.backend = "outlines"
	# xgrammar only supports GBNF grammars, so we must convert Lark.
	# We must check if the grammar is likely Lark and if that
	# grammar is convertible to GBNF
	elif (guided_params.grammar is not None
	and grammar_is_likely_lark(guided_params.grammar)):
	try:
	convert_lark_to_gbnf(guided_params.grammar)
	except Exception:
	logger.warning(
	"xgrammar does not support Lark grammars and the "
	"grammar failed to convert to GBNF. "
	"Falling back to use outlines instead.")
	guided_params.backend = "outlines"

• non-x86 architectures
• regex
  • related: [Feature] Support regex and repetition range mlc-ai/xgrammar#144
  • Failure to parse regex mlc-ai/xgrammar#175
• choice \
  • [Core] choice-based structured output with xgrammar #12632
• jsonschema support is incomplete
  • Support for array minItems and maxItems constraints mlc-ai/xgrammar#160
• lark grammars

[Ubospica]

Hi @russellb , thanks for raising the issue. XGrammar has a project to enhance the quality of the json schema converter and plan to support most of the features. We will track this issue and enhance it accordingly.

[saattrupdan]

@Ubospica Any news on supporting JSON features in XGrammar?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[saattrupdan]

This is still an issue and shouldn't become stale.