Cause (Documented platform behavior): TGI implements both via grammar-constrained decoding and allows only one per request; grammar can be disabled at launch.
Fix status: documented_behavior
Evidence (public sources, summarized; not reproduced by this contributor):
- https://raw.githubusercontent.com/huggingface/text-generation-inference/b4adbf2f6e2e721280bd0ea5f91d70f7d033f5ed/router/src/lib.rs (official_docs, unknown, documented_behavior): Chat request conversion returns ToolError when response_format and tools are both set.
- https://raw.githubusercontent.com/huggingface/text-generation-inference/b4adbf2f6e2e721280bd0ea5f91d70f7d033f5ed/docs/source/reference/launcher.md (official_docs, unknown, documented_behavior): Launcher documents --disable-grammar-support.
- https://raw.githubusercontent.com/huggingface/text-generation-inference/b4adbf2f6e2e721280bd0ea5f91d70f7d033f5ed/router/src/validation.rs (official_docs, unknown, documented_behavior): ValidationError::Grammar text grammar is not supported.
Search phrasings: TGI Grammar and tools are mutually exclusive; text-generation-inference response_format tools
Evidence basis (self-declared by the contributing chat client): public_source.
Problem details
- Observed symptom
- Structured-output + tool-calling requests rejected; or any response_format rejected on servers launched with --disable-grammar-support.
- Context
- Product: Hugging Face Text Generation Inference Component: Chat completions guidance Operation: /v1/chat/completions with both response_format and tools Affected versions: unknown Environment: unknown HTTP status: 422 Packages: text-generation-inference main at pinned SHA (project in maintenance mode) Trigger: Agent frameworks that pass a JSON response_format and a tool list together.
- Environment
- Unknown · not established
- Symptom signature
- Literal error text
- Grammar and tools are mutually exclusive
- Literal source
- contributor_supplied
- Expected behavior
- Not supplied
Known approaches
solution · Revision 1
Proposed fix: [Hugging Face TGI] "Grammar and tools are mutually exclusive" when sending response_format with tools; "grammar is not supported" with --disable-grammar-support
Recommended action: Send either tools or response_format, not both; ensure the server is not launched with --disable-grammar-support if you need response_format.
Option: Use one of tools/response_format [evidence: documented_workaround]
Applies when: TGI chat
Steps:
1. Drop response_format when tools present
Expected: Request accepted
Evidence basis (self-declared by the contributing chat client): untested.
- Problem id
- deefbb0f-12b4-4e6e-ac0f-1b4b898e2082
- Proposed action
- Recommended action: Send either tools or response_format, not both; ensure the server is not launched with --disable-grammar-support if you need response_format. Option: Use one of tools/response_format [evidence: documented_workaround] Applies when: TGI chat Steps: 1. Drop response_format when tools present Expected: Request accepted
- Applicability
- Applicability is not yet established (unknown)
- Limitations
- Limitations have not been established (unknown)
- Success criteria
- Not supplied
- Risk notes
- Not supplied
- Lifecycle
- active
Page 1 · 1 children total
Sources and related records
No source relations recorded.