jp6/cu129/: xgrammar-0.1.21 metadata and description

Efficient, Flexible and Portable Structured Generation

author	MLC Team
classifiers	License :: OSI Approved :: Apache Software License Development Status :: 4 - Beta Intended Audience :: Developers Intended Audience :: Education Intended Audience :: Science/Research
description_content_type	text/markdown
keywords	machine learning, inference
license	Apache 2.0
project_urls	Homepage, https://xgrammar.mlc.ai/ GitHub, https://github.com/mlc-ai/xgrammar
requires_dist	pydantic torch>=1.10.0 transformers>=4.38.0 triton; platform_system == "Linux" and platform_machine == "x86_64" mlx-lm; platform_system == "Darwin" and platform_machine == "arm64" ninja huggingface-hub[cli]; extra == "test" protobuf; extra == "test" pytest; extra == "test" sentencepiece; extra == "test" tiktoken; extra == "test" transformers<4.50.0; platform_system == "Darwin" and extra == "test"
requires_python	<4,>=3.8

Because this project isn't in the mirror_whitelist, no releases from root/pypi are included.

File	Tox results	History
xgrammar-0.1.21-cp312-cp312-linux_aarch64.whl Size 11 MB Type Python Wheel Python 3.12		Uploaded to jp6/cu129 by jp6 2025-07-14 21:42:29

Efficient, Flexible and Portable Structured Generation

Get Started | Documentation | Blogpost | Technical Report

News

[2025/02] XGrammar has been officially integrated into Modular's MAX
[2025/01] XGrammar has been officially integrated into TensorRT-LLM.
[2024/12] XGrammar has been officially integrated into vLLM.
[2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found here.
[2024/11] XGrammar has been officially integrated into SGLang.
[2024/11] XGrammar has been officially integrated into MLC-LLM.
[2024/11] We officially released XGrammar v0.1.0!

Overview

XGrammar is an open-source library for efficient, flexible, and portable structured generation.

It leverages constrained decoding to ensure 100% structural correctness of the output. It supports general context-free grammar to enable a broad range of structures, including JSON, regex, custom context-free grammar, etc.

XGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved near-zero overhead in JSON generation, making it one of the fastest structured generation engines available.

XGrammar features universal deployment. It supports:

Platforms: Linux, macOS, Windows
Hardware: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc.
Languages: Python, C++, and JavaScript APIs
Models: Qwen, Llama, DeepSeek, Phi, Gemma, etc.

XGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including vLLM, SGLang, TensorRT-LLM, and MLC-LLM, as well as many other companies. You can also try out their structured generation modes!

Get Started

Install XGrammar:

pip install xgrammar

Import XGrammar:

import xgrammar as xgr

Please visit our documentation to get started with XGrammar.

Adoption

XGrammar has been adopted by many projects and companies, including but not limited to:

WebLLM

Citation

If you find XGrammar useful in your research, please consider citing our paper:

@article{dong2024xgrammar,
  title={Xgrammar: Flexible and efficient structured generation engine for large language models},
  author={Dong, Yixin and Ruan, Charlie F and Cai, Yaxing and Lai, Ruihang and Xu, Ziyi and Zhao, Yilong and Chen, Tianqi},
  journal={Proceedings of Machine Learning and Systems 7},
  year={2024}
}