[Core] Upgrade to xgrammar 0.1.18, add cache size limit #16283
Conversation
xgrammar 0.1.18 includes a fix for security vulnerability where a malicious user can cause vllm to consume all of the host's RAM with the knowledge that each unique grammar adds an entry to an unbounded cache in memory. The cache was implemented in xgrammar PR 243: mlc-ai/xgrammar#243 This commit makes use of the new option to limit the cache size. The default is 512MB, which should be reasonable for most use cases. It will cache roughly 1000 json schemas. The cache size can be changed using the `VLLM_XGRAMMAR_CACHE_MB` environment variable. The security vulnerability was reported here: GHSA-389x-67px-mjg3 Signed-off-by: Russell Bryant <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
…#16283) Signed-off-by: Russell Bryant <[email protected]>
…#16283) Signed-off-by: Russell Bryant <[email protected]> Signed-off-by: Yang Wang <[email protected]>
…#16283) Signed-off-by: Russell Bryant <[email protected]>
…#16283) Signed-off-by: Russell Bryant <[email protected]>
…#16283) Signed-off-by: Russell Bryant <[email protected]> Signed-off-by: Mu Huai <[email protected]>
…#16283) Signed-off-by: Russell Bryant <[email protected]>
3 participants
xgrammar 0.1.18 includes a fix for security vulnerability where a
malicious user can cause vllm to consume all of the host's RAM with
the knowledge that each unique grammar adds an entry to an unbounded
cache in memory.
The cache was implemented in xgrammar PR 243:
mlc-ai/xgrammar#243
This commit makes use of the new option to limit the cache size.
The default is 512MB, which should be reasonable for most use cases.
It will cache roughly 1000 json schemas. The cache size can be changed
using the
VLLM_XGRAMMAR_CACHE_MBenvironment variable.The security vulnerability was reported here:
GHSA-389x-67px-mjg3
Signed-off-by: Russell Bryant [email protected]