tokenizers 1.0.0rc3

Source: https://github.com/huggingface/tokenizers
Tag scanned: v1.0.0-rc.3
Directory: bindings/python (only this part of the repository was scanned)
Compared with: v1.0.0-rc.2 (0 signals introduced since)
Files: 34 scanned, 8 skipped (binary, oversized, or unrecognised)
Signals: 0 critical · 0 warn · 0 info
Scan time: 0.1 s
LLM review: LLM review by claude-haiku-5-5 (Anthropic): nothing at warn or critical to review.
Disclude version: 4.0.0
Scanned (UTC): 2026-10-20 07:11

Signals

No signals flagged.

Reproduce This Scan

Run disclude locally to verify. Add --format sarif for SARIF output. Add --llm (with an API key in the environment) to include the LLM review.

cargo install disclude --version 4.0.0
git clone https://github.com/huggingface/tokenizers tokenizers && cd tokenizers
git checkout v1.0.0-rc.3
disclude scan --severity info --diff v1.0.0-rc.2 bindings/python