tiktoken 0.14.0

Source: https://github.com/openai/tiktoken
Tag scanned: 0.14.0
Compared with: 0.13.0 (0 signals introduced since)
Files: 22 scanned, 6 skipped (binary, oversized, or unrecognised)
Signals: 0 critical · 0 warn · 1 info
Scan time: 1.8 s
LLM review: Reviewed by claude-haiku-5-5 (Anthropic): 1 of 1 warn and critical signal; 1 demoted to info.
Disclude version: 4.0.0
Scanned (UTC): 2026-10-14 18:41

Signals

Ordered by severity, then signals introduced in this release. Hidden characters are shown as ⟨U+…⟩.

info ~~warn~~ → info by review dynamic-execution tiktoken/_educational.py:212:10

`open(__file__)` — file reads its own source, common pattern for extracting payloads hidden in comments (ast pass, confidence 0.85)
```
    )
    with open(__file__) as f:
        data = f.read()
```
likely benign LLM, confidence 0.90 The module opens its own source file and reads it into a variable, a common self-introspection pattern.
Reasoning
The snippet shows open(__file__) followed by a plain read, with no decoding, exec, or eval of the data in view. In an educational tokenizer module, reading its own source is most plausibly for demonstration or introspection. The finding's claim about payload extraction is speculative given only this snippet, and the rule fires on a common, benign idiom.

Reproduce This Scan

Run disclude locally to verify. Add --format sarif for SARIF output. Add --llm (with an API key in the environment) to include the LLM review.

cargo install disclude --version 4.0.0
git clone https://github.com/openai/tiktoken tiktoken && cd tiktoken
git checkout 0.14.0
disclude scan --severity info --diff 0.13.0 .