nltk 3.10.4

Source: https://github.com/nltk/nltk
Tag scanned: v3.10.4
Compared with: v3.10.3 (41 signals introduced since)
Files: 749 scanned, 97 skipped (binary, oversized, or unrecognised)
Signals: 0 critical · 0 warn · 103 info
Scan time: 28.4 s
LLM review: Reviewed by claude-haiku-5-5 (Anthropic): 31 of 31 warn and critical signals; 31 demoted to info.
Disclude version: 4.1.0
Scanned (UTC): 2026-10-22 07:29

Signals

Ordered by severity, then signals introduced in this release. Hidden characters are shown as ⟨U+…⟩.

info ~~warn~~ → info by review dynamic-execution nltk/decorators.py:327:15 new in this release

`eval` called on a non-literal expression (ast pass, confidence 0.70)
```
    src = f"lambda {signature}: _wrapper_({callargs})"
    funcopy = eval(
        src, dict(_wrapper_=wrapper)
    )  # bare-exec ok: _assert_safe_signature fenced src (CVE-2026-14727)
```
likely benign LLM, confidence 0.85 eval of a generated lambda source that forwards arguments to a decorator wrapper.
Reasoning
This is the well-known decorator-generation pattern that builds a lambda from a validated signature string. The comment notes the source is fenced by a signature-safety check, and the eval namespace is restricted to a small dict.
info ~~warn~~ → info by review dynamic-execution nltk/decorators.py:398:20 new in this release

`eval` called on a non-literal expression (ast pass, confidence 0.70)
```
        # import sys; print >> sys.stderr, src # for debugging purposes
        dec_func = eval(
            src, dict(_func_=func, _call_=caller)
        )  # bare-exec ok: _assert_safe_signature fenced src (CVE-2026-14727)
```
likely benign LLM, confidence 0.85 eval of generated decorator source that wraps a function with a caller.
Reasoning
Same generated-function pattern as the other decorators.py eval. The source is built from the decorated function's signature with a restricted globals dict, and the comment cites a safety check.
info ~~warn~~ → info by review dynamic-execution nltk/redos.py:320:12 new in this release

`compile` called on a non-literal expression (ast pass, confidence 0.70)
```
    :data:`DEFAULT_TIMEOUT` at match time."""
    return compile(src, flags)
```
likely benign LLM, confidence 0.80 Compiles a regex source string for timeout-guarded matching.
Reasoning
This compiles a regex source string built internally, with flags, for ReDoS-timeout-guarded matching. Compiling regex is not code execution, and the source comes from library logic rather than untrusted input.
info ~~warn~~ → info by review dynamic-execution nltk/redos.py:335:12 new in this release

`compile` called on a non-literal expression (ast pass, confidence 0.70)
```
        return _cached_compile(pattern, flags)
    return compile(pattern, flags)
```
likely benign LLM, confidence 0.80 Compiles a regex pattern with flags as a fallback path.
Reasoning
Plain re.compile-style call on a pattern argument after a cache check. This is ordinary regex compilation, not dynamic code execution.
info ~~warn~~ → info by review dynamic-execution nltk/redos.py:592:12 new in this release

`compile` called on a non-literal expression (ast pass, confidence 0.70)
```
        flags = getattr(pattern, "flags", 0)
    return compile(source_of(pattern), flags)
```
likely benign LLM, confidence 0.80 Compiles a regex derived from an existing pattern's source.
Reasoning
Compiles a regex derived from a pattern's source with its flags. This is regex compilation within a ReDoS analysis utility, not code execution.
info ~~warn~~ → info by review high-complexity nltk/test/unit/security_probes/ghsa_xv54_447f_mj22.py:1:4 new in this release

string literal compression ratio 0.961 (129 bytes) (raw pass, confidence 0.60)
```
"""GHSA-xv54-447f-mj22 [low]: Symlink-follow sandbox bypass in chat80.sql_query allows out-of-root SQLite file read (CWE-59, CWE-73)"""

import os
```
likely benign LLM, confidence 0.90 Docstring describing a symlink sandbox-bypass advisory in a security test probe.
Reasoning
The raw finding is on a docstring describing a security advisory, which is a long, dense text literal. It is a test probe file with no executable payload.
info ~~critical~~ → info by review dynamic-import nltk/test/unit/test_advisory_probes.py:294:68 new in this release

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
    try:
        AllowlistUnpickler.find_class = lambda self, module, name: __import__(
            module, fromlist=[name.split(".")[0]]
        )
```
likely benign LLM, confidence 0.80 Test monkeypatch of Unpickler.find_class that calls __import__ during an allowlist probe.
Reasoning
Within a test, the unpickler's find_class is temporarily replaced with a lambda that imports a module, as part of checking an allowlist restriction. This is test scaffolding for a security probe rather than a payload delivered to users.
info ~~warn~~ → info by review decoder-import-with-exec nltk/test/unit/test_advisory_probes.py:984:23 new in this release

file imports decoder module(s) pickle and calls `exec`/`eval`/`compile` — multi-stage payload shape (ast pass, confidence 0.80)
```
    old_nextprim_re = compile(r"""([A-Za-z]+(?:\[[A-Za-z,]+\])?)(.*)""")
    old_app_re = compile(r"""([\\/])([.,_]?)([.,]?)(.*)""")
    old_prim_re = compile(r"""([A-Za-z]+)(\[[A-Za-z,]+\])?""")
```
dismissed LLM, confidence 0.85 Module-level compile of literal regex patterns in an advisory test file.
Reasoning
The finding matches a multi-stage shape because the file imports pickle and uses compile, but the flagged calls are compile of literal regex strings for test expectations. There is no decode-then-exec chain.

info unicode-mixed-script nltk/test/unit/test_attack_bllip_model_paths_expanded.py:443:53 new in this release

identifier `Key` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)

        root, outside = pathsec_sandbox
        _assert_parser_model_refused(str(outside / "Key"), bllip_spy)

    def test_case_variant_of_in_root_model_resolves_in_root_and_loads(

info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_attack_ccg_expanded.py:1065:17 new in this release

`exec` called on a non-literal expression (ast pass, confidence 0.70)
```
            with contextlib.redirect_stdout(io.StringIO()):
                exec(compile(example.source, doctest_name, "single"), namespace)
        except Exception as exc:
            assert example.exc_msg, (example.source, repr(exc))
```
likely benign LLM, confidence 0.85 Test harness exec of doctest example source with captured output.
Reasoning
This is a doctest-runner pattern that executes example source from docstrings in a namespace, with stdout captured and exceptions checked. The source comes from the project's own docstrings during testing.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_attack_ccg_expanded.py:1065:22 new in this release

`compile` called on a non-literal expression (ast pass, confidence 0.70)
```
            with contextlib.redirect_stdout(io.StringIO()):
                exec(compile(example.source, doctest_name, "single"), namespace)
        except Exception as exc:
            assert example.exc_msg, (example.source, repr(exc))
```
likely benign LLM, confidence 0.85 Compiles doctest example source inside the test harness.
Reasoning
The compile call is the inner half of the same doctest-execution harness, compiling docstring example source in single mode. It is standard test tooling and not a hidden payload.
info ~~critical~~ → info by review dynamic-import nltk/test/unit/test_attack_chat80_store_expanded.py:50:16 new in this release

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
    try:
        return __import__(name, fromlist=["open"])
    except ImportError:
        pytest.skip(f"{name} not available on this interpreter")
```
likely benign LLM, confidence 0.90 Test helper that dynamically imports an optional module and skips if unavailable.
Reasoning
A try/except around __import__ in a test helper that skips the test when an optional module is missing. The name is a test parameter, not attacker input, and the snippet shows no payload or exfiltration.
info ~~critical~~ → info by review dynamic-import nltk/test/unit/test_attack_chat80_val_load_expanded.py:40:16 new in this release

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
    try:
        return __import__(name, fromlist=["open"])
    except ImportError:
        pytest.skip(f"{name} not available on this interpreter")
```
likely benign LLM, confidence 0.90 Test helper that dynamically imports an optional module and skips if unavailable.
Reasoning
Same pattern as the sibling test: a dynamic import with ImportError handling that calls pytest.skip. It is test scaffolding for optional dependencies.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_attack_core_guards_expanded.py:169:9 new in this release

`exec` called on a non-literal expression (ast pass, confidence 0.70)
```
        exec_locals = {}
        exec(  # bare-exec ok: builds a test function with a non-ASCII parameter
            "def f(caf" + chr(0xE9) + "=1):\n    return caf" + chr(0xE9), exec_locals
        )
```
likely benign LLM, confidence 0.90 Test code using exec to define a function with a non-ASCII parameter name.
Reasoning
exec builds a test function with a non-ASCII parameter name from chr(0xE9). The inline comment states the purpose, and the code only defines and returns a function in a test namespace.

info unicode-zero-width nltk/test/unit/test_attack_data_readline_expanded.py:66:45 new in this release

U+200D ZERO WIDTH JOINER in source (in string/comment) (raw pass, confidence 0.45)

            pieces.append(
                "".join(rng.choice("ab cé⟨U+00A0⟩⟨U+200D⟩") for _ in range(rng.randint(0, 200)))
            )
            pieces.append(rng.choice(BOUNDARIES + [""]))

info ~~warn~~ → info by review decoder-decompress-payload nltk/test/unit/test_attack_gzip_names_expanded.py:264:12 new in this release

file imports decoder module(s) gzip and calls `.decompress()` — embedded payload decompressed at runtime (ast pass, confidence 0.75)
```
    assert _read_text(_buffered(name, "rb")) == b"".join(ans).decode()
    assert gzip.decompress(open(name, "rb").read()) == b"".join(ans)
```
likely benign LLM, confidence 0.90 Test that decompresses a gzip fixture and compares it to expected content.
Reasoning
gzip.decompress is applied to a test fixture file and compared with expected bytes. This is a round-trip check of gzip handling, not a runtime-hidden payload.
info ~~warn~~ → info by review decoder-decompress-payload nltk/test/unit/test_attack_json_loaders_expanded.py:1132:20 new in this release

file imports decoder module(s) gzip and calls `.decompress()` — embedded payload decompressed at runtime (ast pass, confidence 0.75)
```
    assert len(compressed) < len(inflated) // 10  # genuinely a bomb
    decompressed = gzip.decompress(compressed)
    with pytest.raises(ValueError, match="over the"):
        safe_json_loads(decompressed, max_bytes=1000)
```
likely benign LLM, confidence 0.90 Test that feeds a decompressed gzip bomb to safe_json_loads and expects a size-limit error.
Reasoning
The test deliberately decompresses a gzip bomb and checks that safe_json_loads rejects it with a size error. It is a security regression test for a defensive loader.

info unicode-homoglyph nltk/test/unit/test_attack_path_sandbox_expanded.py:508:55 new in this release

identifier `corpus_ελληνικά` contains homoglyph candidates: η (03B7)→n, ν (03BD)→v, κ (03BA)→k (in string/comment) (raw pass, confidence 0.35)

    def test_unicode_directory_is_accepted_by_the_dir_guard(self, restricted_sandbox):
        directory = os.path.join(restricted_sandbox, "corpus_ελληνικά")
        os.makedirs(directory)
        assert validate_tool_dir(directory)

info unicode-mixed-script nltk/test/unit/test_attack_path_sandbox_expanded.py:508:55 new in this release

identifier `corpus_ελληνικά` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)

    def test_unicode_directory_is_accepted_by_the_dir_guard(self, restricted_sandbox):
        directory = os.path.join(restricted_sandbox, "corpus_ελληνικά")
        os.makedirs(directory)
        assert validate_tool_dir(directory)

info ~~critical~~ → info by review dynamic-import nltk/test/unit/test_attack_pickle_candidates_expanded.py:212:24 new in this release

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
        module, name = pair
        real = getattr(__import__(module, fromlist=[name]), name)
        resolved = _broad_loader().find_class(module, name)
        assert callable(resolved)
```
likely benign LLM, confidence 0.85 Test that resolves module/name pairs dynamically to validate pickle class lookup.
Reasoning
The dynamic import resolves module and name pairs from a test list to check that a pickle loader's find_class behaves correctly. The inputs are fixed test data, not untrusted runtime input.

info unicode-homoglyph nltk/test/unit/test_attack_pickle_candidates_expanded.py:358:30 new in this release

identifier `оs` contains homoglyph candidates: о (043E)→o (in string/comment) (raw pass, confidence 0.35)

    # Cyrillic small letter 'o' (U+043E) and 'ѕ' (U+0455) resemble ASCII o / s.
    _CONFUSABLE_VARIANTS = ["оs", "oѕ", "оѕ", "subprocesѕ"]

    @pytest.mark.parametrize("module", _CASE_VARIANTS)

info unicode-mixed-script nltk/test/unit/test_attack_pickle_candidates_expanded.py:358:30 new in this release

identifier `оs` mixes scripts: Cyrillic + Latin (in string/comment) (raw pass, confidence 0.40)

    # Cyrillic small letter 'o' (U+043E) and 'ѕ' (U+0455) resemble ASCII o / s.
    _CONFUSABLE_VARIANTS = ["оs", "oѕ", "оѕ", "subprocesѕ"]

    @pytest.mark.parametrize("module", _CASE_VARIANTS)

info unicode-mixed-script nltk/test/unit/test_attack_pickle_candidates_expanded.py:358:37 new in this release

identifier `oѕ` mixes scripts: Latin + Cyrillic (in string/comment) (raw pass, confidence 0.40)

    # Cyrillic small letter 'o' (U+043E) and 'ѕ' (U+0455) resemble ASCII o / s.
    _CONFUSABLE_VARIANTS = ["оs", "oѕ", "оѕ", "subprocesѕ"]

    @pytest.mark.parametrize("module", _CASE_VARIANTS)

info unicode-mixed-script nltk/test/unit/test_attack_pickle_candidates_expanded.py:358:52 new in this release

identifier `subprocesѕ` mixes scripts: Latin + Cyrillic (in string/comment) (raw pass, confidence 0.40)

    # Cyrillic small letter 'o' (U+043E) and 'ѕ' (U+0455) resemble ASCII o / s.
    _CONFUSABLE_VARIANTS = ["оs", "oѕ", "оѕ", "subprocesѕ"]

    @pytest.mark.parametrize("module", _CASE_VARIANTS)

info ~~warn~~ → info by review frame-introspection nltk/test/unit/test_attack_punkt_model_available_expanded.py:102:5 new in this release

`sys.setprofile` — call-stack frame introspection (ast pass, confidence 0.70)
```
    sys.setprofile(profile)
    try:
        yield seen
```
likely benign LLM, confidence 0.85 Test fixture that temporarily installs a profiler to record calls.
Reasoning
sys.setprofile is installed in a test context manager to record which functions are called, and the previous profiler is restored in a finally block. This is a test instrumentation technique with no exfiltration or hiding.
info ~~warn~~ → info by review frame-introspection nltk/test/unit/test_attack_punkt_model_available_expanded.py:106:9 new in this release

`sys.setprofile` — call-stack frame introspection (ast pass, confidence 0.70)
```
    finally:
        sys.setprofile(previous)
```
likely benign LLM, confidence 0.85 Cleanup restoring the previous profiler after a test fixture.
Reasoning
This is the restore step of the same profiler fixture, resetting the previous profile in finally. It is cleanup of benign test instrumentation.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_attack_translate_expanded.py:348:12 new in this release

`eval` called on a non-literal expression (ast pass, confidence 0.70)
```
    assert str(alignment) == " ".join(f"{i}-{j}" for i, j in aligned)
    assert eval(repr(alignment), {"Alignment": Alignment}) == alignment
    assert alignment.range() == sorted({j for _, j in aligned})
    for i in {p[0] for p in pairs}:
```
likely benign LLM, confidence 0.90 Test that evaluates the repr of an Alignment to verify round-tripping.
Reasoning
eval runs on repr(alignment) with only the Alignment class in scope, as a round-trip equality check in a test. The input is generated by the code under test, not external data.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_attack_translate_expanded.py:444:12 new in this release

`eval` called on a non-literal expression (ast pass, confidence 0.70)
```
    assert str(a) == f"0-0 {_HUGE_INDEX}-{_HUGE_INDEX}"
    assert eval(repr(a), {"Alignment": Alignment}) == a
    assert a.invert() == a
```
likely benign LLM, confidence 0.90 Test that evaluates an Alignment repr with a restricted namespace to verify round-tripping.
Reasoning
Same repr round-trip check with a restricted namespace, using a large index value in a test. It verifies the Alignment repr and is not a dropper.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_attack_tree_repr_recursion.py:110:16 new in this release

`eval` called on a non-literal expression (ast pass, confidence 0.70)
```
        assert rendered == _recursive_repr(tree)
        assert eval(rendered, {"Tree": Tree}) == tree
        assert Tree.fromstring(str(tree)) == tree
```
likely benign LLM, confidence 0.90 Test that evaluates a rendered Tree repr to verify round-tripping.
Reasoning
eval on the rendered Tree repr with only Tree in the namespace, checked against the original tree. It is a serialization round-trip test for recursion handling.
info ~~critical~~ → info by review dynamic-import nltk/test/unit/test_chat80_val_round_trip.py:52:18 new in this release

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
    try:
        module = __import__(name, fromlist=["open"])
    except ImportError:
        pytest.skip(f"{name} is not available on this interpreter")
```
likely benign LLM, confidence 0.90 Test helper that dynamically imports an optional module and skips if unavailable.
Reasoning
A try/except around __import__ with pytest.skip when the module is unavailable. This is optional-dependency handling in a test.

info unicode-homoglyph nltk/test/unit/test_corenlp_options_security.py:248:8 new in this release

identifier `рort` contains homoglyph candidates: р (0440)→p (in string/comment) (raw pass, confidence 0.35)

    ["-mx2g"],  # JVM sizing flag does not belong on the server command
    ["-рort", "9000"],  # Cyrillic homoglyph of -port
    ["-port", "９０００"],  # fullwidth digits
    ["-server_id", "a" * 5000],  # oversize token

info unicode-mixed-script nltk/test/unit/test_corenlp_options_security.py:248:8 new in this release

identifier `рort` mixes scripts: Cyrillic + Latin (in string/comment) (raw pass, confidence 0.40)

    ["-mx2g"],  # JVM sizing flag does not belong on the server command
    ["-рort", "9000"],  # Cyrillic homoglyph of -port
    ["-port", "９０００"],  # fullwidth digits
    ["-server_id", "a" * 5000],  # oversize token

info unicode-homoglyph nltk/test/unit/test_corenlp_options_security.py:1157:14 new in this release

identifier `ро` contains homoglyph candidates: р (0440)→p, о (043E)→o (in string/comment) (raw pass, confidence 0.35)
```
            "ｆｕ",  # fullwidth
            "ро",  # cyrillic homoglyph of "po"
            "@/tmp/argfile",
            "a;b",
```
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_decorators_security.py:292:5 new in this release

`exec` called on a non-literal expression (ast pass, confidence 0.70)
```
    ns = {}
    exec(  # bare-exec ok: builds the victim function with the reserved name
        f"def victim({shape.format(name=reserved_name)}):\n    return 1", ns
    )
```
likely benign LLM, confidence 0.90 Test code using exec to define a victim function with a reserved name.
Reasoning
exec builds a victim function with a reserved parameter name to test decorator security handling. The comment explains the purpose, and the code only defines a function in a test namespace.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_decorators_security.py:461:5 new in this release

`exec` called on a non-literal expression (ast pass, confidence 0.70)
```
    ns = {}
    exec(  # bare-exec ok: builds a victim with the soft-keyword/dunder name
        f"def victim({name}):\n    return {name}", ns
    )
```
likely benign LLM, confidence 0.90 Test code using exec to define a victim function with a soft-keyword or dunder name.
Reasoning
exec defines a victim function with a soft-keyword or dunder name for a decorator regression test. The comment explains the purpose, and the snippet shows no payload.
info ~~warn~~ → info by review dynamic-execution nltk/test/unit/test_decorators_security.py:474:9 new in this release

`exec` called on a non-literal expression (ast pass, confidence 0.70)
```
        ns = {}
        exec(  # bare-exec ok: builds the two victims
            f"def long_one({long_name}):\n    return {long_name}\n"
            f"def many_one({many}):\n    return p0 + p4999",
```
likely benign LLM, confidence 0.90 Test code using exec to define functions with long or many parameter names for decorator testing.
Reasoning
The exec call builds Python source from f-string parameter names inside a unit test that deliberately creates functions with very long or many parameters to exercise decorator security. The inline comment explains the intent. The inputs are test-generated identifiers, not attacker-controlled data, so this is a legitimate test fixture pattern.

info unicode-zero-width nltk/test/unit/test_pathsec_sweep_tag.py:745:16 new in this release

U+200D ZERO WIDTH JOINER in source (in string/comment) (raw pass, confidence 0.45)

    ("emoji", "robot_\U0001f916.model"),
    ("zwj", "co⟨U+200D⟩op.model"),  # zero-width joiner inside the name
    ("mixed-space", "my café 模型.model"),
    ("bidi-rlo", "file\u202edoc.model"),  # right-to-left override, a valid char

info unicode-homoglyph nltk/test/unit/test_pathsec_trusted_exec.py:588:16 new in this release

identifier `α` contains homoglyph candidates: α (03B1)→a (in string/comment) (raw pass, confidence 0.35)
```
    0x4E2D,
    0x65E5,  # α 中 日
    0x905,
    0x5D0,
```
info unicode-mixed-script nltk/test/unit/test_termsec_attack_matrix.py:245:13 new in this release

identifier `bשלום` mixes scripts: Latin + Hebrew (in string/comment) (raw pass, confidence 0.40)
```
    "user \u2066مرحبا\u2069 posted",
    "x \u202bשלום\u202c y",
]
ATTACKS.update({f"from-3850-{i:02d}": p for i, p in enumerate(EXTRA_ATTACKS)})
```
info ~~warn~~ → info by review frame-introspection nltk/test/unit/test_ycoe_streamview_pathsec.py:180:39 new in this release

`sys._getframe` — call-stack frame introspection (ast pass, confidence 0.70)
```
        def spy(path, *args, **kwargs):
            caller = os.path.realpath(sys._getframe(1).f_code.co_filename)
            if caller in own:
                seen.append(os.fspath(path) if not isinstance(path, int) else path)
```
likely benign LLM, confidence 0.90 Test spy inspects its caller's file via sys._getframe to verify path handling.
Reasoning
The test uses sys._getframe(1) inside a spy function to check whether the caller is the module under test, which is a common technique for path-security tests that intercept open calls. It is confined to a test file and does not exfiltrate or hide behavior.

info unicode-mixed-script nltk/test/unit/test_zip_member_collision.py:61:40 new in this release

identifier `ﬁle` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)

        ),
        ([("pkg/file.txt", "x"), ("pkg/ﬁle.txt", "y")], "ligature"),
        ([("a/x.json", "1"), ("A/x.json", "2")], "case-dir"),
    ],

info ~~warn~~ → info by review encoding-base64 nltk/app/nemo_app.py:26:14

4 rows of base64-like lines, lines 26-29 (first: base64-like blob (2228 bytes, compression ratio 0.72)) (raw pass, confidence 0.60)
```
    "FIND": "R0lGODlhMAAiAPcAMf/////37//35//n1v97Off///f/9/f37/fexvfOvfeEQvd7QvdrQvdrKfdaKfdSMfdSIe/v9+/v7+/v5+/n3u/e1u/Wxu/Gre+1lO+tnO+thO+Ua+97Y+97Oe97Me9rOe9rMe9jOe9jMe9jIe9aMefe5+fe3ufezuece+eEWudzQudaIedSIedKMedKIedCKedCId7e1t7Wzt7Oxt7Gvd69vd69rd61pd6ljN6UjN6Ue96EY95zY95rUt5rQt5jMd5SId5KIdbn59be3tbGztbGvda1rdaEa9Z
```
likely benign LLM, confidence 0.85 Base64-encoded GIF image data embedded as a string constant in an NLTK app module.
Reasoning
The raw-pass finding is a base64 blob that begins with the GIF header (R0lGOD), consistent with embedded image assets for a GUI demo. It is a raw-pass finding, which has a higher false-positive rate, and nothing shows it being decoded and executed.
info ~~critical~~ → info by review dynamic-import nltk/internals.py:1541:9

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
    sys.path = [d for d in sys.path if d not in ("", ".")]
    m = __import__(module)
    sys.path = old_path
    return m
```
likely benign LLM, confidence 0.75 Dynamic __import__ of a named module after temporarily adjusting sys.path.
Reasoning
This is a temporary sys.path manipulation used to import a module by name, then restores the path. Dynamic import by a caller-supplied name is a legitimate utility, though the module argument should be trusted.

info unicode-mixed-script nltk/langnames.py:697:13

identifier `ǂKxʼauǁʼein` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    "adp": "Adap",
    "aue": "ǂKxʼauǁʼein",
    "bmy": "Bemba (Democratic Republic of Congo)",
    "bxx": "Borna (Democratic Republic of Congo)",

info ~~critical~~ → info by review dynamic-import nltk/lazyimport.py:93:57

`__import__` called on a non-literal expression (ast pass, confidence 0.70)
```
            safe_print("LazyModule: Loading module %r" % full_name)
        self.__lazymodule_locals[local_name] = module = __import__(
            full_name, self.__lazymodule_locals, self.__lazymodule_globals, "*"
        )
```
likely benign LLM, confidence 0.85 Lazy module loader calling __import__ to load the real module on demand.
Reasoning
A lazy-module loader that imports the underlying module on first attribute access. Dynamic __import__ with a fixed full_name from the lazy wrapper is standard for this design.
info unicode-mixed-script nltk/metrics/aline.py:33:12

identifier `θin` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)
```
>>> align('θin', 'tenwis') # doctest: +SKIP
[[('θ', 't'), ('i', 'e'), ('n', 'n'), ('-', 'w'), ('-', 'i'), ('-', 's')]]
```
info unicode-mixed-script nltk/metrics/aline.py:1599:5

identifier `ən` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)
```
todos,tu
una,ən
dos,dø
tres,trwa
```
info unicode-mixed-script nltk/metrics/aline.py:1603:7

identifier `arbrə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
ombre,om
arbol,arbrə
pluma,plym
kabeθa,kap
```
info unicode-mixed-script nltk/metrics/aline.py:1605:1

identifier `kabeθa` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)
```
pluma,plym
kabeθa,kap
boka,buʃ
pje,pje
```
info unicode-mixed-script nltk/metrics/aline.py:1606:6

identifier `buʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
kabeθa,kap
boka,buʃ
pje,pje
koraθon,kœr
```
info unicode-mixed-script nltk/metrics/aline.py:1608:1

identifier `koraθon` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)
```
pje,pje
koraθon,kœr
ber,vwar
benir,vənir
```
info unicode-mixed-script nltk/metrics/aline.py:1610:7

identifier `vənir` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
ber,vwar
benir,vənir
deθir,dir
pobre,povrə
```
info unicode-mixed-script nltk/metrics/aline.py:1611:1

identifier `deθir` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)
```
benir,vənir
deθir,dir
pobre,povrə
ðis,dIzes
```
info unicode-mixed-script nltk/metrics/aline.py:1612:7

identifier `povrə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
deθir,dir
pobre,povrə
ðis,dIzes
ðæt,das
```
info unicode-mixed-script nltk/metrics/aline.py:1619:1

identifier `fleʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
mæn,man
fleʃ,flajʃ
bləd,blyt
feðər,fEdər
```
info unicode-mixed-script nltk/metrics/aline.py:1619:7

identifier `flajʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
mæn,man
fleʃ,flajʃ
bləd,blyt
feðər,fEdər
```
info unicode-mixed-script nltk/metrics/aline.py:1620:1

identifier `bləd` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
fleʃ,flajʃ
bləd,blyt
feðər,fEdər
hær,hAr
```
info unicode-mixed-script nltk/metrics/aline.py:1621:1

identifier `feðər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
bləd,blyt
feðər,fEdər
hær,hAr
ir,Or
```
info unicode-mixed-script nltk/metrics/aline.py:1621:9

identifier `fEdər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
bləd,blyt
feðər,fEdər
hær,hAr
ir,Or
```
info unicode-mixed-script nltk/metrics/aline.py:1624:4

identifier `awgə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
ir,Or
aj,awgə
nowz,nAzə
mawθ,munt
```
info unicode-mixed-script nltk/metrics/aline.py:1625:6

identifier `nAzə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
aj,awgə
nowz,nAzə
mawθ,munt
təŋ,tsuŋə
```
info unicode-mixed-script nltk/metrics/aline.py:1626:1

identifier `mawθ` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)
```
nowz,nAzə
mawθ,munt
təŋ,tsuŋə
fut,fys
```
info unicode-mixed-script nltk/metrics/aline.py:1627:1

identifier `təŋ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
mawθ,munt
təŋ,tsuŋə
fut,fys
nij,knI
```
info unicode-mixed-script nltk/metrics/aline.py:1627:7

identifier `tsuŋə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
mawθ,munt
təŋ,tsuŋə
fut,fys
nij,knI
```
info unicode-mixed-script nltk/metrics/aline.py:1632:1

identifier `livər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
hart,herts
livər,lEbər
ænd,ante
æt,ad
```
info unicode-mixed-script nltk/metrics/aline.py:1632:8

identifier `lEbər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
hart,herts
livər,lEbər
ænd,ante
æt,ad
```
info unicode-mixed-script nltk/metrics/aline.py:1638:1

identifier `fiʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
ijt,edere
fiʃ,piʃkis
flow,fluere
staɾ,stella
```
info unicode-mixed-script nltk/metrics/aline.py:1638:6

identifier `piʃkis` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
ijt,edere
fiʃ,piʃkis
flow,fluere
staɾ,stella
```
info unicode-mixed-script nltk/metrics/aline.py:1640:1

identifier `staɾ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
flow,fluere
staɾ,stella
ful,plenus
græs,gramen
```
info unicode-mixed-script nltk/metrics/aline.py:1647:1

identifier `məðər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
nij,genU
məðər,mAter
mawntən,mons
nejm,nomen
```
info unicode-mixed-script nltk/metrics/aline.py:1648:1

identifier `mawntən` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
məðər,mAter
mawntən,mons
nejm,nomen
njuw,nowus
```
info unicode-mixed-script nltk/metrics/aline.py:1651:1

identifier `wən` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
njuw,nowus
wən,unus
rawnd,rotundus
sow,suere
```
info unicode-mixed-script nltk/metrics/aline.py:1655:1

identifier `θrij` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)
```
sit,sedere
θrij,tres
tuwθ,dentis
θin,tenwis
```
info unicode-mixed-script nltk/metrics/aline.py:1656:1

identifier `tuwθ` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)
```
θrij,tres
tuwθ,dentis
θin,tenwis
kinwawa,kenuaʔ
```
info unicode-mixed-script nltk/metrics/aline.py:1657:1

identifier `θin` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)
```
tuwθ,dentis
θin,tenwis
kinwawa,kenuaʔ
nina,nenah
```
info unicode-mixed-script nltk/metrics/aline.py:1658:9

identifier `kenuaʔ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
θin,tenwis
kinwawa,kenuaʔ
nina,nenah
napewa,napɛw
```
info unicode-mixed-script nltk/metrics/aline.py:1660:8

identifier `napɛw` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
nina,nenah
napewa,napɛw
wapimini,wapemen
namesa,namɛʔs
```
info unicode-mixed-script nltk/metrics/aline.py:1662:8

identifier `namɛʔs` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
wapimini,wapemen
namesa,namɛʔs
okimawa,okemaw
ʃiʃipa,seʔsep
```
info unicode-mixed-script nltk/metrics/aline.py:1664:1

identifier `ʃiʃipa` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)
```
okimawa,okemaw
ʃiʃipa,seʔsep
ahkohkwa,ahkɛh
pematesiweni,pematesewen
```
info unicode-mixed-script nltk/metrics/aline.py:1664:10

identifier `seʔsep` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
okimawa,okemaw
ʃiʃipa,seʔsep
ahkohkwa,ahkɛh
pematesiweni,pematesewen
```
info unicode-mixed-script nltk/metrics/aline.py:1665:10

identifier `ahkɛh` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
ʃiʃipa,seʔsep
ahkohkwa,ahkɛh
pematesiweni,pematesewen
asenja,aʔsɛn"""
```
info unicode-mixed-script nltk/metrics/aline.py:1667:8

identifier `aʔsɛn` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)
```
pematesiweni,pematesewen
asenja,aʔsɛn"""
```
info identifier-low-length nltk/misc/sort.py:1:1

103 of 249 non-conventional identifiers are single-character (41%) (token pass, confidence 0.50)
info ~~warn~~ → info by review dynamic-attribute nltk/stem/snowball.py:107:24

reach-by-name through `globals` (ast pass, confidence 0.60)
```
            raise ValueError(f"The language '{language}' is not supported.")
        stemmerclass = globals()[language.capitalize() + "Stemmer"]
        self.stemmer = stemmerclass(ignore_stopwords)
        self.stem = self.stemmer.stem
```
likely benign LLM, confidence 0.90 Selects a Snowball stemmer class by language name from module globals.
Reasoning
The stemmer class is looked up by name from the module's globals using a language string. The language is checked against a supported list earlier, and the target is a known stemmer class, so this is a plain dispatch table pattern.

info long-line nltk/test/floresta.txt:224:1

line length 600 characters (raw pass, confidence 0.50)

Os temas escolhidos s�o cinco: �Patrim�nio Virtual� (explora��o dos monumentos portugueses usando as tecnologias da realidade virtual); �Portugal Global� (Sagres como antena de expans�o e comunica��o da expans�o global e local dos portugueses); �Sacra Saturni� (Sagres como lugar de mist�ri… (341 bytes total)

info long-line nltk/test/floresta.txt:1361:1

line length 561 characters (raw pass, confidence 0.50)

Poucos minutos depois do final da reuni�o que, durante cerca de tr�s horas, juntou o Provedor demission�rio e os deputados da Comiss�o Parlamentar de Direitos, Liberdades e Garantias, era divulgado um comunicado onde se real�a �a actua��o de elevado m�rito do dr. M�rio Raposo, marcada por crit�rio… (341 bytes total)

info long-line nltk/test/floresta.txt:1591:1

line length 565 characters (raw pass, confidence 0.50)

Mas para al�m dos temas desse �lbum, e em particular da vers�o minimalista de �All apologies�, um original de Kurt Cobain que Sin�ad adoptou j� depois da morte do l�der dos Nirvana, espera-se ainda a interpreta��o de algumas das outras can��es que a tornaram notada, em particular as do �lbum … (340 bytes total)

info long-line nltk/test/floresta.txt:2155:1

line length 533 characters (raw pass, confidence 0.50)

A demiss�o de Hans-Dietrich Genscher e a crise pol�tica que atravessa o Governo de Bona (tendo em pano de fundo uma �derrapagem� econ�mica, resultado da unifica��o, que est� a afectar a Alemanha e a ser inevitavelmente �exportada� para os outros pa�ses europeus) �, talvez, o maior rev�s que a … (341 bytes total)

info long-line nltk/test/floresta.txt:2211:1

line length 681 characters (raw pass, confidence 0.50)

Porque, depois deste registo de 1972, ficaria imortalizada em som e imagem -- na realiza��o visual de Jean-Pierre Ponnelle -- uma aproxima��o dos finais dos anos 70 deliberadamente expressionista e, agora, em 1993, em Salzburgo, Harnoncourt surpreende (e abre uma inflamada pol�mica) ao explorar at� aos limi… (341 bytes total)

info long-line nltk/test/floresta.txt:2475:1

line length 749 characters (raw pass, confidence 0.50)

Ao longo de uma obra original em que tem procurado figurar a g�nese e os momentos mais marcantes da nossa modernidade filos�fica e est�tica, o fil�sofo Manfred Frank (ainda desconhecido do leitor portugu�s, mas j� muito traduzido em Fran�a e na It�lia) serve-se, entre outros, do mito do judeu errante, n… (341 bytes total)

info long-line nltk/test/floresta.txt:2476:1

line length 896 characters (raw pass, confidence 0.50)

As duas vertentes -- Aasvero e o Holand�s -- cruzam-se e fundem-se a partir do s�culo XVI, com um predom�nio, no plano da mat�ria dieg�tica, da deriva mar�tima sobre a err�ncia terrestre, desde �Os Lus�adas� e as narrativas de viagens inglesas e holandesas, at� ao �Ancient Mariner� de Coleridg… (341 bytes total)

info long-line nltk/test/floresta.txt:2608:1

line length 566 characters (raw pass, confidence 0.50)

Quanto a Portugal, Ferreira do Amaral reafirmou ontem que as vias priorit�rias de liga��o � Galiza s�o a conclus�o da auto-estrada Porto-Valen�a- e bem assim a conclus�o do IP1, entre a ponte que ontem inaugurou com Jos� Borrell e a ponte que ambos tamb�m inauguraram h� dois anos no Guadiana-, a a… (341 bytes total)

info long-line nltk/test/floresta.txt:2698:1

line length 548 characters (raw pass, confidence 0.50)

Analisando estes dados, a AIP constata que, �se � um facto que em todos os subconjuntos (empresas industriais, de constru��o, com�rcio e servi�os, exportadoras e n�o exportadoras) se verifica uma maioria de empresas em que os encargos financeiros s�o inferiores a cinco por cento do volume de vendas, �… (341 bytes total)

info long-line nltk/test/floresta.txt:3081:1

line length 556 characters (raw pass, confidence 0.50)

Assim, encorajado pela total disponibilidade do meu interlocutor, que igualmente admito, tentaria que entendesse, pelo menos, as vantagens econ�micas, em termos de desenvolvimento local (tur�stico, por exemplo), da salvaguarda da invulgarmente importante, mesmo � escala mundial, jazida com pegadas de dinoss�uri… (341 bytes total)

info long-line nltk/test/floresta.txt:3940:1

line length 546 characters (raw pass, confidence 0.50)

Da lista fazem parte filmes como �Au nom du Christ�, do realizador Roger Gnoan M'Bala (Costa do Marfim), �Beijing Zazhong�, de Zhang Yuan, considerado o primeiro filme �punk-rock� chin�s, �Bhaji on the Beach�, de Gurinder Chadha (Gr�-Bretanha), �Koraku Zaru�, do japon�s Kenchi Iwamoto, �L'… (339 bytes total)

info long-line nltk/test/floresta.txt:5178:1

line length 508 characters (raw pass, confidence 0.50)

�Adeus America� de Jan Schutte, �O Casamento de Muriel� de P.J. Hogan, �71 Fragmentos da Cronologia de Um Acaso�, de Michael Haneke, �Fresh� de Boaz Yakin, �Venus de Neve� de Sotiris Gortisas, �Costas �s Costas , Cara a Cara� de Huang Jianxin, �Rainha Bandida� de Shekkar Kapur, �Sem Pele… (341 bytes total)

info unicode-mixed-script nltk/test/unit/test_aline.py:9:27

identifier `θin` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)
```
def test_aline():
    result = aline.align("θin", "tenwis")
    expected = [[("θ", "t"), ("i", "e"), ("n", "n")]]
```
info unicode-mixed-script nltk/test/unit/test_aline.py:38:27

identifier `tuwθ` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)
```
    result = aline.align("tuwθ", "dentis")
    expected = [[("t", "t"), ("u", "i"), ("w", "-"), ("θ", "s")]]
```

info high-complexity nltk/test/unit/test_tokenize.py:441:18

compression ratio 0.879 is >2σ above file mean 0.523 (raw pass, confidence 0.50)

        # A string containing both skin tone modifier and ZWJ Sequences
        test3 = "🤔 🙈 me así, se😌 ds 💕👭👙 hello 👩🏾⟨U+200D⟩🎓 emoji hello 👨⟨U+200D⟩👩⟨U+200D⟩👦⟨U+200D⟩👦 how are 😊 you today🙅🏽🙅🏽"
        expected = [
            "🤔",

info unicode-mixed-script nltk/tokenize/nist.py:50:36

identifier `ˈæməzɒn` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)

    >>> amz = u'Amazon.com, Inc. (/ˈæməzɒn/) is an American electronic commerce...'
    >>> rkt = u'Rakuten, Inc. (楽天株式会社 Rakuten Kabushiki-gaisha) is a Japanese electronic commerce and Internet company based in Tokyo.'

info ~~warn~~ → info by review high-complexity nltk/tokenize/nist.py:51:17

string literal compression ratio 0.954 (130 bytes) (raw pass, confidence 0.60)
```
    >>> amz = u'Amazon.com, Inc. (/ˈæməzɒn/) is an American electronic commerce...'
    >>> rkt = u'Rakuten, Inc. (楽天株式会社 Rakuten Kabushiki-gaisha) is a Japanese electronic commerce and Internet company based in Tokyo.'

    # Expected tokens.
```
dismissed LLM, confidence 0.95 Docstring doctest sample text in the NIST tokenizer, flagged by a raw-pass compression heuristic.
Reasoning
The flagged content is a docstring example containing ordinary Unicode text (Japanese and IPA characters) used as tokenizer test input. The high compression ratio is an artifact of a short string literal with repeated characters in a doctest, not obfuscation.
info unicode-mixed-script nltk/translate/ibm4.py:96:20

identifier `Δj` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)
```
:p0: 1 - p1
:dj: Displacement, Δj

References
```
info unicode-mixed-script nltk/translate/ibm5.py:99:26

identifier `Δv` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)
```
:max_v: Maximum vacancy
:dv: Vacancy difference, Δv

The definition of v_head here differs from GIZA++, section 4.7 of
```

Reproduce This Scan

Run disclude locally to verify. Add --format sarif for SARIF output. Add --llm (with an API key in the environment) to include the LLM review.

cargo install disclude --version 4.1.0
git clone https://github.com/nltk/nltk nltk && cd nltk
git checkout v3.10.4
disclude scan --severity info --diff v3.10.3 .