PyPI / nltk / 3.10.4

nltk 3.10.4

Source
https://github.com/nltk/nltk
Artifact scanned
nltk-3.10.4-py3-none-any.whl (wheel · 1.6 MB · SHA-256 769ba5413f26…)
Compared with
nltk-3.10.3-py3-none-any.whl (5 signals introduced since)
Files
319 scanned, 63 skipped (binary, oversized, or unrecognised)
Signals
0 critical · 0 warn · 52 info
Scan time
12.3 s
LLM review
Reviewed by claude-haiku-5-5 (Anthropic): 10 of 10 warn and critical signals; 10 demoted to info.
Disclude version
4.3.0
Scanned (UTC)
2026-10-23 12:24

Signals

Ordered by severity, then signals introduced in this release. Hidden characters are shown as ⟨U+…⟩.

  1. info warn → info by review dynamic-execution nltk/decorators.py:327:15 new in this release

    `eval` called on a non-literal expression (ast pass, confidence 0.70)

        src = f"lambda {signature}: _wrapper_({callargs})"
        funcopy = eval(
            src, dict(_wrapper_=wrapper)
        )  # bare-exec ok: _assert_safe_signature fenced src (CVE-2026-14727)
    likely benign LLM, confidence 0.75 Builds a lambda source string from a function signature and evals it to create a signature-preserving wrapper.
    Reasoning

    This is the standard decorator-library pattern: a lambda is built from the wrapped function's signature and eval'd with a restricted globals dict. The comment says the source is checked by a signature-safety function first. The eval input is derived from code signatures, not network or user input, so the risk is low, though eval is still a sensitive sink.

  2. info warn → info by review dynamic-execution nltk/decorators.py:398:20 new in this release

    `eval` called on a non-literal expression (ast pass, confidence 0.70)

            # import sys; print >> sys.stderr, src # for debugging purposes
            dec_func = eval(
                src, dict(_func_=func, _call_=caller)
            )  # bare-exec ok: _assert_safe_signature fenced src (CVE-2026-14727)
    likely benign LLM, confidence 0.75 Evals generated wrapper source to build a decorated function that preserves the original signature.
    Reasoning

    Same decorator-factory pattern as the sibling finding: src is generated from the decorated function's signature and eval'd with a fixed namespace of _func_ and _call_. The inline comment cites a validation step on the source. Nothing here reaches external input or does anything beyond defining a wrapper.

  3. info warn → info by review dynamic-execution nltk/redos.py:320:12 new in this release

    `compile` called on a non-literal expression (ast pass, confidence 0.70)

        :data:`DEFAULT_TIMEOUT` at match time."""
        return compile(src, flags)
    likely benign LLM, confidence 0.85 Compiles a regex source string into a pattern object, with flags.
    Reasoning

    compile() here turns a regex source string into a pattern object with flags. Compiling dynamic regex text is ordinary library behavior and is not code execution. The surrounding docstring describes a match-time timeout, consistent with ReDoS protection.

  4. info warn → info by review dynamic-execution nltk/redos.py:335:12 new in this release

    `compile` called on a non-literal expression (ast pass, confidence 0.70)

            return _cached_compile(pattern, flags)
        return compile(pattern, flags)
    likely benign LLM, confidence 0.90 Compiles a regex pattern with flags as a fallback to the cached compile.
    Reasoning

    This is a fallback that compiles a regex pattern with flags after a cache path. Regex compilation is the expected behavior and carries no execution risk.

  5. info warn → info by review dynamic-execution nltk/redos.py:592:12 new in this release

    `compile` called on a non-literal expression (ast pass, confidence 0.70)

            flags = getattr(pattern, "flags", 0)
        return compile(source_of(pattern), flags)
    likely benign LLM, confidence 0.85 Compiles a pattern's source text back into a regex object with its flags.
    Reasoning

    This compiles the textual form of a pattern object, obtained through source_of, with its flags. It is regex handling, not code execution, and fits the module's purpose of analyzing and guarding regex patterns.

  6. info warn → info by review encoding-base64 nltk/app/nemo_app.py:26:14

    4 rows of base64-like lines, lines 26-29 (first: base64-like blob (2228 bytes, compression ratio 0.72)) (raw pass, confidence 0.60)

        "FIND": "R0lGODlhMAAiAPcAMf/////37//35//n1v97Off///f/9/f37/fexvfOvfeEQvd7QvdrQvdrKfdaKfdSMfdSIe/v9+/v7+/v5+/n3u/e1u/Wxu/Gre+1lO+tnO+thO+Ua+97Y+97Oe97Me9rOe9rMe9jOe9jMe9jIe9aMefe5+fe3ufezuece+eEWudzQudaIedSIedKMedKIedCKedCId7e1t7Wzt7Oxt7Gvd69vd69rd61pd6ljN6UjN6Ue96EY95zY95rUt5rQt5jMd5SId5KIdbn59be3tbGztbGvda1rdaEa9Z
    likely benign LLM, confidence 0.90 Embedded base64-encoded GIF image data in the NLTK NeMo demo app.
    Reasoning

    Base64 blob inside a GUI demo app storing GIF image data for button/label images; this is ordinary embedded image resource data in a Tkinter demo.

  7. info critical → info by review dynamic-import nltk/internals.py:1541:9

    `__import__` called on a non-literal expression (ast pass, confidence 0.70)

        sys.path = [d for d in sys.path if d not in ("", ".")]
        m = __import__(module)
        sys.path = old_path
        return m
    likely benign LLM, confidence 0.80 Dynamically imports a module by name after temporarily adjusting sys.path.
    Reasoning

    This is a helper that temporarily strips '' and '.' from sys.path, imports a named module, and restores sys.path. Dynamic import by module name is a normal plugin/stdlib-loading technique, and the module name comes from the library's own callers rather than untrusted data.

  8. info unicode-mixed-script nltk/langnames.py:697:13

    identifier `ǂKxʼauǁʼein` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

        "adp": "Adap",
        "aue": "ǂKxʼauǁʼein",
        "bmy": "Bemba (Democratic Republic of Congo)",
        "bxx": "Borna (Democratic Republic of Congo)",
  9. info critical → info by review dynamic-import nltk/lazyimport.py:93:57

    `__import__` called on a non-literal expression (ast pass, confidence 0.70)

                safe_print("LazyModule: Loading module %r" % full_name)
            self.__lazymodule_locals[local_name] = module = __import__(
                full_name, self.__lazymodule_locals, self.__lazymodule_globals, "*"
            )
    likely benign LLM, confidence 0.85 Lazily imports a wrapped module on first attribute access.
    Reasoning

    This is the standard lazy-module loader: on first attribute access it calls __import__ with the stored full module name and the module's own globals and locals. This is a well-known legitimate design, and the name is fixed by the library's module structure.

  10. info unicode-mixed-script nltk/metrics/aline.py:33:12

    identifier `θin` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)

    
    >>> align('θin', 'tenwis') # doctest: +SKIP
    [[('θ', 't'), ('i', 'e'), ('n', 'n'), ('-', 'w'), ('-', 'i'), ('-', 's')]]
  11. info unicode-mixed-script nltk/metrics/aline.py:1599:5

    identifier `ən` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)

    todos,tu
    una,ən
    dos,dø
    tres,trwa
  12. info unicode-mixed-script nltk/metrics/aline.py:1603:7

    identifier `arbrə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    ombre,om
    arbol,arbrə
    pluma,plym
    kabeθa,kap
  13. info unicode-mixed-script nltk/metrics/aline.py:1605:1

    identifier `kabeθa` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)

    pluma,plym
    kabeθa,kap
    boka,buʃ
    pje,pje
  14. info unicode-mixed-script nltk/metrics/aline.py:1606:6

    identifier `buʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    kabeθa,kap
    boka,buʃ
    pje,pje
    koraθon,kœr
  15. info unicode-mixed-script nltk/metrics/aline.py:1608:1

    identifier `koraθon` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)

    pje,pje
    koraθon,kœr
    ber,vwar
    benir,vənir
  16. info unicode-mixed-script nltk/metrics/aline.py:1610:7

    identifier `vənir` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    ber,vwar
    benir,vənir
    deθir,dir
    pobre,povrə
  17. info unicode-mixed-script nltk/metrics/aline.py:1611:1

    identifier `deθir` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)

    benir,vənir
    deθir,dir
    pobre,povrə
    ðis,dIzes
  18. info unicode-mixed-script nltk/metrics/aline.py:1612:7

    identifier `povrə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    deθir,dir
    pobre,povrə
    ðis,dIzes
    ðæt,das
  19. info unicode-mixed-script nltk/metrics/aline.py:1619:1

    identifier `fleʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    mæn,man
    fleʃ,flajʃ
    bləd,blyt
    feðər,fEdər
  20. info unicode-mixed-script nltk/metrics/aline.py:1619:7

    identifier `flajʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    mæn,man
    fleʃ,flajʃ
    bləd,blyt
    feðər,fEdər
  21. info unicode-mixed-script nltk/metrics/aline.py:1620:1

    identifier `bləd` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    fleʃ,flajʃ
    bləd,blyt
    feðər,fEdər
    hær,hAr
  22. info unicode-mixed-script nltk/metrics/aline.py:1621:1

    identifier `feðər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    bləd,blyt
    feðər,fEdər
    hær,hAr
    ir,Or
  23. info unicode-mixed-script nltk/metrics/aline.py:1621:9

    identifier `fEdər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    bləd,blyt
    feðər,fEdər
    hær,hAr
    ir,Or
  24. info unicode-mixed-script nltk/metrics/aline.py:1624:4

    identifier `awgə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    ir,Or
    aj,awgə
    nowz,nAzə
    mawθ,munt
  25. info unicode-mixed-script nltk/metrics/aline.py:1625:6

    identifier `nAzə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    aj,awgə
    nowz,nAzə
    mawθ,munt
    təŋ,tsuŋə
  26. info unicode-mixed-script nltk/metrics/aline.py:1626:1

    identifier `mawθ` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)

    nowz,nAzə
    mawθ,munt
    təŋ,tsuŋə
    fut,fys
  27. info unicode-mixed-script nltk/metrics/aline.py:1627:1

    identifier `təŋ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    mawθ,munt
    təŋ,tsuŋə
    fut,fys
    nij,knI
  28. info unicode-mixed-script nltk/metrics/aline.py:1627:7

    identifier `tsuŋə` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    mawθ,munt
    təŋ,tsuŋə
    fut,fys
    nij,knI
  29. info unicode-mixed-script nltk/metrics/aline.py:1632:1

    identifier `livər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    hart,herts
    livər,lEbər
    ænd,ante
    æt,ad
  30. info unicode-mixed-script nltk/metrics/aline.py:1632:8

    identifier `lEbər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    hart,herts
    livər,lEbər
    ænd,ante
    æt,ad
  31. info unicode-mixed-script nltk/metrics/aline.py:1638:1

    identifier `fiʃ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    ijt,edere
    fiʃ,piʃkis
    flow,fluere
    staɾ,stella
  32. info unicode-mixed-script nltk/metrics/aline.py:1638:6

    identifier `piʃkis` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    ijt,edere
    fiʃ,piʃkis
    flow,fluere
    staɾ,stella
  33. info unicode-mixed-script nltk/metrics/aline.py:1640:1

    identifier `staɾ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    flow,fluere
    staɾ,stella
    ful,plenus
    græs,gramen
  34. info unicode-mixed-script nltk/metrics/aline.py:1647:1

    identifier `məðər` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    nij,genU
    məðər,mAter
    mawntən,mons
    nejm,nomen
  35. info unicode-mixed-script nltk/metrics/aline.py:1648:1

    identifier `mawntən` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    məðər,mAter
    mawntən,mons
    nejm,nomen
    njuw,nowus
  36. info unicode-mixed-script nltk/metrics/aline.py:1651:1

    identifier `wən` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    njuw,nowus
    wən,unus
    rawnd,rotundus
    sow,suere
  37. info unicode-mixed-script nltk/metrics/aline.py:1655:1

    identifier `θrij` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)

    sit,sedere
    θrij,tres
    tuwθ,dentis
    θin,tenwis
  38. info unicode-mixed-script nltk/metrics/aline.py:1656:1

    identifier `tuwθ` mixes scripts: Latin + Greek (in string/comment) (raw pass, confidence 0.40)

    θrij,tres
    tuwθ,dentis
    θin,tenwis
    kinwawa,kenuaʔ
  39. info unicode-mixed-script nltk/metrics/aline.py:1657:1

    identifier `θin` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)

    tuwθ,dentis
    θin,tenwis
    kinwawa,kenuaʔ
    nina,nenah
  40. info unicode-mixed-script nltk/metrics/aline.py:1658:9

    identifier `kenuaʔ` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    θin,tenwis
    kinwawa,kenuaʔ
    nina,nenah
    napewa,napɛw
  41. info unicode-mixed-script nltk/metrics/aline.py:1660:8

    identifier `napɛw` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    nina,nenah
    napewa,napɛw
    wapimini,wapemen
    namesa,namɛʔs
  42. info unicode-mixed-script nltk/metrics/aline.py:1662:8

    identifier `namɛʔs` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    wapimini,wapemen
    namesa,namɛʔs
    okimawa,okemaw
    ʃiʃipa,seʔsep
  43. info unicode-mixed-script nltk/metrics/aline.py:1664:1

    identifier `ʃiʃipa` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)

    okimawa,okemaw
    ʃiʃipa,seʔsep
    ahkohkwa,ahkɛh
    pematesiweni,pematesewen
  44. info unicode-mixed-script nltk/metrics/aline.py:1664:10

    identifier `seʔsep` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    okimawa,okemaw
    ʃiʃipa,seʔsep
    ahkohkwa,ahkɛh
    pematesiweni,pematesewen
  45. info unicode-mixed-script nltk/metrics/aline.py:1665:10

    identifier `ahkɛh` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    ʃiʃipa,seʔsep
    ahkohkwa,ahkɛh
    pematesiweni,pematesewen
    asenja,aʔsɛn"""
  46. info unicode-mixed-script nltk/metrics/aline.py:1667:8

    identifier `aʔsɛn` mixes scripts: Latin + Other (in string/comment) (raw pass, confidence 0.40)

    pematesiweni,pematesewen
    asenja,aʔsɛn"""
  47. info identifier-low-length nltk/misc/sort.py:1:1

    103 of 249 non-conventional identifiers are single-character (41%) (token pass, confidence 0.50)

  48. info warn → info by review dynamic-attribute nltk/stem/snowball.py:107:24

    reach-by-name through `globals` (ast pass, confidence 0.60)

                raise ValueError(f"The language '{language}' is not supported.")
            stemmerclass = globals()[language.capitalize() + "Stemmer"]
            self.stemmer = stemmerclass(ignore_stopwords)
            self.stem = self.stemmer.stem
    likely benign LLM, confidence 0.85 Looks up the language-specific stemmer class in module globals by constructed name.
    Reasoning

    The language argument is checked against supported languages earlier (ValueError is raised for unsupported ones), so the globals lookup is limited to the stemmer classes in this module. Resolving a class by name from a validated set is an ordinary dispatch pattern.

  49. info unicode-mixed-script nltk/tokenize/nist.py:50:36

    identifier `ˈæməzɒn` mixes scripts: Other + Latin (in string/comment) (raw pass, confidence 0.40)

        >>> amz = u'Amazon.com, Inc. (/ˈæməzɒn/) is an American electronic commerce...'
        >>> rkt = u'Rakuten, Inc. (楽天株式会社 Rakuten Kabushiki-gaisha) is a Japanese electronic commerce and Internet company based in Tokyo.'
  50. info warn → info by review high-complexity nltk/tokenize/nist.py:51:17

    string literal compression ratio 0.954 (130 bytes) (raw pass, confidence 0.60)

        >>> amz = u'Amazon.com, Inc. (/ˈæməzɒn/) is an American electronic commerce...'
        >>> rkt = u'Rakuten, Inc. (楽天株式会社 Rakuten Kabushiki-gaisha) is a Japanese electronic commerce and Internet company based in Tokyo.'
    
        # Expected tokens.
    likely benign LLM, confidence 0.90 Doctest sample text containing ordinary multilingual prose flagged for compressibility.
    Reasoning

    This is a doctest string literal of natural-language text with accented and CJK characters. Its high compression ratio reflects ordinary English and Japanese prose in a short sample, not an encoded payload. The raw pass has a high false-positive rate for this kind of content.

  51. info unicode-mixed-script nltk/translate/ibm4.py:96:20

    identifier `Δj` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)

    :p0: 1 - p1
    :dj: Displacement, Δj
    
    References
  52. info unicode-mixed-script nltk/translate/ibm5.py:99:26

    identifier `Δv` mixes scripts: Greek + Latin (in string/comment) (raw pass, confidence 0.40)

    :max_v: Maximum vacancy
    :dv: Vacancy difference, Δv
    
    The definition of v_head here differs from GIZA++, section 4.7 of

Reproduce This Scan

Run disclude locally to verify. These commands scan the published file; the "new in this release" marks come from comparing it with the previous release's file, which they don't repeat. Add --format sarif for SARIF output. Add --llm (with an API key in the environment) to include the LLM review.

cargo install disclude --version 4.3.0
curl -fsSL -o nltk-3.10.4-py3-none-any.whl https://files.pythonhosted.org/packages/a2/04/e7816813e409ca874ba2fcc35455db0e4130b79eb277e1f9edd270ca3c69/nltk-3.10.4-py3-none-any.whl
printf '%s  %s\n' 769ba5413f265b7d3e87cd163cc28efa01e29eeef8bb6080bc907949d9fcfb51 nltk-3.10.4-py3-none-any.whl | shasum -a 256 -c -
unzip -q nltk-3.10.4-py3-none-any.whl -d pkg
disclude scan --untrusted --severity info pkg