• 
      

    Tree Sitter: Add the update-queries script.

    Review Request #14519 — Created July 26, 2025 and submitted

    Information

    Review Board
    master

    Reviewers

    This change adds contrib/internal/treesitter/update-queries.py, which
    can go into an nvim-treesitter checkout and pull out highlights and
    injections queries.

    neovim supports an "inherits" key in queries files, which will merge
    queries at runtime. This script will do that same inheritance, but it
    just merges at the time of the script invocation. This makes our runtime
    a bit faster and simpler at the expense of slightly inflated file sizes.

    There's a fairly tight correlation between a grammar implementation and
    queries for that grammar. Because nvim-treesitter and
    tree-sitter-language-pack don't always agree on which version of a
    grammar to use (or even the specific implementation of a grammar for a
    given language), some queries may not work properly. This script
    therefore has some extra complexity:

    • We have a blacklist of languages which we know we don't want to get
      from nvim-treesitter. In most cases these are cases where there are
      multiple grammar implementations for a given language and the one
      used in nvim is different from the one in tree-sitter-language-pack.
    • For languages where it's using the same grammar implementation, the
      versions may be incompatible (for example, the highlights may depend
      on node types that are introduced in a grammar version later than the
      one tree-sitter-language-pack ships). In these cases, we'll look
      through the commit history and attempt to find a historical version of
      the queries that will load. This doesn't work for all languages, so
      I'm still working through what to do for those cases.

    Ran the script and verified that the resulting queries files are
    correct.

    Summary ID
    Tree Sitter: Add the update-queries script.
    This change adds `contrib/internal/treesitter/update-queries.py`, which can go into an nvim-treesitter checkout and pull out highlights and injections queries. neovim supports an "inherits" key in queries files, which will merge queries at runtime. This script will do that same inheritance, but it just merges at the time of the script invocation. This makes our runtime a bit faster and simpler at the expense of slightly inflated file sizes. There's a fairly tight correlation between a grammar implementation and queries for that grammar. Because nvim-treesitter and tree-sitter-language-pack don't always agree on which version of a grammar to use (or even the specific implementation of a grammar for a given language), some queries may not work properly. This script therefore has some extra complexity: - We have a blacklist of languages which we know we don't want to get from nvim-treesitter. In most cases these are cases where there are multiple grammar implementations for a given language and the one used in nvim is different from the one in tree-sitter-language-pack. - For languages where it's using the same grammar implementation, the versions may be incompatible (for example, the highlights may depend on node types that are introduced in a grammar version later than the one tree-sitter-language-pack ships). In these cases, we'll look through the commit history and attempt to find a historical version of the queries that will load. This doesn't work for all languages, so I'm still working through what to do for those cases. Testing Done: Ran the script and verified that the resulting queries files are correct.
    tklxmwstuurvkszttumsvznrrrtrquqr
    Description From Last Updated

    continuation line missing indentation or outdented Column: 13 Error code: E122

    reviewbotreviewbot

    Can you add , optional

    maubinmaubin

    In this case we're returning the incompatible queries. Is that what we want to do, or should we return None, …

    maubinmaubin

    Nit: Instead of mixing sys.stderr.write and logger.error should we standardize on using only the logger instead?

    maubinmaubin
    Checks run (1 failed, 1 succeeded)
    flake8 failed.
    JSHint passed.

    flake8

    david
    maubin
    1. 
        
    2. Show all issues

      Can you add , optional

    3. contrib/internal/treesitter/update-queries.py (Diff revision 2)
       
       
       
       
       
      Show all issues

      In this case we're returning the incompatible queries. Is that what we want to do, or should we return None, None instead?

    4. Show all issues

      Nit: Instead of mixing sys.stderr.write and logger.error should we standardize on using only the logger instead?

    5. 
        
    david
    maubin
    1. Ship It!
    2. 
        
    david
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to master (6c62676)