• 
  
    
 
    
 
    

    



    
    
    
    
     
    




    
 
    

  
    



    
 

 

  

  

 

    


  
    





    
 
    



  

    
 
 
    
  
    Tree Sitter: Add query utils.
    
 
    

    



 
    
 
    

  Review Request #14517
  —
  Created July 26, 2025 and updated 

 
    


    




    




    






 
    
  
    
   Information

  
    



    
 
 
    
  david
 
    

    





    
 
 
    
  Review Board
 
    

    





    
 
 
    
  master
 
    

    





    
 
 
    
  
 
    

    





    
 
 
    
  
 
    

    



 
    



 
    
  
    
   Reviewers

  
    



    
 
 
    
  reviewboard
 
    

    





    
 
 
    
  
 
    

    



 
    



    


    
 
    




  

    
 
 
    
  
    This change adds some utilities for manipulating Tree Sitter queries.
    
This new module has two main public-facing functions:
    

    get_all_predicate_names is a method that simply returns a list of all
    
the predicate names in a query file. This is used in the update-queries
    
script (coming in a later change) to print a warning if any of the
    
queries include predicates which we do not have implemented in Review
    
Board.
    

    apply_standard_query_edits is used to make edits to query files to
    
make them compatible with Review Board.
    

    For doing syntax highlighting with tree sitter, we're going to be
    
shipping many of the highlights files from the nvim-treesitter package.
    
These highlights files are often much, much better than the ones that
    
are shipped with the grammars themselves, but they do have a few things
    
that need to be worked around.
    

    Our "standard edits" include:
    

      

    • Removing any queries that use a set! directive that takes three
      
  arguments. This is something entirely custom inside neovim's
      
  implementation, and because set! is implemented in the
      
  py-tree-sitter C code, there's no way for us to do it. This is used in
      
  very few places in the actual queries files, and doesn't seem to
      
  impact syntax highlighting in any meaningful way, so we just get rid
      
  of them.
      • 

    • Rewriting lua-match? and not-lua-match? predicates to use Python
      
  regex syntax and change the predicate name to match? or not-match?
      
  as appropriate.
      • 

    • Rewriting gsub! directives to use Python regex syntax.
      • 

    

    The accompanying tests include unit tests for each of the edits, as well
    
as two parameterized tests for loading highlights and injections queries
    
for each language. These include a few xfail marks for languages where
    
the bundled queries are currently known to fail; as I work through
    
those, hopefully those lists will become completely empty.
    
 
    

    





  

    
 
 
    
  
      

    • Ran unit tests.
      • 

    • Used these from the update-queries script to rewrite queries files
      
  from nvim-treesitter to make them compatible with Review Board.
      • 

    • Verified that rewritten queries worked correctly, especially with
      
  rewritten match queries.
      • 

    
 
    

    




 
    

    





    
 
    




  
    
 
 
    
  


    
 
    
  
   
    

    Summary
    ID

   
    
      
  

   
    

    

     
    
      Tree Sitter: Add query utils.
      
    This change adds some utilities for manipulating Tree Sitter queries.
This new module has two main public-facing functions:

`get_all_predicate_names` is a method that simply returns a list of all
the predicate names in a query file. This is used in the update-queries
script (coming in a later change) to print a warning if any of the
queries include predicates which we do not have implemented in Review
Board.

`apply_standard_query_edits` is used to make edits to query files to
make them compatible with Review Board.

For doing syntax highlighting with tree sitter, we're going to be
shipping many of the highlights files from the nvim-treesitter package.
These highlights files are often much, much better than the ones that
are shipped with the grammars themselves, but they do have a few things
that need to be worked around.

Our "standard edits" include:

- Removing any queries that use a `set!` directive that takes three
  arguments. This is something entirely custom inside neovim's
  implementation, and because `set!` is implemented in the
  py-tree-sitter C code, there's no way for us to do it. This is used in
  very few places in the actual queries files, and doesn't seem to
  impact syntax highlighting in any meaningful way, so we just get rid
  of them.
- Rewriting `lua-match?` and `not-lua-match?` predicates to use Python
  regex syntax and change the predicate name to `match?` or `not-match?`
  as appropriate.
- Rewriting `gsub!` directives to use Python regex syntax.

The accompanying tests include unit tests for each of the edits, as well
as two parameterized tests for loading highlights and injections queries
for each language. These include a few xfail marks for languages where
the bundled queries are currently known to fail; as I work through
those, hopefully those lists will become completely empty.

Testing Done:
- Ran unit tests.
- Used these from the update-queries script to rewrite queries files
  from nvim-treesitter to make them compatible with Review Board.
- Verified that rewritten queries worked correctly, especially with
  rewritten match queries.
    
     
    

        		
    qvvsmvtyozynwrzswmwktwvyvvsluyzu

   
    


      
 
    

    
    
 
    

    






 









 
    

    



  
    
 
    

    


    












    
 




 

    


 













    
 



 
    
  
    
   
    



david




   
    
   
    
  
    
 
    


 

    












    
 



 
    
  
    
   
    



david




   
    
   
    
  
    
 
    


 
    
  
    

   
    
   
    
    
     Review request changed
    
    
   
    

  
    
  
    
  
    




    



  
    
   
    
    Change Summary:
   
    
   
    
    
    Add to coderef.
    
   
    
  
    







  
    
   
    
    Commits:
   
    
   
    
    
    



    
 
    
  
   
    

    

    Summary
    ID

   
    
      
  

   
    

    

    

     
    
      Tree Sitter: Add query utils.
      
    This change adds some utilities for manipulating Tree Sitter queries.
This new module has two main public-facing functions:

`get_all_predicate_names` is a method that simply returns a list of all
the predicate names in a query file. This is used in the update-queries
script (coming in a later change) to print a warning if any of the
queries include predicates which we do not have implemented in Review
Board.

`apply_standard_query_edits` is used to make edits to query files to
make them compatible with Review Board.

For doing syntax highlighting with tree sitter, we're going to be
shipping many of the highlights files from the nvim-treesitter package.
These highlights files are often much, much better than the ones that
are shipped with the grammars themselves, but they do have a few things
that need to be worked around.

Our "standard edits" include:

- Removing any queries that use a `set!` directive that takes three
  arguments. This is something entirely custom inside neovim's
  implementation, and because `set!` is implemented in the
  py-tree-sitter C code, there's no way for us to do it. This is used in
  very few places in the actual queries files, and doesn't seem to
  impact syntax highlighting in any meaningful way, so we just get rid
  of them.
- Rewriting `lua-match?` and `not-lua-match?` predicates to use Python
  regex syntax and change the predicate name to `match?` or `not-match?`
  as appropriate.
- Rewriting `gsub!` directives to use Python regex syntax.

The accompanying tests include unit tests for each of the edits, as well
as two parameterized tests for loading highlights and injections queries
for each language. These include a few xfail marks for languages where
the bundled queries are currently known to fail; as I work through
those, hopefully those lists will become completely empty.

Testing Done:
- Ran unit tests.
- Used these from the update-queries script to rewrite queries files
  from nvim-treesitter to make them compatible with Review Board.
- Verified that rewritten queries worked correctly, especially with
  rewritten match queries.
    
     
    

        		
    qvvsmvtyozynwrzswmwktwvyvvsluyzu

   
    



   
    

    

    

     
    
      Tree Sitter: Add query utils.
      
    This change adds some utilities for manipulating Tree Sitter queries.
This new module has two main public-facing functions:

`get_all_predicate_names` is a method that simply returns a list of all
the predicate names in a query file. This is used in the update-queries
script (coming in a later change) to print a warning if any of the
queries include predicates which we do not have implemented in Review
Board.

`apply_standard_query_edits` is used to make edits to query files to
make them compatible with Review Board.

For doing syntax highlighting with tree sitter, we're going to be
shipping many of the highlights files from the nvim-treesitter package.
These highlights files are often much, much better than the ones that
are shipped with the grammars themselves, but they do have a few things
that need to be worked around.

Our "standard edits" include:

- Removing any queries that use a `set!` directive that takes three
  arguments. This is something entirely custom inside neovim's
  implementation, and because `set!` is implemented in the
  py-tree-sitter C code, there's no way for us to do it. This is used in
  very few places in the actual queries files, and doesn't seem to
  impact syntax highlighting in any meaningful way, so we just get rid
  of them.
- Rewriting `lua-match?` and `not-lua-match?` predicates to use Python
  regex syntax and change the predicate name to `match?` or `not-match?`
  as appropriate.
- Rewriting `gsub!` directives to use Python regex syntax.

The accompanying tests include unit tests for each of the edits, as well
as two parameterized tests for loading highlights and injections queries
for each language. These include a few xfail marks for languages where
the bundled queries are currently known to fail; as I work through
those, hopefully those lists will become completely empty.

Testing Done:
- Ran unit tests.
- Used these from the update-queries script to rewrite queries files
  from nvim-treesitter to make them compatible with Review Board.
- Verified that rewritten queries worked correctly, especially with
  rewritten match queries.
    
     
    

        		
    qvvsmvtyozynwrzswmwktwvyvvsluyzu

   
    


      
 
    

    


    

   
    
  
    




  
    
   
    
    Diff:
   
    
   
    
    
     Revision 3 (+1760)
    Show changes
     
        docs/manual/extending/coderef/index.rst 
        reviewboard/treesitter/core.py 
        reviewboard/treesitter/query_utils.py 
        reviewboard/treesitter/tests/test_queries.py 
     
    
   
    
  
    





    



    
 
    
  
    Checks run (2 succeeded)
    


  
    
 flake8
 passed.
 


    


  
    
 JSHint
 passed.