Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.

 






 

 


  Review Request #14524
  —
  Created July 28, 2025 and updated 

 




















 

  

   Information

  





 
 

  david
 









 
 

  Review Board
 









 
 

  master
 









 
 

  
 









 
 

  
 






 




 

  

   Reviewers

  





 
 

  reviewboard
 









 
 

  
 






 








 





  


 
 

  
This change makes it so we attempt to use tree sitter (if available) for

diffs. If the tree sitter highlighting is unavailable (for example, we

don't have a matching grammar), or fails for some reason, we'll then

fall back on Pygments.

 








  


 
 

  
    

  • Ran unit tests.
    • 

  • Viewed a variety of diffs of different file types and verified that
    
  the new highlighting worked correctly.
    • 


 







 









 





  

 


 
 

  



 

  
   


    Summary
    ID

   

  
  

   


    

     

      Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.
      
This change makes it so we attempt to use tree sitter (if available) for
diffs. If the tree sitter highlighting is unavailable (for example, we
don't have a matching grammar), or fails for some reason, we'll then
fall back on Pygments.

Testing Done:
- Ran unit tests.
- Viewed a variety of diffs of different file types and verified that
  the new highlighting worked correctly.

     


    		
    b73da32276df9dc1073f0b9e70ae2065e0284f35

   



  
 




 













 






 
 
 

  

   

    

     
    

    
    
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
     
  • 
      
     
    • 
    

   

   

    
     

      Description
      From
      Last Updated
     

    
    



     

      
       
       
too many blank lines (2) Column: 5 Error code: E303

      		
      
       reviewbotreviewbot
      
      
       
      
     


    
   

  

 







 






  

 



















 




 




 














 



 

  

   




david




   

   

  

 



 

  


   

   

    
 Review request changed

    
   


  

  

  









  

   

    Change Summary:
   

   

    
    

  • Add some additional unit tests focused on syntax highlighting.
    • 

  • Fix falling back to Pygments when we encounter an exception in tree sitter.
    • 


   

  








  

   

    Commits:
   

   

    





 

  
   


    

    Summary
    ID

   

  
  

   


    

    

     

      Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.
      
This change makes it so we attempt to use tree sitter (if available) for
diffs. If the tree sitter highlighting is unavailable (for example, we
don't have a matching grammar), or fails for some reason, we'll then
fall back on Pygments.

Testing Done:
- Ran unit tests.
- Viewed a variety of diffs of different file types and verified that
  the new highlighting worked correctly.

     


    		
    be9c99de161c73dfb34df57a1b7923c80f454047

   




   


    

    

     

      Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.
      
This change makes it so we attempt to use tree sitter (if available) for
diffs. If the tree sitter highlighting is unavailable (for example, we
don't have a matching grammar), or fails for some reason, we'll then
fall back on Pygments.

Testing Done:
- Ran unit tests.
- Viewed a variety of diffs of different file types and verified that
  the new highlighting worked correctly.

     


    		
    470fc53c64eff5ef83e5597ec77cc03f7cfe9ce6

   



  
 







   

  





  

   

    Diff:
   

   

    
 Revision 2 (+1242 -66)
Show changes
 
    reviewboard/diffviewer/chunk_generator.py 
    reviewboard/diffviewer/tests/test_raw_diff_chunk_generator.py 
    reviewboard/webapi/tests/test_filediff.py 
 

   

  











 

  
Checks run (1 failed, 1 succeeded)



  

 flake8
 failed.
 





  

 JSHint
 passed.
 





 







 
flake8

 


 















  

 
















 



 

  

   




david




   

   

  

 



 















 



 

  

   




david




   

   

  

 



 

  


   

   

    
 Review request changed

    
   


  

  

  















  

   

    Commits:
   

   

    





 

  
   


    

    Summary
    ID

   

  
  

   


    

    

     

      Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.
      
This change makes it so we attempt to use tree sitter (if available) for
diffs. If the tree sitter highlighting is unavailable (for example, we
don't have a matching grammar), or fails for some reason, we'll then
fall back on Pygments.

Testing Done:
- Ran unit tests.
- Viewed a variety of diffs of different file types and verified that
  the new highlighting worked correctly.

     


    		
    2db8b540eaf69f9f970c7d9f8f166f9f06c068fa

   




   


    

    

     

      Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.
      
This change makes it so we attempt to use tree sitter (if available) for
diffs. If the tree sitter highlighting is unavailable (for example, we
don't have a matching grammar), or fails for some reason, we'll then
fall back on Pygments.

Testing Done:
- Ran unit tests.
- Viewed a variety of diffs of different file types and verified that
  the new highlighting worked correctly.

     


    		
    b73da32276df9dc1073f0b9e70ae2065e0284f35

   



  
 







   

  





  

   

    Diff:
   

   

    
 Revision 4 (+1248 -68)
Show changes
 
    reviewboard/diffviewer/chunk_generator.py 
    reviewboard/diffviewer/tests/test_raw_diff_chunk_generator.py 
    reviewboard/webapi/tests/test_filediff.py 
 

   

  











 

  
Checks run (2 succeeded)



  

 flake8
 passed.
 





  

 JSHint
 passed.