Summary

Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.

Review Request #14524 — Created July 28, 2025 and updated March 31, 2026, 10:16 a.m.

Information

Owner

david

Repository

Review Board

Branch

master

Bugs

Depends On

Reviewers

Groups

reviewboard

People

Description

Testing Done


Ran unit tests.
Viewed a variety of diffs of different file types and verified that

  the new highlighting worked correctly.

Commits

Summary	ID
Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	b73da32276df9dc1073f0b9e70ae2065e0284f35

Summary

Tree Sitter: Use Tree Sitter for highlighting in the chunk generator.

This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.

b73da32276df9dc1073f0b9e70ae2065e0284f35

Issues

Description	From	Last Updated
too many blank lines (2) Column: 5 Error code: E303	reviewbot	July 28, 2025, 8:47 a.m.

flake8 passed.

JSHint passed.

Change Summary:


Add some additional unit tests focused on syntax highlighting.
Fix falling back to Pygments when we encounter an exception in tree sitter.

Commits:

	Summary	ID
	Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	be9c99de161c73dfb34df57a1b7923c80f454047
	Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	470fc53c64eff5ef83e5597ec77cc03f7cfe9ce6

Diff:

Revision 2 (+1242 -66)

Show changes

	reviewboard/diffviewer/chunk_generator.py
	reviewboard/diffviewer/tests/test_raw_diff_chunk_generator.py
	reviewboard/webapi/tests/test_filediff.py

Checks run (1 failed, 1 succeeded)

flake8 failed.

JSHint passed.

flake8

reviewboard/diffviewer/tests/test_raw_diff_chunk_generator.py (Diff revision 2)
The issue has been dropped. Show all issues
```
too many blank lines (2)

Column: 5
Error code: E303
```

Commits:

	Summary	ID
	Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	470fc53c64eff5ef83e5597ec77cc03f7cfe9ce6
	Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	2db8b540eaf69f9f970c7d9f8f166f9f06c068fa

Diff:

Revision 3 (+1242 -68)

Show changes

	reviewboard/diffviewer/chunk_generator.py
	reviewboard/diffviewer/tests/test_raw_diff_chunk_generator.py
	reviewboard/webapi/tests/test_filediff.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Commits:

	Summary	ID
	Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	2db8b540eaf69f9f970c7d9f8f166f9f06c068fa
	Tree Sitter: Use Tree Sitter for highlighting in the chunk generator. This change makes it so we attempt to use tree sitter (if available) for diffs. If the tree sitter highlighting is unavailable (for example, we don't have a matching grammar), or fails for some reason, we'll then fall back on Pygments. Testing Done: - Ran unit tests. - Viewed a variety of diffs of different file types and verified that the new highlighting worked correctly.	b73da32276df9dc1073f0b9e70ae2065e0284f35

Diff:

Revision 4 (+1248 -68)

Show changes

	reviewboard/diffviewer/chunk_generator.py
	reviewboard/diffviewer/tests/test_raw_diff_chunk_generator.py
	reviewboard/webapi/tests/test_filediff.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Ship it!