Summary

Tree Sitter: Add update-language-info script and resulting data.

Review Request #14514 — Created July 26, 2025 and updated July 26, 2025, 10:53 a.m.

Information

Owner

david

Repository

Review Board

Branch

master

Bugs

Depends On

Reviewers

Groups

reviewboard

People

Description

This change adds a script that loads information about supported
languages from a tree-sitter-language-pack checkout. This creates a
file, reviewboard/treesitter/_languages.py, which has two mappings,
MIME_TYPE_TO_LANGUAGE and FILE_SUFFIX_TO_LANGUAGES, which are used
for determining the tree sitter language name to use for a given file.

The compiled grammars that are shipped in tree-sitter-language-pack
don't include any of the grammar metadata, so this script will go into a
a checkout and grab the filename suffixes. This also fetches the
"first_line_regex" key, although that is not currently used for
detection.

Testing Done

Ran script to create the _languages.py file.

Commits

Summary	ID
Tree Sitter: Add update-language-info script and resulting data. This change adds a script that loads information about supported languages from a tree-sitter-language-pack checkout. This creates a file, `reviewboard/treesitter/_languages.py`, which has two mappings, `MIME_TYPE_TO_LANGUAGE` and `FILE_SUFFIX_TO_LANGUAGES`, which are used for determining the tree sitter language name to use for a given file. The compiled grammars that are shipped in tree-sitter-language-pack don't include any of the grammar metadata, so this script will go into a a checkout and grab the filename suffixes. This also fetches the "first_line_regex" key, although that is not currently used for detection. Testing Done: Ran script to create the _languages.py file.	d79179711d6e9d6845a3bbf7b28b15c7e5b2ef56

Summary

Tree Sitter: Add update-language-info script and resulting data.

This change adds a script that loads information about supported languages from a tree-sitter-language-pack checkout. This creates a file, `reviewboard/treesitter/_languages.py`, which has two mappings, `MIME_TYPE_TO_LANGUAGE` and `FILE_SUFFIX_TO_LANGUAGES`, which are used for determining the tree sitter language name to use for a given file. The compiled grammars that are shipped in tree-sitter-language-pack don't include any of the grammar metadata, so this script will go into a a checkout and grab the filename suffixes. This also fetches the "first_line_regex" key, although that is not currently used for detection. Testing Done: Ran script to create the _languages.py file.

d79179711d6e9d6845a3bbf7b28b15c7e5b2ef56

Issues

Description	From	Last Updated
continuation line over-indented for visual indent Column: 10 Error code: E127	reviewbot	July 26, 2025, 10:53 a.m.
continuation line under-indented for visual indent Column: 33 Error code: E128	reviewbot	July 26, 2025, 10:53 a.m.
line too long (83 > 79 characters) Column: 80 Error code: E501	reviewbot	July 26, 2025, 10:53 a.m.
line too long (83 > 79 characters) Column: 80 Error code: E501	reviewbot	July 26, 2025, 10:53 a.m.

flake8 failed.

JSHint passed.

flake8

contrib/internal/treesitter/update-language-info.py (Diff revision 1)
The issue has been dropped. Show all issues
```
continuation line over-indented for visual indent

Column: 10
Error code: E127
```
contrib/internal/treesitter/update-language-info.py (Diff revision 1)
The issue has been dropped. Show all issues
```
continuation line under-indented for visual indent

Column: 33
Error code: E128
```
reviewboard/treesitter/_languages.py (Diff revision 1)
The issue has been dropped. Show all issues
```
line too long (83 > 79 characters)

Column: 80
Error code: E501
```

Commits:

	Summary	ID
	Tree Sitter: Add update-language-info script and resulting data. This change adds a script that loads information about supported languages from a tree-sitter-language-pack checkout. This creates a file, `reviewboard/treesitter/_languages.py`, which has two mappings, `MIME_TYPE_TO_LANGUAGE` and `FILE_SUFFIX_TO_LANGUAGES`, which are used for determining the tree sitter language name to use for a given file. The compiled grammars that are shipped in tree-sitter-language-pack don't include any of the grammar metadata, so this script will go into a a checkout and grab the filename suffixes. This also fetches the "first_line_regex" key, although that is not currently used for detection. Testing Done: Ran script to create the _languages.py file.	43fd40d0beb3c46b3edf86d5aa7a87e8cdc58486
	Tree Sitter: Add update-language-info script and resulting data. This change adds a script that loads information about supported languages from a tree-sitter-language-pack checkout. This creates a file, `reviewboard/treesitter/_languages.py`, which has two mappings, `MIME_TYPE_TO_LANGUAGE` and `FILE_SUFFIX_TO_LANGUAGES`, which are used for determining the tree sitter language name to use for a given file. The compiled grammars that are shipped in tree-sitter-language-pack don't include any of the grammar metadata, so this script will go into a a checkout and grab the filename suffixes. This also fetches the "first_line_regex" key, although that is not currently used for detection. Testing Done: Ran script to create the _languages.py file.	d79179711d6e9d6845a3bbf7b28b15c7e5b2ef56

Diff:

Revision 2 (+2142)

Show changes

	contrib/internal/treesitter/update-language-info.py
	reviewboard/treesitter/_languages.py

Checks run (1 failed, 1 succeeded)

flake8 failed.

JSHint passed.

flake8

reviewboard/treesitter/_languages.py (Diff revision 2)
The issue has been dropped. Show all issues
```
line too long (83 > 79 characters)

Column: 80
Error code: E501
```