Summary

Move to JSON as the metadata format for DiffX.

Review Request #11702 — Created July 5, 2021 and submitted July 6, 2021, 9:37 p.m.

Information

Owner

chipx86

Repository

DiffX

Branch

master

Bugs

Depends On

Blocks

11703

Reviewers

Groups

diffx

People

Description

We couldn't depend on YAML itself, due to numerous parser
inconsistencies. A home-grown grammar was defined, but this would be yet
another custom structured data format, and this seemed to be the wrong
place to introduce this.

Instead, we're now using JSON as the metadata format. This has a couple
significant advantages:

JSON is widely available in nearly any language, and through command
line tools, making it very easy to support.
No part of JSON output should conflict with the syntax of DiffX,
unified diffs, or any known custom diff format.

This change updates the spec to make it clear that JSON is the official
format used for DiffX, and to update all the example diffs and metadata
to use it.

Testing Done

Built the docs. Checked all the example diffs and metadata examples to
ensure they had correct syntax and was rendering correctly.

Commits

Summary	ID
Move to JSON as the metadata format for DiffX. Initially, DiffX was envisioned as having a YAML-style metadata format. This was an attempt to keep the appearance of metadata similar to that of several diff variants, such as Git-style diffs. However, none of these variants had any sort of grammar defined for their format. They were just entirely ad-hoc. We couldn't depend on YAML itself, due to numerous parser inconsistencies. A home-grown grammar was defined, but this would be yet another custom structured data format, and this seemed to be the wrong place to introduce this. Instead, we're now using JSON as the metadata format. This has a couple significant advantages: 1. JSON is widely available in nearly any language, and through command line tools, making it very easy to support. 2. No part of JSON output should conflict with the syntax of DiffX, unified diffs, or any known custom diff format. This change updates the spec to make it clear that JSON is the official format used for DiffX, and to update all the example diffs and metadata to use it.	a936062472b27c83d13722c817abfe34c39d8fab

Summary

Move to JSON as the metadata format for DiffX.

Initially, DiffX was envisioned as having a YAML-style metadata format. This was an attempt to keep the appearance of metadata similar to that of several diff variants, such as Git-style diffs. However, none of these variants had any sort of grammar defined for their format. They were just entirely ad-hoc. We couldn't depend on YAML itself, due to numerous parser inconsistencies. A home-grown grammar was defined, but this would be yet another custom structured data format, and this seemed to be the wrong place to introduce this. Instead, we're now using JSON as the metadata format. This has a couple significant advantages: 1. JSON is widely available in nearly any language, and through command line tools, making it very easy to support. 2. No part of JSON output should conflict with the syntax of DiffX, unified diffs, or any known custom diff format. This change updates the spec to make it clear that JSON is the official format used for DiffX, and to update all the example diffs and metadata to use it.

a936062472b27c83d13722c817abfe34c39d8fab

Issues

Description	From	Last Updated
Maybe put "json" first in the list? We should probably also highly recommend that this option is included, even though …	david	July 6, 2021, 5:59 p.m.
F401 'pygments.token.Punctuation' imported but unused	reviewbot	July 6, 2021, 5:21 p.m.
F401 'pygments.token.Operator' imported but unused	reviewbot	July 6, 2021, 5:21 p.m.
F401 'pygments.token.String' imported but unused	reviewbot	July 6, 2021, 5:21 p.m.

flake8 failed.

JSHint passed.

flake8

python/diffx/integrations/pygments_lexer.py (Diff revision 1)
The issue has been resolved. Show all issues
```
F401 'pygments.token.Punctuation' imported but unused
```
python/diffx/integrations/pygments_lexer.py (Diff revision 1)
The issue has been resolved. Show all issues
```
F401 'pygments.token.Operator' imported but unused
```
python/diffx/integrations/pygments_lexer.py (Diff revision 1)
The issue has been resolved. Show all issues
```
F401 'pygments.token.String' imported but unused
```

docs/spec/section-format.rst (Diff revision 1)

The issue has been resolved. Show all issues

Maybe put "json" first in the list?

We should probably also highly recommend that this option is included, even though there's only the one format for now.

chipx86 July 6, 2021, 5:19 p.m.

Gonna make me redo all the examples and length calculations? ;)

Maybe worth doing. I'm also okay with it just being the default if nothing is specified, and calling that out specifically here.

chipx86 July 6, 2021, 5:30 p.m.

Thinking about:

Ditching the format list and just specifying json as a valid option.
Specifying that it's optional, and if ignored, json is implied.
Recommending all diff parsers to explicitly check this and, for now, ensure that it's either json or blank, in order to avoid breaking if dealing with a future spec'd version of a file.

I think that would prevent the biggest issue (bad assumptions by diff parsers), but I don't know. I can include it in all of our examples just in case. What do you think? Are there advantages to recommending it be explicit?

chipx86 July 6, 2021, 5:59 p.m.

Scratch that last part. I'll make it a recommendation and clear up the requirements for diff generators and parsers.

Change Summary:

Changes to the format option:

It's now a recommendation instead of a reserved option.
Removed the list of possible options, in favor of a single valid option: json.
Added information on the requirements of diff parsers and generators when it comes to this option, or the lack of the option in a meta section.
Specified that new format options will only be introduced when bumping the spec version.

Commits:

	Summary	ID
	Move to JSON as the metadata format for DiffX. Initially, DiffX was envisioned as having a YAML-style metadata format. This was an attempt to keep the appearance of metadata similar to that of several diff variants, such as Git-style diffs. However, none of these variants had any sort of grammar defined for their format. They were just entirely ad-hoc. We couldn't depend on YAML itself, due to numerous parser inconsistencies. A home-grown grammar was defined, but this would be yet another custom structured data format, and this seemed to be the wrong place to introduce this. Instead, we're now using JSON as the metadata format. This has a couple significant advantages: 1. JSON is widely available in nearly any language, and through command line tools, making it very easy to support. 2. No part of JSON output should conflict with the syntax of DiffX, unified diffs, or any known custom diff format. This change updates the spec to make it clear that JSON is the official format used for DiffX, and to update all the example diffs and metadata to use it.	5331db232b2d8a6e19637cbc93dfd8554dc3b61f
	Move to JSON as the metadata format for DiffX. Initially, DiffX was envisioned as having a YAML-style metadata format. This was an attempt to keep the appearance of metadata similar to that of several diff variants, such as Git-style diffs. However, none of these variants had any sort of grammar defined for their format. They were just entirely ad-hoc. We couldn't depend on YAML itself, due to numerous parser inconsistencies. A home-grown grammar was defined, but this would be yet another custom structured data format, and this seemed to be the wrong place to introduce this. Instead, we're now using JSON as the metadata format. This has a couple significant advantages: 1. JSON is widely available in nearly any language, and through command line tools, making it very easy to support. 2. No part of JSON output should conflict with the syntax of DiffX, unified diffs, or any known custom diff format. This change updates the spec to make it clear that JSON is the official format used for DiffX, and to update all the example diffs and metadata to use it.	a936062472b27c83d13722c817abfe34c39d8fab

Diff:

Revision 2 (+800 -614)

Show changes

	docs/spec/section-format.rst
	docs/spec/sections.rst
	docs/spec/example-diffs/commit.diff
	docs/spec/example-diffs/local-file.diff
	docs/spec/example-diffs/multi-commit.diff
	docs/spec/example-diffs/repo-file.diff
	docs/spec/example-diffs/wrapped-cvs-diff.diff
	docs/spec/example-diffs/wrapped-git-diff.diff
	3 more

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Ship it!

```
Ship It!
```

Status: Closed (submitted)

Change Summary:

Pushed to master (401df9f)