[Outdated] Basic Util Files For GoDiffX

Review Request #11830 — Created Oct. 1, 2021 and updated




All of the basic files have now been created along with some tests. This includes errors,
sections, options, text, and unified diffs.

I added the outdated tag because I started overwriting some of these basic things while
working on reader.go. It also does not make much sense to ship this on its own since
there aren't really any useful functions.

A test file was created on my machine in another directiory (~/Desktop/GoTest) and the module
was imported there. Since it does not make sense for semi-functional code to be pushed to Github,
I setup the local test file to think it's getting its information from
github.com/beanbaginc/diffx/go/godiffx when in reality, it is just checking my local copy. I can
overwrite this whenever code gets published to make sure it still works properly, but that is likely
something that can come towards the end of the project.

There is testing written for every file now and Go reports it having 98.1% coverage. The remaining 1.9%
is due to small redundancies in error handling where I don't expect a function to ever error at a certain
point, but I have code to handle it in case something odd happens.

The only other piece that still needs to be tested is the Go docs themselves. I have not had enough time
to spin up a local server where I can see what the documentation looks like, but I plan to do that soon
and will update this description once that is finished. I don't expect any issues but this is my first time
creating go docs so we will see.

Summary ID Author
Create LICENSE and README for project
First commit for godiffx, just boilerplate based off of pydiffx. Testing done: NA
4552441df26f5bc1f71e0957b915eb5d294e2c88 Jordan
Create basic support files for diffx.
Created errors.go, sections.go, options.go, and text.go. These will all be based on their python equivalents from pydiffx. At the moment, there is no code within them besides stating which package they are apart of. No functionality introduced so no testing done.
53615b11759d77a06b2a0d6aa8f387beeb1eced7 Jordan
Write basic constants for options.go
Added the basic functions to options.go based on what is in options.py for pydiffx. This was accomplished via enums using iota in go. The implementation may be tweaked overtime once I actually need to start using them. The functionality of these was tested in another go file. This was also the first time I tried testing the godiffx module locally and it worked!
ed55e923e78ead35565ee3f344a234516cfc3921 Jordan
Wrote code for sections.go
sections.go is based off of sections.py, and the functionality was replicated as close as possible to that file. One hiccup was that Go does not support sets natively, but an easy work around is to use sets and just set all of the keys to true. Code was tested by creating the struct in another file that imported this module. The default parameter method was tested and properly adds in the default values and the getters were all functioning as expected.
3659b78ffe8b8cc27d23a5d1f497f2bb0ab00a21 Jordan
Update documentation for sections.go
Basic documentation via comments was added to the code in sections.go. No new functionality was introduced so there was no additional testing.
398e1cd37ede5380265ee716bb93f7217bfa9996 Jordan
Write all custom errors for godiffx.
These are based on the errors that were created for pydiffx. Go does not use errors in the same way, but an error struct was created and an error can be returned whenever needed by other functions. One note is that at the moment, there is no unicode support since I am not very familiar with it or implementing it, but I assume it will be as simple as changing the type and possibly importing the library in the future. This was tested by importing the module locally and testing each of the different types of errors. The init function was validated to ensure it would not overwrite an existing error and the Err function was tested to ensure it printed properly. I also tested to ensure the baseDiffXError was not accessible.
37da0122f047f79a5cb92e0aeba4d0b5711196ec Jordan
Add documentation and fix line value for DiffXParseError
Basic documentation was added to all of the functions and errors. They are all pretty simple definitions based off of what was used in the pydiffx version. There was also a modification made to the way line and column values are treated in DiffXParseError. Originally this was set to just use the values as they are passed in. However, pydiffx assumes these are 0 based so the message increments it by 1. This change was tested by importing the module locally in another file.
b9ade0ac8f01e150291ac24e2bebc38b01276cbc Jordan
Create text values and write SplitLines
Following the example from text.py, the maps have been created for godiffx as well. One thing to note is the utils folder is now gone. This is because when creating a module in Go, you actually don't use nested directories since those will then belong to a new package. Another change is that instead of using a map for the objects like in python, structs were created along with a way to initialize their default values. The reasoning behind this is that BOMS uses tuples to store multiple values in pydiffx, but tuples don't exist in Go. The SplitLines method was also written. This took a fair bit of time while I experimented and learned about the best way to handle bytes. The bytes.Buffer seems to be a very efficient data structure that comes with some functionality. Time will tell if this is the right way to go! All of the objects have been tested in the separate project using this module and I have run SplitLines with what I believe to be the proper behaviour.
ddb24d486e20a4f3365cb1186c647575d2ff3458 Jordan
Fix Errors and Write StripBom Method.
I made a silly mistake when creating the errors by naming the error functions 'Err()' instead of 'Error()' so they were not quite working like errors yet, so that has been resolved. I also wrote the StripBom method which will remove a given BOM from a byte buffer if it exists. In order to do this, I made a method that can retrieve the BOM given a string of the encoding type. I tested out the methods again in main. To test StripBom, I created a buffer that held "\uFEFF is a string that starts with a Byte Order Mark" and compared its length to before and after running the function. The buffer that the function returns will be shorter but it will not mutate the buffer passed in.
87a9119601141817aa81cef3e8b163a7d20b4f7b Jordan
Created test files and wrote init tests for Errors.go
All of the test files were created and can be run with the command 'go test'. I have written all of the Init tests for errors.go except for the base class since I am not 100% sure how to test private methods. However, I don't think it is too important since all of the other test cases work.
0576404dacb4b9f6ac9f5eceb9c29ee97c022b2a Jordan
Finish tests for errors.go
Finished writing the rest of the tests for errors.go, specifically all of the tests for the Error() methods. The only testing that was done was running the test suite and seeing all of the new tests pass.
40257068d62ca72958a447221529a04876ca908a Jordan
Add testing for sections.go
The testing itself is pretty trivial, since it just ensure the init methods works properly and that the set functions return the proper sets. The ValidSectionStates function has a very long test since there are so many different pieces that need to be checked. Not sure if this can be simplified some way in the future.
cb86e91b7b51ac9be3ed52c35fd9aab20baefa90 Jordan
Wrote unit tests for finished functions in text.go
While there is still a bit of work to be done for text.go, the unit tests for the finished functions have now been added in. These tests try to test for every edge case I can think of, but suggestions are welcome! I also do not know if there is a better way to run these tests. Currently, each test is rather large with a number of small pieces it tests for a function. It seems like Go tests usually have one test per function instead of multiple tests for different edge cases of a function, but if I find something new then I will make adjustments. The ony strategy I saw is to make a struct with test data, but I feel it would bloat the code a bit due to the number of byte buffers needed.
6f73cf4159218d416aa97234e19481982e6c2c94 Jordan
Create new error for invalid encoding.
Since it seemed relatively straightforward to add my own encoding and there weren't any great go modules I found with a quick search, I have started creating a function to encode in some given utf scheme. It makes sense to begin by creating an error that can be thrown in case someone provides an invalid string. A go test has been created as well in the same manner as all of the other error tests to ensure InvalidEncodingError works properly.
13bebc89fe5764f1da87e795230cab2eaddd1cb4 Jordan
Create encode method for text.go
I created an encode function based off of what Python does. There weren't any libraries I found quick that seemed to cover the different utf styles. I based their implementation off of the returns for unix and dos newlines. I have not created tests for the function yet, but I manually tested it by using the function in another project and comparing the output to Python
b5ef19b54466ccd7e3b3a961188974203b0c15f4 Jordan
Add GetNewlineForType function in text.go
This function is the same as the version in pydiffx. One note is that it returns a byte slice instead of a buffer, but I thought this would make more sense since it will usually only be a few bytes at most. There is no testing created for this function yet, it will be written at a later time. But it was manually tested with utf8 and utf16.
0951ce3c23a99b2d1b5e53128ebdd85feb28e413 Jordan
Finish manually testing all functions for text.go
All functions have now been written and manually tested in text.go. I thought I had things working on Saturday, but there was still an issue related to an EOF error that I had to sort out. Manual testing was done by creating a buffer that had both unix and dos newlines as well as an empty one where it assumed we used unix. No tests have been written for the newline guessing or getting.
5678c06d947b2c40ff5e73814a2ae1af36812da4 Jordan
Write tests for line ending methods.
This commit just adds a couple tests to ensure the GuessLineEnding and GetNewlineForType functions work properly.
3225e18cd1e33cf1f473cedfff7eb37fdb9df982 Jordan
Wrote unit test for encoding function
Just a quick unit test for encoding function. It's very inclusive and covers every possible return. I noticed that Go only thinks I have ~90% coverage for my tests, so I will likely spend some time trying to identify the edge cases I missed.
1a455560529844e0eb504cb130b8136534fbf337 Jordan
Test a few more edge cases in the code.
The Go test coverage is now at 98%, so almost everything is accounted for. However, there are a couple of pieces that I'm not sure if I can test. These are mainly catching an EOF error from ReadByte() functions because I use a function before that should ensure it never sees this issue. I don't know how I could test some of the pieces of code even with a mocker but I am very confident that those small areas should never have an issue. Their error is handled somehow earlier by another method, but it is still good style to double check that error in the code.
6e56198f6904b19e5d0de6b4bc1009313bc234bc Jordan
Write initial documentation for unifiedDiffs.go.
There isn't any functionality in this commit, mainly just getting all of the documentation down for this file. At the top, I did create the regex function but it is commented since it is not being used yet and I am a bit unsure of what the expression is all doing (but I've asked a question in the slack). Because Go is not as flexible as Python, I can not just create a dictionary of whatever I would like, so I had to problem solve a bit to figure out what I could return. The solution I am going with for now is to create a couple structs that can hold all relevant information. This is the way I've seen Go handle nested JSON before so I figure it would be an appropriate approach. There is not testing since this commit does not come with any funcitonality.
705e1aef8ebc6cb0211aa9be22940aef383e5dc1 Jordan
WIP update for unifiedDiffs.go
The unifiedDiffs.go now has the start of its functionality, namely it now can properly deal with seeing the header of a hunk. There was a lot of experimentation to understand how regexp works within go along with understanding how the implementation works for unified_diffs.py. I have a pretty good idea of how the whole python program works now, so it is just a matter of understanding how that should best be translated. The only testing that has been done is to ensure the hunk is initially created properly (which it is). No tests have been written yet.
96622a38479aa2cf29a4823e54b2d4dca1ff81f4 Jordan
Finished unified_diffs.go
Finshed writing all of the code for unified_diffs.go. You can pass it a list of byte buffers and it can create a proper return. So far, it has only been tested with the first test case from the python version, but I will write tests next that will include the other types to ensure the functionality is good.
184fc152fcd8cff3ac006902960d403363c2a01d Jordan
Write basics for unifiedDiffs testing and fix bugs encountered.
All of the tests that need to be written have been created for unifiedDiffs.go based on pydiffx. Because it takes so long to write out all of the different test cases and because it does not seem as straightforward to compare my custom structs as I had hoped, the testing is currently very minimal. Most test cases ensure that the correct number of hunks are returned or the proper error. There are also a lot of other small tweaks to other files like unifiedDiffs.go and errors.go as I had small mistakes in my understanding of how the Python version was supposed to work.
27fdcf5139556f0750682ce728bf285f8b80c89a Jordan
Update unifiedDiff tests to ensure proper returns and tweaked bugs.
All of the testing is finally complete for unifiedDiffs.go. I had to actually validate all of the UnifiedDiffHunks that were returned to ensure the values were correct. In the process, I found a few minor bugs that I was able to fix up.
430153ba69177c03209a3bc7a382d3379e911e53 Jordan
Review request changed


-Basic Util Files For GoDiffX
+[Outdated] Basic Util Files For GoDiffX



All of the basic files have now been created along with some tests. This includes errors,

    sections, options, text, and unified diffs.


I added the outdated tag because I started overwriting some of these basic things while

  + working on reader.go. It also does not make much sense to ship this on its own since
  + there aren't really any useful functions.