Summary

Add support for posting changes to Cliosoft SOS repositories.

Review Request #12237 — Created April 18, 2022 and submitted May 3, 2022, 9:14 p.m.

Information

Owner

chipx86

Repository

RBTools

Branch

release-3.x

Bugs

Depends On

Reviewers

Groups

rbtools

People

Description

Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack
5+, and SOS 7.20+, users will be able to configure Review Board to talk
to Cliosoft SOS repositories and post changes for review.

Users can post explicit selections for review, such as:

$ rbt post select:-scm
$ rbt post 'select:-sunm -sor -scm'

Or they can post SOS changelists (new in 7.20) by name:

$ rbt post my-changelist-name

The generated diff uses the DiffX (https://diffx.org) format, with
metadata specific to SOS. This is used to identify the workarea, SOS
Project, SOS Server, changelist ID, and RSO (Revision Search Order) that
was posted.

Some of this information is also returned during diff generation for
storage in the review request. This enables custom SOS fields in the
review request to show up, which will contain information useful to
reviewers and the owner to help determine which workarea and changelist
contains the changes.

That logic (along with get_tree_matches_review_request()) will also
enable smart matching of review requests for rbt post -u, which is
another change in development.

At the moment, this does not support posting files bundled as SOS
packages for review. That is planned for a subsequent release.

SOS 7.20 is required for this release, though the code has been written
to not strictly enforce this version yet, as some customers are using
patched builds of earlier releases that contain this support. In time,
we'll want to make 7.20 a hard requirement.

This work is a collaboration between Beanbag and Cliosoft. A big thanks
to them for all the testing, support, and all the new functionality
added to SOS 7.20 to make this integration happen.

Testing Done

All unit tests pass.

Extensively tested with a variety of types of changes in local SOS
workareas. This included:

Selection-based posting (default and custom selections)
Changeset-based posting
Included files and excluded patterns
Changed, added, and deleted files (referenced in selections and changesets)
Symlink changes (though this support still requires further work, and based
on discussions with Cliosoft will be moved to a future release)
Verification that existing SOS selections are preserved after the
operation completes.
Tested posting with rbt post -u (when combined with in-progress changes
for smarter matching)

Commits

Summary	ID
Add support for posting changes to Cliosoft SOS repositories. Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an enterprise-grade source code management solution used throughout the hardware industry. Its features are tailored to large, distributed teams working with inter-dependent projects. It supports large files, "packages" of related files that must be managed under a single revision, general attribute metadata support, file locking, ACLs, sparse checkouts, snapshots, and other feaures useful in enterprise settings. Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack 5+, and SOS 7.20+, users will be able to configure Review Board to talk to Cliosoft SOS repositories and post changes for review. Users can post explicit selections for review, such as: $ rbt post select:-scm $ rbt post 'select:-sunm -sor -scm' Or they can post SOS changelists (new in 7.20) by name: $ rbt post my-changelist-name The generated diff uses the DiffX (https://diffx.org) format, with metadata specific to SOS. This is used to identify the workarea, SOS Project, SOS Server, changelist ID, and RSO (Revision Search Order) that was posted. Some of this information is also returned during diff generation for storage in the review request. This enables custom SOS fields in the review request to show up, which will contain information useful to reviewers and the owner to help determine which workarea and changelist contains the changes. That logic (along with `get_tree_matches_review_request()`) will also enable smart matching of review requests for `rbt post -u`, which is another change in development. At the moment, this does not support posting files bundled as SOS packages for review. That is planned for a subsequent release. SOS 7.20 is required for this release, though the code has been written to not strictly enforce this version yet, as some customers are using patched builds of earlier releases that contain this support. In time, we'll want to make 7.20 a hard requirement. This work is a collaboration between Beanbag and Cliosoft. A big thanks to them for all the testing, support, and all the new functionality added to SOS 7.20 to make this integration happen.	ede4735c9cf7ed3c6363bac9cd69cc521cfde306

Summary

Add support for posting changes to Cliosoft SOS repositories.

Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an enterprise-grade source code management solution used throughout the hardware industry. Its features are tailored to large, distributed teams working with inter-dependent projects. It supports large files, "packages" of related files that must be managed under a single revision, general attribute metadata support, file locking, ACLs, sparse checkouts, snapshots, and other feaures useful in enterprise settings. Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack 5+, and SOS 7.20+, users will be able to configure Review Board to talk to Cliosoft SOS repositories and post changes for review. Users can post explicit selections for review, such as: $ rbt post select:-scm $ rbt post 'select:-sunm -sor -scm' Or they can post SOS changelists (new in 7.20) by name: $ rbt post my-changelist-name The generated diff uses the DiffX (https://diffx.org) format, with metadata specific to SOS. This is used to identify the workarea, SOS Project, SOS Server, changelist ID, and RSO (Revision Search Order) that was posted. Some of this information is also returned during diff generation for storage in the review request. This enables custom SOS fields in the review request to show up, which will contain information useful to reviewers and the owner to help determine which workarea and changelist contains the changes. That logic (along with `get_tree_matches_review_request()`) will also enable smart matching of review requests for `rbt post -u`, which is another change in development. At the moment, this does not support posting files bundled as SOS packages for review. That is planned for a subsequent release. SOS 7.20 is required for this release, though the code has been written to not strictly enforce this version yet, as some customers are using patched builds of earlier releases that contain this support. In time, we'll want to make 7.20 a hard requirement. This work is a collaboration between Beanbag and Cliosoft. A big thanks to them for all the testing, support, and all the new functionality added to SOS 7.20 to make this integration happen.

ede4735c9cf7ed3c6363bac9cd69cc521cfde306

Issues

Description	From	Last Updated
So right now we're only using one SOSChangeSet per invocation, but if for some reason someone were to use this …	david	April 26, 2022, 3:03 p.m.
This is called from numerous places. Can we use self._workarea instead of those calls, or make this cache it internally?	david	April 26, 2022, 3:09 p.m.

flake8 passed.

JSHint passed.

I admit my eyes started to glaze over but for the most part this looks very good.

chipx86 April 26, 2022, 3:02 p.m.

I wish I could have broken it up more. It's a big implementation :/

rbtools/clients/sos.py (Diff revision 1)

The issue has been resolved. Show all issues

So right now we're only using one SOSChangeSet per invocation, but if for some reason someone were to use this multiple times, they'd all share the same instances for adds, deletes, and modifications since these are created at function definition time. It's uglier, but this should probably be changed to default to None and then create the set instances inside the __init__ body.

rbtools/clients/sos.py (Diff revision 1)

Is this new?

chipx86 April 26, 2022, 3:04 p.m.

Oh yes, I meant to mention that. It's still in development in my beanbag-docutils tree, but it works (well enough, anyway) as a replacement for the pattern we've been using.

We can use it now and it'll just be a little bit uglier, but I'll circle back to beanbag-docutils when everything else is wrapped up.

rbtools/clients/sos.py (Diff revision 1)

The issue has been resolved. Show all issues

This is called from numerous places. Can we use self._workarea instead of those calls, or make this cache it internally?

chipx86 April 26, 2022, 3:06 p.m.

It's cached through _query_sos_info, but yeah, I'll update the other call sites where I can (some need to go through the original this function, depending on what's called when).

chipx86 April 26, 2022, 3:08 p.m.

Oh, I take that back. Those would all have to assume that get_local_path() is called first to set that attribute. That's true in a standard rbt post invocation, but not in unit tests, and there's no guarantee of it if writing custom RBTools code. It's not safe to depend on values there.

Given the caching in _query_sos_info(), I'm going to remove self._workarea. It's not really used, and is actually a holdover from the original implementation anyway.

Change Summary:


Fixed SOSChangeSet to use None defaults instead of set() to avoid shared state.
Removed SCMClient._workarea in favor of the caching in _get_wa_root() calls, to avoid function call order assumptions.

Commits:

	Summary	ID
	Add support for posting changes to Cliosoft SOS repositories. Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an enterprise-grade source code management solution used throughout the hardware industry. Its features are tailored to large, distributed teams working with inter-dependent projects. It supports large files, "packages" of related files that must be managed under a single revision, general attribute metadata support, file locking, ACLs, sparse checkouts, snapshots, and other feaures useful in enterprise settings. Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack 5+, and SOS 7.20+, users will be able to configure Review Board to talk to Cliosoft SOS repositories and post changes for review. Users can post explicit selections for review, such as: $ rbt post select:-scm $ rbt post 'select:-sunm -sor -scm' Or they can post SOS changesets (new in 7.20) by name: $ rbt post my-changeset-name The generated diff uses the DiffX (https://diffx.org) format, with metadata specific to SOS. This is used to identify the workarea, SOS Project, SOS Server, changeset ID, and RSO (Revision Search Order) that was posted. Some of this information is also returned during diff generation for storage in the review request. This enables custom SOS fields in the review request to show up, which will contain information useful to reviewers and the owner to help determine which workarea and changeset contains the changes. That logic (along with `get_tree_matches_review_request()`) will also enable smart matching of review requests for `rbt post -u`, which is another change in development. At the moment, this does not support posting files bundled as SOS packages for review. That is planned for a subsequent release. SOS 7.20 is required for this release, though the code has been written to not strictly enforce this version yet, as some customers are using patched builds of earlier releases that contain this support. In time, we'll want to make 7.20 a hard requirement. This work is a collaboration between Beanbag and Cliosoft. A big thanks to them for all the testing, support, and all the new functionality added to SOS 7.20 to make this integration happen.	cca1692a58dddd68cf39455eaf1d525dffe18d69
	Add support for posting changes to Cliosoft SOS repositories. Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an enterprise-grade source code management solution used throughout the hardware industry. Its features are tailored to large, distributed teams working with inter-dependent projects. It supports large files, "packages" of related files that must be managed under a single revision, general attribute metadata support, file locking, ACLs, sparse checkouts, snapshots, and other feaures useful in enterprise settings. Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack 5+, and SOS 7.20+, users will be able to configure Review Board to talk to Cliosoft SOS repositories and post changes for review. Users can post explicit selections for review, such as: $ rbt post select:-scm $ rbt post 'select:-sunm -sor -scm' Or they can post SOS changesets (new in 7.20) by name: $ rbt post my-changeset-name The generated diff uses the DiffX (https://diffx.org) format, with metadata specific to SOS. This is used to identify the workarea, SOS Project, SOS Server, changeset ID, and RSO (Revision Search Order) that was posted. Some of this information is also returned during diff generation for storage in the review request. This enables custom SOS fields in the review request to show up, which will contain information useful to reviewers and the owner to help determine which workarea and changeset contains the changes. That logic (along with `get_tree_matches_review_request()`) will also enable smart matching of review requests for `rbt post -u`, which is another change in development. At the moment, this does not support posting files bundled as SOS packages for review. That is planned for a subsequent release. SOS 7.20 is required for this release, though the code has been written to not strictly enforce this version yet, as some customers are using patched builds of earlier releases that contain this support. In time, we'll want to make 7.20 a hard requirement. This work is a collaboration between Beanbag and Cliosoft. A big thanks to them for all the testing, support, and all the new functionality added to SOS 7.20 to make this integration happen.	f4114e1e0d4e6f7d6f9b2ea8e017b85cd9ffad76

Diff:

Revision 2 (+10650 -2)

Show changes

	setup.py
	rbtools/clients/sos.py
	rbtools/clients/tests/test_sos.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Change Summary:

Based on a discussion with Cliosoft on final terminology, all references to "changeset" has been replaced with "changelist", including in diff metadata.

Description:

		Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an
		enterprise-grade source code management solution used throughout the
		hardware industry. Its features are tailored to large, distributed teams
		working with inter-dependent projects. It supports large files,
		"packages" of related files that must be managed under a single
		revision, general attribute metadata support, file locking, ACLs, sparse
		checkouts, snapshots, and other feaures useful in enterprise settings.

		Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack
		5+, and SOS 7.20+, users will be able to configure Review Board to talk
		to Cliosoft SOS repositories and post changes for review.

		Users can post explicit selections for review, such as:

		`$ rbt post select:-scm`
		$ rbt post 'select:-sunm -sor -scm'


~		Or they can post SOS changesets (new in 7.20) by name:
	~	Or they can post SOS changelists (new in 7.20) by name:

~		`$ rbt post my-changeset-name`
	~	`$ rbt post my-changelist-name`


		The generated diff uses the DiffX (https://diffx.org) format, with
		metadata specific to SOS. This is used to identify the workarea, SOS
~		Project, SOS Server, changeset ID, and RSO (Revision Search Order) that
	~	Project, SOS Server, changelist ID, and RSO (Revision Search Order) that
		was posted.

		Some of this information is also returned during diff generation for
		storage in the review request. This enables custom SOS fields in the
		review request to show up, which will contain information useful to
~		reviewers and the owner to help determine which workarea and changeset
	~	reviewers and the owner to help determine which workarea and changelist
		contains the changes.

		That logic (along with `get_tree_matches_review_request()`) will also
		enable smart matching of review requests for `rbt post -u`, which is
		another change in development.

		At the moment, this does not support posting files bundled as SOS
		packages for review. That is planned for a subsequent release.

		SOS 7.20 is required for this release, though the code has been written
		to not strictly enforce this version yet, as some customers are using
		patched builds of earlier releases that contain this support. In time,
		we'll want to make 7.20 a hard requirement.

		This work is a collaboration between Beanbag and Cliosoft. A big thanks
		to them for all the testing, support, and all the new functionality
		added to SOS 7.20 to make this integration happen.

Commits:

	Summary	ID
	Add support for posting changes to Cliosoft SOS repositories. Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an enterprise-grade source code management solution used throughout the hardware industry. Its features are tailored to large, distributed teams working with inter-dependent projects. It supports large files, "packages" of related files that must be managed under a single revision, general attribute metadata support, file locking, ACLs, sparse checkouts, snapshots, and other feaures useful in enterprise settings. Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack 5+, and SOS 7.20+, users will be able to configure Review Board to talk to Cliosoft SOS repositories and post changes for review. Users can post explicit selections for review, such as: $ rbt post select:-scm $ rbt post 'select:-sunm -sor -scm' Or they can post SOS changesets (new in 7.20) by name: $ rbt post my-changeset-name The generated diff uses the DiffX (https://diffx.org) format, with metadata specific to SOS. This is used to identify the workarea, SOS Project, SOS Server, changeset ID, and RSO (Revision Search Order) that was posted. Some of this information is also returned during diff generation for storage in the review request. This enables custom SOS fields in the review request to show up, which will contain information useful to reviewers and the owner to help determine which workarea and changeset contains the changes. That logic (along with `get_tree_matches_review_request()`) will also enable smart matching of review requests for `rbt post -u`, which is another change in development. At the moment, this does not support posting files bundled as SOS packages for review. That is planned for a subsequent release. SOS 7.20 is required for this release, though the code has been written to not strictly enforce this version yet, as some customers are using patched builds of earlier releases that contain this support. In time, we'll want to make 7.20 a hard requirement. This work is a collaboration between Beanbag and Cliosoft. A big thanks to them for all the testing, support, and all the new functionality added to SOS 7.20 to make this integration happen.	f4114e1e0d4e6f7d6f9b2ea8e017b85cd9ffad76
	Add support for posting changes to Cliosoft SOS repositories. Cliosoft SOS (https://www.cliosoft.com/products/sos/) is an enterprise-grade source code management solution used throughout the hardware industry. Its features are tailored to large, distributed teams working with inter-dependent projects. It supports large files, "packages" of related files that must be managed under a single revision, general attribute metadata support, file locking, ACLs, sparse checkouts, snapshots, and other feaures useful in enterprise settings. Through a combination of RBTools 3.1+, Review Board 4.0.6+, Power Pack 5+, and SOS 7.20+, users will be able to configure Review Board to talk to Cliosoft SOS repositories and post changes for review. Users can post explicit selections for review, such as: $ rbt post select:-scm $ rbt post 'select:-sunm -sor -scm' Or they can post SOS changelists (new in 7.20) by name: $ rbt post my-changelist-name The generated diff uses the DiffX (https://diffx.org) format, with metadata specific to SOS. This is used to identify the workarea, SOS Project, SOS Server, changelist ID, and RSO (Revision Search Order) that was posted. Some of this information is also returned during diff generation for storage in the review request. This enables custom SOS fields in the review request to show up, which will contain information useful to reviewers and the owner to help determine which workarea and changelist contains the changes. That logic (along with `get_tree_matches_review_request()`) will also enable smart matching of review requests for `rbt post -u`, which is another change in development. At the moment, this does not support posting files bundled as SOS packages for review. That is planned for a subsequent release. SOS 7.20 is required for this release, though the code has been written to not strictly enforce this version yet, as some customers are using patched builds of earlier releases that contain this support. In time, we'll want to make 7.20 a hard requirement. This work is a collaboration between Beanbag and Cliosoft. A big thanks to them for all the testing, support, and all the new functionality added to SOS 7.20 to make this integration happen.	ede4735c9cf7ed3c6363bac9cd69cc521cfde306

Diff:

Revision 3 (+10650 -2)

Show changes

	setup.py
	rbtools/clients/sos.py
	rbtools/clients/tests/test_sos.py

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Ship it!

```
Ship It!
```

Status:: Completed
Change Summary:: Pushed to release-3.x (fec9b56)