Summary

Add backends for managing certificates, CA bundles, and fingerprints.

Review Request #13163 — Created July 28, 2023 and submitted Aug. 7, 2023, 1:14 p.m.

Information

Owner

chipx86

Repository

Review Board

Branch

release-6.x

Bugs

Depends On

Reviewers

Groups

reviewboard

People

Description

These can all be tied to a specific Local Site or to the global site.
Each object has an ID specific to the backend, which can be used for
lookup or deletions. IDs must be set by the backend, and consumers of
the backends must take care not to allow user-provided input to be used
without validation.

Backends are given a space within the site's data directory where they
can store files. Most libraries and tools expect a file path to PEMs, so
the backend is responsible for ensuring that those files exist when
needed.

Each object type being managed has a "stored" counterpart class. These
are subclassed by the backend and used to associate any necessary
information needed to identify or place the object somewhere in a
storage system.

By default, we ship a file-based storage backend. This is designed for
most single-server or managed-sitedir needs. All data is managed on the
filesystem, using known paths and filenames. These files can be
hand-added to the correct directory, so long as they follow the correct
naming and file format schemes.

For most standard deployments, this will more than suffice. However,
we'll be providing a database-managed solution in Power Pack, for
handling very large numbers of certificates across unlimited numbers of
Local Sites in a manner that allows for synchronizing across servers.

For Review Board 6.0.0, we may not include any UI for interfacing with
storage, but rather can provide documentation for file-based storage. UI
would come in a follow-up release.

Testing Done

Unit tests pass.

Commits

Summary	ID
Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	283b256ea9ba74a0c829240001318b46eeb7dcf3

Summary

Add backends for managing certificates, CA bundles, and fingerprints.

This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.

283b256ea9ba74a0c829240001318b46eeb7dcf3

Issues

Description	From	Last Updated
'typing.TYPE_CHECKING' imported but unused Column: 1 Error code: F401	reviewbot	July 28, 2023, 9:30 p.m.
'typing.Union' imported but unused Column: 1 Error code: F401	reviewbot	July 28, 2023, 9:30 p.m.
'djblets.testing.decorators.add_fixtures' imported but unused Column: 1 Error code: F401	reviewbot	July 28, 2023, 9:30 p.m.
This should have an "or" in it. Same for several other instances throughout this change.	david	Aug. 7, 2023, 12:57 p.m.

flake8 failed.

JSHint passed.

flake8

reviewboard/certs/storage/__init__.py (Diff revision 1)
The issue has been resolved. Show all issues
```
'typing.TYPE_CHECKING' imported but unused

Column: 1
Error code: F401
```
reviewboard/certs/tests/test_file_storage_backend.py (Diff revision 1)
The issue has been resolved. Show all issues
```
'typing.Union' imported but unused

Column: 1
Error code: F401
```
reviewboard/certs/tests/test_file_storage_backend.py (Diff revision 1)
The issue has been resolved. Show all issues
```
'djblets.testing.decorators.add_fixtures' imported but unused

Column: 1
Error code: F401
```

Change Summary:

Removed unused imports.

Commits:

	Summary	ID
	Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	aa66d8bc794e060f58562b057430adaa7d5ff997
	Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	a5619bf0c1c29f0fc25bf7c940ce16c9d2c30592

Diff:

Revision 2 (+12634)

Show changes

	reviewboard/certs/storage/__init__.py
	reviewboard/certs/storage/base.py
	reviewboard/certs/storage/file_storage.py
	reviewboard/certs/storage/registry.py
	reviewboard/certs/tests/test_file_storage_backend.py
	reviewboard/certs/tests/testcases.py
	reviewboard/certs/tests/testdata/file_storage/make-cert.sh
	reviewboard/certs/tests/testdata/file_storage/cabundles/comodo.pem
	21 more

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Change Summary:


Added a get_ca_bundles_dir() method for storage backends.
Moves some imports to TYPE_CHECKING-only.

Commits:

	Summary	ID
	Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	a5619bf0c1c29f0fc25bf7c940ce16c9d2c30592
	Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	e2e29e170ad1d4a9f0e56356bb84771e08b79b39

Diff:

Revision 3 (+12714)

Show changes

	reviewboard/certs/storage/__init__.py
	reviewboard/certs/storage/base.py
	reviewboard/certs/storage/file_storage.py
	reviewboard/certs/storage/registry.py
	reviewboard/certs/tests/test_file_storage_backend.py
	reviewboard/certs/tests/testcases.py
	reviewboard/certs/tests/testdata/file_storage/make-cert.sh
	reviewboard/certs/tests/testdata/file_storage/cabundles/comodo.pem
	21 more

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

reviewboard/certs/storage/base.py (Diff revision 3)

The issue has been resolved. Show all issues

This should have an "or" in it. Same for several other instances throughout this change.

Change Summary:


Added a missing "or" in some docstrings.

Commits:

	Summary	ID
	Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	e2e29e170ad1d4a9f0e56356bb84771e08b79b39
	Add backends for managing certificates, CA bundles, and fingerprints. This introduces support for storage backends for certificate-related data. Backends are responsible for adding, deleting, fetching, and iterating through SSL/TLS certificates, CA bundles, and verified certificate fingerprints, along with providing stats on the data stored. These can all be tied to a specific Local Site or to the global site. Each object has an ID specific to the backend, which can be used for lookup or deletions. IDs must be set by the backend, and consumers of the backends must take care not to allow user-provided input to be used without validation. Backends are given a space within the site's data directory where they can store files. Most libraries and tools expect a file path to PEMs, so the backend is responsible for ensuring that those files exist when needed. Each object type being managed has a "stored" counterpart class. These are subclassed by the backend and used to associate any necessary information needed to identify or place the object somewhere in a storage system. By default, we ship a file-based storage backend. This is designed for most single-server or managed-sitedir needs. All data is managed on the filesystem, using known paths and filenames. These files can be hand-added to the correct directory, so long as they follow the correct naming and file format schemes. For most standard deployments, this will more than suffice. However, we'll be providing a database-managed solution in Power Pack, for handling very large numbers of certificates across unlimited numbers of Local Sites in a manner that allows for synchronizing across servers. For Review Board 6.0.0, we may not include any UI for interfacing with storage, but rather can provide documentation for file-based storage. UI would come in a follow-up release.	283b256ea9ba74a0c829240001318b46eeb7dcf3

Diff:

Revision 4 (+12714)

Show changes

	reviewboard/certs/storage/__init__.py
	reviewboard/certs/storage/base.py
	reviewboard/certs/storage/file_storage.py
	reviewboard/certs/storage/registry.py
	reviewboard/certs/tests/test_file_storage_backend.py
	reviewboard/certs/tests/testcases.py
	reviewboard/certs/tests/testdata/file_storage/make-cert.sh
	reviewboard/certs/tests/testdata/file_storage/cabundles/comodo.pem
	21 more

Checks run (2 succeeded)

flake8 passed.

JSHint passed.

Ship it!

```
Ship It!
```

Status: Closed (submitted)

Change Summary:

Pushed to release-6.x (0aea6b0)