Use modern default root SSL certs and improve SSL error handling.

Review Request #13049 — Created May 16, 2023 and submitted

Information

RBTools
release-4.x

Reviewers

One of the common pain points in Python is SSL management. Python
depends on OpenSSL to provide certs, and these certs are not always
up-to-date. They also don't use system-managed SSL certs on all
platforms. The decent SSL options out there are somewhat dependent on
which HTTP library used. And when something goes wrong, the SSL errors
are generally obtuse and unhelpful.

This all carries over to RBTools, and we've seen many users have trouble
debugging this on their setup.

This change aims to fix most of that.

We now depend on certifi, which provides the latest Mozilla top-level
certs. We point to these by default when setting up SSL, meaning we can
now help manage how up-to-date the root certs are.

We now catch SSL errors and try to turn the most common SSL cert
verification errors into useful sets of instructions. These describe the
error and, when it's something the user can correct, we provide those
steps. This includes listing the paths where SSL certs can go, and how
to update certifi to get the latest SSL certs.

Some initialization code has been updated to ensure we're catching
connection errors during the initialization process, and logging useful
command line options before they occur, further helping us display and
diagnose these kinds of problems.

This should take care of the most common SSL issues people hit, and give
us a good starting point for improving future SSL handling in RBTools.

Unit tests passed.

Tested each of these with domains on badssl.com. These are documented
within the unit tests. Due to time constraints, integration tests with
badssl.com were skipped for now.

Summary ID
Use modern default root SSL certs and improve SSL error handling.
One of the common pain points in Python is SSL management. Python depends on OpenSSL to provide certs, and these certs are not always up-to-date. They also don't use system-managed SSL certs on all platforms. The decent SSL options out there are somewhat dependent on which HTTP library used. And when something goes wrong, the SSL errors are generally obtuse and unhelpful. This all carries over to RBTools, and we've seen many users have trouble debugging this on their setup. This change aims to fix most of that. We now depend on `certifi`, which provides the latest Mozilla top-level certs. We point to these by default when setting up SSL, meaning we can now help manage how up-to-date the root certs are. We now catch SSL errors and try to turn the most common SSL cert verification errors into useful sets of instructions. These describe the error and, when it's something the user can correct, we provide those steps. This includes listing the paths where SSL certs can go, and how to update `certifi` to get the latest SSL certs. Some initialization code has been updated to ensure we're catching connection errors during the initialization process, and logging useful command line options before they occur, further helping us display and diagnose these kinds of problems. This should take care of the most common SSL issues people hit, and give us a good starting point for improving future SSL handling in RBTools.
d903c12d30270d4db865689c5faeb1986a9ab4b8
Description From Last Updated

Do we want to mention --disable-ssl-verification here?

daviddavid
david
  1. 
      
  2. rbtools/api/errors.py (Diff revision 1)
     
     
     
    Show all issues

    Do we want to mention --disable-ssl-verification here?

    1. I think I want to avoid that, because people will use it without understanding the problem or addressing it correctly. If they're using self-signed certs internally, they should get those certs set up in anything accessing the servers.

  3. 
      
david
  1. Ship It!
  2. 
      
chipx86
Review request changed
Status:
Completed
Change Summary:
Pushed to release-4.x (64bc84d)