Rework extension package metadata handling for Python 3 support.

Review Request #9732 — Created March 4, 2018 and submitted


When loading information for an extension, we attempt to look up one of
the metadata files, normalize the string content, and parse it. On
Python 2, this worked fine no matter what type of string format we got
from pkg_resources, because both binary and Unicode strings happily
let you run decode(). This isn't the case on Python 3. We always have
Unicode strings there, which don't support decode().

We weren't really set up to test this too well, because in the unit
tests, we were overriding too much of the metadata fetching logic,
making it hard to really test these differences.

This change updates the metadata handling to check the string type
before deciding whether we need to decode anything, and to override the
lowest-level function we can to return string content, allowing more of
pkg_resources's parsing code to run. To do this, we no longer create
our own stub for a Distribution, but instead use Distribution as-is
and give it a fake Provider, which is the backend class that looks up
metadata from a package for the distribution to use.

There's a couple of other small things that were also incorrect in this
process. We had to deal with the result type from
get_metadata_lines(), which is a generator on Python 3. Things were
also more sensitive when it came to including the admin site's URLs, and
so a test was updated to do this correctly.

Relevant unit tests pass on Python 2.7, 3.4, 3.5, and 3.6.

  1. Ship It!
  1. Ship It!
Review request changed

Status: Closed (submitted)

Change Summary:

Pushed to release-2.0.x (9cd509f)