Add API caching support to rbtools.

Review Request #6504 — Created Oct. 24, 2014 and submitted — Latest diff uploaded




Add the rbtools.api.cache module. This module includes the
APICache class, which allows the results of HTTP GET requests to
be cached locally to achieve faster lookup. The APICache understands
the following headers:

  • Cache-Control
  • Pragma
  • Vary
  • Last-Modified
  • ETag
  • Expires

The APICache is backed by a SQLite database in the user's cache
directory. If the cache cannot be accessed, the APICache will pass
all requests to the specified urlopen (defaulting to
urllib2.urlopen) method.

The APICache allows for a custom urlopen function so that it can
be tested without making actual HTTP requests.

The APICache stores the version of its schema in the database. If
the APICache is instantiated and the versions of the schema in the
class doesn't match the version of the schema in the database, then
the current cache will be invalidted and the schema will be updated.

Add the ActiveState appdirs module (at version 1.4.0) as
rbtools.utils.appdirs. This module is used to find the user cache
directory in platform-dependant ways.

Add the rbtools.api.cache.Response class which emulates the
behaviour of a urllib2 response. This class allows the read
method to be called more than once.

Add the rbtools.api.cache.CachedResponse class, which is the result
of a cach entry being found the the APICache. It also emulates the
behaviour of a urllib2 response.

Add the clear-cache command. This command manually clears the API

Added unit tests for the APICache.

Unit tests pass.

The following commands were tested and were successful:

  • rbt status
  • rbt post
  • rbt diff
  • rbt patch
  • rbt close
  • rbt clear-cache

Upon clearing the cache with clear-cache, a new cache is created.
The first run of a command always creates cache entries (where
appropriate, i.e., not for PUT or POST requests).

The /api/ endpoint is successfully cached and the cached version is

When using the patch command for the second time on the same review
request, if the diff has not been updated, then the cached version of
the /api/review-requests/<id>/diff/<diff_revision>/ endpoint is

This review request was posted with the APICache enabled.