• 
      

    Sandbox cache backend errors in cache_memoize().

    Review Request #13446 — Created Dec. 4, 2023 and submitted

    Information

    Djblets
    release-3.x

    Reviewers

    If a cache backend was acting up or was down, calls to cache_memoize()
    or cache_memoize_iter() would easily fail, propagating errors and
    breaking the application. This could interfere with state generation or
    just halt operations on a server.

    This is now fully-insulated from any errors coming from the backend. The
    _CacheContext.load_value(), store_value(), and store_many()
    internal helpers now catch exceptions and log them, providing some
    useful information to help immediately show any issues in logs.

    The functions calling these then catch the exceptions, optionally adding
    more context for logs, and then gracefully handle and discard the
    exceptions.

    For a standard cache_memoize(), this means that generated data will be
    returned but any failure to get/set data will be succeed. This is
    actually the same behavior we already had, with one exception (no pun
    intended): Upon exception, a variable would be undefined, breaking. This
    is now set correctly.

    The bulk of the new sandboxing comes from working with large or iterable
    data. We already had protection from loading data from cache (mostly to
    take into account missing chunked cache items), in that we'd fall back
    on generating new data. But we now protect when setting cached data as
    well. If we fail to set a cache key, we give up on setting any prior
    keys for that data, and just yield the newly-generated data as before.

    This should all protect from bad cache server behavior,
    misconfigurations, and outages in code that uses cache_memoize() and
    friends.

    All Djblets and Review Board unit tests pass.

    Summary ID
    Sandbox cache backend errors in cache_memoize().
    If a cache backend was acting up or was down, calls to `cache_memoize()` or `cache_memoize_iter()` would easily fail, propagating errors and breaking the application. This could interfere with state generation or just halt operations on a server. This is now fully-insulated from any errors coming from the backend. The `_CacheContext.load_value()`, `store_value()`, and `store_many()` internal helpers now catch exceptions and log them, providing some useful information to help immediately show any issues in logs. The functions calling these then catch the exceptions, optionally adding more context for logs, and then gracefully handle and discard the exceptions. For a standard `cache_memoize()`, this means that generated data will be returned but any failure to get/set data will be succeed. This is actually the same behavior we already had, with one exception (no pun intended): Upon exception, a variable would be undefined, breaking. This is now set correctly. The bulk of the new sandboxing comes from working with large or iterable data. We already had protection from loading data from cache (mostly to take into account missing chunked cache items), in that we'd fall back on generating new data. But we now protect when setting cached data as well. If we fail to set a cache key, we give up on setting any prior keys for that data, and just yield the newly-generated data as before. This should all protect from bad cache server behavior, misconfigurations, and outages in code that uses `cache_memoize()` and friends.
    4b0db2cd509d5c2572c36de8574d15c87b2121f4
    david
    1. Ship It!
    2. 
        
    maubin
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-3.x (20278e2)