• 
      

    Fix handling the same key at multiple depths with API resource expansion.

    Review Request #11667 — Created June 21, 2021 and submitted

    Information

    Djblets
    release-2.x

    Reviewers

    API resource expansion, using ?expand=, could end up matching keys at
    the wrong level of depth when serializing trees of objects. If there's a
    resource hierarchy that looks like:

    A
    |- B
    |  |-C
    |- C
    

    and expand=B,C, with A.B serialized before A.C, then we'd end up
    with A.B.C expanded but not necessarily A.C.

    This is because we were keeping a record of keys we could expand at each
    new depth, removing any we were handling from the list, and if we ended
    up recursing into a child resource that had the same key as a resource
    we intended to expand on a parent resource, we'd skip the second
    occurrence on the parent.

    This only happened if B and C were listed in fields, rather than
    item_child_resources.

    This manifested in Review Board as an inability to expand
    target_groups on a review_request_draft if we were also expanding
    depends_on. The target_groups in depends_on would be matched
    first, and we'd then skip the intended target_groups.

    There were two causes for this:

    1. We defaulted the list of expanded resource keys usable for child
      resources to be the same exact list of expanded resources for the
      current object being serializes, which meant any time we removed one
      from the child-specific list, we also removed it for the current
      object's list.

    2. Manipulating this list during field iteration meant that we could end
      up serializing a child with an incorrect list of eligible expanded
      fields.

    The fix for this is to build the list of viable expansion fields for
    child resources up-front. We start off with a copy of the list made in
    the request, and then we remove anything we could possibly expand from
    the object being serialized. This is done before we begin serializing
    any object contents, guaranteeing we have a correct list at all times.

    Unit tests pass on Python 2 and 3.

    Reproduced the issue in Review Board where a depends_on would prevent
    target_groups from being expanded correctly. Verified this was fixed
    with this change.

    Summary ID
    Fix handling the same key at multiple depths with API resource expansion.
    API resource expansion, using `?expand=`, could end up matching keys at the wrong level of depth when serializing trees of objects. If there's a resource hierarchy that looks like: A |- B | |-C |- C and `expand=B,C`, with `A.B` serialized before `A.C`, then we'd end up with `A.B.C` expanded but not necessarily `A.C`. This is because we were keeping a record of keys we could expand at each new depth, removing any we were handling from the list, and if we ended up recursing into a child resource that had the same key as a resource we intended to expand on a parent resource, we'd skip the second occurrence on the parent. This only happened if `B` and `C` were listed in `fields`, rather than `item_child_resources`. This manifested in Review Board as an inability to expand `target_groups` on a `review_request_draft` if we were also expanding `depends_on`. The `target_groups` in `depends_on` would be matched first, and we'd then skip the intended `target_groups`. There were two causes for this: 1. We defaulted the list of expanded resource keys usable for child resources to be the same exact list of expanded resources for the current object being serializes, which meant any time we removed one from the child-specific list, we also removed it for the current object's list. 2. Manipulating this list during field iteration meant that we could end up serializing a child with an incorrect list of eligible expanded fields. The fix for this is to build the list of viable expansion fields for child resources up-front. We start off with a copy of the list made in the request, and then we remove anything we could possibly expand from the object being serialized. This is done before we begin serializing any object contents, guaranteeing we have a correct list at all times.
    9f40f2d7f96be5a7b57a4cbe7aac92cbd5c5c407
    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to release-2.x (33cc134)