• 
      

    Allow spies to replicate function signatures for introspection.

    Review Request #9327 — Created Oct. 25, 2017 and submitted

    Information

    kgb
    master
    fe3e42c...

    Reviewers

    kgb

    Attempting to use functions like inspect.getargspec() with function or
    method spies would result in all sors of failures, depending on what the
    spy was representing. Method spies didn't have any of the attributes
    needed for introspection (like func_defaults) and didn't identify as a
    method (preventing inspect.ismethod() from working). Function spies
    had the attributes, but the signature of the forwarding function didn't
    match that of the original function, negating the usefulness of
    getargspec().

    This change fixes all that by better impersonating the functions being
    spied on.

    When spying on a method, a FunctionSpy instance will report its class
    as types.MethodType, allowing isinstance(spy, types.MethodType)
    (and, by extension, inspect.ismethod()) to work. This is the case for
    Python 2.x and 3.x. It's not really important for FunctionSpy to
    report its true class, so this is completely fine to do.

    Method spies also forward any attribute accesses not otherwise handled
    by the spy up to the original function, allowing func_defaults and
    others to work.

    These two things combined allow getargspec() to work for method spies.
    Function spies, though, are a whole different beast.

    For function spies, we override the bytecode and some other data for the
    function, since we can't replace the function's instance itself. In the
    past, the new method just accepted *args, **kwargs and passed those
    on, but this meant that the function's signature wouldn't show any
    specific argument names, breaking introspection.

    To fix that, we now dynamically build the function definition using
    exec() and inspect.formatargspec(). The resulting definition exactly
    matches the positional and keyword arguments of the function being spied
    on (with the exception that the keyword arguments' default values are
    set to a special value used to identify provided vs. default keyword
    arguments -- this does not break introspection as getargspec() reads
    the original function's func_defaults).

    However, the forwarding function still needs to call the original
    function in exactly the same way that it was called, in order for
    argument tracking to continue to work on the spy. To do this, we grab
    the caller's frame from the current frame and inspect the bytecode of
    the call (which is available to us in the frame data). The values of a
    function call's opcode are the number of positional and keyword
    arguments provided. The positional argument data allows us to determine
    how many of the positional arguments in the function signature need to
    be passed to the original function. We then check all keyword arguments
    to see if any were provided and pass those as well. Finally, if the
    signature takes *args and/or **kwargs, those are also passed.

    To make this work, the forwarding function needs to build the call using
    another exec(), meaning we have two layers of nested exec()s. Which
    is excellent.

    This allows both types of spies to accurately represent the function
    signatures and faithfully forward on the signature of the call, allowing
    introspection to fully work.

    Note that it's now more important that function signatures for fake
    functions match the original function. That is, a fake function expected
    to be called with 3 positional arguments should retain the same names
    and should include *args and **kwargs if they exist in the original
    function.

    All kgb unit tests pass, including new tests that pass arguments in
    different orders and run getargspec.

    Ran the test suites of several large projects that use kgb. All tests
    pass, including those that previously failed due to getargspec use.

    david
    1. Ship It!
    2. 
        
    chipx86
    Review request changed
    Status:
    Completed
    Change Summary:
    Pushed to master (a024f3d)