Fix crashes that could occur when detecting file mimetypes.

Review Request #14197 — Created Oct. 11, 2024 and updated

Information

Review Board
release-7.x

Reviewers

We had a few issues that occurred in some environments when trying to
detect the mimetype of a file from its contents. These were rare,
dependent on Python version behavior and the implementation of file on
the system, but have been verified with one customer in particular:

  1. Some versions of Python break if you try to close an already-closed
    pipe, and this had happened on one customer's machine. file may
    close the input stream once it's read enough data to make a
    determination, and our attempt to close would then fail.

  2. We didn't set the mimetype variable if file returned a non-0 exit
    code, which resulted in later usage of an undefined variable.

  3. Errors executing or communicating with file would crash, and even
    if caught higher up it'd leave the uploaded file truncated (the file
    pointer wouldn't be at 0).

This change fixes all these through enhanged exception handlers and
checks, an unconditional reset of the uploaded file's file pointer, and
a fallback value for the mimetype.

All unit tests pass.

Notably, I cannot reproduce this bug in my environment, but I can verify
the cause in the affected customer's environment and that this would
address what we know about that crash.

Summary ID
Fix crashes that could occur when detecting file mimetypes.
We had a few issues that occurred in some environments when trying to detect the mimetype of a file from its contents. These were rare, dependent on Python version behavior and the implementation of `file` on the system, but have been verified with one customer in particular: 1. Some versions of Python break if you try to close an already-closed pipe, and this had happened on one customer's machine. `file` may close the input stream once it's read enough data to make a determination, and our attempt to close would then fail. 2. We didn't set the `mimetype` variable if `file` returned a non-0 exit code, which resulted in later usage of an undefined variable. 3. Errors executing or communicating with `file` would crash, and even if caught higher up it'd leave the uploaded file truncated (the file pointer wouldn't be at 0). This change fixes all these through enhanged exception handlers and checks, an unconditional reset of the uploaded file's file pointer, and a fallback value for the mimetype.
fe1389aef25e7e454cfd81570180327abed8b57b
Description From Last Updated

This looks like it's left over from an earlier iteration? We don't use i anywhere.

daviddavid
david
  1. 
      
  2. reviewboard/attachments/mimetypes.py (Diff revision 1)
     
     
    Show all issues

    This looks like it's left over from an earlier iteration? We don't use i anywhere.

    1. Yeah, some debugging code I had in while trying to make this reproduce in my environment.

  3. 
      
chipx86
Review request changed
Testing Done:
   

All unit tests pass.

  +
  +

Notably, I cannot reproduce this bug in my environment, but I can verify

  + the cause in the affected customer's environment and that this would
  + address what we know about that crash.

Commits:
Summary ID
Fix crashes that could occur when detecting file mimetypes.
We had a few issues that occurred in some environments when trying to detect the mimetype of a file from its contents. These were rare, dependent on Python version behavior and the implementation of `file` on the system, but have been verified with one customer in particular: 1. Some versions of Python break if you try to close an already-closed pipe, and this had happened on one customer's machine. `file` may close the input stream once it's read enough data to make a determination, and our attempt to close would then fail. 2. We didn't set the `mimetype` variable if `file` returned a non-0 exit code, which resulted in later usage of an undefined variable. 3. Errors executing or communicating with `file` would crash, and even if caught higher up it'd leave the uploaded file truncated (the file pointer wouldn't be at 0). This change fixes all these through enhanged exception handlers and checks, an unconditional reset of the uploaded file's file pointer, and a fallback value for the mimetype.
2c3542402c5b4c86bd95b64960bf5de872d806bf
Fix crashes that could occur when detecting file mimetypes.
We had a few issues that occurred in some environments when trying to detect the mimetype of a file from its contents. These were rare, dependent on Python version behavior and the implementation of `file` on the system, but have been verified with one customer in particular: 1. Some versions of Python break if you try to close an already-closed pipe, and this had happened on one customer's machine. `file` may close the input stream once it's read enough data to make a determination, and our attempt to close would then fail. 2. We didn't set the `mimetype` variable if `file` returned a non-0 exit code, which resulted in later usage of an undefined variable. 3. Errors executing or communicating with `file` would crash, and even if caught higher up it'd leave the uploaded file truncated (the file pointer wouldn't be at 0). This change fixes all these through enhanged exception handlers and checks, an unconditional reset of the uploaded file's file pointer, and a fallback value for the mimetype.
fe1389aef25e7e454cfd81570180327abed8b57b

Checks run (2 succeeded)

flake8 passed.
JSHint passed.
david
  1. Ship It!
  2. 
      
maubin
  1. Ship It!
  2.