Python "zipfile" can't detect "quoted-overlap" zipbomb that can be used as a DoS attack

[dyingc]

Bug report

Bug description:

Just found this vulnerability in the latest Python 3.11.5 (and previous 3.10.10).

If we craft a zipbomb using the "quoted-overlap" way (as mentioned https://www.bamsoftware.com/hacks/zipbomb/), this can't be detected by Python's zip file and the zip will be extracted and thus potentially cause a DoS attack by consuming all the storage.

This issue is related to CVE-2019-9674 but not the same. CVE-2019-9674 is talking about the "normal" overlap-zipbomb which is a "full" overlap. This can already be detected by Python's new version of zipfile. However, when we craft a "quoted-overlap" zip, as indicated by https://www.bamsoftware.com/hacks/zipbomb/, python can't detect and happily starts to extract.

For example, the following is the python to extract a zip file, 116 KB before extraction, goes to as large as 17GB after extraction. The size after extraction can be easily increased to multi TBs or even PBs by adjusting the zip-creation.

import zipfile
import sys
import os

def extract_zip(zip_path):
    """
    Extracts the contents of a ZIP file to the current directory.

    :param zip_path: Path to the ZIP file
    """
    if not os.path.exists(zip_path):
        print(f"Error: {zip_path} does not exist.")
        return

    with zipfile.ZipFile(zip_path, 'r') as zip_ref:
        zip_ref.extractall()
        print(f"Extracted contents of {zip_path} to the current directory.")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python extract_zip.py <path_to_zip_file>")
        sys.exit(1)

    zip_file_path = sys.argv[1]
    extract_zip(zip_file_path)

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Linked PRs

• gh-109858: Protect zipfile from "quoted-overlap" zipbomb #110016
• [3.12] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113912
• [3.11] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113913
• [3.10] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113914
• [3.9] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113915
• [3.8] gh-109858: Protect zipfile from "quoted-overlap" zipbomb (GH-110016) #113916
• Add @requires_zlib() decorator for gh-109858 tests #113918

[gpshead]

Additional context: This was reported to us on the Python Security Response Team on 2023-09-06. We ultimately concluded it should be handled in public. The form of zip file attack is not "new" (as evidence by that zipbomb link) and several other zip implementations have had to deal with this over the years. Ours still needs to.

I'll summarize more of our discussions there in a followup comment.

[gpshead]

While we do at least have an existing warning on ZipFile.extractall() in our docs "Warning: Never extract archives from untrusted sources without prior inspection." - that text goes on to focus on other reasons. We expect that there are people who don't actually understand the full implications of handling untrusted compressed complex-referential data like zips and just call an API like .extractall() without sufficient checks regardless.

@SethMichaelLarson & @di looked into PyPI's warehouse to check how PyPI handles this situation with regards to uploaded packages. It already has a decent seeming zip file checking loop in place that rejects zip files who's extracted size would be huge compared to the input as seen in pypi/warehouse#13877 - that seems to reject the example zipbombs already. @pradyunsg later confirmed that while pip itself doesn't do such checks (it probably should) "we do benefit from PyPI's checks in the default configuration and those benefits transfer to mirrored PyPI indexes as well." - so in effect we don't currently believe PyPI can host zipbomb wheels - ameliorating this for the community's pip install invocations by default.

Workarounds & Recommendations

Document the workaround

We should centralize our guidance for Python users on how to avoid the problem when processing untrusted zip files, perhaps with a code snippet such as the overall extraction size checking loop from Warehouse as an example of the kinds of protection users of zipfile today should employ. Having applications and libraries do that is good defense in depth and doesn't require a patched Python runtime.

We also recommend that code handling untrusted zip files always run in a resource constrained container regardless.

Fixing

Improving our zipfile internals to do consistency checks to raise an exception when processing zip files with overlapping sections or inconsistent metadata or whatnot (all of the kinds of things that enable this form of malicious structured file).

How feasible it'll be to backport that is unknown. But as this is only a DoS, for something that most code isn't doing (processing untrusted zips), and it can be worked around, I don't think focusing on designing this backporting this all the way back to 3.8 is worth spending extra time on if it requires any.

Reaching out to specific applications that process arbitrary zip files to make sure they're aware of this issue is a good idea.