What is the maximum file size we can open using Python?

In Python, there is no fixed maximum file size that we can open. Python can open files of any size as long as the operating system and file system support until there is enough memory and disk space available.

The limit of maximum file size comes with the system architecture, i.e., 32-bit, 64-bit, the file system, such as FAT32, NTFS, ext4, and how the file is processed in the code.

System and File System Limits

Following are the system and file system limits -

• FAT32 - Maximum file size: 4 GB
• NTFS (Windows) - Supports files up to 16 EB (Exabytes)
• ext4 (Linux) - Supports files up to 16 TB (Terabytes)
• 32-bit OS - May limit process memory usage to ~2-4 GB
• 64-bit OS - Supports much larger files, depending on RAM and storage

Checking File Size Using Python

We need to check the file size before working with the file by using the os.stat() function, which provides metadata about the file, including its size in bytes.

Syntax

Here is the syntax of using the os.stat() function -

os.stat(filepath).st_size

Here, filepath is the location of the file, and st_size returns its size in bytes.

Example: Retrieving File Size

Following is the example which shows how to get the size of a file and convert it to megabytes for easier reading -

import os

file_path = r"D:\Tutorialspoint\Articles\largefile.txt"
try:
    file_size_bytes = os.stat(file_path).st_size
    file_size_mb = file_size_bytes / (1024 * 1024)
    print(f"File size: {file_size_mb:.2f} MB")
except FileNotFoundError:
    print("The file was not found.")

Here is the output of the above example -

File size: 128.75 MB

Reading Large Files

When working with large files in Python, reading the entire file at a time may occupy large memory and affect the performance. To avoid this, it's better to read the file in smaller parts or chunks. This method is useful for processing big log files, large datasets, or binary files.

Following is the example which reads a large file in 1 MB chunks using binary mode "rb" in the open() method. We can adjust the chunk size based on our requirements -

with open("D:\Tutorialspoint\Articles\file1.txt", "rb") as f:
   chunk_size = 1024 * 1024  # read 1 MB at a time
   while True:
      chunk = f.read(chunk_size)
      if not chunk:
         break
      # process each chunk of the file here
      print(len(chunk))

Here is the output of the above program -

Chunk size: 32 bytes