Command Line File Downloader in Python
Last Updated :
24 Jan, 2021
Python is one of the most popular general-purpose programming languages with a wide range of use cases from general coding to complex fields like AI. One of the reasons for such popularity of python as a programming language is the availability of many built-in as well as third-party libraries and packages.
In this article, we are going to build a simple command-line file downloader, which you can download a file if you have the download link.
Approach:
We are going to download files over HTTP instead of FTP. Once we have made the request for the file, we will use the response as an input stream to write or save the file to the file system. While Downloading, details like download speed, time, and amount of file downloaded will be shown.
- Take the file URL via command line
- Open the file input or download stream over HTTP via requests.
- Download the file by writing the file in chunks from the download/input stream into the file system.
- Display the details
- Save the file.
Step-by-step implementation:
Step 1: Libraries required
Python3
import requests
import sys
import time
Step 2: a dictionary for detecting file size in proper units
While Downloading the file, parameters like speed, size, time will be shown. So let's see how to do this. Dictionary for getting file size and download speed unit depending on the amount downloaded takes sizes in bytes.
Python3
units = {
'B':{'size':1, 'speed':'B/s'},
'KB':{'size':1024, 'speed':'KB/s'},
'MB':{'size':1024*1024, 'speed':'MB/s'},
'GB':{'size':1024*1024*1024, 'speed':'GB/s'}
}
Function for checking units of the length of the file downloaded. Length is in bytes so corresponding units are checked.
Python3
def check_unit(length): # length in bytes
if length < units['KB']['size']:
return 'B'
elif length >= units['KB']['size'] and length <= units['MB']['size']:
return 'KB'
elif length >= units['MB']['size'] and length <= units['GB']['size']:
return 'MB'
elif length > units['GB']['size']:
return 'GB'
Step 3: Downloading file over HTTP
We will open a file stream over HTTP using requests and then save the data in chunks to the local file. Let's see how the code will look and then put it all together.
Python3
# Opening file stream
r = requests.get(link_to_file, stream = True)
# writing file data in chunks.
# for examples, A file of 10 MB written in
# chunk size of 8096 Bytes.
with open(file_name, 'wb') as f:
for chunk in r.iter_content(chunk_size):
f.write(chunk)
Step 4: Printing output
The output will consist of a dynamic progress bar with downloading details. For this we use stdout.write() and stdout.flush() methods
Python3
sys.stdout.write(format_string % (list_of_variables))
# for printing details in single
# line which updates without going to the next line
# once the details are updated.
sys.stdout.flush()
Below is the full implementation:
Python3
import requests
import sys
import time
units = {
'B' : {'size':1, 'speed':'B/s'},
'KB' : {'size':1024, 'speed':'KB/s'},
'MB' : {'size':1024*1024, 'speed':'MB/s'},
'GB' : {'size':1024*1024*1024, 'speed':'GB/s'}
}
def check_unit(length): # length in bytes
if length < units['KB']['size']:
return 'B'
elif length >= units['KB']['size'] and length <= units['MB']['size']:
return 'KB'
elif length >= units['MB']['size'] and length <= units['GB']['size']:
return 'MB'
elif length > units['GB']['size']:
return 'GB'
# takes download link and directory where file to be saved.
def downloadFile(url, directory) :
localFilename = url.split('/')[-1] # files name
with open(directory + '/' + localFilename, 'wb') as f:
print ("Downloading . . .\n")
start = time.time() # start time
r = requests.get(url, stream=True)
# total length in bytes of the file
total_length = float(r.headers.get('content-length'))
d = 0 # counter for amount downloaded
# when file is not available
if total_length is None:
f.write(r.content)
else:
for chunk in r.iter_content(8192):
d += float(len(chunk))
f.write(chunk) # writing the file in chunks of 8192 bytes
# amount downloaded in proper units
downloaded = d/units[check_unit(d)]['size']
# converting the unit of total length or size of file from bytes.
tl = total_length / units[check_unit(total_length)]['size']
trs = d // (time.time() - start) # speed in bytes per sec
#speed in proper unit
download_speed = trs/units[check_unit(trs)]['size']
speed_unit = units[check_unit(trs)]['speed'] # speed in proper units
done = 100 * d / total_length # percentage downloaded or done.
fmt_string = "\r%6.2f %s [%s%s] %7.2f%s / %4.2f %s %7.2f %s"
set_of_vars = ( float(done), '%',
'*' * int(done/2),
'_' * int(50-done/2),
downloaded, check_unit(d),
tl, check_unit(total_length),
download_speed, speed_unit)
sys.stdout.write(fmt_string % set_of_vars)
sys.stdout.flush()
return (time.time() - start) # total time taken for download
def main() :
directory = '.'
if len(sys.argv) > 1 :
url = sys.argv[1] # url from cmd line arg
if len(sys.argv) > 2:
directory = sys.argv[2]
total_time = downloadFile(url, directory)
print ('')
print ("Download complete...")
print ("\rTime Elapsed: %.2fs" % total_time)
else :
print("No link found!")
if __name__ == "__main__" :
main()
Save the code in a python file and use it as follows
python <program_name>.py <file_link> <save_location(by default '.')>
Output:
Similar Reads
Close a File in Python
In Python, a file object (often denoted as fp) is a representation of an open file. When working with files, it is essential to close the file properly to release system resources and ensure data integrity. Closing a file is crucial to avoid potential issues like data corruption and resource leaks.
2 min read
Check end of file in Python
In Python, checking the end of a file is easy and can be done using different methods. One of the simplest ways to check the end of a file is by reading the file's content in chunks. When read() method reaches the end, it returns an empty string.Pythonf = open("file.txt", "r") # Read the entire cont
2 min read
Check a File is Opened or Closed in Python
In computer programming, working with files is something we often do. Python, a programming language, gives us useful tools to handle files. One important thing to know when dealing with files is whether a file is currently open or closed. This is crucial to avoid problems and make sure the data sta
4 min read
Print the Content of a Txt File in Python
Python provides a straightforward way to read and print the contents of a .txt file. Whether you are a beginner or an experienced developer, understanding how to work with file operations in Python is essential. In this article, we will explore some simple code examples to help you print the content
3 min read
Check if a File Exists in Python
When working with files in Python, we often need to check if a file exists before performing any operations like reading or writing. by using some simple methods we can check if a file exists in Python without tackling any error. Using pathlib.Path.exists (Recommended Method)Starting with Python 3.4
3 min read
How Use Linux Command In Python Using System.Os
Using the system module from the os library in Python allows you to interact with the Linux command line directly from your Python script. This module provides a way to execute system commands, enabling you to automate various tasks by integrating Linux commands seamlessly into your Python code. Whe
3 min read
Create A File If Not Exists In Python
In Python, creating a file if it does not exist is a common task that can be achieved with simplicity and efficiency. By employing the open() function with the 'x' mode, one can ensure that the file is created only if it does not already exist. This brief guide will explore the concise yet powerful
2 min read
Python Loop through Folders and Files in Directory
File iteration is a crucial process of working with files in Python. The process of accessing and processing each item in any collection is called File iteration in Python, which involves looping through a folder and perform operation on each file. In this article, we will see how we can iterate ove
4 min read
Python Delete File
When any large program is created, usually there are small files that we need to create to store some data that is needed for the large programs. when our program is completed, so we need to delete them. In this article, we will see how to delete a file in Python. Methods to Delete a File in Python
4 min read
Check If A File is Writable in Python
When it comes to Python programming it is essential to work with files. One important aspect is making sure that a file can be written before you try to make any changes. In this article, we will see how we can check if a file is writable in Python. Check If a File Is Writable in PythonBelow are som
3 min read