Scan Through a Directory Recursively in Python

A directory is simply defined as a collection of subdirectories and single files or either one of them. A directory hierarchy is constructed by organizing all the files and subdirectories within a main directory and this is also known as root directory. These subdirectories are separated using a / operator in a directory hierarchy.

In python, we have different methods to scan through a directory recursively. In this article we are going to see about each method.

• Using os.walk() method
• Using glob.glob() method
• Using os.listdir() method

Note: The directories are handled by the operating system; therefore, whenever one needs a status update on any directory it needs to be done using the os module.

Using os.walk() Method

The os.walk() function generates file names in a directory tree by walking it top-down or bottom-up. It returns a three-tuple for each directory in the tree rooted at directory top such as path, names and filenames.

Example - 1

Here, in this example we are using the os.walk() method to display all the files and subdirectories present in the current root directory -

import os
path = "."
for root, d_names, f_names in os.walk(path):
   print(root, d_names, f_names)

Following is the output of the above program, which gives the file names in the directory -

. ['old_articles', '__pycache__'] ['config.py', 'file1.txt', 'file2.txt', 'helper.py',] []

Example - 2

We can also make use of the full path for each file to scan through a directory. For that, we must use the os.path.join() method. This method will create a path for a file. These paths of each file can be appended together using the append() method as shown below -

import os
path = "./TEST"
fname = []
for root,d_names,f_names in os.walk(path):
   for f in f_names:
      fname.append(os.path.join(root, f))
print("fname = %s" %fname)

The output for the program above is as follows -

fname = []

Using glob.glob() Method

The glob module is used to get the pathnames which are matching to a specific pattern in a specific directory. The glob.glob() method is used to search for all the pathnames containing the given path specification as an argument.

If the path specification is passed as an *(Asterisk) then this method matches zero or more characters in the pathname, so it returns all the files present in the directory.

Example

In this example, we are going to print the names of all the files and subdirectories present in the root directory using the glob() method -

from pathlib import Path

root_directory = Path('.')
size = 0
for f in root_directory.glob("*"):
   print(f)

Output

main.py

Using os.listdir() Method

The os.listdir() method in the os module is used to list all files and directories present in a specified directory. This method returns a list containing the names of the entries in the given path.

If no path is specified, then this method lists the contents of the current working directory. It does not return full paths but only the names of the files or directories.

Example

In this example, we are going to print the names of all the files and subdirectories present in the current directory using the os.listdir() method -

import os

directory_contents = os.listdir(".")
for item in directory_contents:
    print(item)

Below is the output of the above program -

main.py
data.txt
logs