Pandas isnull() and notnull() Method

The pandas.isnull() and pandas.notnull() methods are used to detect missing (or null) values in a DataFrame or Series. In real-world datasets, it's common to have incomplete or missing data, and these functions help identify such entries for cleaning or analysis. The isnull() method returns True for missing values and False otherwise.

Similarly, notnull() is the opposite of isnull(). It returns True for non-missing values, making it useful for filtering valid data. These methods are essential tools in data preprocessing, allowing users to quickly locate, remove, or fill missing values to ensure cleaner and more accurate analysis.

Pandas DataFrame isnull() Method

To download the CSV file used, Click Here.

Let see an example: In the following example, The Team column is checked for NULL values and a boolean series is returned by the isnull() method which stores True for ever NaN value and False for a Not null value.

Syntax:

pd.isnull(dataframe)

Parameters: Object to check null values for (DataFrame)

Returns: DataFrame of Boolean values where True indicates the presence of NaN (Not a Number) values in the specified DataFrame. 

# importing pandas package
import pandas as pd

# making data frame from csv file
data = pd.read_csv("/content/employees.csv")

# creating bool series True for NaN values
bool_series = pd.isnull(data["Team"])

# filtering data
data[bool_series]

Output

Count NaN values in Pandas DataFrame

This example demonstrates how to count missing (NaN) values in a specific column of a Pandas DataFrame. It reads data from a CSV file (employees.csv) into a DataFrame called data. It then uses the pd.isnull() function to create a boolean series that flags rows where the "Team" column is NaN. Using this boolean series, it counts and displays how many rows have missing values in that column and prints the filtered rows where the "Team" value is missing.

Syntax:

pd.isnull(obj)

pd.notnull(obj)

Parameters: obj: A Pandas Series, DataFrame, or scalar value to check for NaN.

Returns: A boolean Series or DataFrame of the same shape as obj, where True represents missing (NaN) values for isnull(), and non-missing values for notnull().

import pandas as pd

data = pd.read_csv("employees.csv")

bool_series = pd.isnull(data["Team"])

missing_values_count = bool_series.sum()

filtered_data = data[bool_series]

print("Count of missing values in the 'Team' column:", missing_values_count)

Output:

Count of missing values in the 'Team' column: 43

Pandas DataFrame notnull() Method

In the following example, the Gender column is checked for NULL values and a boolean series is returned by the notnull() method which stores True for every NON-NULL value and False for a null value.

Syntax:

pd.notnull(dataframe)

Parameters: Object to check null values for (DataFrame)

Returns: DataFrame of Boolean values where False indicates the presence of NaN (Not a Number) values in the specified DataFrame.

# importing pandas package
import pandas as pd

# making data frame from csv file
data = pd.read_csv("/content/employees.csv")

# creating bool series False for NaN values
bool_series = pd.notnull(data["Gender"])

# displayed data only with team = NaN
data[bool_series]

Output:

Filter a Pandas Dataframe Based on Null Values

This example reads a CSV file (employees.csv) into a Pandas DataFrame called data. It then uses the pd.notnull() function to generate a boolean series, bool_series, which marks True for all rows where the "Gender" column is not null (i.e., has valid data). The code filters the DataFrame using this series to create a subset that contains only rows with known gender values. Finally, it prints this cleaned subset of the data.

Syntax:

pd.notnull(obj)

Parameters: obj: Can be a scalar value, Series, or DataFrame. It is the input to check for non-missing (non-NaN) values.

Returns: A boolean Series or DataFrame of the same shape as the input, where True indicates non-null (valid) entries.

import pandas as pd

# Making a DataFrame from a CSV file
data = pd.read_csv("employees.csv")

# Creating a boolean series marking False for NaN values in the "Gender" column
bool_series = pd.notnull(data["Gender"])

# Displaying data only with non-null "Gender" values
filtered_data = data[bool_series]

print("Data with non-null 'Gender' values:")
print(filtered_data)

Output