Pandas is a data analysis and manipulation library for Python. It provides numerous functions and methods to manage tabular data. The core data structure of pandas is DataFrame, which stores data in tabular form with labeled columns and rows.
How to Add a Column to a pandas DataFrame
How to add a column onto the end of a pandas DataFrame:
df["new column"] = 1or
df["new column"] = [1, 2, 3]In this code, the first set of brackets represents the name of the new column, while values after the = sign is the value(s) assigned under the column.
How to insert a column in a pandas DataFrame:
df.insert(1, "new column", 1)or
df.insert(1, "new column", [1, 2, 3])This code uses insert(), which requires three parameters: the index of where the new column will be added, the name of the new column and the new value(s) under the column.
From a data perspective, rows represent observations or data points. Columns represent features or attributes about the observations. Consider a DataFrame of house prices. Each row is a house and each column is a feature about the house such as age, number of rooms, price and so on.
Adding or dropping columns is a common operation in data analysis. We’ll go over four different ways of adding a new column to a DataFrame.
First, let’s create a simple DataFrame to use in the examples.
import numpy as np
import pandas as pd
df = pd.DataFrame({"A": [1, 2, 3, 4],
"B": [5, 6, 7, 8]})
print(df)
5 Pandas Add Column Methods
Below are four methods for adding columns to a pandas DataFrame.
Method 1: Add Columns on the End
This might be the most commonly used method for creating a new column.
#new column on the end of DataFrame
df["C"] = [10, 20, 30, 40]
print(df)
We specify the column name like we are selecting a column in the DataFrame. Then, the values are assigned to this column. A new column is added as the last column, i.e. the column with the highest index.
Adding Multiple Columns to the End of a Pandas DataFrame
We can also add multiple columns at once with this method. Column names are passed in a list and values need to be two-dimensional compatible with the number of rows and columns. For instance, the following code adds three columns filled with random integers between 0 and 10.
#multiple new columns on the end of DataFrame
df[["1of3", "2of3", "3of3"]] = np.random.randint(10, size=(4,3))
print(df)
Let’s drop these three columns before going to the next method.
df.drop(["1of3", "2of3", "3of3"], axis=1, inplace=True)
Method 2: Add Columns at a Specific Index With DataFrame.Insert()
In the first method, the new column is added at the end. Pandas also allows for adding new columns at a specific index. The insert() function can be used to customize the location of the new column. Let’s add a column next to column A.
#new column using df.insert()
df.insert(1, "D", 5)
print(df)
The insert function takes three parameters that are the index, the name of the column and the values. The column indices start from zero, so we set the index parameter as one to add the new column next to column A. We can pass a constant value to be filled in all rows.
Method 3: Add Columns With DataFrame.Loc[]
The loc[] method allows you to select rows and columns using their labels. It’s also possible to create a new column with this method.
#new column using df.loc[]
df.loc[:, "E"] = list("abcd")
print(df)
In order to select rows and columns, we pass the desired labels. The colon indicates that we want to select all the rows. In the column part, we specify the labels of the columns to be selected. Since the DataFrame does not have column E, pandas creates a new column.
Method 4: Add Columns With DataFrame.Assign()
The last method is the assign() function.
#new column using df.assign()
df = df.assign(F = df.C * 10)
print(df)
We specify both the column name and values inside the assign() function. You may notice that we derive the values using another column in the DataFrame. The previous methods also allow for similar derivations.
Method 5: Add Columns With a Python Dictionary
A Python dictionary can also be used to add a new column in a pandas DataFrame. This method is best to use when needing to add multiple columns at once or if you need to have columns in a dictionary format.
#new column using Python dictionary
g_dict = {0: 9, 1: 10, 2: 11, 3: 12}
df["G"] = df.index.map(g_dict)
print(df)
#Output:
A D B C E F G
0 1 5 5 10 a 100 9
1 2 5 6 20 b 200 10
2 3 5 7 30 c 300 11
3 4 5 8 40 d 400 12
Insert() vs. Assign() for Adding Columns in Pandas
There is an important difference between the insert() and assign() functions in pandas. The insert() function works in place, which means adding a new column is saved in the DataFrame.
The situation is a little different with the assign() function. It returns the modified DataFrame but does not change the original one. In order to use the modified version with the new column, we need to explicitly assign it.
We’ve now covered four different methods for adding new columns to a pandas DataFrame, a common operation in data analysis and manipulation. One of the things I like about pandas is that it usually provides multiple ways to perform a given task, making it a flexible and versatile tool for analyzing and manipulating data.
Frequently Asked Questions
How do you add columns in Pandas?
A column can be added in a Pandas DataFrame by creating a new column on the end or by using the insert(), loc[] or assign() methods.
- To add a column on the end of a DataFrame:
df[“A”] = [1, 2, 3] - To add a column at a specific index in a DataFrame:
df.insert(1, “A”, [1, 2, 3]) - To add a column at a specific index and/or modify values in an existing column:
df.loc[:, “A”] = [1, 2, 3] - To add a new column on the end and return a new DataFrame object:
df = df.assign(A = [1,2,3])
How do you add up a column in Pandas?
The sum() method can be used to add up values of a column in a Pandas DataFrame, such as total = df[“A”].sum().
How do I add multiple columns in Pandas?
To add multiple columns to a Pandas DataFrame, you can add them directly or use the assign() method.
- Adding multiple columns with one value each:
df[[“A”, “B”, “C”]] = pd.DataFrame([[1, 2, 3]], index=df.index) - Adding multiple columns using assign():
df = df.assign(A = [1, 2, 3], B = [4, 5, 6])
How do I avoid SettingWithCopyWarning when adding a DataFrame column?
To avoid the SettingWithCopyWarning, use .loc[] when assigning new columns to a DataFrame. This ensures you're modifying the original DataFrame directly:
df.loc[:, 'new_column'] = values
Avoid chained assignments like df[df['A'] > 0]['B'] = value, which can trigger the warning.
What is the difference between insert() and assign() in Pandas?
In pandas in Python, insert() adds a column at a specific position and modifies the DataFrame in place:
df.insert(1, 'new_col', values)
assign() returns a new DataFrame with the added column, leaving the original unchanged:
df = df.assign(new_col=values)
Use insert() for positional control, assign() for chaining.
