Python groupby method to remove all consecutive duplicates

In Python, the groupby method from the itertools module provides a powerful way to group consecutive identical elements in an iterable. This functionality can be effectively utilized to remove all consecutive duplicates from a sequence. By retaining only the first occurrence of each group of identical elements, we can clean up a list or string while preserving its overall structure.

Let’s start with a simple example to see how the groupby method can help us remove consecutive duplicates:

from itertools import groupby

# Example list with consecutive duplicates
s = [1, 1, 2, 3, 3, 4]

# Removing consecutive duplicates using groupby
res = [key for key, _ in groupby(s)]
print(res) 

[1, 2, 3, 4]

Explanation:

• groupby function groups consecutive identical elements in the sequence.
• For each group, the key (unique element) is extracted and added to the result list.
• The final output is a list with all consecutive duplicates removed.

Syntax of groupby method

itertools.groupby(iterable, key=None)

Parameters

• iterable: The input iterable to be grouped.
• key (optional): A function to specify a criterion for grouping. By default, consecutive identical elements are grouped.

Return Type

• Returns an iterator that generates groups of consecutive identical elements from the input iterable.

Example 1: Removing consecutive duplicates from a list

Before diving into the implementation, consider a list with repetitive elements:

from itertools import groupby

# List with consecutive duplicates
s = ["a", "a", "b", "b", "c"]

# Remove consecutive duplicates
res = [key for key, _ in groupby(s)]
print(res)  

['a', 'b', 'c']

Explanation:

• groupby method groups consecutive identical elements.
• Key variable represents the unique elements in each group, forming the final cleaned list.

Example 2: Removing duplicates from a string

Let’s apply the groupby method to a string:

from itertools import groupby

# String with consecutive duplicates
s = "aabbcc"

# Remove consecutive duplicates
res = "".join(key for key, _ in groupby(s))
print(res)  

abc

Explanation:

• By using "".join(), the unique keys are concatenated into a single string without consecutive duplicates.
• The process works similarly to lists, treating characters as elements.

Example 3: Handling case sensitivity

Strings can be processed in a case-sensitive manner by default. Here’s how it works:

from itertools import groupby

# String with mixed-case duplicates
s = "aaAAaabb"

# Remove consecutive duplicates
res = "".join(key for key, _ in groupby(s))
print(res)  

aAab

Explanation:

• Consecutive duplicates are removed without altering the case.
• The method groups identical elements as they appear, maintaining case sensitivity.

Example 4: Custom grouping with a key function

Key parameter can be used for advanced grouping scenarios. For instance, treating uppercase and lowercase letters as identical:

from itertools import groupby

# String with mixed-case duplicates
s = "aaAAaabb"

# Remove consecutive duplicates ignoring case
res = "".join(key for key, _ in groupby(s, key=str.lower))
print(res) 

ab

Explanation:

• str.lower function is passed as the key, ensuring that grouping ignores case differences.
• This creates a more flexible approach to grouping elements.

Frequently Asked Questions (FAQs) on groupby method

1. Can the groupby method be used with data types other than lists and strings?

Yes, the groupby method works with any iterable, including tuples and generators:

from itertools import groupby
# Tuple with consecutive duplicates
a = (1, 1, 2, 2, 3)
res = [key for key, _ in groupby(a)]
print(res)  # Output: [1, 2, 3]

2. How does the groupby method handle non-consecutive duplicates?

The groupby method only groups consecutive identical elements. Non-consecutive duplicates remain in the result:

from itertools import groupby
# List with non-consecutive duplicates
s = [1, 2, 1, 2]
res = [key for key, _ in groupby(s)]
print(res)  # Output: [1, 2, 1, 2]

3. Can we use groupby for numeric processing?

Absolutely. The method works seamlessly with numeric data:

from itertools import groupby
# Numeric data
n = [1, 1, 2, 2, 3, 3]
res = [key for key, _ in groupby(n)]
print(res)  # Output: [1, 2, 3]