Python program to remove duplicate elements index from other list

Given two lists, the task is to write a Python program to remove all the index elements from 2nd list which are duplicate element indices from 1st list.

Examples:

Input : test_list1 = [3, 5, 6, 5, 3, 7, 8, 6], test_list2 = [1, 7, 6, 3, 7, 9, 10, 11] 
Output : [1, 7, 6, 9, 10] 
Explanation : 3, 7 and 11 correspond to 2nd occurrence of 5, 3 and 6, hence removed.

Input : test_list1 = [3, 5, 6, 5, 3, 7, 8], test_list2 = [1, 7, 6, 3, 7, 9, 10] 
Output : [1, 7, 6, 9, 10] 
Explanation : 3 and 7 correspond to 2nd occurrence of 5 and 3 hence removed. 
 

Method 1: Using list comprehension + loop + enumerate()

In this, we perform task of getting all the indices using enumerate() and loop using set to store already occurred elements. Then, same indices from other list are omitted.

# Python3 code to demonstrate working of
# Remove duplicate elements index from other list
# Using list comprehension + loop + enumerate()

# initializing lists
test_list1 = [3, 5, 6, 5, 3, 7, 8, 6]
test_list2 = [1, 7, 6, 3, 7, 9, 10, 11]
             
# printing original lists
print("The original list 1 is : " + str(test_list1))
print("The original list 2 is : " + str(test_list2))

oc_set = set() 
temp = [] 

# getting duplicate elements list
for idx, val in enumerate(test_list1): 
    if val not in oc_set: 
        oc_set.add(val)          
    else: 
        temp.append(idx)

# excluding duplicate indices from other list
res = [ele for idx, ele in enumerate(test_list2) if idx not in temp]

# printing result
print("The list after removing duplicate indices : " + str(res))

 Output:

The original list 1 is : [3, 5, 6, 5, 3, 7, 8, 6]
The original list 2 is : [1, 7, 6, 3, 7, 9, 10, 11]
The list after removing duplicate indices : [1, 7, 6, 9, 10]

Time Complexity: O(n*n), where n is the number of elements in the list “test_list”.
Auxiliary Space: O(n), where n is the number of elements in the list “test_list”.

 Method 2: Using list comprehension + enumerate()

In this, we perform similar task as above just in a shorthand manner, extracting indices using enumerate() and list comprehension.

# Python3 code to demonstrate working of
# Remove duplicate elements index from other list
# Using list comprehension + enumerate()

# initializing lists
test_list1 = [3, 5, 6, 5, 3, 7, 8, 6]
test_list2 = [1, 7, 6, 3, 7, 9, 10, 11]
             
# printing original lists
print("The original list 1 is : " + str(test_list1))
print("The original list 2 is : " + str(test_list2))

# getting duplicate elements list using list comprehension
temp = [idx for idx, val in enumerate(test_list1) if val in test_list1[:idx]]

# excluding duplicate indices from other list
res = [ele for idx, ele in enumerate(test_list2) if idx not in temp]

# printing result
print("The list after removing duplicate indices : " + str(res))

Output:

The original list 1 is : [3, 5, 6, 5, 3, 7, 8, 6]
The original list 2 is : [1, 7, 6, 3, 7, 9, 10, 11]
The list after removing duplicate indices : [1, 7, 6, 9, 10]

Method 3: Using a dictionary to store the index of each element in the first list

Step-by-step algorithm:

1. Initialize two lists, test_list1 and test_list2.
2. Create a dictionary index_dict to store the index of each element in test_list1.
3. Initialize a list res to store the filtered elements from test_list2.
4. Loop through each element x and its index i in test_list2.
5. Check if the index i is not present in the list of indexes of duplicates in test_list1.
6. If i is not present, then add the element x to the list res.
7. Print the list res.

test_list1 = [3, 5, 6, 5, 3, 7, 8, 6]
test_list2 = [1, 7, 6, 3, 7, 9, 10, 11]

# create a dictionary to store the index of each element in test_list1
index_dict = {val: i for i, val in enumerate(test_list1)}

# remove elements from test_list2 that have the same index as a duplicate in test_list1
res = [x for i, x in enumerate(test_list2) if i not in [index_dict[val] for val in test_list1 if test_list1.count(val) > 1]]
print(res)

[1, 7, 6, 9, 10]

Time complexity:

The time complexity of the algorithm is O(n^2), where n is the length of test_list1. This is because the list comprehension [index_dict[val] for val in test_list1 if test_list1.count(val) > 1] has a time complexity of O(n^2) due to the count() function, and this comprehension is executed for each element in test_list2.
Auxiliary space:

The auxiliary space complexity of the algorithm is O(n), where n is the length of test_list1. This is because the dictionary index_dict has a space complexity of O(n), and the list res can have a maximum of n elements.

Method 4 :  using the set() and difference() method

Initialize two lists test_list1 and test_list2 with integer values.
Print the original lists using the print() function and str() conversion.
Use the set() function to get the unique values from test_list1.
Use the range() function to generate a sequence of integers that correspond to the indices of test_list2.
Use list comprehension to create a set of indices to be removed from test_list2. This is done by checking whether an element in test_list1 is present in unique_values, and whether its index is equal to the current index of the list comprehension (idx). If both conditions are met, the index idx is added to the set of indices to remove.
Use the set difference operation (-) to obtain the indices of elements that are not to be removed from test_list2.
Create a new list final_list using list comprehension. The elements of test_list2 are selected by their indices, but only if the index is not in the set of indices to remove.
Print the final list using the print() function and str() conversion.

# Initializing lists
test_list1 = [3, 5, 6, 5, 3, 7, 8, 6]
test_list2 = [1, 7, 6, 3, 7, 9, 10, 11]

# Printing original lists
print("The original list 1 is : " + str(test_list1))
print("The original list 2 is : " + str(test_list2))

# Using set() and difference() method to remove duplicate elements index from other list
unique_values = set(test_list1)
indices_to_remove = set(range(len(test_list2))) - set(idx for idx, val in enumerate(test_list1) if val in unique_values and test_list1.index(val) == idx)
final_list = [test_list2[i] for i in range(len(test_list2)) if i not in indices_to_remove]

# Printing result
print("The list after removing duplicate indices : " + str(final_list))

The original list 1 is : [3, 5, 6, 5, 3, 7, 8, 6]
The original list 2 is : [1, 7, 6, 3, 7, 9, 10, 11]
The list after removing duplicate indices : [1, 7, 6, 9, 10]

Time complexity: O(n) + O(n) + O(n) = O(n), where n is the length of the input lists.

Auxiliary space: O(n) + O(n) = O(n), where n is the length of the input lists.

Method 6: Using the Counter() function from the collections module

Step-by-Step approach:

Import the Counter() function from the collections module.
Initialize two lists, test_list1 and test_list2.
Use Counter() to count the occurrences of each element in test_list1.
Iterate over test_list2 and use the count of each element in test_list1 to determine if it has a duplicate index.
Add any element with a non-duplicate index to final_list.
Print the resulting final_list.

# Initializing lists
test_list1 = [3, 5, 6, 5, 3, 7, 8, 6]
test_list2 = [1, 7, 6, 3, 7, 9, 10, 11]

# Printing original lists
print("The original list 1 is : " + str(test_list1))
print("The original list 2 is : " + str(test_list2))

# Using set() and difference() method to remove duplicate elements index from other list
unique_values = set(test_list1)
indices_to_remove = set(range(len(test_list2))) - set(idx for idx, val in enumerate(test_list1) if val in unique_values and test_list1.index(val) == idx)
final_list = [test_list2[i] for i in range(len(test_list2)) if i not in indices_to_remove]

# Printing result
print("The list after removing duplicate indices : " + str(final_list))

The original list 1 is : [3, 5, 6, 5, 3, 7, 8, 6]
The original list 2 is : [1, 7, 6, 3, 7, 9, 10, 11]
The list after removing duplicate indices : [1, 7, 6, 9, 10]

Time complexity: O(n) where n is the length of the longer list (either test_list1 or test_list2).
Auxiliary space: O(n) where n is the length of test_list1, to store the counts of each element.

manjeet_04

Follow

Improve

Article Tags :

• Python
• Python Programs
• Python list-programs

Practice Tags :

• python