How to Remove List Duplicates in Python

Removing duplicates from a list is a common task in Python. There are several ways to accomplish this, depending on whether you want to maintain the original order of the list or not.

Using a Set to Remove Duplicates

One of the simplest ways to remove duplicates is to convert the list to a set, which automatically removes duplicates. However, this method does not preserve the order of elements.

Python 
my_list = [1, 2, 2, 3, 4, 4, 5]
unique_list = list(set(my_list))
print(unique_list)  # Output may be unordered

Preserving Order While Removing Duplicates

If you need to preserve the order of elements, you can use a loop or a dictionary (since Python 3.7, dictionaries maintain insertion order).

Python 
my_list = [1, 2, 2, 3, 4, 4, 5]
unique_list = list(dict.fromkeys(my_list))
print(unique_list)  # Output: [1, 2, 3, 4, 5]

Using List Comprehension

You can also use list comprehension to remove duplicates while maintaining the order:

Python 
my_list = [1, 2, 2, 3, 4, 4, 5]
unique_list = []
[unique_list.append(x) for x in my_list if x not in unique_list]
print(unique_list)  # Output: [1, 2, 3, 4, 5]

Each of these methods provides a different approach to removing duplicates, depending on your specific needs.

Import Links

Here are some useful import links for further reading: