Get all Unique Keys from a List of Dictionaries - Python

Our task is to get all unique keys from a list of dictionaries and we are given a list where each element is a dictionary, we need to extract and return a list of keys that appear across all dictionaries. The result should contain each key only once regardless of how many times it appears. For example, if we have the following list of dictionaries: a = [{'my': 1, 'name': 2}, {'is': 1, 'my': 3}, {'ria': 2}] then the unique keys in the list would be: ['ria', 'my', 'is', 'name'].

Using a loop and update()

We can iterate through each dictionary in the list and add the keys to a set and since a set automatically handles uniqueness we don’t need to worry about duplicate keys.

a = [{'my': 1, 'name': 2}, {'is': 1, 'my': 3}, {'ria': 2}]

s = set()
for d in a:
    s.update(d.keys())

print(list(s)) 

['name', 'is', 'ria', 'my']

Explanation:

• We initialize an empty set s then for each dictionary d in the list "a" we update the set with the dictionary's keys using s.update(d.keys()).
• Since sets only store unique values hence any duplicate keys are automatically discarded.

Using itertools.chain()

We can use itertools.chain() to chain together the keys of all dictionaries in the list and convert them into a set to ensure uniqueness.

from itertools import chain

a = [{'my': 1, 'name': 2}, {'is': 1, 'my': 3}, {'ria': 2}]

s = set(chain.from_iterable(d.keys() for d in a))

print(list(s)) 

['my', 'is', 'name', 'ria']

Explanation:

• We use itertools.chain() to chain the keys from all dictionaries into a single iterable.
• from_iterable() allows us to flatten the iterable of key lists into a single sequence and we then convert the result into a set to ensure all keys are unique.

Using reduce()

We can also use reduce() from the functools module to combine all the dictionaries and extract the unique keys.

from functools import reduce

a = [{'my': 1, 'name': 2}, {'is': 1, 'my': 3}, {'ria': 2}]

s = reduce(lambda acc, d: acc.union(d.keys()), a, set())

print(list(s)) 

['is', 'name', 'ria', 'my']

Explanation:

• reduce() applies a function (in this case, acc.union(d.keys())) cumulatively to the items in "a".
• It starts with an empty set (set()) as the initial value (acc) and adds the keys from each dictionary.
• union() ensures that only unique keys are kept in the set.

Using collections.Counter

We can use collections.Counter to count the occurrence of each key across all dictionaries, the keys with any count will be considered unique as all keys are stored in the dictionary.

from collections import Counter

a = [{'my': 1, 'name': 2}, {'is': 1, 'my': 3}, {'ria': 2}]

s = set(Counter(k for d in a for k in d.keys()))

print(list(s)) 

['is', 'name', 'my', 'ria']

Explanation:

• Counter is used here to count the occurrences of keys across all dictionaries.
• The keys are generated by iterating through each dictionary and using k for d in arr for k in d.keys().