dedupe

sknano.core.itertools.dedupe(items, key=None)

Remove duplicate values in a sequence, but preserve order of remaining items.

Parameters:

• items (sequence) –
• key ({None, function}, optional) – function that converts sequence items into a hashable type for the purposes of duplicate detection.

Returns:

items

Return type:

set

Examples

>>> a = [{'x': 1, 'y': 2}, {'x': 1, 'y': 3},
...      {'x': 1, 'y': 2}, {'x': 2, 'y': 4}]
>>> list(dedupe(a, key=lambda d: (d['x'], d['y'])))
[{'x': 1, 'y': 2}, {'x': 1, 'y': 3}, {'x': 2, 'y': 4}]
>>> list(dedupe(a, key=lambda d: d['x']))
[{'x': 1, 'y': 2}, {'x': 2, 'y': 4}]