dedupe

sknano.core.dedupe(items, key=None)[source] [edit on github][source]

Remove duplicate values in a sequence, but preserve order of remaining items.

Parameters:
  • items (sequence) –
  • key ({None, function}, optional) – function that converts sequence items into a hashable type for the purposes of duplicate detection.
Returns:

items

Return type:

set

Examples

>>> a = [{'x': 1, 'y': 2}, {'x': 1, 'y': 3},
...      {'x': 1, 'y': 2}, {'x': 2, 'y': 4}]
>>> list(dedupe(a, key=lambda d: (d['x'], d['y'])))
[{'x': 1, 'y': 2}, {'x': 1, 'y': 3}, {'x': 2, 'y': 4}]
>>> list(dedupe(a, key=lambda d: d['x']))
[{'x': 1, 'y': 2}, {'x': 2, 'y': 4}]