sknano.core.dedupe

sknano.core.dedupe(items, key=None)

Remove duplicate values in a sequence, but preserve order of remaining items.

Parameters:

items : sequence

key : {None, function}, optional

function that converts sequence items into a hashable type for the purposes of duplicate detection.

Returns:

items : set

Examples

>>> a = [{'x': 1, 'y': 2}, {'x': 1, 'y': 3},
...      {'x': 1, 'y': 2}, {'x': 2, 'y': 4}]
>>> list(dedupe(a, key=lambda d: (d['x'], d['y'])))
[{'x': 1, 'y': 2}, {'x': 1, 'y': 3}, {'x': 2, 'y': 4}]
>>> list(dedupe(a, key=lambda d: d['x']))
[{'x': 1, 'y': 2}, {'x': 2, 'y': 4}]