Deduplication keeps the first occurrence of each distinct line and discards every later repeat. It is probably the single most-used list operation in marketing work, because almost every list assembled from more than one source contains repeats.
What counts as a duplicate here
Exact matching: the entire line must be character-for-character identical. Running Shoes and running shoes are two distinct lines under this rule.
Why exact matching is the right default
It is predictable. You know precisely what was removed and why. Looser matching catches more but occasionally removes something you wanted, and that failure is silent.
When to reach for something looser
If capitalisation is inconsistent across your sources, case-insensitive deduplication catches what exact matching misses. If word order varies, sort the words within each line first.
Try it: Remove Duplicates (Exact) on SeoWolf's Notepad.