sort -u deduplicates, and it also reorders the list. awk '!seen[$0]++' deduplicates while preserving the original order. Which you want is a real decision, not a stylistic one.
Why order often matters
Keyword exports arrive ranked by volume. Rank order is information. Running sort -u over that list destroys it and hands back an alphabetised list where the most important terms are scattered — technically deduplicated, practically less useful than before.
The memory profile differs
awk '!seen[$0]++' holds every distinct line in a hash map, so peak memory scales with the number of unique lines. For millions of long lines that becomes real. sort spills to temporary files on disk and handles inputs larger than memory, at the cost of the ordering.
Hashing for very large inputs
Where lines are long and numerous, storing a hash of each line rather than the line itself cuts memory substantially. It introduces a theoretical collision risk, which with a 64-bit hash is negligible for any list a marketer will ever produce.
Try it: Remove Duplicates (Exact) on SeoWolf's Notepad.