The awk command can make it easy to remove duplicate characters from a string even when those characters aren’t sequential, especially when the process is turned into a script. First, the awk command ...

I use awk all the time, but generally only to conveniently pull a particular field out of data that I’m workin with. Regardless of the separator used, awk makes it easy to extract just what you need.