Uniq command Linux Filter adjacent matching lines from INPUT

Uniq Command Linux is used to report or omit repeated lines. Uniq is useful and handy tool to remove duplicate data from N number of lines of file. Its not only used to remove duplicate data it also helps in shell scripting to make reports.

Options Uniq Command Linux

• -c,  Option will count number of occurrences
• -d,  Only print duplicate lines, one for each group
• -D, Print all duplicate lines groups can be delimited with an empty line METHOD={none(default),pretend,separate}
• -f,  Avoid comparing the first N fields
• -i, Ignore differences in case when comparing
• -s, Avoid comparing the first N characters
• –u, Only print unique lines
• -z, End lines with 0 byte, not newline
• -w, Compare no more than N characters in lines

Normal Uniq Command without options

[root@localhost Linux_Commands]# uniq testingdata

Test File data used for this article explanation
[root@localhost Linux_Commands]# cat testingdata

Uniq command without any options will print uniq lines

Count Recurrences using Uniq

If you have an data with multiple duplicates, you can even count the recurrences using Uniq command below is the best example

[root@localhost Linux_Commands]# uniq -c testingdata

[root@localhost Linux_Commands]# uniq -c testingdata |sort -nr

Print Only each line which has duplicates

Only print duplicate lines, one for each group

[root@localhost Linux_Commands]# uniq -d testingdata

Duplicate data group

Using Uniq command print group of duplicate lines using -D

[root@localhost Linux_Commands]# uniq --all-repeated=separate testingdata

[root@localhost Linux_Commands]# uniq -D testingdata

Uniq Lines Printing out of all

Simple method to separate Uniq lines (which lines does not have any duplicates) using -u option

[root@localhost Linux_Commands]# uniq -u testingdata

N Characters in lines

Compare no more than N characters specified using -w option

[root@localhost Linux_Commands]# uniq -c -w 2 testingdata

Avoid Comparing N Number of fields

-f option eliminate comparing N number of fileds

[root@localhost Linux_Commands]# uniq -f1 testingdata

[root@localhost Linux_Commands]# uniq -f2 testingdata

Uniq Command Linux Conclusion : 

Linux Command is used to find Uniq values out of number of duplicate lines. Count duplicate occurrences and print only files by specified numer.

Related Articles

25 Linux basic commands

sed command 20 practical examples

bc command in linux examples

netstat command Linux

100 Linux Commands Video Tutorial