Find and count duplicate lines in files


Search and count text files for duplicate or multiple content

When editing text or configuration files in the Linux shell, there can often be the requirement that identical entries in the files occur only once, so that to check how many times a line was duplicated, especially in files with a larger number of lines, this does not have to be done manually, help provide the use of the filters sort and uniq in the Linux bash.

This command counts duplicate lines and sorts the output in the bash.

$ sort FILE | uniq --count

  Replace the FILE placeholder with the real file name.

Only duplicate lines should be shown.

$ sort FILE | uniq --count --repeated

Nothing is displayed if there are no duplicate lines in the file.

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Leave a Reply

Your email address will not be published. Required fields are marked *