How to Delete Empty Lines in Files Using Grep, Sed, and Awk

An experienced Linux user knows exactly what kind of a nuisance blank lines can be in a processable file. These empty/blank lines not only get in the way of correctly processing such files but also make it difficult for a running program to read and write the file.

On a Linux operating system environment, it is possible to implement several text manipulation expressions to get rid of these empty/blank lines from a file. In this article, empty/blank lines refer to the whitespace characters.

Create a File with Empty/Blank Lines in Linux

We need to create a reference file with some empty/blank lines. We will later amend it in the article through several techniques we will be discussing. From your terminal, create a text file of your choice with a name like “i_have_blanks” and populate it with some data and some blank spaces.

$ nano i_have_blanks.txt
Or
$ vi i_have_blanks.txt

Throughout the article, we will be outputting the contents of a file on our terminal using the cat command for flexible referencing.

$ cat i_have_blanks.txt

The three Linux commands that will propel us towards an ideal solution to this empty/blank lines problem are grep, sed, and awk.

Therefore, create three copies of your i_have_blanks.txt file and save them with different names so that each can be accommodated by one of the stated three Linux commands.

Through regex (regular expressions), we can identify blank lines with the POSIX standard character “[:space:]”.

How to Remove Blank/Empty Lines in Files

With this problem statement, we are considering the elimination of all existing empty/blank lines from a given readable file using the following commands.

1. Delete Empty Lines Using Grep Command

Supported use of shorthand character classes can reduce the grep command to a simple one like:

$ grep -v '^[[:space:]]*$' i_have_blanks.txt
OR
$ grep '\S' i_have_blanks.txt

To fix a file with blank/empty lines, the above output needs to pass through a temp file before overwriting the original file.

$ grep '\S' i_have_blanks.txt > tmp.txt 
$ mv tmp.txt i_have_blanks.txt
$ cat i_have_blanks.txt

As you can see, all the blank lines that spaced the content of this text file are gone.

2. Delete Empty Lines Using Sed Command

The d action in the command tells it to delete any existing whitespace from a file. This command’s blank line matching and deleting mechanism can be represented in the following manner.

$ sed '/^[[:space:]]*$/d' i_have_blanks_too.txt

The above command scans through the text file lines for non-blank characters and deletes all the other remaining characters. Through its non-blank character class support, the above command can be simplified to the following:

$ sed '/\S/!d' i_have_blanks_too.txt

Also, because of the command’s in-place editing support, we don’t need a temp file to temporarily hold our converted file before overwriting the original text file like with the case of the grep command. You however need to use this command with an -i option as an argument.

$ sed -i '/\S/!d' i_have_blanks_too.txt i_have_blanks_too.txt 
$ cat i_have_blanks_too.txt

3. Delete Empty Lines Using Awk Command

The awk command runs a non-white character check on each line of a file and only prints them if this condition is true. The flexibility of this command comes with various implementation routes. Its straightforward solution is as follows:

$ awk '!/^[[:space:]]*$/' i_have_blanks_first.txt

The interpretation of the above command is straightforward, only the file lines that do not exist as whitespaces are printed. The longer version of the above command will look similar to the following:

$ awk '{ if($0 !~ /^[[:space:]]*$/) }' i_have_blanks_first.txt

Through awk non-blank character class support, the above command can also be represented in the following manner:

$ awk -d '/\S/' i_have_blanks_first.txt

The -d an option lets awk dump the final file lines on the system terminal. As you can see, the file no longer has whitespaces.

The three discussed and implemented solutions to dealing with blank lines from files through grep, sed, and awk commands will take us a long way into implementing stable and efficient read and write file operations on a Linux system.