There is information about the “awk” command in Linux-based operating systems, which is used to search within the file, list it, and print the content appropriately. Since everything is a file in Linux, it may take time to find and list the settings in the file. In this case, you can use the “awk” command, which is widely used to get the desired value in the file and list it. As the basic function, words are parsed according to the spaces in the line and take reference value according to their order. For example; first variable is “$1” , second variable is “$2” and n variable is “$n”

“awk” Example Usage

Let’s create a .txt file to show in the example. We will understand the use of the awk command by making examples on this file.

 

As can be seen in the example, we have very simple data. In this way, we can easily use the spaces in the line as a parser. When we evaluate the rows and columns according to the spaces, it will be First = $1, Last Name = $2, Age = $3 and City = $4.

Name Surname Age City Gender


Ömer Şıvka 38 Çroum M
Ömer Kara 20 Edirne M 
Hale Yilmaz 33 Kars W

Kemal Durmus 27 Antalya M

Fatma Kara 22 Mardin W

30 M
Hasan Ali 28 Samsun M

 

Example 1:

Our first example would be to display the $1 values in the name column, i.e. contact names.

┌──(kali㉿kali)-[~]
└─$ awk '{print $1}' axkexlorer.txt                                      4 ⚙
Name


Ömer
Ömer
Hale

Kemal

Fatma

30
Hasan

 

Example 2:

If we want to print the Name and Surname information together, it will be as follows.

 

┌──(kali㉿kali)-[~]
└─$ awk '{print $1 $2}' axkexlorer.txt                                   4 ⚙
NameSurname


ÖmerŞıvka
ÖmerKara
HaleYilmaz

KemalDurmus

FatmaKara

30M
HasanAli

 

When we apply the above code, you will see that the data is displayed without any spaces between First Name and Last Name values. To correct this situation, double quotes (” “) denoting the use of string expressions will be useful.

┌──(kali㉿kali)-[~]
└─$ awk '{print $1 " " $2}' axkexlorer.txt                               4 ⚙
Name Surname
 
 
Ömer Şıvka
Ömer Kara
Hale Yilmaz
 
Kemal Durmus
 
Fatma Kara
 
30 M
Hasan Ali

 

Example 3:

You can use the commands below to view all columns.

┌──(kali㉿kali)-[~]
└─$ awk '{print $0}' axkexlorer.txt                                      4 ⚙
Name Surname Age City Gender


Ömer Şıvka 38 Çroum M 
Ömer Kara 20 Edirne M
Hale Yilmaz 33 Kars W

Kemal Durmus 27 Antalya M

Fatma Kara 22 Mardin W

30 M
Hasan Ali 28 Samsun M
                                                                             
┌──(kali㉿kali)-[~]
└─$ awk '{print}' axkexlorer.txt                                         4 ⚙
Name Surname Age City Gender


Ömer Şıvka 38 Çroum M 
Ömer Kara 20 Edirne M
Hale Yilmaz 33 Kars W

Kemal Durmus 27 Antalya M

Fatma Kara 22 Mardin W

30 M
Hasan Ali 28 Samsun M

 

In Commands length, NR and NF

We will talk about length, NR and NF that you can use in commands. “length” will give the number of characters in the specified line, “NR” will give the number of lines, and “NF” will give the number of space-separated fields in the line. When we apply the code below, the output we will get will be as follows.

──(kali㉿kali)-[~]
└─$ awk '{print NF}' axkexlorer.txt                                      4 ⚙
5
0
0
5
5
5
0
5
0
5
0
2
5

 

Example 4:

Our data is under 13 columns (the first number indicates the headings). We know the headers of the columns and we want the data to return to us, except for the headers, when listing the data. Then we do row control (NR) with “if”. We use the (NR!=1) command so that the first line does not appear. In this way, we can ensure that the remaining ones are listed.

┌──(kali㉿kali)-[~]
└─$ awk '{ if (NR!=1) print}' axkexlorer.txt                             4 ⚙


Ömer Şıvka 38 Çroum M 
Ömer Kara 20 Edirne M
Hale Yilmaz 33 Kars W

Kemal Durmus 27 Antalya M

Fatma Kara 22 Mardin W

30 M
Hasan Ali 28 Samsun M

 

Example 5:

If we want to display only the rows with all fields filled in the data stack, you can use the following command.

┌──(kali㉿kali)-[~]
└─$ awk '{ if (NF==5) print}' axkexlorer.txt                             4 ⚙
Name Surname Age City Gender
Ömer Şıvka 38 Çroum M 
Ömer Kara 20 Edirne M
Hale Yilmaz 33 Kars W
Kemal Durmus 27 Antalya M
Fatma Kara 22 Mardin W
Hasan Ali 28 Samsun M

 

Filtering and Mathematical Operations Using “awk”

Let’s say we have a table consisting of 3 columns, for example, Continent, Country, Population. Let’s take a value in the 2nd column of this table (countries with a Country value of X) and get the Population sum of this value. The name of our file is awkworkbook.csv, ‘;’ in the table where the columns are separated as comma-separated Let’s specify the character we use for parsing with. We can search for the value X in the $2 column and the Country column and return the sum of the values with {sum += $3}.

 

Example 6:

Based on the code and table below, the result of the command will be “345533“.

┌──(kali㉿kali)-[~]
└─$ awk -F ';' '$2 ~ /X/ {sum += $3} END {print sum}' awkworkbook.csv    4 ⚙
345533
                                                                             
┌──(kali㉿kali)-[~]
└─$

 

Special Characters Using “awk”

“\n” is a horizontal new line. “\t” Tab character. “\v” Line in vertical. \” Double quotes.
Since double quotes are used as string identifiers, you can prevent the data from being treated as a string by putting \ in front of double-quotes. Apart from these special characters, which we have limited in relation to the examples, there are of course different characters that you can use.

Editing Content

Example 7:

The following code will change the column order (by swapping 5 and 4) and transfer the new output to the new.txt file. However, we have seen in the examples above that as a result of this process, the gap between the columns will also disappear. In this case, we can insert spaces with double quotes, commas to sort columns, or assign the existing order to a variable.

awk '{ print $1 $2 $3 $5 $4 }' ~/example.txt > new.txt
awk '{ print $1 " " $2 " " $3 " " $5 " " $4 }' ~/example.txt > new.txt
awk '{ print $1,$2,$3,$5,$4 }' ~/example.txt > new.txt
awk '{age = $3; $3 = ""; print $0, age; }' ~/example.txt > new.txt

Example 8:

After some simple changes to column-by-column sorting, we continue with cross-column adjustments. With the command below, we will both change the order of the columns and change the space between the columns with “—” and save this edited data as a new.txt document.

awk '"} {print--BEGIN {OFS="- $1,$2,$3,$5,$4}' ~/example.txt > new.txt

 

Example 9:

In the code below, the files in the current directory will be listed and the name and extension information of the files with JPG extension will be arranged as name and “*.doc”. That is, JPG files will be saved as DOC with the same name. -F” in the code.” Specifies that the column separator character will be a period (.) instead of a space. The next bash will have the output processed as a command.

ls | awk -F"." '/JPG/{print "mv "$1"."$2" "$1".doc"}' | bash

 

Example 10:

It’s helpful to use a backslash (\) to write long “awk” commands on a new line.

awk -F: \ '{print $1 " -> " $6}' /etc/passwd

 

Example 11:

In the last example, the BEGIN keyword is used to give information at the beginning of the command when processing with the “Awk” command.

awk -F: 'BEGIN {print "### Userlist ###"} {print $1}' /etc/passwd