Login

Awk utility in Fedora

<div>Fedora provides awk as part of its default installation, including all its editions, including the immutable ones like Silverblue. But you may be asking, what is awk and why would you need it?
Awk is a data driven programming language that acts when it matches a pattern. On Fedora, and most other distributions, GNU awk or gawk is used. Read on for more about this language and how to use it.
 
<h2>A brief history of awk</h2>
Awk began at Bell Labs in 1977. Its name is an acronym from the initials of the designers: Alfred V. Aho, Peter J. Weinberger, and Brian W. Kernighan. 
<blockquote class="wp-block-quote">
The specification for awk in the POSIX Command Language and Utilities standard further clarified the language. Both the gawk designers and the original awk designers at Bell Laboratories provided feedback for the POSIX specification. 
<cite>From <a href="https://www.gnu.org/software/gawk/manual/gawk.html#Foreword3">The GNU Awk User’s Guide</a></cite></blockquote>
For a more in-depth look at how awk/gawk ended up being as powerful and useful as it is, follow the link above. Numerous individuals have contributed to the current state of gawk. Among those are:
<ul>
<li>Arnold Robbins and David Trueman, the creators of gawk</li>
<li>Michael Brennan, the creator of mawk, which later was merged with gawk</li>
<li>Jurgen Kahrs, who added networking capabilities to gawk in 1997</li>
<li>John Hague, who rewrote the gawk internals and added an awk-level debugger in 2011</li>
</ul>
<h2>Using awk</h2>
The following sections show various ways of using awk in Fedora.
<h3>At the command line</h3>
The simples way to invoke awk is at the command line. You can search a text file for a particular pattern, and if found, print out the line(s) of the file that match the pattern anywhere. As an example, use cat to take a look at the command history file in your home director:
<pre class="wp-block-preformatted">$ cat ~/.bash_history</pre>
 There are probably many lines scrolling by right now. 
Awk helps with this type of file quite easily. Instead of printing the entire file out to the terminal like cat, you can use awk to find something of specific interest. For this example, type the following at the command line if you’re running a standard Fedora edition:
<pre class="wp-block-preformatted">$ awk '/dnf/' ~/.bash_history </pre>
 If you’re running Silverblue, try this instead:
<pre class="wp-block-preformatted">$ awk '/rpm-ostree/' ~/.bash_history</pre>
In both cases, more data likely appears than what you really want. That’s no problem for awk since it can accept regular expressions. Using the previous example, you can change the pattern to more closely match search requirements of wanting to know about installs only. Try changing the search pattern to one of these:
<pre class="wp-block-preformatted">$ awk '/rpm-ostree install/' ~/.bash_history $ awk '/dnf isntall/' ~/.bash_history </pre>
All the entries of your bash command line history appear that have the pattern specified at any position along the line. Awk works on one line of a data file at a time. It matches pattern, then performs an action, then moves to next line until the end of file (EOF) is reached. 
<h3>From an awk program</h3>
Using awk at the command line as above is not much different than piping output to grep, like this:
<pre class="wp-block-preformatted">$ cat .bash_history | grep 'dnf install'</pre>
The end result of printing to standard output (stdout) is the same with both methods. 
Awk is a programming language, and the command awk is an interpreter of that language. The real power and flexibility of awk is you can make programs with it, and combine them with shell scripts to create even more powerful programs. For more feature rich development with awk, you can also incorporate C or C++ code using <a href="https://www.gnu.org/software/gawk/manual/gawk.html#Dynamic-Extensions">Dynamic-Extensions</a>. 
Next, to show the power of awk, let’s make a couple of program files to print the header and draw five numbers for the first row of a bingo card. To do this we’ll create two awk program files. 
The first file prints out the header of the bingo card. For this example it is called bingo-title.awk. Use your favorite editor to save this text as that file name: 
<pre class="wp-block-code"> <div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px"><div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">BEGIN { 
    print "B\tI\tN\tG\tO" 
}</div></div> </pre>
Now the title program is ready. You could try it out with this command:
<pre class="wp-block-preformatted">$ awk -f bingo-title.awk</pre>
The program prints the word BINGO, with a tab space (\t) between the characters. For the number selection, let’s use one of awk’s builtin numeric functions called rand() and use two of the control statements, for and switch. (Except the editor changed my program, so no switch statement used this time).
The title of the second awk program is bingo-num.awk. Enter the following into your favorite editor and save with that file name:
<pre class="wp-block-code"> <div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px"><div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">@include "bingo-title.awk" 
BEGIN { 
    for (i = 1; i < = 5; i++) { 
    b = int(rand() * 15) + (15*(i-1)) 
    printf "%s\t", b 
    } 
    print 
}</div></div> </code></pre>
The @include statement in the file tells the interpreter to process the included file first. In this case the interpreter processs the bingo-title.awk file so the title prints out first.
<h3>Running the test program</h3>
Now enter the command to pick a row of bingo numbers:
<pre class="wp-block-preformatted">$ awk -f bingo-num.awk</pre>
Output appears similar to the following. Note that the rand() function in awk is not ideal for truly random numbers. It’s used here only as for example purposes.
<pre class="wp-block-code"> <div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px"><div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$ awk -f bingo-num.awk 
B   I   N   G   O 
13  23  34  53  71</div></div> </pre>
In the example, we created two programs with only beginning sections that used actions to manipulate data generated from within the awk program. In order to satisfy the rules of Bingo, more work is needed to achieve the desirable results. The reader is encouraged to fix the programs so they can reliably pick bingo numbers, maybe look at the awk function srand() for answers on how that could be done.
<h2>Final examples</h2>
Awk can be useful even for mundane daily search tasks that you encounter, like listing all flatpak’s on the Flathub repository from org.gnome (providing you have the Flathub repository setup). The command to do that would be:
<pre class="wp-block-preformatted">$ flatpak remote-ls flathub --system | awk /org.gnome/</pre>
A listing appears that shows all output from remote-ls that matches the org.gnome pattern. To see flatpaks already installed from org.gnome, enter this command:
<pre class="wp-block-preformatted">$ flatpak list --system | awk /org.gnome/</pre>
Awk is a powerful and flexible programming language that fills a niche with text file manipulation exceedingly well. 
</div>

xSicKxBot