Monthly Archives: December 2014

Use awk to print the last N columns of a file or a pipe

 in Bash  tagged awk / column / new line / newline / print by

In this post we will describe a way to print the last N number of columns in awk.

We will use this code as example, where we will print the last 2 columns only:


awk '{n = 2; for (--n; n >= 0; n--){ printf "%s\t",$(NF-n)} print ""}';
'

In the awk script we use the variable n to control how many columns we want to print. In the above example we initialized it  to the value 2 as that is the number of columns we want printed.

After, we use a for loop to iterate over the fields (in this case the last two fields) and we print them to the screen using printf "%s\t",$(NF-n) to avoid printing the new line character and to separate them with a tab character.

NF is a special variable in awk that holds the total number of fields available on that line. If you do not change the delimiter, then it will hold the number of words on the line.

$(NF-n) is the way we ask awk to gives us the variable value that is n places before the last.

Outside the loop we print "" to print the new line character between input rows.

Examples:

If we want to print the last two columns of the ls -l command we can do it as follows:


ls -l | awk '{i = 2; for (--i; i >= 0; i--){ printf "%s\t",$(NF-i)} print ""}';

If we want to print the last two columns of the /etc/passwd file we can do it as follows:


awk -F ':' '{i = 2; for (--i; i >= 0; i--){ printf "%s\t",$(NF-i)} print ""}' /etc/passwd;

Note that we change the delimiter with the command line argument -F ":"

Pull all Git repositories you have access to

 in Bash  tagged cut / git / gitolite / pull / repository / tail / xargs by 

ssh [email protected] info | cut -f 2 | tail -n +3 | xargs -I {} -n 1 -I_repository -- sh -c 'cd _repository; git pull; cd ..;'

The above command will connect to the git server (git.bytefreaks.net) using  gitolite and get a list of all the repositories you have access to using ssh [email protected] info

The command should return a list similar to this:

hello bytefreaks, this is git@git running gitolite3 v3.5.3.1-1-gf8776f5 on git 1.7.1

 R W	Repo1
 R W	Repo2
 R W	Repo3
 R  	Repo4

From the results, we remove the first 3 lines as they contain no useful information to cloning all the repositories. From the rest of the lines, where each line contains the information for a repository we have access to, we keep the third column only as it is the one that holds the repository name as it is stored on the server.

Afterwards it will remove all columns except the second to filter the column with the repository names and will remove the first 3 lines to keep only the data we are interested in.

On the last stage of the pipe we have a list of the names of the repositories, using xargs, we assign each repository name to the _repository variable and using one result at a time, we navigate into the folder of the repository using cd and call the pull command.

Note: We assume that all repositories are in the current folder as children and each one is in a sub-folder of its own which is named as the repository is.