Yearly Archives: 2011

Get the first column of a file in bash

 in Bash / GNU/Linux / HowTos  tagged awk / bash / column / GNU/Linux / howtos by 
awk '{print $1}' someFile;

The command awk '{print $1}' someFile is a command that is used to extract specific data from a file in Unix/Linux systems. It uses the awk scripting language, which is a powerful tool for text processing and data manipulation.

The syntax of the command is as follows: awk ‘{print $1}’ someFile. Here, the awk command is followed by a set of instructions in single quotes. The instructions specify what to do with the data in the file someFile. In this case, the instruction is print $1 which means that awk will print the first field or column of each line in the file.

The $1 in the instruction refers to the first field of each line in the file. Fields in an awk file are separated by whitespace or any other specified delimiter. In this case, the default delimiter is whitespace, so each field is separated by a space or tab.

The someFile in the command is the name of the file that the awk command will process. The file can be any text file and can contain any type of data. The awk command will extract the first field of each line in the file and print it on the screen.

In conclusion, the command “awk ‘{print $1}’ someFile” is a powerful tool for extracting specific data from a file in Unix/Linux systems. The awk scripting language provides a flexible and efficient way to process text data and manipulate it to meet specific requirements.

*NOTE: If you want the second column change $1 to $2 etc. $1 can be replaced by a variable and used in a more elaborate way that applies to more cases/problems.

Kill all processes of a user (Or kill almost all using an exception list) in linux

 in Bash / GNU/Linux / HowTos  tagged awk / egrep / kill / ps / xargs by

Following is a command a root can use to stop all active threads of a user with an exception list (you can replace someApplication) with specific commands you wish to keep alive.

ps -U useraccount | egrep -v "someApplication|someCommand" | awk '{print $2}' | xargs -t kill;

In this command, the following actions are being taken:

  1. The ps Command: The first command in this line, ps, is a process status command. It shows the current status of all processes on the system. The -U option is used to specify the user account. So, in this case, the command will show the status of all processes that are running under the specified user account.
  2. The | Operator: The | symbol, also known as the pipe operator, is used to redirect the output of one command to the input of another command. So, in this case, the output of the ps command is being passed as the input to the next command in the line.
  3. The egrep Command: The next command in the line is egrep, which is a variation of the grep command. Grep is a tool used to search for patterns in files, and egrep is an extended version of it that supports regular expressions. The -v option is used to invert the match. So, the egrep command in this line is searching for all processes that do not contain the specified strings, “someApplication” or “someCommand”.
  4. The awk Command: The next command in the line is awk, which is a programming language used for text processing and data extraction. In this line, the awk command is being used to extract the second column of the output, which is the process ID. The {print $2} part of the command specifies that the second column should be printed.
  5. The xargs Command: The final command in the line is xargs, which is used to execute commands based on the input. In this case, the -t option is used to display the commands that are being executed. The xargs command is being used to execute the kill command for each process ID, which is passed as the input. The kill command is used to terminate a process.

In summary, this command is used to find all processes running under the specified user account, excluding the processes that contain the specified strings “someApplication” or “someCommand”. The process IDs are extracted and passed as the input to the ‘kill’ command, which terminates the processes. The ‘-t’ option is used to display the commands being executed by xargs.

The next example is very similar to the first one but it used to kill all of the processes of your account.

ps x | egrep -v "ssh|screen|ps|bash|awk|tail" | awk '{print $1}' | tail -n +2 | xargs -t kill;

NOTE: USE WITH CAUTION!

Resolve IPs for Servers listed in a file using /etc/hosts

 in Bash / GNU/Linux / HowTos  tagged /etc/hosts / awk / IP / xargs by 
cat $NODEFILE | xargs -L 1 -I xx grep xx /etc/hosts | awk '{print $1}';

The command above is a combination of several Linux commands that are used to extract specific information from the /etc/hosts file.

Here is a step-by-step explanation of what the command does:

  1. cat $NODEFILE: This command reads the contents of the file specified by the environment variable $NODEFILE. This file can contain a list of hostnames or IP addresses.
  2. | xargs -L 1 -I xx grep xx /etc/hosts: This command takes the output from the previous step and passes it as an argument to the grep command. The xargs command is used to execute a command for each line of the input. The -L 1 option specifies that only one line from the input should be used as an argument for each execution of the grep command. The -I xx option specifies that the placeholder xx should be used to represent each argument passed to the grep command. The grep command is then used to search for the specified hostnames or IP addresses in the /etc/hosts file.
  3. awk '{print $1}': This command takes the output from the previous step and uses the awk utility to extract specific columns of data. The '{print $1}' option specifies that the first column of data (which is the IP address in this case) should be printed.

The final output of this command will be a list of IP addresses that correspond to the hostnames or IP addresses specified in the $NODEFILE file and found in the /etc/hosts file.

In summary, the command is a pipeline of multiple commands that are used to extract specific information from a file. The combination of the cat, xargs, grep, and awk commands allows for powerful text processing and manipulation, and this kind of command is a common pattern used in many Linux shell scripts.

*NOTES:$NODEFILE contains a list of Hostnames that you want their IPs resolved.
xargs is used to get each Hostname and use on its own as a filter for the grep command that will parse the /etc/hosts file. In other words for each hostname the commands xx grep xx /etc/hosts | awk ‘{print $1}’ are issued. Also it is important to explain what xx is: xx is a variable name that we use, in order to show to the grep command where and how we want it to use the hostname that we got from the /etc/hosts file.
awk is removing all columns but the first where the IPs should be listed there.