This post will be related to the use of grep, sed, and awk to process text streams and files.
GREP
Is a command searches a file for lines containing a match to the given strings and prints the matching lines.
It is mainly used in the following combinations:
grep 'some string' given_file cat given_file | grep 'some string' command | grep 'some string'
Example:
We want to see the line containing our ip address, so we run ifconfig and look for inetsting:
[email protected]:~# ifconfig eth0 | grep inet inet addr:10.1.0.1 Bcast:10.1.255.255 Mask:255.255.0.0 [email protected]
Example:
We would like to see if there is a user named cristian on our system. We list the content of the file and then search for cristian in /etc/passwd:
[email protected]:~# cat /etc/passwd | grep cristian cristian:x:1000:1000:Cristian Huza,,,:/home/cristian:/bin/bash [email protected]:~#
Example:
Let’s say that we have a file named students.txt and we want to see if we have a student named Anne.
This is the content of the file:
- John
- Danny
- Mary
- Bill
- Anne
Now, we will grep for “anne”:
[email protected]:~# grep anne students.txt [email protected]:~#
We have no result, but why? well, we searched for anne, but all the names in the file begin with a capital letter. We will have to make the search case insensitive using -i parameter:
[email protected]:~# grep -i anne students.txt Anne [email protected]:~#
One other interesting option is -v. It inverses the match, so if you search for John in a certain text file, it will provide you all the lines that DO NOT contain John:
[email protected]:~# grep -v John students.txt Danny Mary Bill Anne [email protected]:~#
SED
Is defined as a stream editor. Actually it is mostly used to replace, substitute, delete a certain string.
Example:
[email protected]:~# echo “John has three apples” | sed s/John/Anne/
Anne has three apples
[email protected]:~#
Now, if we have a certain string that appears multiple times in a stream, we will have to use the global replacement parameter g, otherwise only the first occurrence will be replaced:
[email protected]:~# echo "John has three apples. John has 5 candies" | sed s/John/Anne/ Anne has three apples. John has 5 candies [email protected]:~# echo "John has three apples. John has 5 candies" | sed s/John/Anne/g Anne has three apples. Anne has 5 candies [email protected]:~#
You can also use -i parameter to make the substitution inline for example we can replace John with Mike in a certain text file:
[email protected]:~# cat students.txt John Danny Mary Bill Anne [email protected]:~# sed -i 's/John/Mike/' students.txt [email protected]:~# cat students.txt Mike Danny Mary Bill Anne [email protected]:~#
For more options please check the man page.
AWK
Is defined as a pattern scanning and text processing language and it is used in combination with grep and cut or in standalone scripts.
We would like to print only the Month and the year of the current date for example:
[email protected]:~# date Mon Aug 1 23:07:09 EEST 2011 [email protected]:~# date | awk '{print $2 " " $6}' Aug 2011 [email protected]:~#
Or, to print the line number and then the actual line:
[email protected]:~# awk '{printf("%5d : %s\n", NR,$0)}' students.txt 1 : Mike 2 : Danny 3 : Mary 4 : Bill 5 : Anne [email protected]:~#
For more information check out the man page for awk/gawk.