Bash bits, nibbles and bytes: Cut script inefficiency

One of the most common uses I use scripting for is parsing log files. You know, those incredibly verbose files that we should read to find out if we have been h4x0r3d or not. Unfortunately there is a lot of information in those logs that we are not interested in at the time. So we write scripts to filter out the required information.

A very good program for parsing out required fields is cut (read 'man cut') where you can "cut" out any required fields, character or bytes according to specified delimiters. However calling an external program brings in extra overhead which, in a loop processing hundreds of thousands of lines, slows down the script a lot. Yes I know there is perl, python, gawk, awk, mawk and probably even squawk but this is about bash Smiling

While the bash man page (man bash) is probably about one of the best examples of a completely written man page there is, and it is certainly comprehensive. It is sometimes not a very comprehensible, or easy to understand, page and there are some real gems of scripting information in there that are hard to find.

More Here