Temp Files – Bash Scripting

If you use bash scripts on a regular basis for file manipulations you would have come across times where the output had to be written to some temporary files. From which further handling can be done. This is quite common. But surprisingly after years, I have come to know the use of temporary files that the system itself safely creates in the /tmp directory. You do not have to worry about the naming or the location or such. You can use the mktemp command to create a temporary file which can be assigned to a file handler.

Lets look at a very common use. You have some text ready after some processing. You want to write it to a file. What you do before hand is say,
temp_file=$(mktemp)

This will create a tempfile and assign it to the file handler temp_file. Then you will probably want to write some stuff to this file. you can do this by just sending the outputs like this,
echo "my output" > $temp_file

Just using echo here for explanation purposes. You can continue appending your outputs of course, using >>.

You can use this file for further processing in a for loop or something just like you would do any other file. Like for example,

for line in `cat $temp_file`; do

This was one really useful tip because I do not have to worry about the file naming or location of the file or about removing the file later on.

Thanks for reading.

Advertisements

tr – First Use

tr – Translate of Delete. This is the first time i am using this command and it was pretty nice. Used the gui repalce functionality on text editors so far and, sed recently. But this seems pretty nice.

tr 'abc' 'xyz' will replace the occurence of abc with xyz. Takes input from the standard input. Meaning if you run this command and type something in and press enter. You will see the results down below. Common use is to cat and pipeline the content though.

Sed – Quick Reference

Another personal reference

Two basic forms of use.
sed [-n] [-e] 'command(s)' files
sed [-n] -f scriptfile files

First one uses inline commands. The second one uses a script file. A command like delete or print can be executed based on an address/line range or a pattern. Lets look at the commands themselves in no particular order just to recollect things. All concepts are mixed and mashed.

  1. p stands for print and it prints the lines that are specified. ‘1p’ will print the first line.
  2. '1,5 p' will print the lines from one to five.
  3. '2, +3 p' will print line 2 and the next 3 lines as well.
  4. '2, ~4 p' will print line 2 and every fourth line from that. Meaning, 2, 6, 10, 14, 18, 22…
  5. Address and Pattern matching can be used together.'/hello/, 5 p' looks for the line that has ‘hello’ and then prints five lines that follow it.
  6. '/hello/, $ p' prints from the line that has the first occurence of hello till the end
  7. '/pattern1/ /pattern2/ p' prints all the lines that contains pattern 1 upto the line that contains pattern 2.
  8. d is for deleting a line. Similar to p, for printing.
  9. w can be used to write to a file after filtering some lines. Or with no filters at all, which would make it work like a cp command.
  10. 'w newfile' will make a copy of the target file into the new file
  11. '2,5 w newfile' will just write the lines 2 to 5 of the target file to the new file. Pattern matching can also be used instead of Address matching.
  12. a can be used to append a line to a file.
  13. '4 a text to be appended' appends the text to be appended to the file after line 4.
  14. c can be used for replacing or changing patterns or lines.
  15. '4 c text line to be replaced' will replace the fourth line with the text line to be replaced. Multiple lines can be replaced with single lines as well.
  16. i stands for insert.
  17. y for translate.
  18. Syntax for y [address1[,address2]]y/list-1/list-2/.
  19. [address1[,address2]]l shows hidden characters
  20. [address]q [value] for quit
  21. [address]r file read and insert contents of file into address of the target file specified
  22. [address1[,address2]]e [command] execute bash command on the address
  23. [address1[,address2]]s/pattern/replacement/[flags] substitute command
  24. 's/Paulo Coelho/PAULO COELHO/w junk.txt' replace and write to new file

More to be added to this post.

xargs – possible uses

Xargs is one of those commands I have never used. Recently I was looking into how I can use them in the scripts I use on a daily basis.

Here are some. These are for personal references. If you find it useful. Please go ahead and use them.

  1. xargs, was actually used to get over the limitations of the number of arguments that a certain command can handle. But then it has been used for other purposes as well now I guess.
  2. ls somefilepattern | xargs can convert the list that would have been displayed vertically into a list that is displayed on a single line seperated by spaces, since it is actually converting it to an argument list for some command. This can be a really useful oneliner.
  3. When you want it to act as an argument list for some command then ls somefilepattern | xargs command will do. What this is doing is that it is taking a list of those files, converting them into an argument list and passing it to the command. Good!
  4. A good derivation of the last two points is that it can be used with commands like the cut command to get some vertical list, which can then be passed on to a command or just convert it to a oneline list. Ex: ls * | cut -d "." -f1 | xargs

They say it is mostly used with find command but then I cannot relate to those uses.

Loop the way you want it – Bash – IFS

You have a text fine containing several lines of text. You want to loop through each line so write a for loop in your bash script saying,

for lines in `cat somefile`;do
#do something
done

But wait a minute, your code is now processing word by word and not line by line. Whats happening? It is basically using space as the separator by default. So what do you have to do now. Use line break(\n) as your separator. How do you do that, IFS. IFS stands for Internal Field Separator.

So now if I add the line,
IFS=$'\n'
to my script before my for loop. Bash would now start looping through the lines rather than words.