How to read from two input files using while loop

Question

I wanted to know if there is any way of reading from two input files in a nested while loop one line at a time. For example, lets say I have two files FileA and FileB.

FileA:

[jaypal:~/Temp] cat filea this is File A line1 this is File A line2 this is File A line3 

FileB:

[jaypal:~/Temp] cat fileb this is File B line1 this is File B line2 this is File B line3 

Current Sample Script:

[jaypal:~/Temp] cat read.sh #!/bin/bash while read lineA do echo $lineA while read lineB do echo $lineB done < fileb done < filea 

Execution:

[jaypal:~/Temp] ./read.sh this is File A line1 this is File B line1 this is File B line2 this is File B line3 this is File A line2 this is File B line1 this is File B line2 this is File B line3 this is File A line3 this is File B line1 this is File B line2 this is File B line3 

Problem and desired output:

This loops over FileB completely for each line in FileA. I tried using continue, break, exit but none of them are meant for achieving the output I am looking for. I would like the script to read just one line from File A and then one line from FileB and exit the loop and continue with second line of File A and second line of File B. Something similar to the following script -

[jaypal:~/Temp] cat read1.sh #!/bin/bash count=1 while read lineA do echo $lineA lineB=`sed -n "$count"p fileb` echo $lineB count=`expr $count + 1` done < filea [jaypal:~/Temp] ./read1.sh this is File A line1 this is File B line1 this is File A line2 this is File B line2 this is File A line3 this is File B line3 

Is this possible to achieve with while loop?

Answer 1

If you can guarantee that some character will never occur in the first file then you can use paste.

For example you know for sure that @ will never occur:

paste -d@ file1 file2 | while IFS="@" read -r f1 f2 do printf 'f1: %s\n' "$f1" printf 'f2: %s\n' "$f2" done 

Note that it is enough if the character is guaranteed to not occur in the first file. This is because read will ignore IFS when filling the last variable. So even if @ occurs in the second file it will not be split.

Example using some bash features for arguably cleaner code and paste using default delimiter tab:

while IFS=$'\t' read -r f1 f2 do printf 'f1: %s\n' "$f1" printf 'f2: %s\n' "$f2" done < <(paste file1 file2) 

Bash features used: ansi c string ($'\t') and process substitution (<(...)) to avoid the while loop in a subshell problem.

If you cannot be certain that any character will never occur in both files then you can use two file descriptors.

while true do read -r f1 <&3 || break read -r f2 <&4 || break printf 'f1: %s\n' "$f1" printf 'f2: %s\n' "$f2" done 3<file1 4<file2 

Not tested much. Might break on empty lines.

File descriptors number 0, 1, and 2 are already used for stdin, stdout, and stderr, respectively. File descriptors from 3 and up are (usually) free. The bash manual warns from using file descriptors greater than 9, because they are "used internally".

Note that open file descriptors are inherited to shell functions and external programs. Functions and programs inheriting an open file descriptor can read from (and write to) the file descriptor. You should take care to close all file descriptors which are not required before calling a function or external program.

Here is the same program as above with the actual work (the printing) separated from the meta-work (reading line by line from two files in parallel).

work() { printf 'f1: %s\n' "$1" printf 'f2: %s\n' "$2" } while true do read -r f1 <&3 || break read -r f2 <&4 || break work "$f1" "$f2" done 3<file1 4<file2 

Now we pretend that we have no control over the work code and that code, for whatever reason, tries to read from file descriptor 3.

unknowncode() { printf 'f1: %s\n' "$1" printf 'f2: %s\n' "$2" read -r yoink <&3 && printf 'yoink: %s\n' "$yoink" } while true do read -r f1 <&3 || break read -r f2 <&4 || break unknowncode "$f1" "$f2" done 3<file1 4<file2 

Here is an example output. Note that the second line from the first file is "stolen" from the loop.

f1: file1 line1 f2: file2 line1 yoink: file1 line2 f1: file1 line3 f2: file2 line2 

Here is how you should close the file descriptors before calling external code (or any code for that matter).

while true do read -r f1 <&3 || break read -r f2 <&4 || break # this will close fd3 and fd4 before executing anycode anycode "$f1" "$f2" 3<&- 4<&- # note that fd3 and fd4 are still open in the loop done 3<file1 4<file2 

Answer 2

Open the two files on different file descriptors. Redirect the input of the read built-in to the descriptor that the file you want is connected to. In bash/ksh/zsh, you can write read -u 3 instead of read <&3.

while IFS= read -r lineA && IFS= read -r lineB <&3; do echo "$lineA"; echo "$lineB" done <fileA 3<fileB 

This snippet stops when the shortest file has been processed. See Reading two files into an IFS while loop -- Is there a way to get a zero diff result in this case? if you want to keep processing until the end of both files.

See also When would you use an additional file descriptor? for additional information on file descriptors, and Why is `while IFS= read` used so often, instead of `IFS=; while read..`? for an explanation of IFS= read -r.

Answer 3

Try the command below:

paste -d '\n' inp1.txt inp2.txt > outfile.txt 

Answer 4

I know you want a shell script, but you might want to take a look at the paste command.

Answer 5

Alternatively, I suppose you could slurp the file into an array variable tying each line of the file into array[line_of_file_index] using bash's mapfile command. However, I am not sure if it is only for Bash3 higher or Bash4.

http://wiki.bash-hackers.org/commands/builtin/mapfile