Bash script wait for processes and get return code

Question

I am trying to create a script which will start many background command. For each background command I need to get the return code.

I have been trying the following script :

 #!/bin/bash set -x pid=() return=() for i in 1 2 do echo start $i ssh mysql "/root/test$i.sh" & pid[$i]=$! done for i in ${#pid[@]} do echo ${pid[$i]} wait ${pid[$i]} return[$i]=$? if [ ${return[$i]} -ne 0 ] then echo mail error fi done echo ${return[1]} echo ${return[2]} 

My issue is during the wait loop, if the second pid finish before the first one, I'll not be able to get the return code.

I know that I can run wait pid1 pid2, but with this command I can't get the return code of all commands.

Any idea ?

Answer 1

The issue is more with your

for i in ${#pid[@]} 

Which is for i in 2.

It should rather be:

for i in 1 2 

or

for ((i = 1; i <= ${#pid[@]}; i++)) 

wait "$pid"will return the exit code of the job with bash (and POSIX shells, but not zsh) even if the job had already terminated when wait was started.

Answer 2

You can do this by using a temporary directory.

# Create a temporary directory to store the statuses dir=$(mktemp -d) # Execute the backgrouded code. Create a file that contains the exit status. # The filename is the PID of this group's subshell. for i in 1 2; do { ssh mysql "/root/test$i.sh" ; echo "$?" > "$dir/$BASHPID" ; } & done # Wait for all jobs to complete wait # Get return information for each pid for file in "$dir"/*; do printf 'PID %d returned %d\n' "${file##*/}" "$(<"$file")" done # Remove the temporary directory rm -r "$dir" 

Answer 3

A generic implementation without temporary files.

#!/usr/bin/env bash ## associative array for job status declare -A JOBS ## run command in the background background() { eval $1 & JOBS[$!]="$1" } ## check exit status of each job ## preserve exit status in ${JOBS} ## returns 1 if any job failed reap() { local cmd local status=0 for pid in ${!JOBS[@]}; do cmd=${JOBS[${pid}]} wait ${pid} ; JOBS[${pid}]=$? if [[ ${JOBS[${pid}]} -ne 0 ]]; then status=${JOBS[${pid}]} echo -e "[${pid}] Exited with status: ${status}\n${cmd}" fi done return ${status} } background 'sleep 1 ; false' background 'sleep 3 ; true' background 'sleep 2 ; exit 5' background 'sleep 5 ; true' reap || echo "Ooops! Some jobs failed" 

Answer 4

Bash 4.3 added -n to the wait builtin, and -p was added in version 5.1.

From https://www.gnu.org/software/bash/manual/html_node/Job-Control-Builtins.html

wait -n

If the -n option is supplied, wait waits for a single job from the list of pids or jobspecs or, if no arguments are supplied, any job, to complete and returns its exit status. [...]

wait -p

If the -p option is supplied, the process or job identifier of the job for which the exit status is returned is assigned to the variable varname named by the option argument. [...]

The combination of the two options means Bash 5.1+ is actually quite decent at basic multiprocessing. The main drawback now is really just tracking/managing stdout/stderr.

_job1 () { sleep "$( shuf -i 1-3 -n 1 )"s ; true ; } _job2 () { sleep "$( shuf -i 1-3 -n 1 )"s ; return 42 ; } limit="2" i="0" set -- _job1 _job2 while [ "$#" -gt "0" ] ;do until [ "$i" -eq "$limit" ] ;do printf 'starting %s\n' "$1" "$1" & pids[$!]="$1" i="$(( i + 1 ))" shift done if wait -n -p ended_pid ;then return_code="$?" printf '%s succeeded, returning "%s"\n' "${pids[ended_pid]}" "$return_code" else return_code="$?" printf '%s FAILED, returning "%s"\n' "${pids[ended_pid]}" "$return_code" fi unset 'pids[ended_pid]' i="$(( i - 1 ))" done while [ "${#pids[@]}" -gt "0" ] ;do if wait -n -p ended_pid ;then printf '%s succeeded, returning "%s"\n' "${pids[ended_pid]}" "$?" else printf '%s FAILED, returning "%s"\n' "${pids[ended_pid]}" "$?" fi unset 'pids[ended_pid]' done 

More information (though not on wait -p): http://mywiki.wooledge.org/ProcessManagement

Answer 5

Stéphane's answer is good, but I would prefer

for i in ${!pid[@]} do wait "${pid[i]}" return_status[i]=$? unset "pid[$i]" done 

which will iterate over the keys of the pid array, regardless of which entries still exist, so you can adapt it, break out of the loop, and re-start the whole loop and it'll just work. And you don't need consecutive values of i to begin with.

Of course, if you're dealing with thousands of processes then perhaps Stépane's approach would be fractionally more efficient when you have a non-sparse list.