How to check the status of bash shell script while executing from Python script?

Question

I have a simple Python script which will execute a shell script using subprocess module in Python.

Below is my Python shell script which is calling testing.sh shell script and it works fine.

import os import json import subprocess jsonData = '{"pp": [0,3,5,7,9]}' jj = json.loads(jsonData) os.putenv( 'jj3', ' '.join( str(v) for v in jj['pp'] ) ) print "start" proc = subprocess.Popen('testing.sh', stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = proc.communicate() if stderr: print "Shell script gave some error" print stderr else: print stdout print "end" # Shell script ran fine. 

And below is my testing.sh shell script -

#!/bin/bash dir1=some_directory dir2=some_directory length1=some_number length2=some_number if [ "$dir1" = "$dir2" ] && [ "$length1" -gt 0 ] && [ "$length2" -gt 0 ] then for el in $jj3 do scp david@machineB:/data/be_t1_snapshot/20131215/t1_"$el"_5.data /data01/primary/. || scp david@machineC:/data/be_t1_snapshot/20131215/t1_"$el"_5.data /data01/primary/. done fi 

What my above shell script does is, it will copy the files from machineB OR machineC to machineA. If the files are not there in machineB then it should be there in machineC always. So it will try to copy the files from machineB to machineA but if the files are not there in machineB then it will try to copy the files from machineC to machineA.

Now my above Python script (which I am running from machineA) tries to execute my above shell script and see whether my script got executed successfully or not. As I am storing the stderr of my shell script and stdout as well.

Problem Statement:-

With the above approach there is one problem that I am seeing. As I mentioned if the files are not there in machineB, then it will try to copy the files from machineC to machineA. So whenever I run my Python script which calls my shell script, what happens is that, some files are not there in machineB so it throws some exception and then it tries copying the files from machineC but when the call comes back to Python script, it always go inside if stderr: block as there was an error while copying the files from machineB (which is not what I want) and end print statement doesn't gets printed out.

So the question is if the files are not there in machineB, it will try copying the files from machineC and if the file is there is machineC and got successfully copied to machineA without any error, then I want to call that as a success instead of failure. In current scenario what is happening, if the files are not there in machineB and got successfully copied from machineC then it still counts as a failure and end print statement doesn't gets printed out.

I also want to see whether my shell script has any problem while executing from Python. If yes, then I don't want to print end statement.

How should I overcome this problem?

UPDATE:-

What will happen with the below shell script? As the first for loop will failed because david is not a linux command, but my scp command works fine. So still I will see status code as not equal to 0?

for i in $( david ); do echo item: $i done dir1=some_directory dir2=some_directory length1=some_number length2=some_number if [ "$dir1" = "$dir2" ] && [ "$length1" -gt 0 ] && [ "$length2" -gt 0 ] then for el in $jj3 do scp david@machineB:/data/be_t1_snapshot/20131215/t1_"$el"_5.data /data01/primary/. || scp david@machineC:/data/be_t1_snapshot/20131215/t1_"$el"_5.data /data01/primary/. done fi 

Answer 1

From Python Subprocess Doc :

Popen.returncode

The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet. A negative value -N indicates that the child was terminated by signal N (Unix only). 

So you can check exit status using :

proc.returncode

but it's bit ironic thing is zero 0 means False in Python.

Example :

if (var == False) : # this will execute if var is False or 0 (or 0.0, 0L, 0j) 

Read Python Pitfalls