Running Python script using shell script redirecting python output

Question

I am running my python script using a shell script run.sh to restart it if crashes :

#!/bin/sh COMMAND='nohup python3 -u test.py run >> py.log &' LOGFILE=restart.txt writelog() { now=`date` echo "$now $*" >> $LOGFILE } writelog "Starting" while true ; do $COMMAND writelog "Exited with status $?" writelog "Restarting" done 

I'm running

nohup sh -u run.sh > output& 

The problem is python would only stream the first output(print()) to py.log. How can I get out the output streamed instantly. I have also tried COMMAND=./test.py run >> py.log & . I would appreciate the help. Thank you.

Answer 1

Your shell script is passing the string 'run >> py.log &' to your test.py script, which is probably causing it not to run correctly.

You need to change you script slightly:

#!/bin/sh COMMAND='nohup python3 -u test.py run' LOGFILE=restart.txt writelog() { now=`date` echo "$now $*" >> $LOGFILE } writelog "Starting" while true ; do $COMMAND >> py.log & writelog "Exited with status $?" writelog "Restarting" done 

Now your $COMMAND will run in the background and append it's output to py.log.

The reason your original script does not work is because the shell variables are expanded after output redirections are set, so the >> and & were no longer recognized as part of the shell command and passed as the argument string to your Python script. You can reference LESS='+/SIMPLE COMMAND EXPANSION' man bash for steps used in processing commands.

Answer 2

I ended up using the module redirect_stdout to redirect output to a file and then using sys.stdout.flush() after each print() and for the shell script COMMAND='./test.py run'. Now it's streaming the output in realtime.