0

I have a python script and input data that the python script takes.

I want to execute the python script on remote server without copying python script and data to the remote server.

I tried the script:

ssh [email protected] "cd /home/dong/fold python -u -" < script.py arg1 

It execute the script but the arg1 (which is input data) python script doesn't take from local machine but when the input data present in remote server it works.

I hope experts may show some way to overcoming this problem.

1
  • Comments are not for extended discussion; this conversation has been moved to chat.
    – terdon
    CommentedMay 2, 2022 at 14:19

1 Answer 1

1

There is a way, but there's also a caveat. Read the whole answer before you attempt anything.

Create a new named pipe (fifo) on the server:

ssh [email protected] 'cd /home/dong/fold && mkfifo myfifo' 

The command should return almost immediately, make sure it succeeded. Now let your remotely running script treat the named pipe as a file to work with. This command should block, this is expected, let it run:

ssh [email protected] 'cd /home/dong/fold && python -u - myfifo' < script.py 

While the above command waits, feed the named pipe from your local computer (obviously you need another local console for this):

ssh [email protected] 'cd /home/dong/fold && cat > myfifo' < arg1 

This should unblock the previous command, the two commands should now transfer data.

Notes and caveats:

  • cd does not consume its stdin, so all the content of local script.py will go to the remote python and all the content of local arg1 will go to the remote cat. If you're not sure and you absolutely want to be on the safe side, you can always redirect the stdin of cd away from important data (</dev/null cd …). An overkill in case of cd, but if you insert yet another tool for any reason and the tool may read its stdin, then such redirection may solve the problem.

  • Reading from the named pipe blocks until there's a writing process; writing to the named pipe blocks until there's a reading process. This means cat and python may start in any order, you don't need to synchronize them additionally. If you want to implement the solution in a (local) shell script or do everything in a single local console then it makes sense to run ssh … cat … first asynchronously (i.e. in the background). The important thing is to create the named pipe before anything on the server tries to use it; that's why I placed mkfifo myfifo inside a separate ssh command.

  • There's a mechanism of connection sharing, it allows you to run many sshs over a single connection to the server. See man 1 ssh, -M and -S options. This is especially handy when authenticating with a password, wanting to put some ssh(s) in the background and not being able to use ssh -f for this (e.g. because it implies -n and you don't want -n). You start with ssh -fNMS … that handles the authentication and goes to the background, then you invoke as many ssh -S … as you want (possibly in the background), eventually you run ssh -O exit -S … to terminate the master connection. I won't elaborate.

  • When myfifo is no longer needed, you can remove it like any other file (rm myfifo in the right directory on the server).

  • Finally the main caveat.

    The local script.py is transferred via stdin of one local ssh and the data is available as a non-seekable stream on the stdin of the remotely running python. This should not be a problem, it's already this way in your original code and AFAIK python can handle this.

    Similarly the local arg1 is transferred via stdin of another local ssh and this data is available as a non-seekable stream the remotely running python may read from myfifo. This will work only if the python script reads sequentially from the file and does not expect the file to be seekable. What you read from a pipe is not seekable, thus what your script reads from the pipe is not seekable. If the script tries to seek then it will fail. If the script tries to write to myfifo then it will fail or misbehave (the script is absolutely not able to modify the local arg this way). The ability or inability of your script to do the desired job by working with myfifo totally depends on the details of the script which are undisclosed.

    If the python script needs the file to be seekable then you must either provide a copy of the local arg as a regular file on the server (you explicitly stated you don't want this, still a simple copy is the most straightforward solution) or mount the local arg on the server, so it appears as a regular file there. The latter method is a non-trivial task. Without help from the admin of the server, you are limited to FUSE-based solutions like sshfs or curlftpfs (if FUSE is enabled at all). And you need to turn your local computer into a server of the chosen protocol; and you need to make sure the other computer can reach your local computer. I won't elaborate.

5
  • second command gets block and doesnot get unblock
    – simond
    CommentedMay 2, 2022 at 5:20
  • @stuarrty Does the third command exit? Again: the ability or inability of your script to do the desired job by working with myfifo totally depends on the details of the script which are undisclosed. I have tested my solution with the cat-mimicking script from this answer (the one described as "Or to allow arbitrary large lines"). It works. This particular script is designed to be able to work with a pipe and therefore it works with myfifo. I know nothing about your script.CommentedMay 2, 2022 at 5:42
  • script is little confidential but i can share with you if you can provide the mail id or some link, third command executed successfully but no output
    – simond
    CommentedMay 2, 2022 at 5:44
  • 1
    @stuarrty This site does not work like that. You can edit the question and post the script there (with proper formatting) for all to see. Please note I'm not a programmer, I don't know Python at all, I probably won't be able to analyze it. Besides, in general we are not for analyzing code (especially if it's more than a few lines). If the third command exited then either you messed something up and it didn't write to the fifo (check the paths, maybe you created a regular file elsewhere on the server), or your script read from the fifo once and then expected more (reopened it to read again?).CommentedMay 2, 2022 at 5:57
  • ok ...thanks not sucessful finally
    – simond
    CommentedMay 2, 2022 at 6:16

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.