There is a way, but there's also a caveat. Read the whole answer before you attempt anything.
Create a new named pipe (fifo) on the server:
ssh [email protected] 'cd /home/dong/fold && mkfifo myfifo'
The command should return almost immediately, make sure it succeeded. Now let your remotely running script treat the named pipe as a file to work with. This command should block, this is expected, let it run:
ssh [email protected] 'cd /home/dong/fold && python -u - myfifo' < script.py
While the above command waits, feed the named pipe from your local computer (obviously you need another local console for this):
ssh [email protected] 'cd /home/dong/fold && cat > myfifo' < arg1
This should unblock the previous command, the two commands should now transfer data.
Notes and caveats:
cd
does not consume its stdin, so all the content of local script.py
will go to the remote python
and all the content of local arg1
will go to the remote cat
. If you're not sure and you absolutely want to be on the safe side, you can always redirect the stdin of cd
away from important data (</dev/null cd …
). An overkill in case of cd
, but if you insert yet another tool for any reason and the tool may read its stdin, then such redirection may solve the problem.
Reading from the named pipe blocks until there's a writing process; writing to the named pipe blocks until there's a reading process. This means cat
and python
may start in any order, you don't need to synchronize them additionally. If you want to implement the solution in a (local) shell script or do everything in a single local console then it makes sense to run ssh … cat …
first asynchronously (i.e. in the background). The important thing is to create the named pipe before anything on the server tries to use it; that's why I placed mkfifo myfifo
inside a separate ssh
command.
There's a mechanism of connection sharing, it allows you to run many ssh
s over a single connection to the server. See man 1 ssh
, -M
and -S
options. This is especially handy when authenticating with a password, wanting to put some ssh
(s) in the background and not being able to use ssh -f
for this (e.g. because it implies -n
and you don't want -n
). You start with ssh -fNMS …
that handles the authentication and goes to the background, then you invoke as many ssh -S …
as you want (possibly in the background), eventually you run ssh -O exit -S …
to terminate the master connection. I won't elaborate.
When myfifo
is no longer needed, you can remove it like any other file (rm myfifo
in the right directory on the server).
Finally the main caveat.
The local script.py
is transferred via stdin of one local ssh
and the data is available as a non-seekable stream on the stdin of the remotely running python
. This should not be a problem, it's already this way in your original code and AFAIK python
can handle this.
Similarly the local arg1
is transferred via stdin of another local ssh
and this data is available as a non-seekable stream the remotely running python
may read from myfifo
. This will work only if the python script reads sequentially from the file and does not expect the file to be seekable. What you read from a pipe is not seekable, thus what your script reads from the pipe is not seekable. If the script tries to seek then it will fail. If the script tries to write to myfifo
then it will fail or misbehave (the script is absolutely not able to modify the local arg
this way). The ability or inability of your script to do the desired job by working with myfifo
totally depends on the details of the script which are undisclosed.
If the python script needs the file to be seekable then you must either provide a copy of the local arg
as a regular file on the server (you explicitly stated you don't want this, still a simple copy is the most straightforward solution) or mount the local arg
on the server, so it appears as a regular file there. The latter method is a non-trivial task. Without help from the admin of the server, you are limited to FUSE-based solutions like sshfs
or curlftpfs
(if FUSE is enabled at all). And you need to turn your local computer into a server of the chosen protocol; and you need to make sure the other computer can reach your local computer. I won't elaborate.