turning list of file paths from one command into command-line arguments for another

Question

I want to pass a list of file paths produced by one command (the KDE baloosearch6 program) as command-line arguments to another command (the KDE gwenview image viewer program). The age-old problem is the image paths contain spaces, so I need to quote them, e.g.: gwenview '/path/to/my first file' '/path to/another file'. The programs don't have -print0/-0 options.

Claude AI gave me the one-liner to do this:

baloosearch6 foobar | sed "s/.*/'&'/" | tr '\n' ' ' \ | xargs gwenview 

but there should be a better way, e.g. processing an array of file paths. How to properly collect an array of lines in zsh gives the zsh magic to collect output lines in an array:

files=("${(@f)$(baloosearch6 foobar)}") 

but now I have to turn the array of file paths back into quoted parameters on the command line. I read the Substitutions chapter of "A User's Guide to the Z-Shell" and couldn't see it. Maybe I should instead be messing with IFS and such.

Ideally I want a single command/function make_cli_args that turns the output of one command into quoted arguments for another. (Why is this so hard?)

Answer 1

If you have an array and just want to pass the elements intact to a command, it's enough to use $array in zsh, or "${array[@]}" (with quotes!) in Bash/ksh. The former drops empty elements, though, but "${array[@]}" works in either:

zsh:

% l=("foo * bar" '' $'new\nline') % printf '<%s>\n' $l <foo * bar> <new line> % printf '<%s>\n' "${l[@]}" <foo * bar> <> <new line> 

Bash:

$ l=("foo * bar" '' $'new\nline') $ printf '<%s>\n' "${l[@]}" <foo * bar> <> <new line> 

---

On the other hand, if you need to pass the array elements to a shell to be executed as a command line, you'll need to quote them properly. This would happen if you need to embed the values in a shell script, or run them through eval (for whatever reason), or pass them through ssh.

Here, you can use printf %q in Bash or zsh, or the (q) modifier in zsh.

Bash's printf %q tends to use backslashes, so the output is ugly, but it should work (the echo is to add a trailing newline for the printout):

$ printf '%q ' "${l[@]}"; echo foo\ \*\ bar '' $'new\nline' 

zsh's (qqqq) uses the $'...' quoting which looks nicer (opinion, of course):

% printf '%q ' "${l[@]}"; echo foo\ \*\ bar '' new$'\n'line % printf "%s " ${(qqqq)l}; echo $'foo * bar' $'' $'new\nline' 

However, note that $'...' is nonstandard and may have differences between shells.

See: Escape a variable for use as content of another script

In any case, I would try to avoid inserting quoted data inside code. It's usually a pain and has loads of corner cases.

In most cases with filenames, it's enough to pass them raw, one on each line and read them into an array. (And make sure you don't have newlines in the filenames, or check for them and complain.)

Answer 2

Quoting is part of the syntax of the shell language. You only need quotes if you're writing shell code.

The default xargs input format also understands its own form of quoting, one that is different from that of zsh (it's similar to that of the Mashey shell which is contemporary to xargs from PWB Unix in the 70s; not of any modern shell), but wrapping the lines inside '...' like sed "s/.*/'&'/" does is not enough as that won't work if the lines may contain '.

That tr '\n' ' ' is also counter-productive. xargs treats newlines as delimiters just the same as spaces. By making the input of xargs an overly long non-delimited line, you also make it more likely to break with xargs implementations that have a limit on the length of those lines or discard non-delimited lines.

To run a command with each line of the input as separate arguments, you use GNU's xargs -d '\n' cmd -- or even better here xargs -rd '\n' cmd -- to avoid running cmd at all if the input is empty.

baloosearch6 foobar | xargs -rd '\n' gwenview -- 

(-- can likely be omitted if all the paths output by baloosearch6 are absolute and therefore can't start with -).

For commands that may read thing from their stdin:

xargs -rd '\n' -a <(baloosearch6 foobar) gwenview -- 

Would be even better¹

files=( "${(f@)$(cmd)}" ) 

Gets the output of cmd, removes all the trailing newline characters (including empty lines), splits it on linefeeds and preserves empty elements, but makes an array with one empty element if cmd produces no output (or only empty lines). You need "${(@Af)$(cmd)}" to avoid it, but here, you don't want any empty elements as a file path cannot be empty, so you might as well do:

files=( ${(f)"$(cmd)"} ) 

That array assignment will have the exit status of cmd which is therefore not lost which is one advantage over using xargs.

files=( ${(f)"$(baloosearch6 foobar)"} ) && (( $#files > 0 )) && gwenview -- $files 

Would run gwenview only if baloosearch6 succeeds and there's at least one resulting file path.

That means however that the whole list has to be stored in memory and gwenview cannot start displaying them until baloosearch6 has finished and that could also reach the limit on the size of the arguments to a command (see zargs for how to avoid it).

In any case, if there are matching file paths containing newline characters, either baloosearch6 uses some form of encoding to represent them (like \n, or %0A) and you'd need to modify that code to decode the encoding or it skips them, or it prints them as-is which would make it broken by design as without a -0 it would not be possible to post-process its output.

If you want something quick and easy, and if you don't mind running gwenview if baloosearch6 fails or finds nothing, you can also change $IFS to a newline character³:

IFS=$'\n' gwenview -- $(baloosearch6 foobar) 

With IFS=$'\n\n', the empty lines (except for the trailing ones which are discarded by command substitution) are preserved.

Beware, changing that global $IFS parameter affects all command substitutions and $=var IFS-splitting operators going forward.

As mentioned above, you'd use quotes if writing shell code like for instance:

echo "gwenview -- '/path to some file'" | sh eval "gwenview -- '/path to some file'" watch "gwenview -- '/path to some file'" # or use -x to avoid watch # running a shell ssh user@host "gwenview -- '/path to some file'" 

Though beware that in the latter, it's the login shell of user on host that will interpret that code and it may have a quoting syntax different from that of the shell you use on the client (zsh here)².

zsh has a number of quoting operators to quote strings so they can be used in shell code, but not all are equally safe. Among those, the safest and that would be understood by all shells of the Bourne family including sh, zsh and bash which are the most commonly used as login shell these days is the ${(qq)var} one which uses single quotes and \' to quote the single quote itself.

ssh -Y user@host "gwenview -- ${(j[ ])${(qq)files}}" 

For instance would ask the login shell of the remote user to interpret code that looks like:

gwenview -- '/path/to/some file' '/that'\''s another file' 

(not that it would make any sense here, as the remote host would have no reason to have those files).

Which all Bourne-like shells should interpret the same, but here, you might as well do:

print -rNC1 -- $files | ssh -Y user@host 'xargs -r0 gwenview --' 

Which is a simple enough line of code that all shells would interpret it the same.

---

^{¹ Though while with A | B you can get the exit status of A in $pipestatus[1] or via the pipefail option, in B <(A), the exit status of A is lost.}

^{² More on that at How to execute an arbitrary simple command over ssh without knowing the login shell of the remote user?}

^{³ From its default of $' \t\n\0' which are most commonly used separators.}