4

How do I write a script to execute the files in multiple directories?

The problem is this: I have many directories, and each has a data file to be read and analyzed by a python script (say, a.py). I don't want to "cd" to each of the directories and type "a.py". Outputs are saved in each directory.

    2 Answers 2

    8

    find will work magic for you.

    find -name datafile.dat -execdir ~/a.py '{}' \; 

    The find searches recursively in all subdirectories for files that match a set of rules and performs an action on them.

    The -name rule will let you find files with a name that matches what you give it. You can use globbing, for example, "*.dat" would find all the .dat files.

    If necessary, you can use -regex instead of -name to match with a regex pattern instead of a glob pattern, so you could do ".*\.dat$" to match all the .dat files.

    The -execdir will execute whatever command you give it from the directory of the found file, replacing "{}" with the found file.

    3
    • Nice. I should really learn find, it has flags for every conceivable situationCommentedNov 6, 2010 at 4:32
    • The man page for find isn't great, but the info pages are extensive.CommentedNov 6, 2010 at 5:47
    • 1
      @Michael: Zsh has similar features; they're more cryptic but less typing. @Shawn: -execdir is useful, but not portable. @all: In bash 4 or zsh, you can do for x in **/datafile.dat; do (cd "$(dirname "$x")" && ~/a.py "$(basename "$x")"); doneCommentedNov 6, 2010 at 13:16
    7

    You can probably just use a for loop:

    for i in first_dir second_dir third_dir; do pushd $i a.py popd done 

    It will run pushd $i; a.py; popd with $i set to first_dir, then again with $i as second_dir, and finally $i as third_dir. pushd switches to the given directory, and popd switches back to where you were

    7
    • 3
      Instead of pushd and popd you can use a subshell - (cd $i && a.py). This construct also protects against $i not existing - you dont want to popd if pushd did not work.
      – camh
      CommentedNov 5, 2010 at 6:57
    • 3
      @Michael, @camh, @sinyeo: Always put double quotes around variable substitutions., e.g. cd -- "$i". Also, protect commands against files beginning with - by passing -- before file names.CommentedNov 5, 2010 at 8:30
    • 2
      Ok, as always the comments are right, but I can't remember the last time I posted a tiny shell snippet without somebody commenting on something like this -- do you guys seriously write one-off loops with all these redundancies built in? You're protecting against directories not existing or having spaces or starting with dashes as though it needs to accept arbitrary input; it's a one-time for loop that's iterating over the static array "first_dir second_dir third_dir" -- none of those things are a problemCommentedNov 5, 2010 at 14:14
    • 2
      @Michael: I strive to include proper quoting and appropriate generalizations in my public posts. I can not know when someone random reader is going to copy some bit of an answer into a context that is (unbeknownst to the reader) just different enough from the original context. Using quoting and other generalization techniques can greatly expand the breadth of situations where any given code snippet can be correctly applied. I also like to use proper quoting, etc. in my interactive use since I find that such practice forms habits that benefit me when writing full shell programs.CommentedNov 6, 2010 at 2:00
    • 1
      @Michael: I totally agree with Chris; I'm in the habit of quoting even in one-liners, and using && when possible, because you just never know when you'll forget there's a space in something, or make a typo causing a key command to fail, and depending on when it bites you, you could have a mess to unravel.
      – Cascabel
      CommentedNov 6, 2010 at 2:11

    You must log in to answer this question.

    Start asking to get answers

    Find the answer to your question by asking.

    Ask question

    Explore related questions

    See similar questions with these tags.