1

I have a gridded model data (separated by different depth) saved in text files. The structure is like this:

Format of every column in a row in the text file:

x_coordinate y_coordinate density 

There are about 400*400 points in every text file(as a plane in certain depth).

Filename:

dep### 

Here, ### is a number represents the depth(in Z direction). The number can be an integer or fraction. For example, now I have these files: "dep0", "dep0.5", "dep10", "dep300", that means those data are 400*400 gridded data in x-y plane form for depth of 0, 0.5, 10 and 300.

Now, I would like to pick up the number in the filename (aka the depth) and add it to the third column of each row, combine all of them together. Also the depth should be sorted from minima to maxima. So the output file should looks like this(for example):

x_coordinate y_coordinate z_coordinate density 0 0 0 2.5 0 1 0 2.5 ... ... 0 2.6 400 400 0 2.9 0 0 0.5 2.8 ... ... 0.5 2.9 0 0 10 3.2 ... ... 10 3.3 ... ... 300 4.7 ... ... 300 4.8 

At first I was doing it with this script:

for((i=$depmin;i<=$depmax;i++)) do if [ -f "xyp/dep"$i ];then awk '{print $1, $2,'$i',$3}' "xyp/dep"$i >> "xyzp/area1" fi done 

Then I figure out it would miss any files which depth is not integer, since the variable $i in the for loop increase by 1 in every round.

I have tried to use sed and find -exec, but I kept getting errors. The difficulty for me is that I don't quite understand how to use $, '', <<< correctly to redirect or pipe the value to awk or other function. Please help me with this problem.

=====================

I have come up with this script:

depnumbers=$(ls xyp | sed -e 's/dep//g' |sort -n) filecount=$(ls xyp | wc -l) for((i=1;i<=$filecount;i++)) do dep=$(awk '{print $'$i'}' <<< $depnumbers) awk '{print $1, $2,'$dep',$3}' "xyp/dep"$dep >> "xyzp/area1" done 

It works just fine. Is there any way to simplify or improve this script for such task? Actually I'm new to bash and still think something is wrong...not sure

8
  • Please edit your post and add an example of a filename with a fraction.CommentedOct 15, 2014 at 10:17
  • Please edit your question and add an example of the actual data and your desired output. Isn't the order of the rows in the second file important? We can't understand exactly what you need without an example.
    – terdon
    CommentedOct 15, 2014 at 11:26
  • Sorry for the misspell, wrong grammar and unclear statements. Tell me if there are still something not clear.CommentedOct 15, 2014 at 12:26
  • What is the name of the file awk is reading ?
    – user78605
    CommentedOct 15, 2014 at 13:08
  • The accepted answer to this question may help you tidy your loop :-)CommentedOct 15, 2014 at 13:08

1 Answer 1

0

For only adding numbers into the file.

awk 'NR>1{print $1,$2,substr(FILENAME,7),$4 }' xyp/dep* > "xyzp/area1" 

For sorting by numbers.

ls -1v xyp/dep* | xargs awk 'NR>1{print $1,$2,substr(FILENAME,7),$4 }' > "xyzp/area1" 

For sorting from minus numbers.

 ls xyp/dep* | sort -t 'p' -k 3 -n | xargs awk 'NR>1{print $1,$2,substr(FILENAME,7),$4 }' > "xyzp/area1" 

This would regard the character p as separator and make the function sort -n only affect on the following numbers.

8
  • Because the number that follows the filename doesn't have same digits. If I have files at depth of 1, 3, 10, 15, 20, 300, the reading priority will be like 1=>10=>15=>20=>3=>300, since the substr only compares the number from specific position. Also it should be substr(FILENAME,7) because the FILENAME contain path xyp/. Still many thanks :) I'll have to read some manual.CommentedOct 15, 2014 at 14:08
  • @Pakox.Wang Yeah sorry, i didn't test and forgot it included the directory. Will sorting the new file on the third field not work ?
    – user78605
    CommentedOct 15, 2014 at 14:13
  • @Pakox.Wang Check update
    – user78605
    CommentedOct 15, 2014 at 14:25
  • Sorry I just went home...will test tomorrow. Checked the manual and it seems fine... but I want to ask if it's still okay when we have the depth number in negative? Just asking, currently have no such issue but I'm aware of it since the data is from Geology model. BTW, I think we should put a wildcard at ls -1v xyp/dep*CommentedOct 15, 2014 at 14:59
  • Yeah i should have,thanks :) anyway i added another answer for minus/negative numbers
    – user78605
    CommentedOct 15, 2014 at 15:03

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.