Format column-based text file into tree structure using bash

Question

Is there a Unix/Linux command that can turn this:

AMERICA USA NEW_YORK AB-100 AMERICA USA NEW_YORK VF-200 AMERICA USA NEW_YORK XY-243 AMERICA USA LOS_ANGELES UH-198 AMERICA CANADA TORONTO UT-876 AMERICA CANADA TORONTO UT-877 AMERICA CANADA VANCOUVER UT-871 AMERICA CANADA VANCOUVER UT-872 AMERICA CANADA VANCOUVER UT-873 AMERICA MEXICO MEXICO OU-098 AMERICA MEXICO MONTERREY OU-099 AMERICA MEXICO MONTERREY OU-100 EUROPE FRANCE PARIS IV-122 EUROPE FRANCE PARIS AV-112 EUROPE FRANCE PARIS IF-111 EUROPE FRANCE PARIS XX-190 EUROPE FRANCE TOULOUSE TL-654 

Into this:

AMERICA USA NEW_YORK AB-100 VF-200 XY-243 LOS_ANGELES UH-198 CANADA TORONTO UT-876 UT-877 VANCOUVER UT-871 UT-872 UT-873 MEXICO MEXICO OU-098 MONTERREY OU-099 OU-100 EUROPE FRANCE PARIS IV-122 AV-112 IF-111 XX-190 TOULOUSE TL-654 

Answer 1

For your example:

dir=$(mktemp -d) sed 's|\t|/|g' file | while read -r line; do mkdir -p "$dir/$line"; done (cd "$dir"; tree) rm -r "$dir" 

Output:

 . ├── AMERICA │   ├── CANADA │   │   ├── TORONTO │   │   │   ├── UT-876 │   │   │   └── UT-877 │   │   └── VANCOUVER │   │   ├── UT-871 │   │   ├── UT-872 │   │   └── UT-873 │   ├── MEXICO │   │   ├── MEXICO │   │   │   └── OU-098 │   │   └── MONTERREY │   │   ├── OU-099 │   │   └── OU-100 │   └── USA │   ├── LOS_ANGELES │   │   └── UH-198 │   └── NEW_YORK │   ├── AB-100 │   ├── VF-200 │   └── XY-243 └── EUROPE └── FRANCE ├── PARIS │   ├── AV-112 │   ├── IF-111 │   ├── IV-122 │   └── XX-190 └── TOULOUSE └── TL-654 

Answer 2

An awk script that works with standard awks, keeps the lines in its original order, and works with an arbitrary number of columns in the data:

awk -F $'\t' ' function indent (n, i) { for (i=1; i<=n; i++) printf "\t" } { for (i=1; i<=NF; i++) if ($i != o[i]) { printf "%s%s\n", indent(i-1), $i o[i] = $i } } ' 

The same program logic can be implemented in shell. For bash:

indent () { ind=$( printf "%*s" "$1" '' ) ; printf "${ind// /$'\t'}" ;} while IFS=$'\t' read -a f do for ((i=0; i<${#f}; i++)) do if [[ "${f[i]}" != "${o[i]}" ]] then printf "%s%s\n" "$( indent "$i" )" "${f[i]}" o[i]=${f[i]} fi done done 

Note: For ksh you have to adjust read -a by read -A.

Answer 3

In awk, you could do:

$ awk '{ a[$1][$2][$3] ? a[$1][$2][$3]=a[$1][$2][$3]"\n\t\t\t"$4 : a[$1][$2][$3]="\t\t\t"$4 ; } END{ for(cont in a){ printf "%s\n", cont; for(count in a[cont]){ printf "\t%s\n", count; for(city in a[cont][count]){ print "\t\t"city"\n"a[cont][count][city] }}}}' file EUROPE FRANCE TOULOUSE TL-654 PARIS IV-122 AV-112 IF-111 XX-190 AMERICA USA NEW_YORK AB-100 VF-200 XY-243 LOS_ANGELES UH-198 CANADA VANCOUVER UT-871 UT-872 UT-873 TORONTO UT-876 UT-877 MEXICO MEXICO OU-098 MONTERREY OU-099 OU-100 

In Perl:

perl -lane 'push @{$k{$F[0]}{$F[1]}{$F[2]}},"\t\t\t".$F[3]; END{ for $cont (keys(%k)){ print "$cont"; for $coun (keys(%{$k{$cont}})){ print "\t$coun"; for $city (keys(%{$k{$cont}{$coun}})){ print "\t\t$city\n", join "\n",@{$k{$cont}{$coun}{$city}} }}}}' file EUROPE FRANCE PARIS XX-190 XX-190 TOULOUSE TL-654 AMERICA USA NEW_YORK XY-243 XY-243 LOS_ANGELES UH-198 MEXICO MONTERREY OU-100 OU-100 MEXICO OU-098 CANADA VANCOUVER UT-873 UT-873 TORONTO UT-877 UT-877