Need to format text file and update based of key value format

Question

I have a file having the below values:

cat data.txt

server1: calv server2: anot log: /u/log/1 server3: calv server4: anot server5: secd server6: calv LIB_TRGT_calv,anot: /tmp/hello.txt LIB_TRGT_secd: /var/del.tmp 

I get the variables starting with LIB_TRGT i.e LIB_TRGT_calv,anot & LIB_TRGT_secd

I need to get the names after TRGT_ from the variable above i.e calv,anot.

considering we got calv & anot; I need to get all the entries having calv & anot and add the entries found in the data.txt as below:

Desired Output:

server1: calv server2: anot log: /u/log/1 server3: calv server4: anot server5: secd server6: calv LIB_TRGT_calv,anot: /tmp/hello.txt LIB_TRGT_secd: /var/del.tmp LIB_server1: /tmp/hello.txt LIB_server2: /tmp/hello.txt LIB_server3: /tmp/hello.txt LIB_server4: /tmp/hello.txt LIB_server6: /tmp/hello.txt LIB_server5: /var/del.tmp 

Likewise for LIB_TRGT_secd

Below is what I did so far:

grep TRGT* data.txt | cut -d: -f1 |cut -d_ -f3 calv,anot secd 

further

grep TRGT* test.txt | cut -d: -f1 |cut -d_ -f3 | sed -n 1'p' | tr ',' '\n' calv anot 

However, secd is missing and i m not sure how to use xargs and further go about it.

Tried the solution by user @Kusalananda but it does not work. See output below:

Answer 1

An awk program to do this, assuming the LIB_TRGT_ lines always occur after the other lines that these refer to:

BEGIN { FS = ":[[:blank:]]*" OFS = ": " } { print } $1 !~ /^LIB_TRGT_/ { # "list" is an associative array that uses the keys "calv", # "anot", and "secd", and for each of these stores the relevant # servers as a SUBSEP-delimited string. list[$2] = (list[$2] == "" ? $1 : list[$2] SUBSEP $1) next } { path = $2 # Pick out the comma-delimited list of keys after "TRGT_", and # split these up into elements of the array "k". sub("^.*TRGT_", "", $1) nk = split($1, k, ",") $0 = "" $2 = path # For each key, get the corresponding list from "list" and # output the path for each of its values. for (i = 1; i <= nk; ++i) { nv = split(list[k[i]], v, SUBSEP) for (j = 1; j <= nv; ++j) { $1 = "LIB_" v[j] print } } } 

Testing, with the above script in the file script, and with your data in the Unix text file called file:

$ awk -f script file server1: calv server2: anot log: /u/log/1 server3: calv server4: anot server5: secd server6: calv LIB_TRGT_calv,anot: /tmp/hello.txt LIB_server1: /tmp/hello.txt LIB_server3: /tmp/hello.txt LIB_server6: /tmp/hello.txt LIB_server2: /tmp/hello.txt LIB_server4: /tmp/hello.txt LIB_TRGT_secd: /var/del.tmp LIB_server5: /var/del.tmp 

The ordering is slightly different from what you show in the question due to reacting to the LIB_TRGT_ lines as soon as they appear rather than saving them and handling them at the end.

Answer 2

We can solve this problem by means of generating code from the data and then applying that generated code to the data itself.

GNU sed with extended regex mode used.

inputf='./data.txt' < "$inputf" \ sed -E ' s/^\s+|\s+$//g s/\s+/\t/g ' | sed -E ' /_TRGT_[^:\t]/{ s/^(([^_]+_)+TRGT_)([^,:]+)[,:](.*(\t\S+))$/\3\5\n\1\4/ h;s/\n.*//;s/.*\t// s:[\/&]:\\&:g;G s/(.*)\n(.*\t).*(\n.*)/\2\1\3/ P } D ' | sed -En ' 1i\ p s#(.*)\t(.*)#/:\\t\1$/{s/^/LIB_/;s/:.*/:\\t\2/;H;ba;}#p $a\ :a\ $!d;g;s/.// ' | sed -Ef - "$inputf" server1: calv server2: anot log: /u/log/1 server3: calv server4: anot server5: secd server6: calv LIB_TRGT_calv,anot: /tmp/hello.txt LIB_TRGT_secd: /var/del.tmp LIB_server1: /tmp/hello.txt LIB_server2: /tmp/hello.txt LIB_server3: /tmp/hello.txt LIB_server4: /tmp/hello.txt LIB_server5: /var/del.tmp LIB_server6: /tmp/hello.txt 

---

• Step0, clean the data by removing leading trailing whitespace. Monospace to TABs.
• Step1 we isolate the key value pairs, put them in one per line.
• Step2 armed with these kv tuples, we generate sed code.
• Step3, we apply the code to data.

Answer 3

The below is a unified solution to both of your scenarios, viz., when the second fields are quoted or not.

awk -v q=\' '1 NF<2{next} !/^([^_]+_)+TRGT_/ { a[FNR] = $2 b[FNR,$2] = $1 if (index($2,q) == 1) quoted++ next } { n = split($1, temp, /[_,:]/) s=0 for (i=1; i<n; i++) { t = temp[i] s += length(t)+1 ch = substr($1, s-1, 1) if (ch == "_") continue key = (!quoted) ? t : q t q c[key] = $2 } split("", temp, ":") } BEGIN { OFS="\t" } END { for (i=1; i in a; i++) if ( (a[i] in c) && ((i SUBSEP a[i]) in b)) print "LIB_"b[i,a[i]], c[a[i]] }' data.txt server1: 'calv' server2: 'anot' log: '/u/log/1' server3: 'calv' server4: 'anot' server5: 'secd' server6: 'calv' LIB_TRGT_calv,anot: '/tmp/hello.txt' LIB_TRGT_secd: '/var/del.tmp' LIB_server1: '/tmp/hello.txt' LIB_server2: '/tmp/hello.txt' LIB_server3: '/tmp/hello.txt' LIB_server4: '/tmp/hello.txt' LIB_server5: '/var/del.tmp' LIB_server6: '/tmp/hello.txt' 

---