shell - how to subtract the two files in linux -


i have 2 files below:

file1

"connect"    connect_id="12" "connect"    connect_id="11" "connect"    connect_id="122" "connect"    connect_id="109" 

file2

"quit"    connect_id="12" "quit"    connect_id="11" 

the file contents not same similar above , number of records minimum 100,000.

now want result show below file1 (means final result should there in file1)

"connect"    connect_id="122" "connect"    connect_id="109" 

i have used while loop below:

awk {'print $2'} file2 | sed "s/connection_id=//g" > sample.txt  while read actual;      grep -w -v $actual file1 > file1_tmp     mv -f file1_tmp file1  done < sample.txt 

here have adjusted code according example. may or may not work.

my problem loop repeating more 1 hour complete process.

so can 1 suggest me how achieve same other ways using diff or comm or sed or awk or other linux command run faster?

here want eliminate big typical while loop.

most unix tools line based , don't have whole line matches means grep, comm , diff out window. extract field based information want awk perfect:

$ awk 'nr==fnr{a[$2];next}!($2 in a)' file2 file1 "connect"    connect_id="122" "connect"    connect_id="109" 

to store results file1 you'll need redict output temporary file , move file file1 so:

$ awk 'nr==fnr{a[$2];next}!($2 in a)' file2 file1 > tmp && mv tmp file1 

explanation:

the awk variable nr increments every record read, each line in every file. fnr variable increments every record gets reset every file.

nr==fnr    # condition true when reading file1 a[$2]      # add second field in file1 array lookup table next       # next line in file1 (skips following blocks) !($2 in a) # looking @ file2 if second field not in            # array execute default block i.e print line  

to modify command need change fields matched. in real case if want match field 1 file1 field 4 file2 do:

$ awk 'nr==fnr{a[$1];next}!($4 in a)' file2 file1 

Comments

Popular posts from this blog

c# - Send Image in Json : 400 Bad request -

jquery - Fancybox - apply a function to several elements -

An easy way to program an Android keyboard layout app -