shell - how to subtract the two files in linux -
i have 2 files below:
file1
"connect" connect_id="12" "connect" connect_id="11" "connect" connect_id="122" "connect" connect_id="109"
file2
"quit" connect_id="12" "quit" connect_id="11"
the file contents not same similar above , number of records minimum 100,000.
now want result show below file1 (means final result should there in file1)
"connect" connect_id="122" "connect" connect_id="109"
i have used while loop below:
awk {'print $2'} file2 | sed "s/connection_id=//g" > sample.txt while read actual; grep -w -v $actual file1 > file1_tmp mv -f file1_tmp file1 done < sample.txt
here have adjusted code according example. may or may not work.
my problem loop repeating more 1 hour complete process.
so can 1 suggest me how achieve same other ways using diff
or comm
or sed
or awk
or other linux command run faster?
here want eliminate big typical while
loop.
most unix tools line based , don't have whole line matches means grep
, comm
, diff
out window. extract field based information want awk
perfect:
$ awk 'nr==fnr{a[$2];next}!($2 in a)' file2 file1 "connect" connect_id="122" "connect" connect_id="109"
to store results file1
you'll need redict output temporary file , move file file1
so:
$ awk 'nr==fnr{a[$2];next}!($2 in a)' file2 file1 > tmp && mv tmp file1
explanation:
the awk
variable nr
increments every record read, each line in every file. fnr
variable increments every record gets reset every file.
nr==fnr # condition true when reading file1 a[$2] # add second field in file1 array lookup table next # next line in file1 (skips following blocks) !($2 in a) # looking @ file2 if second field not in # array execute default block i.e print line
to modify command need change fields matched. in real case if want match field 1 file1
field 4 file2
do:
$ awk 'nr==fnr{a[$1];next}!($4 in a)' file2 file1
Comments
Post a Comment