algorithm - Form a Matrix From a Large Text File Quickly -

hi struggling reading data file enough. ( left 4hrs, crashed) must simpler way.

the text file looks similar this:

from 1    5 3    2 2    1 4    3 

from want form matrix there 1 in according [m,n]

the current code is:

    function [z] = reed (a)      [m,n]=size(a);     i=1;      while (i <= n)         z(a(1,i),a(2,i))=1;         i=i+1;     end 

which output following matrix, z:

    z =   0     0     0     0     1  1     0     0     0     0  0     1     0     0     0  0     0     1     0     0 

my actual file has 280,000,000 links , from, code slow size file. know faster in matlab?

thanks

you can along lines of following:

>> = zeros(4,5); >> b = importdata('testcase.txt'); >> a(sub2ind(size(a),b.data(:,1),b.data(:,2))) = 1; 

my test case, 'testcase.txt' contains sample data:

from 1    5 3    2 2    1 4    3 

the result be:

>>  =       0     0     0     0     1      1     0     0     0     0      0     1     0     0     0      0     0     1     0     0 

edit - 1

after taking @ data, seems if modify code appropriately, may not have enough memory execute matrix a become large.

as such, can use sparse matrices achieve same given below:

>> b = importdata('web-stanford.txt'); >> = sparse(b.data(:,1),b.data(:,2),1,max(max(b.data)),max(max(b.data))); 

this approach i'd recommend a matrix have size of [281903,281903] large handle due memory constraints. sparse matrix on other hand, maintains matrix entries non-zero, saving on lot of space. in cases, can use sparse matrices more-or-less use normal matrices.

more information sparse command given here.

edit - 2

i'm not sure why isn't working you. here's screenshot of how did in case helps:

edit - 3

it seems you're getting double matrix in b while i'm getting struct. i'm not sure why happening; can speculate deleted header lines input file before used importdata.

basically it's b.data same b. such, should able use following instead:

>> = sparse(b(:,1),b(:,2),1,max(max(b)),max(max(b)));