python - mapreduce wordcount example in using Octo module -
i started learn mapreduce octo module word count example. try count words in dir hw3data (as specified below). pc works both server , client.
i started windows cmd 2 terminals
server: octo.py server wordcount.py seems server side started without problem
client: octo.py client localhost seems python can't find txt files stored in hw3data dir, says no work, sleeping. can help?
the wordcount.py code below
wordcount.py
server
import glob text_files=glob.glob('c:/python27/octopy-0.1/hw3data/*.txt') def file_contents(file_name): f=open(file_name) try: return f.read() finally: f.close() source=dict((file_name,file_contents(file_name)) file_name in text_files) f=open('outfile','w') def final(key,value): print key,value f.write(str((key,value)))
client
def mapfn(key,value): line in value.splitlines(): word in line.split(): yield word.lower(),1 def reducefn(key,value): return key,len(value)
verify if data files have ".txt" in names. i'm current working in problem homework #3. luck!
Comments
Post a Comment