GitHub - tom-odon/NGrams: Extract unigrams and bigrams from text, and utilize their frequencies to create models for rating perplexity of new data as well as generating new sentences.

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
HG-heldout50.txt		HG-heldout50.txt
HG-test.txt		HG-test.txt
HG-train.txt		HG-train.txt
HG-wordtypes.txt		HG-wordtypes.txt
NGrams.py		NGrams.py
TESTOUTPUT.txt		TESTOUTPUT.txt
WriteUp.docx		WriteUp.docx
data-output.txt		data-output.txt
data-small.txt		data-small.txt
data-vsmall-output.txt		data-vsmall-output.txt
data-vsmall.txt		data-vsmall.txt
ngramprob-HG-new.txt		ngramprob-HG-new.txt
ngramprob-tiny-new.txt		ngramprob-tiny-new.txt
output-HG-test-new.txt		output-HG-test-new.txt
output-tiny-test.txt		output-tiny-test.txt
output-tiny-train.txt		output-tiny-train.txt
tiny-test.txt		tiny-test.txt
tiny-train.txt		tiny-train.txt
tiny-wordtypes.txt		tiny-wordtypes.txt
~$riteUp.docx		~$riteUp.docx

About

Extract unigrams and bigrams from text, and utilize their frequencies to create models for rating perplexity of new data as well as generating new sentences.