Mustafa Jarrar, profile picture
Uploaded byMustafa Jarrar
1,943 views

Adnan: Introduction to Natural Language Processing

This document provides an introduction to natural language processing (NLP). It discusses key topics in NLP including languages and intelligence, the goals of NLP, applications of NLP, and general themes in NLP like ambiguity in language and statistical vs rule-based methods. The document also previews specific NLP techniques that will be covered like part-of-speech tagging, parsing, grammar induction, and finite state analysis. Empirical approaches to NLP are discussed including analyzing word frequencies in corpora and addressing data sparseness issues.

Embed presentation

Introduction to Natural Language Processing Markus Dickinson Linguistics @georgetown.edu With some modifications
2 Languages and Intelligence • Languages: Natural (mandarin) and Artificial (prolog/lisp), and in between? • Processing of Computer languages is easy: why? • Intelligence: Natural? and Artificial (AI). • What is AI?: behave as humans do! Whatever that means! Turing Test • Language related activities: acquisition, speech,… manifestation of intelligence
3 Languages and Intelligence • Historically: peaks and valleys. • Tendency to underestimate the problem and overestimate the computing power. • NLP Goes along with AI: Rise and Decline • Now is a peak, 20 years ago may have been a valley. • The internet, small devices may be a driving force.
4 What is NLP? • Natural Language Processing (NLP) – Computers use to process (analyze, understand, generate) natural language – A somewhat applied field • Computational Linguistics (CL) – Computational aspects of the human language faculty – More theoretical
5 Why Study NLP? • Human language interesting & challenging – NLP offers insights into language • Language is the medium of the web • Interdisciplinary: Linguistics, CS, psychology, math, EE, • Help in communication – With computers (ASR, TTS)-Interfaces-HCI – With other humans (MT) • Ambitious yet practical
6 Goals of NLP • Scientific Goal –Identify the computational machinery needed for an agent to exhibit various forms of linguistic behavior • Engineering Goal –Design, implement, and test systems that process natural languages for practical applications • Ups and downs-Historically: more with the web
7 Applications • speech processing: get flight information or book a hotel over the phone, TTS (for the blind) • information extraction: discover names of people and events they participate in, from a document • machine translation: translate a document from one human language into another • question answering: find answers to natural language questions in a text collection or database • summarization: generate a short biography of Noam Chomsky from one or more news articles • OCR: both print and handwritten. • More inportant with Web and Mobile devices!
8 General Themes • Ambiguity of Language • Language as a formal system • Rule-based vs. Statistical Methods • The need for efficiency • Syntax/Semantics/Data mining • Multilingual across/languages Support
9 Ambiguity of language • Phonetic – [raIt] = write, right, rite (Soundex) ‫سائد‬-‫صائد‬ • Lexical – can = noun, verb, modal (‫ذهب‬) • Structural – I saw the man with the telescope ‫بالنظارة‬ ‫فقط‬ ‫رأيته‬ • Semantic →dish = physical plate, menu item (‫مشروع‬) • Referential: The son asked the father to drive him home – (‫شعر‬ ‫تصفيف‬ ‫البنت‬ ‫من‬ ‫األم‬ ‫طلبت‬‫ها‬) → All of these make NLP difficult
10 Language as a formal system • We can treat parts of language formally – Language = a set of acceptable strings in an alphabet – Define a model to recognize/generate language (but recall: language before grammar) • Works for different levels of language (phonology, morphology, etc.) • Can use finite-state automata, context-free grammars, etc. to represent language
11 Rule-based & Statistical Methods • Theoretical linguistics captures abstract properties of language • NLP can more or less follow theoretical insights – Rule-based: model system with linguistic rules – Statistical: model system with probabilities of what normally happens: trust what people usually do/write/say • Hybrid models combine the two
12 The need for efficiency • Simply writing down linguistic insights isn’t sufficient to have a working system • Programs need to run in real-time, i.e., be efficient – There are thousands of grammar rules which might be applied to a sentence • Use insights from computer science – To find the best parse, use chart parsing, a form of dynamic programming • Recall: computers are powerful but “never” as powerful as need be.
13 Preview of Topics 1.Finding Syntactic Patterns in Human Languages: Lg. as Formal System 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
14 The Problem of Syntactic Analysis • Assume input sentence S in natural language L • Assume you have rules (grammar G) that describe syntactic regularities (patterns or structures) found in sentences of L • Given S & G, find syntactic structure of S • Such a structure is called a parse tree
15 Example 1 S  NP VP VP  V NP VP  V S VP he NP V slept NP  I NP  he V  slept V  ate V  drinks Grammar Parse Tree S  NS| VS NS  M K VS  V NP NP M  N NP  N|NS K  N
16 Parsing Example 1 • S  NP VP • VP  V NP • VP  V • NP  I • NP  he • V  slept • V  ate • V  drinks
17 More Complex Sentences • I can fish. • I saw the elephant in my pajamas. • These sentences exhibit ambiguity • Computers will have to find the acceptable or most likely meaning(s). • I can fish. (‫يعلب؟‬ ،‫يستطيع‬)
18 Example 2 S NP VP Pronoun V NP NI can fish S NP VP Pronoun V NP NI can fish S NP VP Pronoun Aux V I can fish S NP VP Pronoun Aux V I can fish
19 Example 3 • NP  D Nom • Nom  Nom RelClause • Nom  N • RelClause  RelPro VP • VP  V NP • D  the • D  my • V  is • V  hit • N  dog • N  boy • N  brother • RelPro  who S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy
20 Topics 1.Finding Syntactic Patterns in Human Languages 2. Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
21 Meaning from a Parse Tree • I can fish. • We want to understand – Who does what? • the canner is me, the action is canning, and the thing canned is fish. • e.g. Canning(ME, FishStuff) • This is a logic representation of meaning S NP VP Pronoun V NP NI can fish S NP VP Pronoun V NP NI can fish We can do this by • associating meanings with lexical items in the tree • then using rules to figure out what the S as a whole means
22 Meaning from a Parse Tree (Details) • Let’s augment the grammar with feature constraints • S  NP VP – <S subj> =<NP> – <S>=<VP> • VP V NP – <VP> = <V> – <VP obj> =<NP> S NP VP Pronoun V NP NI can fish S NP VP Pronoun V NP NI can fish *2:[pred: Canning] [subj: *1 pred: *2 obj: *3] *1[sem: ME] *3[sem: Fish Stuff] [pred: *2 obj: *3]
23 Grammar Induction • Start with a tree bank = collection of parsed sentences • Extract grammar rules corresponding to parse trees, estimating the probability of the grammar rule based on its frequency P(Aβ | A) = Count(Aβ) / Count(A) • You then have a probabilistic grammar, derived from a corpus of parse trees • How does this grammar compare to grammars created by human intuition? • How do you get the corpus?
24 Finite-State Analysis A finite-state machine for recognizing NPs:  initial=0; final ={2}  0->N->2  0->D->1  1->N->2  2->N->2 An equivalent regular expression for NP’s /D? N+/ A regular expression for recognizing simple sentences /(Prep D? A* N+)* (D? N) (Prep D? A* N+)* (V_tns|Aux V_ing) (Prep D? A* N+)*/ We can also “cheat” a bit in our linguistic analysis
25 Topics 1.Finding Syntactic Patterns in Human Languages 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
26 Empirical Approaches to NLP • Empiricism: knowledge is derived from experience • Rationalism: knowledge is derived from reason • NLP is, by necessity, focused on ‘performance’, in that naturally-occurring linguistic data has to be processed – Have to process data characterized by false starts, hesitations, elliptical sentences, long and complex sentences, input in a complex format, etc. • The methodology used is corpus-based – linguistic analysis (phonological, morphological, syntactic, semantic, etc.) carried out on a fairly large scale – rules are derived by humans or machines from looking at phenomena in situ (with statistics playing an important role)
27 Which Words are the Most Frequent? Common Words in Tom Sawyer (71,730 words), from Manning & Schutze p.21 • Will these counts hold in a different corpus (and genre, cf. Tom)? • What happens if you have 8-9M words?
28 Data Sparseness • Many low-frequency words • Fewer high-frequency words. • Only a few words will have lots of examples. • About 50% of word types occur only once • Over 90% occur 10 times or less. Word Frequency Number of words of that frequency 1 3993 2 1292 3 664 4 410 5 243 6 199 7 172 8 131 9 82 10 91 11-50 540 51-100 99 >100 102 Frequency of word types in Tom Sawyer, from M&S 22.
29 Zipf’s Law: Frequency is inversely proportional to rank Word Freq f Rank r f.r the 3332 1 3332 and 2972 2 5944 a 1775 3 5325 he 877 10 8770 but 410 20 8200 be 294 30 8820 there 222 40 8880 one 172 50 8600 about 158 60 9480 more 138 70 9660 never 124 80 9920 oh 116 90 10440 two 104 100 10400 turned 51 200 10200 you’ll 30 300 9000 name 21 400 8400 comes 16 500 8000 group 13 600 7800 lead 11 700 7700 friends 10 800 8000 begin 9 900 8100 family 8 1000 8000 brushed 4 2000 8000 sins 2 3000 6000 could 2 4000 8000 applausive 1 8000 8000 Empirical evaluation of Zipf’s Law on Tom Sawyer, from M&S 23.
30 Illustration of Zipf’s Law logarithmic scale (Brown Corpus, from M&S p. 30)
31 Empiricism: Part-of-Speech Tagging • Word statistics are only so useful • We want to be able to deduce linguistic properties of the text • Part-of-speech (POS) Tagging = assigning a POS (lexical category) to every word in a text – Words can be ambiguous – What is the best way to disambiguate?
32 Part-of-Speech Disambiguation Secretariat/NNP is/VBZ expected/VBN to/TO race/VB tomorrow/NN The/DT reason/NN for/IN the/DT race/NN for/IN outer/JJ space/NN is … • Given a sentence W1…Wn and a tagset of lexical categories, find the most likely tag C1..Cn for each word in the sentence • Tagset – e.g., Penn Treebank (45 tags) • Note that many of the words may have unambiguous tags • The tagger also has to deal with unknown words
Penn Tree Bank Tagset
A Statistical Method for POS Tagging MD NN VB PRP he 0 0 0 .3 will .8 .2 0 0 race 0 .4 .6 0 lexical generation probs he|PRP 0.3 will|MD 0.8 race|NN 0.4 race|VB 0.6 will|NN 0.2 .4 .6.3 .7 .8 .2 <s>| 1 C|R MD NN VB PRP MD .4 .6 NN .3 .7 PRP .8 .2  1 POS bigram probs Find the value of C1..Cn which maximizes: i=1, n P(Wi| Ci) * P(Ci| Ci-1) lexical generation probabilities POS bigram probabilities
Chomsky’s Critique of Corpus-Based Methods 1. Corpora model performance, while linguistics is aimed at the explanation of competence If you define linguistics that way, linguistic theories will never be able to deal with actual, messy data 2. Natural language is in principle infinite, whereas corpora are finite, so many examples will be missed Excellent point, which needs to be understood by anyone working with a corpus. But does that mean corpora are useless?  Introspection is unreliable (prone to performance factors, cf. only short sentences), and pretty useless with child data.  Insights from a corpus might lead to generalization/induction beyond the corpus– if the corpus is a good sample of the “text population” 3. Ungrammatical examples won’t be available in a corpus Depends on the corpus, e.g., spontaneous speech, language learners, etc. The notion of grammaticality is not that clear - Who did you see [pictures/?a picture/??his picture/*John’s picture] of?
36 Topics 1.Finding Syntactic Patterns in Human Languages 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
37 The Annotation of Data • If we want to learn linguistic properties from data, we need to annotate the data – Train on annotated data – Test methods on other annotated data • Through the annotation of corpora, we encode linguistic information in a computer-usable way.
An Annotation Tool
39 Knowledge Discovery Methodology Raw Corpus Annotated Corpus Initial Tagger Annotation Editor Annotation Guidelines Machine Learning Program Raw Corpus Learned Rules Annotated Corpus Rule Apply Knowledge Base?
40 Topics 1.Finding Syntactic Patterns in Human Languages 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5. Applications 6.Conclusion
41 Application #1: Machine Translation • Using different techniques for linguistic analysis, we can: – Parse the contents of one language – Generate another language consisting of the same content – If we have an intermediate language then we may be able to translate between pairs – Success modest: check Google translate!
Machine Translation on the Web http://complingone.georgetown.edu/~linguist/GU-CLI/GU-CLI-home.html  ‫اصيب‬ ‫من‬ ‫عليهما‬ ‫االعتداء‬ ‫جراء‬ ‫لحم‬ ‫بيت‬ ‫من‬ ‫الجنوب‬ ‫الى‬ ‫المعصرة‬ ‫قرية‬ ‫في‬ ‫وجروح‬ ‫برضوض‬ ‫متظاهران‬ ‫قوات‬ ‫قبل‬ ‫من‬ ‫عليهما‬ ‫االعتداء‬ ‫جراء‬ ‫لحم‬ ‫بيت‬ ‫من‬ ‫الجنوب‬ ‫الى‬ ‫المعصرة‬ ‫قرية‬ ‫في‬ ‫وجروح‬ ‫برضوض‬ ‫تظاهران‬ ‫العنصري‬ ‫الفصل‬ ‫جدار‬ ‫ضد‬ ‫تنظم‬ ‫التي‬ ‫االسبوعية‬ ‫المسيرة‬ ‫خالل‬ ‫االسرائيلي‬ ‫.االحتالل‬ ‫امام‬ ‫من‬ ‫المسيرة‬ ‫طلقت‬ ‫رفع‬ ‫ذلك‬ ‫وخالل‬ ‫واالجانب‬ ‫العرب‬ ‫المتظاهرين‬ ‫من‬ ‫العشرات‬ ‫بمشاركة‬ ‫شوارعها‬ ‫لتجوب‬ ‫الثانوية‬ ‫القرية‬ ‫مدرسة‬ ‫الشعب‬ ‫بحق‬ ‫المتواصلة‬ ‫االسرائيلية‬ ‫والممارسات‬ ‫باالجراءات‬ ‫المنددة‬ ‫والالفتات‬ ‫الفلسطينية‬ ‫االعالم‬ ‫المتظاهرون‬ ‫الفلسطيني‬ ‫الشعب‬ ‫لدى‬ ‫االسرى‬ ‫قضية‬ ‫محورية‬ ‫الى‬ ‫اشارة‬ ‫في‬ ‫االسرى‬ ‫من‬ ‫عدد‬ ‫صور‬ ‫المتظاهرون‬ ‫رفع‬ ‫كما‬ ، ‫ي‬ ‫االسرائيلي‬ ‫االحتالل‬ ‫جنود‬ ‫من‬ ‫العشرات‬ ‫واجهها‬ ‫العنصري‬ ‫الفصل‬ ‫جدار‬ ‫اقامة‬ ‫مشارف‬ ‫الى‬ ‫المسيرة‬ ‫وصول‬ ‫دى‬  Injured demonstrators sustained bruises and cuts to the mill in the villag south of Bethlehem in the attack on them by the Israeli occupation force during the weekly march organized against the Apartheid Wall.  The march began in front of the village school secondary to roam the str with dozens of demonstrators Arabs and foreigners, and during that demonstrators raised Palestinian flags and banners condemning the actio and practices of Israel's continued right of the Palestinian people, as the protesters portrayed a number of prisoners in reference to the central i of the prisoners to the Palestinian people and the arrival of the march to outskirts of the establishment of the apartheid wall and faced dozens of Israeli soldiers
If languages were all very similar…. … then MT would be easier  Dialects - http://rinkworks.com/dialect/  Spanish to Portuguese….  Spanish to French  English to Japanese  …………..
MT Approaches Interlingua Semantics Semantics Syntax Syntax Morphology MorphologyDirect Syntactic Transfer Semantic Transfer
MT Using Parallel Treebanks S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy S NP VP Nom RelClause Nom VP V butta otootoda N otokonokowa NP inuo S NP VP Nom RelClause Nom VP V butta otootoda N otokonokowa NP inuo
Application #2: Understanding a Simple Narrative (Question Answering) Yesterday Holly was running a marathon when she twisted her ankle. David had pushed her. 1. When did the running occur? Yesterday. 2. When did the twisting occur? Yesterday, during the running. 3. Did the pushing occur before the twisting? Yes. 4. Did Holly keep running after twisting her ankle? Maybe not????
47 Question Answering by Computer (Temporal Questions) Yesterday Holly was running a marathon when she twisted her ankle. David had pushed her. 09042005 09052005 run twist ankle duringfinishes or during before push before during 1. When did the running occur? Yesterday. 2. When did the twisting occur? Yesterday, during the running. 3. Did the pushing occur before the twisting? Yes. 4. Did Holly keep running after twisting her ankle? Maybe not????
Application #3: Information Extraction KEY: Trigger word tagging Named Entity tagging Chunk parsing: NGs, VGs, preps, conjunctions Bridgestone Sports Co. said Friday it has set up a joint venture in Taiwan with a local concern and a Japanese trading house to produce golf clubs to be shipped to Japan. CompanyNG Set-UPVG Joint- VentureNG with CompanyNG ProduceVG ProductNG The joint venture, Bridgestone Sports Taiwan Cp., capitalized at 20 million new Taiwan dollars, will start production in January 1990 with production of 20,000 iron and “metal wood” clubs a month.
Information Extraction: Filling Templates Activity: Type: PRODUCTION Company: Product: golf clubs Start-date: Bridgestone Sports Co. said Friday it has set up a joint venture in Taiwan with a local concern and a Japanese trading house to produce golf clubs to be shipped to Japan. Activity: Type: PRODUCTION Company: Bridgestone Sports Taiwan Co Product: iron and “metal wood” clubs Start-date: DURING 1990 The joint venture, Bridgestone Sports Taiwan Cp., capitalized at 20 million new Taiwan dollars, will start production in January 1990 with production of 20,000 iron and “metal wood” clubs a month.
50 Conclusion • NLP programs can carry out a number of very interesting tasks – Part-of-speech disambiguation – Parsing – Information extraction – Machine Translation – Question Answering • These programs have impacts on the way we communicate • These capabilities also have important implications for cognitive science

More Related Content

PPTX
Transformers AI PPT.pptx
byRahulKumar854607
 
PPTX
Text Classification
byRAX Automation Suite
 
PPTX
Language models
byMaryam Khordad
 
PDF
Natural Language Processing (NLP)
byYuriy Guts
 
PDF
Word2Vec
byhyunyoung Lee
 
PDF
Recurrent Neural Networks
byRakuten Group, Inc.
 
PPTX
Introduction to Transformer Model
byNuwan Sriyantha Bandara
 
PDF
Natural language processing (NLP) introduction
byRobert Lujo
 
Transformers AI PPT.pptx
byRahulKumar854607
 
Text Classification
byRAX Automation Suite
 
Language models
byMaryam Khordad
 
Natural Language Processing (NLP)
byYuriy Guts
 
Word2Vec
byhyunyoung Lee
 
Recurrent Neural Networks
byRakuten Group, Inc.
 
Introduction to Transformer Model
byNuwan Sriyantha Bandara
 
Natural language processing (NLP) introduction
byRobert Lujo
 

What's hot

PDF
An introduction to the Transformers architecture and BERT
bySuman Debnath
 
PPTX
Word2Vec
bymohammad javad hasani
 
PPTX
NLP
byguestff64339
 
PDF
NLP using transformers
byArvind Devaraj
 
PPTX
Bert
byAbdallah Bashir
 
PDF
Introduction to Recurrent Neural Network
byYan Xu
 
PDF
Text summarization
bykareemhashem
 
PDF
Convolutional neural network
byYan Xu
 
PPTX
Knowledge representation In Artificial Intelligence
byRamla Sheikh
 
PPTX
A Simple Introduction to Word Embeddings
byBhaskar Mitra
 
PDF
Natural language processing
byAanchal Chaurasia
 
PPTX
Natural language processing and transformer models
byDing Li
 
PDF
Natural Language Processing
byToine Bogers
 
PPTX
NLP
byGirish Khanzode
 
PPTX
Minmax Algorithm In Artificial Intelligence slides
bySamiaAziz4
 
PPTX
Word embedding
byShivaniChoudhary74
 
PDF
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
byMinh Pham
 
PPTX
Deep Learning Explained
byMelanie Swan
 
PPTX
Text similarity measures
byankit_ppt
 
PPTX
[Paper review] BERT
byJEE HYUN PARK
 
An introduction to the Transformers architecture and BERT
bySuman Debnath
 
Word2Vec
bymohammad javad hasani
 
NLP
byguestff64339
 
NLP using transformers
byArvind Devaraj
 
Bert
byAbdallah Bashir
 
Introduction to Recurrent Neural Network
byYan Xu
 
Text summarization
bykareemhashem
 
Convolutional neural network
byYan Xu
 
Knowledge representation In Artificial Intelligence
byRamla Sheikh
 
A Simple Introduction to Word Embeddings
byBhaskar Mitra
 
Natural language processing
byAanchal Chaurasia
 
Natural language processing and transformer models
byDing Li
 
Natural Language Processing
byToine Bogers
 
NLP
byGirish Khanzode
 
Minmax Algorithm In Artificial Intelligence slides
bySamiaAziz4
 
Word embedding
byShivaniChoudhary74
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
byMinh Pham
 
Deep Learning Explained
byMelanie Swan
 
Text similarity measures
byankit_ppt
 
[Paper review] BERT
byJEE HYUN PARK
 

Viewers also liked

PPTX
Jarrar: Introduction to Information Retrieval
byMustafa Jarrar
 
PPTX
Introduction to Artificial Intelligence
byMustafa Jarrar
 
PPTX
Jarrar: Un-informed Search
byMustafa Jarrar
 
PPTX
Jarrar: Informed Search
byMustafa Jarrar
 
PPTX
Jarrar: Description Logic
byMustafa Jarrar
 
PPTX
Jarrar: Introduction to Natural Language Processing
byMustafa Jarrar
 
PDF
Natural Language Processing
byJaganadh Gopinadhan
 
PDF
Dependency Parsing
byJinho Choi
 
PPT
Information retreival, By Hadi Mohammadzadeh
byHadi Mohammadzadeh
 
PDF
Всеволод Демкин "Natural language processing на практике"
byGeeksLab Odessa
 
PPTX
Business Process Implementation
byMustafa Jarrar
 
PPT
Natural Language Processing
byYasir Khan
 
PPT
Jarrar: Games
byMustafa Jarrar
 
PDF
Natural Language Processing Crash Course
byCharlie Greenbacker
 
PDF
Natural language processing with python and amharic syntax parse tree by dani...
byDaniel Adenew
 
PDF
Crash Course in Natural Language Processing (2016)
byVsevolod Dyomkin
 
PPTX
Natural Language Processing: Parsing
byRushdi Shams
 
PPTX
Jarrar: Propositional Logic Inference Methods
byMustafa Jarrar
 
PDF
Parsing Natural Scenes and Natural Language with Recursive Neural Networks
byjie cao
 
PPTX
Jarrar: Probabilistic Language Modeling - Introduction to N-grams
byMustafa Jarrar
 
Jarrar: Introduction to Information Retrieval
byMustafa Jarrar
 
Introduction to Artificial Intelligence
byMustafa Jarrar
 
Jarrar: Un-informed Search
byMustafa Jarrar
 
Jarrar: Informed Search
byMustafa Jarrar
 
Jarrar: Description Logic
byMustafa Jarrar
 
Jarrar: Introduction to Natural Language Processing
byMustafa Jarrar
 
Natural Language Processing
byJaganadh Gopinadhan
 
Dependency Parsing
byJinho Choi
 
Information retreival, By Hadi Mohammadzadeh
byHadi Mohammadzadeh
 
Всеволод Демкин "Natural language processing на практике"
byGeeksLab Odessa
 
Business Process Implementation
byMustafa Jarrar
 
Natural Language Processing
byYasir Khan
 
Jarrar: Games
byMustafa Jarrar
 
Natural Language Processing Crash Course
byCharlie Greenbacker
 
Natural language processing with python and amharic syntax parse tree by dani...
byDaniel Adenew
 
Crash Course in Natural Language Processing (2016)
byVsevolod Dyomkin
 
Natural Language Processing: Parsing
byRushdi Shams
 
Jarrar: Propositional Logic Inference Methods
byMustafa Jarrar
 
Parsing Natural Scenes and Natural Language with Recursive Neural Networks
byjie cao
 
Jarrar: Probabilistic Language Modeling - Introduction to N-grams
byMustafa Jarrar
 

Similar to Adnan: Introduction to Natural Language Processing

PPTX
Artificial Intelligence Notes Unit 4
byDigiGurukul
 
PPTX
Natural Language Processing
byVarunjeet Singh Rekhi
 
PPTX
gdhfjdhjcbdjhvjhdshbajhbvdjbklcbdsjhbvjhsdbvjjv
bysamjohnson7350
 
PPTX
AI UNIT-3 FINAL (1).pptx
byprakashvs7
 
PPTX
Artificial Intelligence_NLP
byThenmozhiK5
 
PPTX
https://www.slideshare.net/amaresimachew/hot-topics-132093738
byAssosa University
 
PDF
Natural Language Processing Course in AI
bySATHYANARAYANAKB
 
PPT
Natural language procssing
byRajnish Raj
 
PPTX
lecture 1 intro NLP_lecture 1 intro NLP.pptx
byMustansarAli20
 
PPTX
NLP
byMohamed El-Serngawy
 
PPT
cs626-449-lect1-intro-2009-7-23-jshdih.ppt
bynidhitechfestiitb
 
PPT
intro.ppt
byssuser77162c
 
PPTX
natural language processing of artificial
bymadhurrajverma
 
PDF
Computational linguistics
byshrey bhate
 
PPTX
NLP_KASHK: Introduction
byHemantha Kulathilake
 
PDF
Introduction to Computational Linguistics
byshaimashimo
 
PPT
NLP-my-lecture (3).ppt
byKrishnaGupta717939
 
PPT
Intro
byGetachew Mamo
 
PPT
Intro
byGetachew Mamo
 
PPTX
Introduction to Natural Language Processing - Stages in NLP Pipeline, Challen...
byresming1
 
Artificial Intelligence Notes Unit 4
byDigiGurukul
 
Natural Language Processing
byVarunjeet Singh Rekhi
 
gdhfjdhjcbdjhvjhdshbajhbvdjbklcbdsjhbvjhsdbvjjv
bysamjohnson7350
 
AI UNIT-3 FINAL (1).pptx
byprakashvs7
 
Artificial Intelligence_NLP
byThenmozhiK5
 
https://www.slideshare.net/amaresimachew/hot-topics-132093738
byAssosa University
 
Natural Language Processing Course in AI
bySATHYANARAYANAKB
 
Natural language procssing
byRajnish Raj
 
lecture 1 intro NLP_lecture 1 intro NLP.pptx
byMustansarAli20
 
NLP
byMohamed El-Serngawy
 
cs626-449-lect1-intro-2009-7-23-jshdih.ppt
bynidhitechfestiitb
 
intro.ppt
byssuser77162c
 
natural language processing of artificial
bymadhurrajverma
 
Computational linguistics
byshrey bhate
 
NLP_KASHK: Introduction
byHemantha Kulathilake
 
Introduction to Computational Linguistics
byshaimashimo
 
NLP-my-lecture (3).ppt
byKrishnaGupta717939
 
Intro
byGetachew Mamo
 
Intro
byGetachew Mamo
 
Introduction to Natural Language Processing - Stages in NLP Pipeline, Challen...
byresming1
 

More from Mustafa Jarrar

PPTX
Clustering Arabic Tweets for Sentiment Analysis
byMustafa Jarrar
 
PPTX
Classifying Processes and Basic Formal Ontology
byMustafa Jarrar
 
PPTX
Discrete Mathematics Course Outline
byMustafa Jarrar
 
PPTX
Business Process Design and Re-engineering
byMustafa Jarrar
 
PPTX
BPMN 2.0 Analytical Constructs
byMustafa Jarrar
 
PPTX
BPMN 2.0 Descriptive Constructs
byMustafa Jarrar
 
PPTX
Introduction to Business Process Management
byMustafa Jarrar
 
PDF
Customer Complaint Ontology
byMustafa Jarrar
 
PPTX
Subset, Equality, and Exclusion Rules
byMustafa Jarrar
 
PPTX
Schema Modularization in ORM
byMustafa Jarrar
 
PPTX
On Computer Science Trends and Priorities in Palestine
byMustafa Jarrar
 
PPTX
Lessons from Class Recording & Publishing of Eight Online Courses
byMustafa Jarrar
 
PPTX
Presentation curras paper-emnlp2014-final
byMustafa Jarrar
 
PPTX
Jarrar: Future Internet in Horizon 2020 Calls
byMustafa Jarrar
 
PPT
Habash: Arabic Natural Language Processing
byMustafa Jarrar
 
PPTX
Riestra: How to Design and engineer Competitive Horizon 2020 Proposals
byMustafa Jarrar
 
PPTX
Bouquet: SIERA Workshop on The Pillars of Horizon2020
byMustafa Jarrar
 
PPTX
Jarrar: Sparql Project
byMustafa Jarrar
 
PPTX
Jarrar: Logical Foundation of Ontology Engineering
byMustafa Jarrar
 
PPTX
Jarrar: Stepwise Methodologies for Developing Ontologies
byMustafa Jarrar
 
Clustering Arabic Tweets for Sentiment Analysis
byMustafa Jarrar
 
Classifying Processes and Basic Formal Ontology
byMustafa Jarrar
 
Discrete Mathematics Course Outline
byMustafa Jarrar
 
Business Process Design and Re-engineering
byMustafa Jarrar
 
BPMN 2.0 Analytical Constructs
byMustafa Jarrar
 
BPMN 2.0 Descriptive Constructs
byMustafa Jarrar
 
Introduction to Business Process Management
byMustafa Jarrar
 
Customer Complaint Ontology
byMustafa Jarrar
 
Subset, Equality, and Exclusion Rules
byMustafa Jarrar
 
Schema Modularization in ORM
byMustafa Jarrar
 
On Computer Science Trends and Priorities in Palestine
byMustafa Jarrar
 
Lessons from Class Recording & Publishing of Eight Online Courses
byMustafa Jarrar
 
Presentation curras paper-emnlp2014-final
byMustafa Jarrar
 
Jarrar: Future Internet in Horizon 2020 Calls
byMustafa Jarrar
 
Habash: Arabic Natural Language Processing
byMustafa Jarrar
 
Riestra: How to Design and engineer Competitive Horizon 2020 Proposals
byMustafa Jarrar
 
Bouquet: SIERA Workshop on The Pillars of Horizon2020
byMustafa Jarrar
 
Jarrar: Sparql Project
byMustafa Jarrar
 
Jarrar: Logical Foundation of Ontology Engineering
byMustafa Jarrar
 
Jarrar: Stepwise Methodologies for Developing Ontologies
byMustafa Jarrar
 

Recently uploaded

PDF
Q3'25 Financial Results and Earnings Presentation
byDropbox
 
PDF
Access Linux File System in RHEL - RHCSA (RH124).pdf
byLinuxCert Guru
 
PDF
Manage Local Users and Groups - RHCSA (RH124)
byLinuxCert Guru
 
PDF
Database Performance at Scale: The TL;DR
byScyllaDB
 
PDF
Manage Files Using CLI - RHCSA (RH124).pdf
byLinuxCert Guru
 
PDF
Career Blueprint: Mentor Tracks & Career Clinic - Part 2
byDianaGray10
 
PDF
Manage Networking in RHEL - RHCSA (RH124).pdf
byLinuxCert Guru
 
PDF
#MakeAIMatter for HR Professionals | AI Transformation Workshop by Tekdi Tech...
byParth Lawate
 
PDF
Upskill to Agentic Automation - Accelerating Your Job Search using AI
byDianaGray10
 
PDF
Revolutionizing SQL Database Design for the Modern Era.pdf
bySQL DBM
 
PPTX
A GREEN BUSINESS TOOLKIT FOR EARLY-STAGE ENTREPRENEURS (1).pptx.pptx
bylearnskillsofficial1
 
PDF
Private Governance: How Decentralized Mechanisms Can Regulate the Crypto Economy
byFederico Ast
 
PDF
Unveiling the Basics of Agentic AI - OSUG Mumbai
bythesubuiyer
 
PPTX
Career Blueprint - Future Career Vision & Success Stories - 2025 - Part 1
byDianaGray10
 
PDF
Do more with the HP Z2 Mini G1a
byPrincipled Technologies
 
PDF
From Pixels to Insights: Getting Started with Imagery in FME
bySafe Software
 
PDF
Analyze and Store Logs - RHCSA (RH124).pdf
byLinuxCert Guru
 
PPTX
Expanding Your Toolbox: Beginners Guide to Controlling Kubernetes Logs
byEric D. Schabell
 
PDF
Wrapping up unowned UTF8 C strings: CStringView
byIgalia
 
PPTX
Sephora UAE API Service Real-Time Product & Price Data Access.pptx
bycreativeclicks1733
 
Q3'25 Financial Results and Earnings Presentation
byDropbox
 
Access Linux File System in RHEL - RHCSA (RH124).pdf
byLinuxCert Guru
 
Manage Local Users and Groups - RHCSA (RH124)
byLinuxCert Guru
 
Database Performance at Scale: The TL;DR
byScyllaDB
 
Manage Files Using CLI - RHCSA (RH124).pdf
byLinuxCert Guru
 
Career Blueprint: Mentor Tracks & Career Clinic - Part 2
byDianaGray10
 
Manage Networking in RHEL - RHCSA (RH124).pdf
byLinuxCert Guru
 
#MakeAIMatter for HR Professionals | AI Transformation Workshop by Tekdi Tech...
byParth Lawate
 
Upskill to Agentic Automation - Accelerating Your Job Search using AI
byDianaGray10
 
Revolutionizing SQL Database Design for the Modern Era.pdf
bySQL DBM
 
A GREEN BUSINESS TOOLKIT FOR EARLY-STAGE ENTREPRENEURS (1).pptx.pptx
bylearnskillsofficial1
 
Private Governance: How Decentralized Mechanisms Can Regulate the Crypto Economy
byFederico Ast
 
Unveiling the Basics of Agentic AI - OSUG Mumbai
bythesubuiyer
 
Career Blueprint - Future Career Vision & Success Stories - 2025 - Part 1
byDianaGray10
 
Do more with the HP Z2 Mini G1a
byPrincipled Technologies
 
From Pixels to Insights: Getting Started with Imagery in FME
bySafe Software
 
Analyze and Store Logs - RHCSA (RH124).pdf
byLinuxCert Guru
 
Expanding Your Toolbox: Beginners Guide to Controlling Kubernetes Logs
byEric D. Schabell
 
Wrapping up unowned UTF8 C strings: CStringView
byIgalia
 
Sephora UAE API Service Real-Time Product & Price Data Access.pptx
bycreativeclicks1733
 
In this document
Powered by AI

Overview of Natural Language Processing presented by Markus Dickinson at Georgetown University.

Explores natural vs artificial languages, the Turing Test, and the historical context of NLP within AI.

Defines Natural Language Processing and Computational Linguistics; discusses interdisciplinary studies and practical insights.

Details scientific and engineering goals of NLP, focusing on linguistic behavior and practical system designs.

Describes various applications of NLP, including speech processing, machine translation, and summarization.

Discusses language ambiguity, formal systems, rule-based vs statistical methods, and the need for computational efficiency.

Introduces syntactic analysis, parse trees, and examples showcasing syntactic ambiguity in sentences.

Examines how parse trees convey meaning and the methodology for grammar induction from parsed sentences.

Contrasts empiricism and rationalism in NLP, emphasizes the importance of part-of-speech tagging and its challenges.

Presents critiques of corpus-based methods in contrast to theoretical linguistics in NLP.

Highlights the importance of annotating NLP data and methodologies for knowledge discovery.

Describes machine translation approaches and the complexities involved with different languages.

Explores applications in question answering and information extraction through NLP.

Summarizes various tasks NLP can accomplish and implications for communication and cognitive science.

Adnan: Introduction to Natural Language Processing

  • 1.
    Introduction to NaturalLanguage Processing Markus Dickinson Linguistics @georgetown.edu With some modifications
  • 2.
    2 Languages and Intelligence •Languages: Natural (mandarin) and Artificial (prolog/lisp), and in between? • Processing of Computer languages is easy: why? • Intelligence: Natural? and Artificial (AI). • What is AI?: behave as humans do! Whatever that means! Turing Test • Language related activities: acquisition, speech,… manifestation of intelligence
  • 3.
    3 Languages and Intelligence •Historically: peaks and valleys. • Tendency to underestimate the problem and overestimate the computing power. • NLP Goes along with AI: Rise and Decline • Now is a peak, 20 years ago may have been a valley. • The internet, small devices may be a driving force.
  • 4.
    4 What is NLP? •Natural Language Processing (NLP) – Computers use to process (analyze, understand, generate) natural language – A somewhat applied field • Computational Linguistics (CL) – Computational aspects of the human language faculty – More theoretical
  • 5.
    5 Why Study NLP? •Human language interesting & challenging – NLP offers insights into language • Language is the medium of the web • Interdisciplinary: Linguistics, CS, psychology, math, EE, • Help in communication – With computers (ASR, TTS)-Interfaces-HCI – With other humans (MT) • Ambitious yet practical
  • 6.
    6 Goals of NLP •Scientific Goal –Identify the computational machinery needed for an agent to exhibit various forms of linguistic behavior • Engineering Goal –Design, implement, and test systems that process natural languages for practical applications • Ups and downs-Historically: more with the web
  • 7.
    7 Applications • speech processing:get flight information or book a hotel over the phone, TTS (for the blind) • information extraction: discover names of people and events they participate in, from a document • machine translation: translate a document from one human language into another • question answering: find answers to natural language questions in a text collection or database • summarization: generate a short biography of Noam Chomsky from one or more news articles • OCR: both print and handwritten. • More inportant with Web and Mobile devices!
  • 8.
    8 General Themes • Ambiguityof Language • Language as a formal system • Rule-based vs. Statistical Methods • The need for efficiency • Syntax/Semantics/Data mining • Multilingual across/languages Support
  • 9.
    9 Ambiguity of language •Phonetic – [raIt] = write, right, rite (Soundex) ‫سائد‬-‫صائد‬ • Lexical – can = noun, verb, modal (‫ذهب‬) • Structural – I saw the man with the telescope ‫بالنظارة‬ ‫فقط‬ ‫رأيته‬ • Semantic →dish = physical plate, menu item (‫مشروع‬) • Referential: The son asked the father to drive him home – (‫شعر‬ ‫تصفيف‬ ‫البنت‬ ‫من‬ ‫األم‬ ‫طلبت‬‫ها‬) → All of these make NLP difficult
  • 10.
    10 Language as aformal system • We can treat parts of language formally – Language = a set of acceptable strings in an alphabet – Define a model to recognize/generate language (but recall: language before grammar) • Works for different levels of language (phonology, morphology, etc.) • Can use finite-state automata, context-free grammars, etc. to represent language
  • 11.
    11 Rule-based & StatisticalMethods • Theoretical linguistics captures abstract properties of language • NLP can more or less follow theoretical insights – Rule-based: model system with linguistic rules – Statistical: model system with probabilities of what normally happens: trust what people usually do/write/say • Hybrid models combine the two
  • 12.
    12 The need forefficiency • Simply writing down linguistic insights isn’t sufficient to have a working system • Programs need to run in real-time, i.e., be efficient – There are thousands of grammar rules which might be applied to a sentence • Use insights from computer science – To find the best parse, use chart parsing, a form of dynamic programming • Recall: computers are powerful but “never” as powerful as need be.
  • 13.
    13 Preview of Topics 1.FindingSyntactic Patterns in Human Languages: Lg. as Formal System 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
  • 14.
    14 The Problem ofSyntactic Analysis • Assume input sentence S in natural language L • Assume you have rules (grammar G) that describe syntactic regularities (patterns or structures) found in sentences of L • Given S & G, find syntactic structure of S • Such a structure is called a parse tree
  • 15.
    15 Example 1 S NP VP VP  V NP VP  V S VP he NP V slept NP  I NP  he V  slept V  ate V  drinks Grammar Parse Tree S  NS| VS NS  M K VS  V NP NP M  N NP  N|NS K  N
  • 16.
    16 Parsing Example 1 •S  NP VP • VP  V NP • VP  V • NP  I • NP  he • V  slept • V  ate • V  drinks
  • 17.
    17 More Complex Sentences •I can fish. • I saw the elephant in my pajamas. • These sentences exhibit ambiguity • Computers will have to find the acceptable or most likely meaning(s). • I can fish. (‫يعلب؟‬ ،‫يستطيع‬)
  • 18.
    18 Example 2 S NP VP PronounV NP NI can fish S NP VP Pronoun V NP NI can fish S NP VP Pronoun Aux V I can fish S NP VP Pronoun Aux V I can fish
  • 19.
    19 Example 3 • NP D Nom • Nom  Nom RelClause • Nom  N • RelClause  RelPro VP • VP  V NP • D  the • D  my • V  is • V  hit • N  dog • N  boy • N  brother • RelPro  who S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy
  • 20.
    20 Topics 1.Finding Syntactic Patternsin Human Languages 2. Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
  • 21.
    21 Meaning from aParse Tree • I can fish. • We want to understand – Who does what? • the canner is me, the action is canning, and the thing canned is fish. • e.g. Canning(ME, FishStuff) • This is a logic representation of meaning S NP VP Pronoun V NP NI can fish S NP VP Pronoun V NP NI can fish We can do this by • associating meanings with lexical items in the tree • then using rules to figure out what the S as a whole means
  • 22.
    22 Meaning from aParse Tree (Details) • Let’s augment the grammar with feature constraints • S  NP VP – <S subj> =<NP> – <S>=<VP> • VP V NP – <VP> = <V> – <VP obj> =<NP> S NP VP Pronoun V NP NI can fish S NP VP Pronoun V NP NI can fish *2:[pred: Canning] [subj: *1 pred: *2 obj: *3] *1[sem: ME] *3[sem: Fish Stuff] [pred: *2 obj: *3]
  • 23.
    23 Grammar Induction • Startwith a tree bank = collection of parsed sentences • Extract grammar rules corresponding to parse trees, estimating the probability of the grammar rule based on its frequency P(Aβ | A) = Count(Aβ) / Count(A) • You then have a probabilistic grammar, derived from a corpus of parse trees • How does this grammar compare to grammars created by human intuition? • How do you get the corpus?
  • 24.
    24 Finite-State Analysis A finite-statemachine for recognizing NPs:  initial=0; final ={2}  0->N->2  0->D->1  1->N->2  2->N->2 An equivalent regular expression for NP’s /D? N+/ A regular expression for recognizing simple sentences /(Prep D? A* N+)* (D? N) (Prep D? A* N+)* (V_tns|Aux V_ing) (Prep D? A* N+)*/ We can also “cheat” a bit in our linguistic analysis
  • 25.
    25 Topics 1.Finding Syntactic Patternsin Human Languages 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
  • 26.
    26 Empirical Approaches toNLP • Empiricism: knowledge is derived from experience • Rationalism: knowledge is derived from reason • NLP is, by necessity, focused on ‘performance’, in that naturally-occurring linguistic data has to be processed – Have to process data characterized by false starts, hesitations, elliptical sentences, long and complex sentences, input in a complex format, etc. • The methodology used is corpus-based – linguistic analysis (phonological, morphological, syntactic, semantic, etc.) carried out on a fairly large scale – rules are derived by humans or machines from looking at phenomena in situ (with statistics playing an important role)
  • 27.
    27 Which Words arethe Most Frequent? Common Words in Tom Sawyer (71,730 words), from Manning & Schutze p.21 • Will these counts hold in a different corpus (and genre, cf. Tom)? • What happens if you have 8-9M words?
  • 28.
    28 Data Sparseness • Manylow-frequency words • Fewer high-frequency words. • Only a few words will have lots of examples. • About 50% of word types occur only once • Over 90% occur 10 times or less. Word Frequency Number of words of that frequency 1 3993 2 1292 3 664 4 410 5 243 6 199 7 172 8 131 9 82 10 91 11-50 540 51-100 99 >100 102 Frequency of word types in Tom Sawyer, from M&S 22.
  • 29.
    29 Zipf’s Law: Frequencyis inversely proportional to rank Word Freq f Rank r f.r the 3332 1 3332 and 2972 2 5944 a 1775 3 5325 he 877 10 8770 but 410 20 8200 be 294 30 8820 there 222 40 8880 one 172 50 8600 about 158 60 9480 more 138 70 9660 never 124 80 9920 oh 116 90 10440 two 104 100 10400 turned 51 200 10200 you’ll 30 300 9000 name 21 400 8400 comes 16 500 8000 group 13 600 7800 lead 11 700 7700 friends 10 800 8000 begin 9 900 8100 family 8 1000 8000 brushed 4 2000 8000 sins 2 3000 6000 could 2 4000 8000 applausive 1 8000 8000 Empirical evaluation of Zipf’s Law on Tom Sawyer, from M&S 23.
  • 30.
    30 Illustration of Zipf’sLaw logarithmic scale (Brown Corpus, from M&S p. 30)
  • 31.
    31 Empiricism: Part-of-Speech Tagging •Word statistics are only so useful • We want to be able to deduce linguistic properties of the text • Part-of-speech (POS) Tagging = assigning a POS (lexical category) to every word in a text – Words can be ambiguous – What is the best way to disambiguate?
  • 32.
    32 Part-of-Speech Disambiguation Secretariat/NNP is/VBZ expected/VBNto/TO race/VB tomorrow/NN The/DT reason/NN for/IN the/DT race/NN for/IN outer/JJ space/NN is … • Given a sentence W1…Wn and a tagset of lexical categories, find the most likely tag C1..Cn for each word in the sentence • Tagset – e.g., Penn Treebank (45 tags) • Note that many of the words may have unambiguous tags • The tagger also has to deal with unknown words
  • 33.
  • 34.
    A Statistical Methodfor POS Tagging MD NN VB PRP he 0 0 0 .3 will .8 .2 0 0 race 0 .4 .6 0 lexical generation probs he|PRP 0.3 will|MD 0.8 race|NN 0.4 race|VB 0.6 will|NN 0.2 .4 .6.3 .7 .8 .2 <s>| 1 C|R MD NN VB PRP MD .4 .6 NN .3 .7 PRP .8 .2  1 POS bigram probs Find the value of C1..Cn which maximizes: i=1, n P(Wi| Ci) * P(Ci| Ci-1) lexical generation probabilities POS bigram probabilities
  • 35.
    Chomsky’s Critique ofCorpus-Based Methods 1. Corpora model performance, while linguistics is aimed at the explanation of competence If you define linguistics that way, linguistic theories will never be able to deal with actual, messy data 2. Natural language is in principle infinite, whereas corpora are finite, so many examples will be missed Excellent point, which needs to be understood by anyone working with a corpus. But does that mean corpora are useless?  Introspection is unreliable (prone to performance factors, cf. only short sentences), and pretty useless with child data.  Insights from a corpus might lead to generalization/induction beyond the corpus– if the corpus is a good sample of the “text population” 3. Ungrammatical examples won’t be available in a corpus Depends on the corpus, e.g., spontaneous speech, language learners, etc. The notion of grammaticality is not that clear - Who did you see [pictures/?a picture/??his picture/*John’s picture] of?
  • 36.
    36 Topics 1.Finding Syntactic Patternsin Human Languages 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5.Applications 6.Conclusion
  • 37.
    37 The Annotation ofData • If we want to learn linguistic properties from data, we need to annotate the data – Train on annotated data – Test methods on other annotated data • Through the annotation of corpora, we encode linguistic information in a computer-usable way.
  • 38.
  • 39.
  • 40.
    40 Topics 1.Finding Syntactic Patternsin Human Languages 2.Meaning from Patterns 3.Patterns from Language in the Large 4.Bridging the Rationalist-Empiricist Divide 5. Applications 6.Conclusion
  • 41.
    41 Application #1: MachineTranslation • Using different techniques for linguistic analysis, we can: – Parse the contents of one language – Generate another language consisting of the same content – If we have an intermediate language then we may be able to translate between pairs – Success modest: check Google translate!
  • 42.
    Machine Translation onthe Web http://complingone.georgetown.edu/~linguist/GU-CLI/GU-CLI-home.html  ‫اصيب‬ ‫من‬ ‫عليهما‬ ‫االعتداء‬ ‫جراء‬ ‫لحم‬ ‫بيت‬ ‫من‬ ‫الجنوب‬ ‫الى‬ ‫المعصرة‬ ‫قرية‬ ‫في‬ ‫وجروح‬ ‫برضوض‬ ‫متظاهران‬ ‫قوات‬ ‫قبل‬ ‫من‬ ‫عليهما‬ ‫االعتداء‬ ‫جراء‬ ‫لحم‬ ‫بيت‬ ‫من‬ ‫الجنوب‬ ‫الى‬ ‫المعصرة‬ ‫قرية‬ ‫في‬ ‫وجروح‬ ‫برضوض‬ ‫تظاهران‬ ‫العنصري‬ ‫الفصل‬ ‫جدار‬ ‫ضد‬ ‫تنظم‬ ‫التي‬ ‫االسبوعية‬ ‫المسيرة‬ ‫خالل‬ ‫االسرائيلي‬ ‫.االحتالل‬ ‫امام‬ ‫من‬ ‫المسيرة‬ ‫طلقت‬ ‫رفع‬ ‫ذلك‬ ‫وخالل‬ ‫واالجانب‬ ‫العرب‬ ‫المتظاهرين‬ ‫من‬ ‫العشرات‬ ‫بمشاركة‬ ‫شوارعها‬ ‫لتجوب‬ ‫الثانوية‬ ‫القرية‬ ‫مدرسة‬ ‫الشعب‬ ‫بحق‬ ‫المتواصلة‬ ‫االسرائيلية‬ ‫والممارسات‬ ‫باالجراءات‬ ‫المنددة‬ ‫والالفتات‬ ‫الفلسطينية‬ ‫االعالم‬ ‫المتظاهرون‬ ‫الفلسطيني‬ ‫الشعب‬ ‫لدى‬ ‫االسرى‬ ‫قضية‬ ‫محورية‬ ‫الى‬ ‫اشارة‬ ‫في‬ ‫االسرى‬ ‫من‬ ‫عدد‬ ‫صور‬ ‫المتظاهرون‬ ‫رفع‬ ‫كما‬ ، ‫ي‬ ‫االسرائيلي‬ ‫االحتالل‬ ‫جنود‬ ‫من‬ ‫العشرات‬ ‫واجهها‬ ‫العنصري‬ ‫الفصل‬ ‫جدار‬ ‫اقامة‬ ‫مشارف‬ ‫الى‬ ‫المسيرة‬ ‫وصول‬ ‫دى‬  Injured demonstrators sustained bruises and cuts to the mill in the villag south of Bethlehem in the attack on them by the Israeli occupation force during the weekly march organized against the Apartheid Wall.  The march began in front of the village school secondary to roam the str with dozens of demonstrators Arabs and foreigners, and during that demonstrators raised Palestinian flags and banners condemning the actio and practices of Israel's continued right of the Palestinian people, as the protesters portrayed a number of prisoners in reference to the central i of the prisoners to the Palestinian people and the arrival of the march to outskirts of the establishment of the apartheid wall and faced dozens of Israeli soldiers
  • 43.
    If languages wereall very similar…. … then MT would be easier  Dialects - http://rinkworks.com/dialect/  Spanish to Portuguese….  Spanish to French  English to Japanese  …………..
  • 44.
    MT Approaches Interlingua Semantics Semantics SyntaxSyntax Morphology MorphologyDirect Syntactic Transfer Semantic Transfer
  • 45.
    MT Using ParallelTreebanks S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy S NP VP NomD Nom RelClause RelPro VP V who NP hit the dog is my brother the N boy S NP VP Nom RelClause Nom VP V butta otootoda N otokonokowa NP inuo S NP VP Nom RelClause Nom VP V butta otootoda N otokonokowa NP inuo
  • 46.
    Application #2: Understandinga Simple Narrative (Question Answering) Yesterday Holly was running a marathon when she twisted her ankle. David had pushed her. 1. When did the running occur? Yesterday. 2. When did the twisting occur? Yesterday, during the running. 3. Did the pushing occur before the twisting? Yes. 4. Did Holly keep running after twisting her ankle? Maybe not????
  • 47.
    47 Question Answering byComputer (Temporal Questions) Yesterday Holly was running a marathon when she twisted her ankle. David had pushed her. 09042005 09052005 run twist ankle duringfinishes or during before push before during 1. When did the running occur? Yesterday. 2. When did the twisting occur? Yesterday, during the running. 3. Did the pushing occur before the twisting? Yes. 4. Did Holly keep running after twisting her ankle? Maybe not????
  • 48.
    Application #3: InformationExtraction KEY: Trigger word tagging Named Entity tagging Chunk parsing: NGs, VGs, preps, conjunctions Bridgestone Sports Co. said Friday it has set up a joint venture in Taiwan with a local concern and a Japanese trading house to produce golf clubs to be shipped to Japan. CompanyNG Set-UPVG Joint- VentureNG with CompanyNG ProduceVG ProductNG The joint venture, Bridgestone Sports Taiwan Cp., capitalized at 20 million new Taiwan dollars, will start production in January 1990 with production of 20,000 iron and “metal wood” clubs a month.
  • 49.
    Information Extraction: FillingTemplates Activity: Type: PRODUCTION Company: Product: golf clubs Start-date: Bridgestone Sports Co. said Friday it has set up a joint venture in Taiwan with a local concern and a Japanese trading house to produce golf clubs to be shipped to Japan. Activity: Type: PRODUCTION Company: Bridgestone Sports Taiwan Co Product: iron and “metal wood” clubs Start-date: DURING 1990 The joint venture, Bridgestone Sports Taiwan Cp., capitalized at 20 million new Taiwan dollars, will start production in January 1990 with production of 20,000 iron and “metal wood” clubs a month.
  • 50.
    50 Conclusion • NLP programscan carry out a number of very interesting tasks – Part-of-speech disambiguation – Parsing – Information extraction – Machine Translation – Question Answering • These programs have impacts on the way we communicate • These capabilities also have important implications for cognitive science