![19
Parsing
• Taking a surface input and identifying its components
and underlying structure
• Morphological parsing: parsing a word into stem and
affixes and identifying the parts and their
relationships
– Stem and features:
• goose goose +N +SG or goose +V
• geese goose +N +PL
• gooses goose +V +3SG
– Bracketing: indecipherable [in [[de [cipher]] able]]](https://image.slidesharecdn.com/lect4-morphology-230320071025-90a88f83/85/lect4-morphology-ppt-19-320.jpg)
lect4-morphology.ppt
- 2. 2 What is Morphology? • The study of how words are composed of morphemes (the smallest meaning-bearing units of a language) – Stems – core meaning units in a lexicon – Affixes (prefixes, suffixes, infixes, circumfixes) – bits and pieces that combine with stems to modify their meanings and grammatical functions (can have multiple ones) • Immaterial • Trying • Absobl**dylutely • agoing • Unreadable
- 3. 3 Why is Morphology Important to the Lexicon? Full listing versus Minimal Redundancy • true, truer, truest, truly, untrue, truth, truthful, truthfully, untruthfully, untruthfulness • Untruthfulness = un- + true + -th + -ful + -ness • These morphemes appear to be productive • By representing knowledge about the internal structure of words and the rules of word formation, we can save room and search time.
- 4. 4 Need to do Morphological Parsing Morphological Parsing (or Stemming) • Taking a surface input and breaking it down into its morphemes • foxes breaks down into the morphemes fox (noun stem) and –es (plural suffix) • rewrites breaks down into re- (prefix) and write (stem) and –s (suffix)
- 5. 5 Two Broad Classes of Morphology • Inflectional Morphology – Combination of stem and morpheme resulting in word of same class – Usually fills a syntactic feature such as agreement – E.g., plural –s, past tense -ed • Derivational Morphology – Combination of stem and morpheme usually results in a word of a different class – Meaning of the new word may be hard to predict – E.g., +ation in words such as computerization
- 6. 6 Word Classes • By word class, we have in mind familiar notions like noun and verb that we discussed a bit in the previous lecture. • We’ll go into more details when we get to parsing (Chapter 12). • Right now we’re concerned with word classes because the way that stems and affixes combine is based to a large degree on the word class of the stem.
- 7. 7 English Inflectional Morphology • Word stem combines with grammatical morpheme – Usually produces word of same class – Usually serves a syntactic function (e.g., agreement) like likes or liked bird birds • Nominal morphology – Plural forms • s or es • Irregular forms (next slide) • Mass vs. count nouns (email or emails) – Possessives
- 8. 8 Complication in Morphology • Ok so it gets a little complicated by the fact that some words misbehave (refuse to follow the rules) • The terms regular and irregular will be used to refer to words that follow the rules and those that don’t. Regular (Nouns) • Singular (cat, thrush) • Plural (cats, thrushes) • Possessive (cat’s thrushes’) Irregular (Nouns) • Singular (mouse, ox) • Plural (mice, oxen)
- 9. 9 • Verbal inflection – Main verbs (sleep, like, fear) are relatively regular • -s, ing, ed • And productive: Emailed, instant-messaged, faxed, homered • But eat/ate/eaten, catch/caught/caught – Primary (be, have, do) and modal verbs (can, will, must) are often irregular and not productive • Be: am/is/are/were/was/been/being – Irregular verbs few (~250) but frequently occurring – English verbal inflection is much simpler than e.g. Latin
- 10. 10 Regular and Irregular Verbs • Regulars… – Walk, walks, walking, walked, walked • Irregulars – Eat, eats, eating, ate, eaten – Catch, catches, catching, caught, caught – Cut, cuts, cutting, cut, cut
- 11. 11 Derivational Morphology • Derivational morphology is the messy stuff that no one ever taught you. – Quasi-systematicity – Irregular meaning change – Changes of word class
- 12. 12 English Derivational Morphology • Word stem combines with grammatical morpheme – Usually produces a word of a different class – More complicated than inflectional • Example: nominalization – -ize verbs -ation nouns – generalize, realize generalization, realization – verb -er nouns – Murder, spell murderer, speller • Example: verbs, nouns adjectives – embrace, pity embraceable, pitiable – care, wit careless, witless
- 13. 13 • Example: adjective adverb – happy happily • More complicated to model than inflection – Less productive: *science-less, *concern-less, *go-able, *sleep-able – Meanings of derived terms harder to predict by rule • clueless, careless, nerveless
- 14. 14 Derivational Examples • Verb/Adj to Noun -ation computerize computerization -ee appoint appointee -er kill killer -ness fuzzy fuzziness
- 15. 15 Derivational Examples • Noun/Verb to Adj -al Computation Computational -able Embrace Embraceable -less Clue Clueless
- 16. 16 Compute • Many paths are possible… • Start with compute – Computer -> computerize -> computerization – Computation -> computational – Computer -> computerize -> computerizable – Compute -> computee
- 17. 17 How do people represent words? • Hypotheses: – Full listing hypothesis: words listed – Minimum redundancy hypothesis: morphemes listed • Experimental evidence: – Priming experiments (Does seeing/hearing one word facilitate recognition of another?) suggest neither • Regularly inflected forms prime stem but not derived forms • But spoken derived words can prime stems if they are semantically close (e.g. government/govern but not department/depart)
- 18. 18 • Speech errors suggest affixes must be represented separately in the mental lexicon – easy enoughly
- 19. 19 Parsing • Taking a surface input and identifying its components and underlying structure • Morphological parsing: parsing a word into stem and affixes and identifying the parts and their relationships – Stem and features: • goose goose +N +SG or goose +V • geese goose +N +PL • gooses goose +V +3SG – Bracketing: indecipherable [in [[de [cipher]] able]]
- 20. 20 Why parse words? • For spell-checking – Is muncheble a legal word? • To identify a word’s part-of-speech (pos) – For sentence parsing, for machine translation, … • To identify a word’s stem – For information retrieval
- 21. 21 What do we need to build a morphological parser? • Lexicon: stems and affixes (w/ corresponding pos) • Morphotactics of the language: model of how morphemes can be affixed to a stem. E.g., plural morpheme follows noun in English • Orthographic rules: spelling modifications that occur when affixation occurs – in il in context of l (in- + legal)
- 22. 22 Morphotactic Models • English nominal inflection q0 q2 q1 plural (-s) reg-n irreg-sg-n irreg-pl-n •Inputs: cats, goose, geese
- 23. 23 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 24. 24 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 25. 25 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 26. 26 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 27. 27 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 28. 28 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 29. 29 • Derivational morphology: adjective fragment q5 q0 q1 q2 un- adj-root -er, -ly, -est • Adj-root: clear, happy, real, big, red
- 30. 30 • Derivational morphology: adjective fragment q5 q0 q1 q2 un- adj-root -er, -ly, -est • Adj-root: clear, happy, real, big, red •BUT: unbig, redly, realest
- 31. 31 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 32. 32 Antworth data on English Adjectives • Big, bigger, biggest • Cool, cooler, coolest, cooly • Red, redder, reddest • Clear, clearer, clearest, clearly, unclear, unclearly • Happy, happier, happiest, happily • Unhappy, unhappier, unhappiest, unhappily • Real, unreal, really
- 33. 33 • Derivational morphology: adjective fragment q3 q5 q4 q0 q1 q2 un- adj-root1 -er, -ly, -est adj-root1 adj-root2 -er, -est • Adj-root1: clear, happy, real • Adj-root2: big, red
- 34. 34 FSAs and the Lexicon • First we’ll capture the morphotactics – The rules governing the ordering of affixes in a language. • Then we’ll add in the actual words
- 35. 35 Using FSAs to Represent the Lexicon and Do Morphological Recognition • Lexicon: We can expand each non-terminal in our NFSA into each stem in its class (e.g. adj_root2 = {big, red}) and expand each such stem to the letters it includes (e.g. red r e d, big b i g) q0 q1 ε r e q2 q4 q3 -er, -est d b g q5 q6 i q7
- 36. 36 Limitations • To cover all of e.g. English will require very large FSAs with consequent search problems – Adding new items to the lexicon means recomputing the FSA – Non-determinism • FSAs can only tell us whether a word is in the language or not – what if we want to know more? – What is the stem? – What are the affixes and what sort are they? – We used this information to build our FSA: can we get it back?
- 37. 37 Parsing/Generation vs. Recognition • Recognition is usually not quite what we need. – Usually if we find some string in the language we need to find the structure in it (parsing) – Or we have some structure and we want to produce a surface form (production/generation) • Example – From “cats” to “cat +N +PL”
- 38. 38 Finite State Transducers • The simple story – Add another tape – Add extra symbols to the transitions – On one tape we read “cats”, on the other we write “cat +N +PL”
- 39. 39 Parsing with Finite State Transducers • cats cat +N +PL • Kimmo Koskenniemi’s two-level morphology – Words represented as correspondences between lexical level (the morphemes) and surface level (the orthographic word) – Morphological parsing :building mappings between the lexical and surface levels c a t +N +PL c a t s
- 40. 40 Finite State Transducers • FSTs map between one set of symbols and another using an FSA whose alphabet is composed of pairs of symbols from input and output alphabets • In general, FSTs can be used for – Translator (Hello:Ciao) – Parser/generator (Hello:How may I help you?) – To map between the lexical and surface levels of Kimmo’s 2-level morphology
- 41. 41 • FST is a 5-tuple consisting of – Q: set of states {q0,q1,q2,q3,q4} – : an alphabet of complex symbols, each an i/o pair s.t. i I (an input alphabet) and o O (an output alphabet) and is in I x O – q0: a start state – F: a set of final states in Q {q4} – (q,i:o): a transition function mapping Q x to Q – Emphatic Sheep Quizzical Cow q0 q4 q1 q2 q3 b:m a:o a:o a:o !:?
- 42. 42 Transitions • c:c means read a c on one tape and write a c on the other • +N:ε means read a +N symbol on one tape and write nothing on the other • +PL:s means read +PL and write an s c:c a:a t:t +N:ε +PL:s
- 43. 43 FST for a 2-level Lexicon • E.g. Reg-n Irreg-pl-n Irreg-sg-n c a t g o:e o:e s e g o o s e q0 q1 q2 q3 c a t q1 q3 q4 q2 s e:o e:o e q0 q5 g
- 44. 44 FST for English Nominal Inflection q0 q7 +PL:^s# Combining (cascade or composition) this FSA with FSAs for each noun type replaces e.g. reg- n with every regular noun representation in the lexicon q1 q4 q2 q5 q3 q6 reg-n irreg-n-sg irreg-n-pl +N: +PL:# +SG:# +SG:# +N: +N:
- 45. 45 The Gory Details • Of course, its not as easy as – “cat +N +PL” <-> “cats” • Or even dealing with the irregulars geese, mice and oxen • But there are also a whole host of spelling/pronunciation changes that go along with inflectional changes
- 46. 46 Multi-Tape Machines • To deal with this we can simply add more tapes and use the output of one tape machine as the input to the next • So to handle irregular spelling changes we’ll add intermediate tapes with intermediate symbols
- 47. 47 Multi-Level Tape Machines • We use one machine to transduce between the lexical and the intermediate level, and another to handle the spelling changes to the surface tape
- 48. 48 Orthographic Rules and FSTs • Define additional FSTs to implement rules such as consonant doubling (beg begging), ‘e’ deletion (make making), ‘e’ insertion (watch watches), etc. Lexical f o x +N +PL Intermediate f o x ^ s # Surface f o x e s
- 49. 49 Lexical to Intermediate Level
- 50. 50 Intermediate to Surface • The add an “e” rule as in fox^s# <-> foxes
- 51. 51 Note • A key feature of this machine is that it doesn’t do anything to inputs to which it doesn’t apply. • Meaning that they are written out unchanged to the output tape.
- 52. 52
- 53. 53
- 54. 54 • Note: These FSTs can be used for generation as well as recognition by simply exchanging the input and output alphabets (e.g. ^s#:+PL)
- 55. 55 Summing Up • FSTs provide a useful tool for implementing a standard model of morphological analysis, Kimmo’s two-level morphology – Key is to provide an FST for each of multiple levels of representation and then to combine those FSTs using a variety of operators (cf AT&T FSM Toolkit) – Other (older) approaches are still widely used, e.g. the rule- based Porter Stemmer