![What is NLP?
Natural Language Generation: Conversation/Dialogue
context
because of your game ?
message
yeah i’m on my
way now
response
ok good luck !
[Sordoni et al., 2015]](https://image.slidesharecdn.com/nlppresentation-210918032003/85/Nlp-presentation-9-320.jpg)
![What is NLP?
Natural Language Generation: Image Captioning
[UT
oronto]](https://image.slidesharecdn.com/nlppresentation-210918032003/85/Nlp-presentation-10-320.jpg)
![What is NLP?
Natural Language Generation: Visual QuestionAnswering
Does it appear to be rainy?
Does this person have 20/20 vision?
Is this person expecting company?
What is just under the tree?
How many slices of pizza are there?
Is this a vegetarian pizza?
What color are her eyes?
What is the mustache made of?
[Antol et al., 2015]](https://image.slidesharecdn.com/nlppresentation-210918032003/85/Nlp-presentation-11-320.jpg)
Nlp presentation
- 1. Trends & Research applications in Natural Language Processing Surya SG (with some vision, robotics, and deep learning)
- 2. About the Course (and its Goals) Research-oriented webinar course! We will understand lots of interesting NLP tasks & some fun novel projects! We’ll start with some basics of NLP. Then cover some specific, latest research topics via several research works. E.g., we will discuss connections of NLP with vision and robotics, and several deep learning for NLP models. No NLP background needed.
- 3. What is NLP? Question answering
- 4. What is NLP? Question answering
- 5. What is NLP? Question answering
- 8. What is NLP? Natural Language Generation: Summarization - Lohan charged with the0 of $2,500 necklace - Pleaded not guilty - Judge set bail at $40,000 - To reappear in court on Feb 23
- 9. What is NLP? Natural Language Generation: Conversation/Dialogue context because of your game ? message yeah i’m on my way now response ok good luck ! [Sordoni et al., 2015]
- 10. What is NLP? Natural Language Generation: Image Captioning [UT oronto]
- 11. What is NLP? Natural Language Generation: Visual QuestionAnswering Does it appear to be rainy? Does this person have 20/20 vision? Is this person expecting company? What is just under the tree? How many slices of pizza are there? Is this a vegetarian pizza? What color are her eyes? What is the mustache made of? [Antol et al., 2015]
- 12. What is NLP? Automatic Speech Recognition
- 13. Some Exciting NLP Challenges 1) Human-like Language Understanding: metaphors/idioms, humor, sarcasm, politeness/rudeness 1) Language Generation and Dialogue: document summarization, database to language summary, coherent and intelligent conversation models 1) Grounded Language with Vision and Speech: image-text alignment, language disambiguation via images, image/video captioning, image/video question answering, text to image generation, visual story entailment 1) Embodied Language for Robotic Tasks: instructions for navigation, articulation, manipulation, skill learning 1) Machine Learning Models: deep+structured models, interpretable models, adversarial models, reward-based models (reinforcement learning)
- 14. Human-likeAmbiguousLanguage You: Siri: I am under the weather today. Here is the weather today… 50 F Non-literal: Idioms, Metaphors
- 15. Human-likeAmbiguousLanguage Break a leg! Non-literal: Idioms, Metaphors
- 16. Human-likeAmbiguousLanguage I bet I can stop gambling! Yeah, right! Please do not … Humor, Sarcasm, Politeness/Rudeness
- 17. Human-likeAmbiguousLanguage Clean the dishes in the sink. Prepositional Attachment, Coreference Ambiguities
- 19. Visually Grounded Language Get the mug on the table with black stripes. Text-Image Alignment: Most of our daily communication language points to several objects in the visual world
- 20. Visually Grounded Language Visual Question Answering: Humans asking machines about pictures/videos, e.g., for visually impaired, in remote/ dangerous scenarios, in household service settings Is there milk in the refrigerator?
- 21. Embodied Language (Robot Instructions) Turn right at the butterfly painting, then go to the end of the hall Task-based instructions, e.g., navigation, grasping, manipulation, skill learning
- 22. Embodied Language (Robot Instructions) Cut some onions, and add to broth, stir it Task-based instructions, e.g., navigation, grasping, manipulation, skill learning
- 23. Grounded Language Generation/Dialogue Both for answering human questions, and to ask questions back, and for casual chit-chat What food is in the refrigerator? Apples and oranges
- 24. Grounded Language Generation/Dialogue Both for answering human questions, and to ask questions back, and for casual chit-chat Crack the window! You mean open it or break it?
- 25. Language Technologies Goal: Deep Understanding Requires context, linguistic structure, meanings… Reality: Shallow Matching Requires robustness and scale Amazing successes, but fundamental limitations
- 26. What is Nearby NLP? Computational Linguistics Using computational methods to learn more about how language works Weend up doing this and using it Cognitive Science Figuring out how the human brain works Includes the bits that do language Humans: the only working NLP prototype! Speech Processing Mapping audio signals to text Traditionally separate from NLP, converging? Two components: acoustic models and language models Language models in the domain of stat NLP
- 27. Opportunities andChallenges in Working with Low-Resource Languages Polyglot AI Part - I
- 28. Key Takeaways 1. To build a Polyglot Enterprise-AI we need techniques that address not only the languages of human but also the language of the enterprise 1. You don’t have to solve language of the humans to perfection to begin to address the language of the enterprise. No shame in shooting for the Narrow AI Vs. Broad AI !! 1. Depending on the availability of labeled data, transparency and explain-ability requirements different techniques may have to be applied to build Polyglot Enterprise-AI
- 94. Some Tips for fellow aspiring researchers A new researcher should be expected to score very low on most of these criteria while one about to graduate should get very high scores on almost all of them. • ability to build evaluation pipelines and perform evaluations for new tasks • ability to locate and read the relevant papers on a "new" problem • ability to come up with "easy" and "reasonable" baselines • ability to find, download, install, and run existing software from third parties • familiarity with machine learning, graph theory, linear algebra, calculus, combinatorics, statistics, and text processing • understanding of linguistic phenomena and annotation • understanding the variability of human judgments • ability to write good narratives of experiments • ability to write good overviews of existing research • ability to develop and give presentations • ability to discuss research with other team members • ability to see a problem or an approach from a very broad perspective
- 95. Some Tips for fellow aspiring researchers • ability to assess the feasibility of a problem or approach • ability to plan a research project and execute it over time • intuition to try alternative methods • understanding of the relative advantages and drawbacks of general methods across problems • ability to implement in code generic algorithms and to make appropriate modifications to them • understanding of related sciences such as bioinformatics, artificial intelligence, etc. • understanding of computational complexity • understanding of the fundamental data structures and algorithms • familiarity with the availability on the Web of relevant corpora, papers, and tools • excellent understanding of UNIX, including process control, scripting, and backup
- 96. Some Tips for fellow aspiring researchers • ability to build web-based and local demonstration systems • ability to describe one's research to others with different levels of overlap in backgrounds with the student's • understanding of project management: CVS, documentation, modularization, portability of code • knowledge of a number of programming languages: C/C++, Java, perl/python,matlab • ability to plan one's time, esp. wrt. courses, travel, committees • ability to read a paper and abstract its main points - both strengths and weaknesses • ability to draw charts, diagrams, screen snapshots, and other illustrations for papers • ability to write quick scripts to convert data from one format to another • ability to write quick scripts to test existing libraries or external software • ability to write quick scripts to evaluate experiments • ability to teach the introductory class, as well as plan it and grade it
- 97. Some Tips for fellow aspiring researchers • ability to relate one's work to similar problems in related research areas • ability to store and retrieve data in a database systems • ability to write interfaces to existing resources: both local and Web-based • ability to network with colleagues • ability to promote oneself • ability to organize events: colloquia, external visits, etc. • ability to build an end to end system • ability to take initiative and to propose new projects • ability to write proposals for funding • ability to elicit assistance from advisers, fellow students, and others • ability to ask intelligent questions at talks • ability to design and perform user studies • ability to request and obtain IRB support for user studies
- 98. Some Tips for fellow aspiring researchers • knowledge of a range of research methods, and an ability to read and give feedback on colleagues' work (that is not necessarily in my own area of interest) • ability to initiate collaboration with others • knowledge of people from whom he or she can ask and receive helpful feedback on my work • knowledge of research communities in which to become an active member, get good feedback on his or her work and get exposure of his or her work to others. • awareness of his or her key strengths as a researcher and future teacher (for people with academic career aspirations) • learn how to emphasize his or her strengths and use them to have impact.
- 99. References • Taylor Berg-‐Kirkpatrick – CMU - Slides: Dan Klein – UC Berkeley • COMP 790.139 (Fall 2016) - Mohit Bansal • Yulia Tsvetkov - http://www.cs.cmu.edu/~ytsvetko/ • http://www.cs.yale.edu/homes/radev/posts/phdskills.txt