[부스트캠프 Tech Talk] 안영진_Tackling Complexity with Easy Stuff
0 likes108 views
The document discusses strategies for learning deep learning without getting bored or fatigued. It recommends finding manageable projects by breaking down complex problems into smaller, more feasible sub-tasks. This involves searching GitHub efficiently by following smart contributors, starring interesting repositories, and using browser extensions. It also suggests discussing self-directed projects with peers and establishing metrics to standardize performance evaluations. By starting with simpler ideas and gradually increasing complexity, deep learning concepts can be tackled effectively in the long run.
![[부스트캠프 Tech Talk] 안영진_Tackling Complexity with Easy Stuff](https://image.slidesharecdn.com/boostcampaitechtechtalkahnyoungjin-211210113216/85/Tech-Talk-_Tackling-Complexity-with-Easy-Stuff-6-320.jpg)
![[부스트캠프 Tech Talk] 안영진_Tackling Complexity with Easy Stuff](https://image.slidesharecdn.com/boostcampaitechtechtalkahnyoungjin-211210113216/85/Tech-Talk-_Tackling-Complexity-with-Easy-Stuff-13-320.jpg)
![[부스트캠프 Tech Talk] 안영진_Tackling Complexity with Easy Stuff](https://image.slidesharecdn.com/boostcampaitechtechtalkahnyoungjin-211210113216/85/Tech-Talk-_Tackling-Complexity-with-Easy-Stuff-18-320.jpg)
![[부스트캠프 Tech Talk] 안영진_Tackling Complexity with Easy Stuff](https://image.slidesharecdn.com/boostcampaitechtechtalkahnyoungjin-211210113216/85/Tech-Talk-_Tackling-Complexity-with-Easy-Stuff-32-320.jpg)
![Simplicity to Complexity
42
https://github.com/lucidrains/DALLE-pytorch
Tested feasibility of text-to-image generation pipeline with simple project.
32 x 32 geometric shapes inputted to
[Discrete Variational Autoencoders & Byte Pair Text Encoder → DALLE]](https://image.slidesharecdn.com/boostcampaitechtechtalkahnyoungjin-211210113216/85/Tech-Talk-_Tackling-Complexity-with-Easy-Stuff-42-320.jpg)
[부스트캠프 Tech Talk] 안영진_Tackling Complexity with Easy Stuff
- 1. Tackling Complexity with Easy Stuff #Learning_Style #Searching_Tips #Development_Log #Boredom_and_Want Ahn, Young Jin
- 2. Executive Summary Why • Deep Learning can find its application in various fields. • But at the same time, Deep Learning requires both in-depth knowledge and sensitivity for the trend. • Boredom and fatigue are the biggest risk for learning Deep Learning in long term. What • Tips for searching without fatigue. • Project Based Learning by doing manageable things. • Breaking down intricated problems into smaller parts. How • Follow smart people on Github. • Star impressive repositories and cluster them later. • Use Chrome extensions effectively on Github. • Find feasible projects related to the course work. • Make your own project, discuss about it with your friendly peers. • Standardize metrics for the performance. • When checking feasibility, break down big problems into small sub-tasks. Tackling Complexity with Easy Stuff
- 3. Boredom is Risk for Learning 3 Arthur Schopenhauer "Want and boredom are indeed the twin poles of human life."
- 4. Boredom is Risk for Learning Setting upper bound does help to reduce the fatigue. Times for programming, Sitting on desk, Listening to lectures… 4
- 5. Table of Contents 5 Table of Contents 01 Self Introduction 02 Tips for Searching 03 Project Based Learning 04 Simplicity to Complexity
- 7. Self Introduction “Inventor" ENTP (Myers-Briggs) Social & Investigative (RIASEC Holland Codes) 7
- 8. 8
- 9. 9
- 10. 10
- 11. 11
- 12. 12
- 14. Tips for Searching 14 Following smart people are informative as Facebook group.
- 15. Tips for Searching 15 Starring(⭐) project that intrigues you helps. https://github.com/lorey/github-stars-by-topic
- 16. 16 https://github.com/lorey/github-stars-by-topic Clustering Starred Topics. • Fetching Github stars with PyGithub API • Clustering topics with TFIDF and NMF
- 17. 17 Useful Github chrome extensions are also available. Tips for Searching
- 19. 19 Identifying 3 different types of classes from images didn’t seem to be easy. https://github.com/note823/boostcamp_ai_tech Project Based Learning
- 20. 20 https://github.com/snoop2head/DeepLearningProject Visually seeing the result from end to end mattered. Practicing with easier classification task helped. Project Based Learning
- 21. Provided code with replaceability, like fitting lego blocks. 21
- 22. Simplicity to Complexity 22 Simple classification using pretrained Resnet50 provided clues for variation.
- 23. Project Based Learning 23 Understanding BERT’s architecture was difficult.
- 24. Project Based Learning 24 Masked Language Model reminded me of something that all Korean students have encountered.
- 25. Project Based Learning 25 https://github.com/snoop2head/machine-suneung Using pretrained BERT model to solve Korean Suneung was good project to grasp the usage of BERT model.
- 26. Project Based Learning 26 https://github.com/om00839/machine-suneung Even without fine-tuning, using huggingface pipeline yielded higher accuracy than random choice.
- 27. 27 https://github.com/snoop2head/machine-suneung Using huggingface pipeline, it became the task of finding the most similar choice.
- 28. 28 However, underperforming human prediction was debatable. Questions from the peers were points to focus on. @waterglasstoon
- 29. 29 https://github.com/boostcampaitech2-happyface/clipboards-nlp Project codes across books and github repos were collected.
- 30. 30 https://github.com/boostcampaitech2-happyface/clipboards-nlp Project codes included Word-in-Context, Corpus of Linguistic Acceptability task finetuning methods.
- 31. 31 These BERT projects in turn helped me to write RBERT dataset code.
- 33. Simplicity to Complexity 33 https://github.com/snoop2head/elastic-stock-prediction Submitting multiple times and praying for good result yields ambiguous result. Setting criteria before the submission mattered.
- 34. 34 https://mccormickml.com/2020/03/10/question-answering-with-a-fine-tuned-BERT/ MRC tasks’ accuracy can be broken down into two scores.
- 35. Simplicity to Complexity 35 https://snoop2head.github.io/Custom-MRC-Reader/ MRC Experimentation were customized to show the following criteria. 1. Start indexing accuracy 2. End indexing accuracy 3. Exact Match(EM) score 4. Evaluation Loss
- 36. Simplicity to Complexity 36 For the final project, teammates suggested sentence generation task for blog posts.
- 37. 37 https://github.com/snoop2head/KoGPT-Joong-2 Finetuning KoGPT-Trinity on Geulstagram dataset enabled teammates to anticipate the outcome of the generation.
- 38. 38 https://github.com/snoop2head/KoGPT-Joong-2 It handled acrostic poem generation as well.
- 39. 39 It was simple project with few lines of code. https://github.com/snoop2head/KoGPT-Joong-2
- 40. Simplicity to Complexity 40 Our team is currently working on Pokemon DALL-E, Model that generates Pokemon design given user’s description.
- 41. Simplicity to Complexity 41 https://github.com/NVlabs/stylegan2-ada-pytorch Working on text-to-image generation, the team was uncertain for the result from small dataset. Making sample generations enabled team to be optimistic for project outcome
- 42. Simplicity to Complexity 42 https://github.com/lucidrains/DALLE-pytorch Tested feasibility of text-to-image generation pipeline with simple project. 32 x 32 geometric shapes inputted to [Discrete Variational Autoencoders & Byte Pair Text Encoder → DALLE]
- 43. Simplicity to Complexity 43 https://github.com/lucidrains/DALLE-pytorch Above is the ground truth from test dataset (ex: Big Halftone Orange Triangle Rotated Counter-clockwise) Below is inferenced shape using dalle-pytorch.
- 44. Thank you 44
- 45. Executive Summary Why • Deep Learning can find its application in various fields. • But at the same time, Deep Learning requires both in-depth knowledge and sensitivity for the trend. • Boredom and fatigue are the biggest risk for learning Deep Learning in long term. What • Tips for searching without fatigue. • Project Based Learning by doing manageable things. • Breaking down intricated problems into smaller parts. How • Follow smart people on Github. • Star impressive repositories and cluster them later. • Use Chrome extensions effectively on Github. • Find feasible projects related to the course work. • Make your own project, discuss about it with your friendly peers. • Standardize metrics for the performance. • When checking feasibility, break down big problems into small sub-tasks. Tackling Complexity with Easy Stuff
- 46. Q&A 46