Introduction to Data Science Presentation
- 1. Introduction to Data Science Prepared by S.L.Swarna AP/AI&DS S.Santhiya AP/AI&DS EXCEL ENGINEERING COLLEGE
- 2. Data All Around • Data, Big Data and Challenges • Data Science – Introduction – Why Data Science • Data Scientists – What do they do? • Major/Concentration in Data Science – What courses to take.
- 3. Data All Around • Lots of data is being collected and warehoused – Web data, e-commerce – Financial transactions, bank/credit transactions – Online trading and purchasing – Social Network
- 4. How Much Data Do We have? • Google processes 20 PB a day (2008) • Facebook has 60 TB of daily logs • eBay has 6.5 PB of user data + 50 TB/day (5/2009) • 1000 genomes project: 200 TB
- 5. Types of Data We Have • Relational Data (Tables/Transaction/Legacy Data) • Text Data (Web) • Semi-structured Data (XML) • Graph Data • Social Network, Semantic Web (RDF), … • Streaming Data • You can afford to scan the data once
- 6. What is Data Science? • Data Science is about data gathering, analysis and decision-making. • Data Science is about finding patterns in data, through analysis, and make future predictions. • By using Data Science, companies are able to make: • Better decisions (should we choose A or B) • Predictive analysis (what will happen next?) • Pattern discoveries (find pattern, or maybe hidden information in the data)
- 7. Where is Data Science Needed? Examples of where Data Science is needed: • For route planning: To discover the best routes to ship • To foresee delays for flight/ship/train etc. (through predictive analysis) • To create promotional offers • To find the best suited time to deliver goods • To forecast the next years revenue for a company • To analyze health benefit of training • To predict who will win elections
- 8. How Does a Data Scientist Work? • A Data Scientist requires expertise in several backgrounds: • Machine Learning • Statistics • Programming (Python or R) • Mathematics • Databases • A Data Scientist must find patterns within the data. Before he/she can find the patterns, he/she must organize the data in a standard format.
- 9. Here is how a Data Scientist works: • Ask the right questions - To understand the business problem. • Explore and collect data - From database, web logs, customer feedback, etc. • Extract the data - Transform the data to a standardized format. • Clean the data - Remove erroneous values from the data. • Find and replace missing values - Check for missing values and replace them with a suitable value (e.g. an average value).
- 10. • Normalize data - Scale the values in a practical range (e.g. 140 cm is smaller than 1,8 m. However, the number 140 is larger than 1,8. - so scaling is important). • Analyze data, find patterns and make future predictions. • Represent the result - Present the result with useful insights in a way the "company" can understand. •