Data analytics with python introductory
- 2. Computational Problems The practice of engineering is applying science to solve a problem. There are 2 kinds of problems: Deterministic: These are the set of problems which have a pre-defined set of steps which solve them, every time. E.g software written to dispense currency from an ATM machine. Whatever be the case, the software executes the same steps everytime to dispense the currency. Non-Deterministic: There are many problems where the solution is not deterministic. This is because either we don’t know enough about the problem or we don’t have enough computing power to model the problem. E.g how to classify whether a mail is spam or not. There is no single formula to determine a spam mail. It depends on the occurrence of certain words used together, length of email and other factors. Another example can be how to measure the happiness of humans. The solution to this problem will differ greatly from 1 person to another. For such cases, STATISTICS will come into play. Now, we can approach to solve Non-deterministic problems by using a pre-defined set of rules but it will not work for all the cases. You can define few rules to classify a mail as SPAM or HAM and it may work on a given set of mails but a new mail may arrive which may not follow the rules. In this case, you will have to modify the rules again. Machine Learning is an approach which uses data to identify patterns(learning) and solves the problem based on this learning. As new data comes in, the machine learning algorithm adjusts itself based on the data and start giving out results as per the new learning.
- 3. Jargons…… Statistics is just about the numbers, and quantifying the data. There are many tools for finding relevant properties of the data but this is pretty close to pure mathematics. Data Mining is about using Statistics as well as other programming methods to find patterns hidden in the data so that you can explain some phenomenon. Data Mining builds intuition about what is really happening in some data and is still little more towards math than programming, but uses both. Machine Learning uses Data Mining techniques and other learning algorithms to build models of what is happening behind some data so that it can predict future outcomes. It’s a particular approach to AI. Deep Learning is one type of Machine Learning that achieves great power and flexibility by learning to represent the world as nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones Artificial Intelligence uses models built by Machine Learning and other ways to reason about the world and give rise to intelligent behavior whether this is playing a game or driving a robot/car. Artificial Intelligence has some goal to achieve by predicting how actions will affect the model of the world and chooses the actions that will best achieve that goal. Very programming based.
- 4. Jargons…… Artificial Intelligence NarrowAI General Intelligence Machine Learning Data Mining NLP Deep Learning Image Processing Voice Recognition
- 5. Machine Learning Machine Learning is the name given to generalizable algorithms that enable a computer to carry out a task by examining data rather than hard programming. Its a subfield of computer science and artificial intelligence that focuses on developing systems that learn from data and help in making decisions and predictions based on that learning. ML enables computers to make data-driven decisions rather than being explicitly programmed to carry out a certain task. Math provides models; understand their relationships and apply them to real-world objects.
- 6. Types of Machine Learning a. Supervised Learning: These are “predictive” in nature. The purpose is to predict the value of a particular variable(target variable) based on values of some other variables(independent or explanatory variables). Classification and Regression are examples of predictive tasks. Classification is used to predict the value of a discrete target variable while regression is used to predict the value of a continuous target variable. To predict whether an email is spam or not is a Classification task while to predict the future price of a stock is a regression task. They are called supervised because we are telling the algorithm what to predict. b. Unsupervised Learning: These are “descriptive” in nature. The purpose is to derive patterns that summarize the underlying relationships in data. Association Analysis, Cluster Analysis and Anomaly detection are examples of Unsupervised Learning. They are called unsupervised because in such cases, the final outcome is not known beforehand. With unsupervised learning there is no feedback based on the prediction results. c. Reinforcement learning: Where evaluations are given about how good or bad a certain situation is: Examples include types of ML that enable computers to learn to play games or drive vehicles
- 7. Types of Machine Learning
- 9. ML – Technical Areas ML Process ML Technical Architecture Required Skills
- 10. ML – Process
- 11. ML – Effort Distribution
- 12. ML – Life Cycle
- 15. Skills Required
- 16. About the Program… Machine Learning Foundations: Mathematics and Science behind Machine Learning Functions and Graphs Statistics and its Applications Introduction to Probability Theory
- 17. About the Program… Machine Learning: Getting Started with Machine Learning • What is Machine Learning – Examples and Applications • Numpy and Pandas Tutorial • Scikit Learn Tutorial • Introduction to Model Evaluation and Validation • Training and Testing • Metrics for Evaluation • 2 Mini-Projects to understand and implement Machine Learning Basics
- 18. About the Program… Supervised Learning • Introduction to Supervised Learning • Linear Regression • Logistic Regression • Decision Trees • Random Forests • Naïve Bayes Classifier • Bayesian Statistics and Inference • K-Nearest Neighbor • Introduction to Neural Networks • Introduction to Natural language Processing • Mini Project to apply Supervised Learning Algorithms
- 19. About the Program… Unsupervised Learning • Introduction to Unsupervised Learning • K-Means Clustering • Hierarchal Clustering • Clustering using DBSCAN • Clustering Mini-Project • Feature Selection • Principal Components Analysis (PCA) • Feature Transformations Reinforcement Learning • Introduction to Reinforcement Learning • Markov decision Processes • Game Theory Fundamentals • Mini Project to implement Reinforcement Learning
- 20. About the Program… Deep Learning • Introduction to Deep Learning • Deep Learning tools • TensorFlow • Deep Neural networks • Convolutional Neural Networks • Neural network Mini-Project Introduction to Kaggle Platform and other Data Science Competitions Industry Project: This will be a industry-specific project to solve a real-world problem using different Machine Learning techniques learned in the overall course.