3171617_introduction_applied machine learning.pptx
- 1. APPLIED MACHINE LEARNING (3171617 ) Prof. Anita Parmar
- 5. CONTENT Human learning and it’s types Machine learning and it’s types Applications of machine learning Issues in machine learning Tools in machine learning
- 6. WHAT IS HUMAN LEARNING Learning is typically referred to as the process of gaining information through observation. With more learning, tasks can be performed more efficiently. Ex. More homework –less mistakes. Types of human learning Learning under expert guidance Learning guided by knowledge gained from experts Learning by self
- 7. Learning under expert guidance • Somebody who is an expert in the subject directly teaches us • Example The baby ‘learns’ things from his parents. He calls his hand, a ‘hand’, because that is the information he gets from his parents. Learns how to form words from the alphabets and numbers from the digits The professional mentors, by virtue of the knowledge that they have gained through years of hands-on experience, help all new comers in the field to learn on-job. So guided learning is the process of gaining information from a person having sufficient knowledge due to the past experience.
- 8. Learning guided by knowledge gained from experts We build our own notion indirectly based on what we have learnt from the expert in the past Learning also happens with the knowledge which has been imparted by teacher or mentor at some point of time in some other form/context For example A baby can group together all objects of same colour even if his parents have not specifically taught him to do so. Kid can select one odd word from a set of words. There is no direct learning. It is some past information shared on some different context, which is used as a learning to make decisions.
- 9. LEARNING BY SELF We do it ourselves, may be after multiple attempts, some being unsuccessful. For example, A baby learning to walk through obstacles. He bumps on to obstacles and falls down multiple times till he learns that whenever there is an obstacle, he needs to cross over it. Learning to ride a cycle as a kid or drive a car as an adult. A lot of things need to be learnt only from mistakes made in the past
- 10. WHAT IS MACHINE LEARNING? Tom M. Mitchell has defined machine learning as ‘A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.’
- 11. Learning to play chess, E represents the experience of playing the game, T represents the task of playing chess and P is the performance measure indicated by the percentage of games won by the player. Image classification, E represents the past data with images having labels or assigned classes (for example whether the image is of a class cat or a class dog or a class elephant etc.), T is the task of assigning class to new, unlabelled images and P is the performance measure indicated by the percentage of images correctly classified. Spam mail filtering ?
- 12. HOW DO MACHINES LEARN? The basic machine learning process can be divided into three parts. 1. Data input: past data or information is utilized as a basis for future decision-making 2. Abstraction: the input data is represented in a broader way through the underlying algorithm 3. Generalization: the abstracted representation is generalized to form a framework for making decisions
- 14. For example, a broad pool of knowledge may consist of all living animals and their characteristics such as whether they live in land or water, whether they lay eggs etc. It is a difficult task for any student to memorize the characteristics of all living animals – no matter how much photographic memory he/she may possess. It is better to draw a notion about the basic groups that all living animals belong to and the characteristics which define each of the basic groups.
- 15. ABSTRACTION During the machine learning process, knowledge is fed in the form of input data. However, the data cannot be used in the original shape and form. abstraction helps in deriving a conceptual map based on the input data. This map, or a model is summarized knowledge representation of the raw data. The model may be in any one of the following forms: Computational blocks like if/else rules Mathematical equations Specific data structures like trees or graphs Logical groupings of similar observations
- 16. The choice of model is taken based on multiple aspects, some of which are listed below: The type of problem to be solved: Whether the problem is related to forecast or prediction, analysis of trend, understanding the different segments or groups of objects, etc. Nature of the input data: whether the data has no values for many fields, the data types, etc. Domain of the problem: e.g. fraud detection problem in banking domain. Once the model is chosen, the next task is to fit the model based on the input data. In a case where the model is represented by a mathematical equation, say y = c1 + c2 x we have to find out the values of c1 and c2 .
- 17. GENERALIZATION To tune up the abstracted knowledge to a form which can be used to take future decisions.
- 18. DEFINING A NEW PROBLEM 1. What is the problem? 2. Why does the problem need to be solved? 3. How to solve the problem? Step 1 : Example : I need a program that will prompt the next word as and when I type a word. Use Tom Mitchell’s machine learning formalism stated above to define the T, P, and E for the problem. For example: Task (T): Prompt the next word when I type a word. Experience (E): A corpus of commonly used English words and phrases. Performance (P): The number of correct words prompted considered as a percentage (which in machine learning paradigm is known as learning accuracy).
- 19. Step 2: Why does the problem need to be solved? What is the motivation for solving the problem? What requirement will it fulfil? Consider the benefits of solving the problem. What capabilities does it enable? Step 3: How would I solve the problem? Try to explore how to solve the problem manually. Detail out step-by-step data collection, data preparation, and program design to solve the problem.
- 21. TYPES OF MACHINE LEARNING 1. Supervised learning – Also called predictive learning. A machine predicts the class of unknown objects based on prior cl ass-related information of similar objects. 2. Unsupervised learning – Also called descriptive learning. A machine finds patterns in unknown objects by grouping similar objects together. 3. Reinforcement learning A machine learns to act on its own to achieve the given goals.
- 24. SUPERVISED LEARNING supervised learning is to learn from past information. In context of the definition of machine learning, this past information is the experience.
- 25. WHAT IS CAT
- 28. Some examples of supervised learning are Predicting the results of a game Predicting whether a tumour is harmful or not Predicting the price of domains like real estate, stocks, etc. Classifying texts such as classifying a set of emails as spam or non-spam
- 30. CLASSIFICATION Assigning a label or category or class to a test data based on the label or category or class information that is imparted by the training data.
- 31. Some typical classification problems include: Image classification Prediction of disease Win–loss prediction of games Recognition of handwriting
- 32. CLASSIFICATION: APPLICATIONS Pattern recognition Face recognition: Character recognition: Speech recognition: Medical diagnosis: Biometrics:
- 33. FACE RECOGNITION Training examples of a person Test images
- 34. REGRESSION In linear regression, the objective is to predict numerical features like real estate or stock price, temperature, marks in an examination, sales revenue, etc. The underlying predictor variable and the target variable are continuous in nature. In case of linear regression, a straight line relationship is ‘fitted’ between the predictor variables and the target variables. Ex. Yearly budgeting exercise of the sales managers. They have to give sales prediction for the next year based on sales figure of previous years. A typical linear regression model can be represented in the form – y = α + β x where ‘x’ is the predictor variable and ‘y’ is the target variable
- 38. UNSUPERVISED LEARNING There is no labelled training data to learn from and no prediction to be made. The objective is to take a dataset as input and try to find natural groupings or patterns within the data elements or records. Therefore, unsupervised learning is often termed as descriptive model and the process of unsupervised learning is referred as pattern discovery or knowledge discovery.
- 39. CLUSTERING Clustering is the main type of unsupervised learning. It intends to group or organize similar objects together. Objective of clustering to discover the intrinsic grouping of unlabelled data and form clusters. Different measures of similarity can be applied for clustering. One of the most commonly adopted similarity measure is distance. Two data items are considered as a part of the same cluster if the distance between them is less. In the same way, if the distance between the data items is high, the items do not generally belong to the same cluster.
- 43. ASSOCIATION ANALYSIS. one more variant of unsupervised learning is association analysis. Example : market basket analysis From past transaction data in a grocery store, it may be observed that most of the customers who have bought item A, have also bought item B and item C or at least one of them. This means that there is a strong association of the event ‘purchase of item A’ with the event purchase of item B’, or ‘purchase of item C’.
- 46. REINFORCEMENT LEARNING We have seen babies learn to walk without any prior knowledge of how to do it. Often we wonder how they really do it. The action tried to be achieved is walking, the child is the agent and the place with hurdles on which the child is trying to walk resembles the environment. It tries to improve its performance of doing the task. When a sub-task is accomplished successfully, a reward is given. When a sub-task is not executed correctly, obviously no reward is given. This continues till the machine is able to complete execution of the whole task. This process of learning is known as reinforcement learning.
- 48. An other example of reinforcement learning is self- driving cars. The critical information which it needs to take care of are speed and speed limit in different road segments, traffic conditions, road conditions, weather conditions, etc. The tasks that have to be taken care of are start/stop, accelerate/decelerate, turn to left / right, etc.
- 50. PROBLES NOT TO BE SOLVED USING MACHINE LEARNING Machine learning should not be applied to tasks in which humans are very effective or frequent human intervention is needed. For example, air traffic control is a very complex task needing intense human involvement At the same time, for very simple tasks which can be implemented using traditional programming paradigms, there is no sense of using machine learning. For example, simple rule-driven or formula-based applications like price calculator engine
- 51. APPLICATIONS OF MACHINE LEARNING Banking and finance In the banking industry, fraudulent transactions, especially the ones related to credit cards, are extremely prevalent. Since the volumes as well as velocity of the transactions are extremely high, high performance machine learning solutions are implemented by almost all leading banks across the globe. The models work on a real-time basis, i.e. the fraudulent transactions are spotted and prevented right at the time of occurrence. Customers of a bank are often offered lucrative proposals by other competitor banks. Proposals like higher bank interest, lower processing charge of loans, zero balance savings accounts, etc. are offered to customers, with the intent that the customer switches over to the competitor bank. Machine learning helps in preventing or at least reducing the customer churn. Using descriptive learning, the specific pockets of problem, i.e. a specific bank or a specific zone or a specific type of offering like car loan, may be spotted where maximum churn is happening. Using predictive learning, the set of vulnerable customers who may leave the bank very soon, can be identified.
- 52. Insurance Two major areas in the insurance industry where machine learning is used are risk prediction during new customer onboarding and claims management. During customer onboarding, based on the past information the risk profile of a new customer needs to be predicted. When a customer claim comes for settlement, past information related to historic claims along with the adjustor notes are considered to predict whether there is any possibility of the claim to be fraudulent.
- 53. Healthcare Wearable device data form a rich source for applying machine learning and predict the health conditions of the person real time. In case there is some health issue which is predicted by the learning model, immediately the person is alerted to take preventive action. Suppose an elderly person goes for a morning walk. Suddenly, while walking, his blood pressure shoots up beyond a certain limit, which is tracked by the wearable. The wearable data is sent to a remote server and a machine learning algorithm is constantly analyzing the streaming data. It also has the history of the elderly person and persons of similar age group. The model predicts some fatality unless immediate action is taken. Alert can be sent to the person to immediately stop walking and take rest.
- 54. TOOLS IN MACHINE LEARNING Python Python is one of the most popular, open source programming language widely adopted by machine learning community. there is a machine learning library named scikitlearn, which has various classification, regression, and clustering algorithms embedded in it.
- 55. R R is a language for statistical computing and data analysis. It is an open source language, extremely popular in the academic community – especially among statisticians and data miners. R is a very simple programming language with a huge set of libraries available for different stages of machine learning. Some of the libraries standing out in terms of popularity are plyr/dplyr (for data transformation), caret (‘Classification and Regression Training’ for classification), RJava (to facilitate integration with Java), tm (for text mining), ggplot2 (for data visualization).
- 56. Matlab MATLAB (matrix laboratory) is a licenced commercial software with a robust support for a wide range of numerical computing. MATLAB also provides extensive support of statistical functions and has a huge number of machine learning algorithms in-built. SAS SAS (earlier known as ‘Statistical Analysis System’) is another licenced commercial software which provides strong support for machine learning functionalities.
- 57. ISSUE IN MACHINE LEARNING Privacy The biggest fear and issue arising out of machine learning is related to privacy and the breach of it. The primary focus of learning is on analyzing data, both past and current, and coming up with insight from the data. This insight may be related to people and the facts revealed might be private enough to be kept confidential.
- 58. SUMMARY Machine learning imbibes the philosophy of human learning, i.e. learning from expert guidance and from experience. The basic machine learning process can be divided into three parts. Data Input Abstraction Generalization. Machine learning can be classified into three broad categories: Supervised learning Unsupervised learning Reinforcement learning Machine learning has been adopted by various industry domains such as Banking and Financial Services, Insurance, Healthcare, etc. to solve problems. Some of the most adopted platforms to implement machine learning include Python, R, MATLAB, SAS, etc. To avoid ethical issues, the critical consideration is required before applying machine learning and using any outcome from machine learning.