Regression analysis algorithm
- 1. 1 University of Technology Computer Science Department Regression Analysis UPrepared by SAMER A.QADER 2017
- 2. 2 Contents: 1.What is Regression Analysis? 2.Types of Regression Analysis? 3.Simple Linear Regression Analysis 4.Regression Equations model data 5.Example of Linear regression Analysis 6.References
- 3. 3 1. UWhat is Regression AnalysisU? The origin of the term "regression" to describe the process of fitting lines to data is rooted in a study of genetics by Sir Francis Galton in the late 19th century. He discovered that fathers who were extremely short or extremely tall tended to have sons whose heights were closer to the average height. He called this phenomenon "regression to the mean". Regression analysis is a form of predictive modeling technique which investigates the relationship between a dependent (target) and independent variable(s) (predictor). This technique is used for forecasting, time series modeling and finding the causal effect relationship between the variables. For example, relationship between rash driving and number of road accidents by a driver is best studied through regression. UBenefits 1. It indicates the strength of impact of multiple independent variables on a dependent variable. 2. It indicates the significant relationships between dependent variable and independent variable. 3. These help market researchers / data analysts / data scientists to eliminate and evaluate the best set of variables to be used for building predictive models
- 4. 4 2. UTypes of Regression Analysis Types of Regression analysis is generally classified into two Kinds: 1. Simple 2. Multiple. It involves only two variables: dependent variable, explanatory (independent) variable. A regression analysis may involve a linear model or a nonlinear model.
- 5. 5 3. Simple Linear regression model :- • Linear regression model is a model with a single regressor x that has a linear relationship with a Response y. • Linear regression can be defined as the line of the best fit You might recall from basic algebra that lines can be defined in a slope-intercept form similar to y = a + bx. In this form, the letter y indicates the dependent variable and x indicates the independent variable. The slope term b specifies how much the line rises for each increase in x. Positive values define lines that slope upward while negative values define lines that slope downward. The term a is known as the intercept because it specifies the point where the line crosses, or intercepts, the vertical y axis. It indicates the value of y when x = 0.
- 6. 6 4.Regression Equations model data :- . Intercept Slope Best fit Regression line Slope DependentVariables Independent Variables Intercept
- 7. 7 5. Example of Linear regression Suppose we have the following 5 data points of population (in hundred thousand) of a medium size city over 20 years (based on every 5 – year’s census) as shown in table 1 you want to predict the population in the year 2005. How you predict that population? Sol Step 1: First we calculate the slope: - So we need to find: • ∑ 𝒙𝒙 and ∑ 𝒚𝒚 . • (XY) and ∑ 𝒙𝒙𝒙𝒙 . • (X2 ) and ∑ 𝟐𝟐𝒙𝒙 . Year 1980 1985 1990 1995 2000 2005 population 2.1 2.9 3.2 4.1 9.4 ?
- 8. 8 x y Year Population XY X2 1980 2.1 4158 3920400 1985 2.9 5756.5 3940225 1990 3.2 6368 3960100 1995 4.1 8179.5 3980025 2000 9.4 9800 4000000 � 𝒙𝒙 � 𝒚𝒚 � 𝒙𝒙𝒙𝒙 � 𝟐𝟐 𝒙𝒙 9950 17.2 34262 19800750 N = 5 Data points Slope = 𝒏𝒏 ∑ 𝒙𝒙𝒙𝒙−∑ 𝒙𝒙 ∑ 𝒚𝒚 𝒏𝒏 ∑ 𝒙𝒙𝟐𝟐−(∑ 𝒙𝒙)𝟐𝟐 = 𝟓𝟓.(𝟑𝟑𝟑𝟑𝟑𝟑𝟑𝟑𝟑𝟑)−(𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗).(𝟏𝟏𝟏𝟏.𝟐𝟐) 𝟓𝟓.(𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏𝟏)−(𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗)𝟐𝟐 = 0.136 Step 2: We calculate the intercept: - So we need to find: • 𝒀𝒀� the average of years • 𝑿𝑿� the average of population 𝒀𝒀� = ∑ 𝒀𝒀 𝑵𝑵 = 𝟏𝟏𝟏𝟏.𝟐𝟐 𝟓𝟓 = 3.44 𝑿𝑿� = ∑ 𝑿𝑿 𝑵𝑵 = 𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗 𝟓𝟓 = 1990 intercept = 3.44 – (0.136). (1990) = 267.2
- 9. 9 Step3:- We find the linear Regression for x = 2005 y = 267.2 + (0.136).(2005) = 5.48 Year 1980 1985 1990 1995 2000 2005 population 2.1 2.9 3.2 4.1 9.4 5.48
- 10. 10 References: • Machine Learning with R , Second Edition , Brett Lantz • Sykes, Alan O. "An introduction to regression analysis." (1993). • Chatterjee, Samprit, and Ali S. Hadi. Regression analysis by example. John Wiley & Sons, • Internet:http://people.revoledu.com/kardi/tutorial/Regression/WhatIsRegre ssion.html