Regression Analysis

WINTERTemplate
SUBJECT: BASIC AND INFERENTIAL STATISTICS
REPORTER: SHIELA ROBETH B. VINARAO
TOPIC: REGRESSION ANALYSIS
PROFESSOR: DR. GLORIA T. MIANO
REGRESSION
ANALYSIS

THE SIMPLE LINEAR
REGESSION ANALYSIS
The simple linear regression analysis
is used when there is a significant
relationship between 𝒙 and 𝒚 variables.
This is used in predicting the value of
a dependent variable 𝒚 given the value of
the independent variable 𝒙.
D
E
F
I
N
I
T
I
O
N

THE SIMPLE LINEAR
REGESSION ANALYSIS
Suppose the advertising cost 𝒙 and
sales (𝒚) are correlated, then we can predict
the future sales (𝒚) in terms of advertising
cost (𝒙).
Another type of problem which uses
regression analysis is when variables
corresponding to years are given, it is
possible to predict the value of that variable
several years hence or several years back.
E
X
A
M
P
L
E

THE SIMPLE LINEAR
REGESSION ANALYSIS
F
O
R
M
U
L
A
𝒚 = 𝒂 + 𝒃𝒙

WINTERTemplate
Example
Consider the following data:
𝑥 𝑦
1 6
2 4
3 3
4 5
5 4
6 2
0
1
2
3
4
5
6
7
0 2 4 6 8
y-axis
x-axis
 Straight line indicates that the two variables are to some
extent LINEARLY RELATED
 The variable we are basing our predictions on is called the
predictor variable and is referred to as 𝑥. When there is only
one predictor variable, the prediction method is called
𝑠𝑖𝑚𝑝𝑙𝑒 𝑟𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛.

𝒙 𝒚 𝒙𝑦 𝒙 𝟐
1 6 6 1
2 4 8 4
3 3 9 9
4 5 20 16
5 4 20 25
6 2 12 36
Σ𝑥 = 21
𝑥 = 3.5
Σ𝑦 = 24
𝒚 = 4
Σ𝑥𝑦 =75 Σ𝑥2 = 91
SIMPLE LINEAR REGESSION

Σ𝑥 = 21
𝑥 = 3.5
Σ𝑦 = 24
𝒚 = 4
Σ𝑥𝑦 =75
𝒏 = 𝟔
Σ𝑥2 = 91
𝒃 =
𝒏 𝒙𝒚 − 𝒙 𝒚
𝒏 𝒙2 − 𝒙 2
=
6(75) − 21 24
6(91) − 21 2
=
450 − 504
546 − 441
=
−54
105
𝒃 = −𝟎. 𝟓𝟏
𝒂 = 𝒚 − 𝒃 𝒙
= 4 − −0.51 3.5
= 4 − (−1.79)
= 4 + 1.79
𝒂 = 𝟓. 𝟕𝟗
𝒚 = 𝟓. 𝟕𝟗 − 𝟎. 𝟓𝟏𝒙

05Example
A study is conducted
on the relationship of
the number of
absences (𝒙) and the
grades (𝒚) of the
students in English.
Determine the
relationship using the
following data.
Number of Absences
𝒙
Grades in English
𝒚
1 90
2 85
2 80
3 75
3 80
8 65
6 70
1 95
4 80
5 80
5 75
1 92
2 89
1 80
9 65

Number of
Absences
𝒙
Grades in
English
𝒚
1 90
2 85
2 80
3 75
3 80
8 65
6 70
1 95
4 80
5 80
5 75
1 92
2 89
1 80
9 65
0
10
20
30
40
50
60
70
80
90
100
0 2 4 6 8 10
GradesinEnglish
y
Number of absences
x
Scatter Diagram

WINTERTemplate
Solving by the Stepwise Method
Problem : Is there a significant relationship between the
number of absences and the grades of 15
students in English class?
Hypotheses :
Level of significance :
𝜶 = 𝟎. 𝟎𝟓
𝑟. 05 = −5.14
There is no significant relationship between the number
of absences and the grades of 15 students in English
class.
Ho:
There is a significant relationship between the number of
absences and the grades of 15 students in English class.H1:
df = n – 2
= 15 – 2
= 13

S
T
A
T
I
S
T
I
C
S
Pearson Product Moment
Coefficient of Correlation 𝒓
𝒙 = 𝟓𝟑 𝒚 = 𝟏𝟐𝟎𝟏 𝒙 𝟐
= 281 𝒚 𝟐
= 𝟗7335 𝒙𝒚 = 3950
𝒏 = 𝟏𝟓 𝒙 = 𝟑. 𝟓𝟑 𝒚 = 𝟖𝟎. 𝟎𝟕

𝒓 =
𝟓𝟗𝟐𝟓𝟎 − 𝟔𝟑𝟔𝟓𝟑
𝟒𝟐𝟏𝟓 − 𝟐𝟖𝟎𝟗 𝟏𝟒𝟔𝟎𝟎𝟐𝟓 − 𝟏𝟒𝟒𝟐𝟒𝟎𝟏
𝒓 =
𝟏𝟓 𝟑𝟗𝟓𝟎 − 𝟓𝟑 𝟏𝟐𝟎𝟏
𝟏𝟓 𝟐𝟖𝟏 − 𝟓𝟑 𝟐 𝟏𝟓 𝟗𝟕, 𝟑𝟑𝟓 − 𝟏𝟐𝟎𝟏 𝟐
𝒓 =
−𝟒𝟒𝟎𝟑
𝟏𝟒𝟎𝟔 𝟏𝟕𝟔𝟐𝟒
𝒓 =
−𝟒𝟒𝟎𝟑
𝟐𝟒𝟕𝟕𝟗𝟑𝟒𝟒
𝒓 =
−𝟒𝟒𝟎𝟑
𝟒𝟗𝟕𝟕. 𝟖𝟖 𝒓 = −𝟎. 𝟖𝟖

The computed r value of −0.88 is beyond the
critical value of −5.14 at 0.05 level of significance with 13
degrees of freedom, so the null hypothesis is rejected.
This means that there is a significant relationship
between the number of absences and the grades of
students in English. Since the value of r is negative, it
implies that students who had more absences had lower
grades.
Decision Rule :
If the r computed value is greater than or
beyond the critical value, reject Ho.
Conclusion :

Suppose we want to predict the grade (𝒚) of the student who
has incurred 7 absences (𝒙). To get the value of x, the simple
linear regression analysis will be used.
𝒂 = 𝒚 − 𝒃 𝒙
= 80.07 − −3.13 3.53
= 80.07 – (−11.05)
= 80.07 + 11.05
𝒂 = 𝟗𝟏. 𝟏𝟐
𝒃 =
𝒏 𝒙𝒚 − 𝒙 𝒚
𝒏 𝒙2 − 𝒙 2
=
15 3950 − 53 1201
15 281 − 53 2
=
59250 − 63653
4215 − 2809
=
−4403
1406
𝒃 = −𝟑. 𝟏𝟑
= 91.12 + −3.13 𝒙
= 91.12 − 3.13 𝟕
= 91.12 − 21.91
= 𝟔𝟗. 𝟐𝟏 𝒐𝒓 𝟔𝟗
69 is the grade of
the student with 7
absences.

WINTERTemplate
Remarks:
 It is important to remember that the
values of a and b are only estimates of the
corresponding parameters of a and b.
 To justify the assumption of linearity, a
test for linearity of regression should be
performed.
 If there are two or more independent
variables, the regression equation becomes
𝑦 = 𝑏0 + 𝑏1 𝑥1+𝑏2 𝑥2 + ⋯ + 𝑏 𝑛 𝑥 𝑛

 The significance of the slope of the regression line is to determine if
the regression model is usable.
 If the slope is not equal to zero, then we can use the regression
model to predict the dependent variable for any value of the
independent variable.
 If the slope is equal to zero, we do not use the model to make
predictions.
The scatter plot amounts to determining whether or not the
slope of the line of the best fit is significantly different from a
horizontal line or not.
A horizontal line means there is no association between two
variables, that is 𝑟 = 0.
 In testing for significance in simple linear regression, the null
hypothesis is H0: 𝑏 = 0 and the alternative hypothesis is H1: 𝑏 ≠ 0
Significance Test in Simple Linear
Regression

0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6
y-axis
x-axis
If 𝒃 = 𝟎
𝑦 = 1 + 0(1)
y = 1 + 0(2)
𝑦 = 1 + 0 3
𝑦 = 1 + 0 4
𝑦 = 1 + 0(5)
A horizontal line means there is no
association between two variables.
Slope of
Linear Regression

Regression
The t-test is conducted for testing the significance of r to
determine if the relationship is not a zero correlation.
𝑡 = 𝑟
𝑛 − 2
1 − 𝑟2
Where:
𝑛 = 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒
F
O
R
M
U
L
A
1

Regression
The t-test is conducted for testing the significance of r to
determine if the relationship is not a zero correlation.
𝑡 =
𝑏
𝑆 𝑏
Where:
𝑆 𝑏 =
𝑠
Σ 𝒙− 𝒙 𝟐
is the estimated standard deviation of 𝑏.
𝑠 =
Σ 𝑦− 𝒚 2
𝑛−2
is the standard deviation of the 𝑦 values about
the regression line.
F
O
R
M
U
L
A
2

Example
Given are two sets of data on the number of customers
(in hundreds) and sales (in thousand of pesos) for a given
period of time from ten eateries. Find the equation of the
regression line which can predict the amount of sales
from the number of customers. Can we conclude that we
can use the model to make such a prediction?
Eatery 1 2 3 4 5 6 7 8 9 10
x 2 6 8 8 12 16 20 20 22 26
y 58 105 88 118 117 137 157 169 149 202

𝑥 𝑦 𝑥𝑦 𝑥2 𝑦2
2 58 116 4 3364
6 105 630 36 11025
8 88 704 64 7744
8 118 944 64 13924
12 117 1404 144 13689
16 137 2192 256 18769
20 157 3140 400 24649
20 169 3380 400 28561
22 149 3278 484 22201
26 202 5252 676 40804
Regression
𝒕 = 𝒓
𝒏 − 𝟐
𝟏 − 𝒓 𝟐
𝑡 = 0.95
10 − 2
1 − 0.95 2
𝑡 = 0.95
8
1 − .90
𝑡 = 0.95
8
.097
𝒕 = 𝟖. 𝟔𝟐
Given: 𝑛 = 10
𝑟 = 0.950122955

𝑥 𝑦 𝑥𝑦 𝑥2 𝑦2
𝑦 = 60 + 5𝑥 𝒙 − 𝒙 𝟐 𝑦 − 𝑦 2
2 58 116 4 3364 70 144 144
6 105 630 36 11025 90 64 225
8 88 704 64 7744 100 36 144
8 118 944 64 13924 100 36 324
12 117 1404 144 13689 120 4 9
16 137 2192 256 18769 140 4 9
20 157 3140 400 24649 160 36 9
20 169 3380 400 28561 160 36 81
22 149 3278 484 22201 170 64 441
26 202 5252 676 40804 190 144 144
Regression
Σ𝑥 = 140
Σ𝑦 = 1300
Σ𝑥𝑦 = 21040Σ𝑥2 = 2528
Σ𝑦2
= 184730
𝒙 − 𝒙
𝟐
= 568
𝑦 − 𝒚
2
= 1530𝒕 =
𝒃
𝑺 𝒃𝑆 𝑏 =
Σ 𝑦 − 𝒚 2
𝑛 − 2
Σ 𝒙 − 𝒙 𝟐
𝑆 𝑏 =
1530
8
568
𝑆 𝑏 = 0.5803
𝒕 =
𝟓
𝟎. 𝟓𝟖𝟎𝟑
𝒕 = 𝟖. 𝟔𝟐
Σ 𝒙 = 14

Since 8.62 is greater than 3.355, we
reject the H0 or accept the H1. Thus, the
obtained relationship is significant or is
non zero using .005 level.
We can conclude that we can use the
model to predict sales from population.
Decision:

COMPUTATION USING
MICROSOFT EXCEL
Pearson r
Slope b
Intercept a
Syntax
+pearson(array1,array2)
or
+correl(array1,array2)
+slope(known_y’s,known_x’s)
+intercept(known_y’s,known_x’s)

Regression Analysis

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Regression Analysis (20)

Recently uploaded (20)

Regression Analysis