Learning analytics to quantize and improve the skills development and attainment in large classes

IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
_______________________________________________________________________________________
Volume: 03 Issue: 10 | OCT-2014, Available @ http://www.ijret.org 358
LEARNING ANALYTICS TO QUANTIZE AND IMPROVE THE SKILLS
DEVELOPMENT AND ATTAINMENT IN LARGE CLASSES
Ishwari Aghav1
, Jagannath Aghav2
1
Cofounder, Saple Software, Pune 411021
2
Professor, Computer and IT Department, College of Engineering Pune 411005
Abstract
The intervention of technology in the teaching and learning processes is bringing change. There is requirement of methods and
techniques to analyze the generated data in these processes. Traditional way of delivery and assessment is becoming more
focused by the application of technology. In this paper, we propose and illustrate an algorithmic methodology that shall allow the
stakeholders in education to focus more on skills attainment effectively.
In large size class, there is huge data generated by registration system, administration system, and the learning management
system together. We propose the methodology that mainly maps and quantizes the learning outcomes, and that relates to the
successful and weaker range of attainments of stakeholders. This algorithmic methodology reads the data viz. course details,
outcomes, and assessment scores of a large class and analyzes the skills attainment with respect to the planned each course
outcome. The detailed report is generated with the average of attainment of the class strength, highest attainment, and lowest
attainment in each of the course outcomes for each student.
Key Words: Learning Analytics, Software tool, System Architecture, Course Outcomes, and Skills Attainment
--------------------------------------------------------------------***----------------------------------------------------------------------
1. INTRODUCTION
Accreditation agencies in the world have outlined the
required attributes of engineering graduates. The attributes
are eventually mapped to the contents of curriculum that
helps in the attainment and in the development skills of
graduates.
Identifying and addressing good and weak attainment
groups especially in large size student classes is a challenge
to the Instructors. With the help of emerging educational
data mining technologies and learning analytics, it is
becoming feasible for focusing effectively more on learning
outcomes. Analytics of data in course management will help
in redesigning the course outcomes. It will produce
quantized evidence for new assessment tools and
implementations.
The delivery of contents is planned by Instructors in various
forms. There are challenging decisions on curriculum,
instruction, assessment strategies, monitoring the student
performance, etc. The decision making is challenging
especially when the size of class is large. Following
parameters if are considered for analyzing then the
challenges become even more formidable”:
(a) Monitoring performance of each stakeholder,
(b) Addressing and identifying the groups that needs
attention,
(c) Averaging the performance over various electives in
different semesters, etc.
(d) Predicting optimal performance,
(e) Planning and redesigning effective course instructions,
(f) Designing, testing, and evaluating the course contents
and relevant resources, and
(g) Identifying the appropriate assessment tools and
techniques.
This motivates the design of an algorithmic solution that
shall facilitate the collecting, presenting, and analyzing the
academic, learning, assessment, data to the meet various
objectives.
2. LITERATURE SURVEY:
Techniques used in analyzing various engineering domains
exist widely in literature. The importance of analytics is
grown recently after proliferation of management
information systems in everyday life. There are various
categories of analytics based on domains, e.g. health
analytics in health care, learning analytics in universities,
business analytics in communication, insurance, and
banking systems.
Learning Analytics [1, 2] is defined as “the measurement,
collection, analysis and reporting of data about learners and
their contexts, for purposes of understanding and optimizing
learning and the environments in which it occurs.”
In order to address the applications of technology in
education, this context requires use of learning analytics.
The usage shall help in decision making at Institute or
administrative level, faculty level, and student level. Some
of the useful gains (not all covered) of applying learning
analytics are: increased productivity, effective resource
allocations, transformation pedagogical approaches,
provides interventions to risk learners, and help to students
in gaining the insight on own learning.

_______________________________________________________________________________________
The online courses are reshaping with advent of technology.
Traditional courses are getting transformed with addition of
growth of tools and frameworks. In order of make learning
effective especially in large classes, it essential to focus on
parameters that shall help in planning lessons, resources,
assessments, and delivery of the course in the class.
There are efforts [3, 4, 6-18] in various directions to capture
the data and apply effectively in order to improve learning.
An e-learning tool [3] based on object oriented approach
wherein objects are text, audio, and graphics. Upon request
from student the tool dynamically creates customized course
based on profile in the learning management system. There
are other tools [19, 28-30] in the learning and academic
analytics that aid in management of courses.
There are various approaches and methods on delivering
contents [4-6]. The delivery methods [4] are transmitted to
remote learning management system, which provides the
contents, based on request from users. The focus is more on
delivering automatically transmitting the contents to remote
learning management system, and maintaining/ updating
automatically the course catalog. The Academic Analysis
Tool [7] is the software that analyzes student behavior
which actually gives the feedback on how students use and
learn in different courses.
There are applications and methods [20-27] of academic
analytics, which are applied for future predictions, and also
some other cases [32-36] of applying analytics in literature
are available.
3. SYSTEM ARCHITECTURE BASED ON ‘LIFE
CYCLE OF SUCCSSIVE SKILLS
ATTAINMENT’
Learning in the classroom has life cycle that aims at
attainment of skills. The components of cycle are: mainly
the learner’s interaction with instructor, content lessons,
assessment tools, and the skills attainment. This is iterative
cycle of improvement with duration of semesters and years.
The target course outcomes that are derived and planned
from the program outcomes are set based on graduate
attributes. On the basis of target outcomes, the content
lessons are delivered for learning. Schematic outline
showing the life cycle on successive skills attainment is
depicted in FIGURE 1. The content lessons include the
resource material, books, videos, etc. The cycle is iterative
and it improves on growth in learning, helps in taking
decisions on delivery, planning, and understanding the
delivery contents.
FIGURE 1: Life Cycle of Successive Skills Attainment
We have implemented the system architecture that will
allow focusing on skills improvement and the attainment by
using learning analytics. The methodology quantizes and
maps the parameters of interest especially assessment and
delivery of course contents. There is flexibility of defining
and planning the different assessment sequences and
weightages. The models built on these defined course
assessment tools provide results on the attainment of each
planned outcome.
FIGURE 2 depicts the system architecture with the layout of
components and its implementation. Following are the
analytical modules or components used in the system
architecture:
 Assessment Design Module
 Course Outcomes Module
 Report Generation Module
 Learning Analytics Module
There two main interactive modules: one for designing
course outcomes and another for designing various types of
assessments. The program outcomes are mapped to course
outcomes requires to be entered, as it forms the basis of
performance evaluation.

_______________________________________________________________________________________
FIGURE 2: System Architecture of Implementation
The course outcomes while being entered are verified with
Bloom’s Taxonomy, and this will be providing the
classification of each design outcome. This will help in
balancing the outcomes in learning of course. The intelligent
course design module shall advise the classification and also
will give short-comings as feedback to the design process.
The process of entering data, classifying outcomes
statements is completely independent of course contents or
the subject matter. The process implementation is automated
with an algorithmic solution. Automated weighted
assessment in the design module is implemented with an
algorithmic methodology. It allows defining the weightage
of each assessment in the final computation of performance.
This is useful when there are multiple assessment tools
applied for continuous evaluation.
The learning analytics module and the report generation
module are designed with an algorithmic methodology that
provides:
(a) Average attainment of each course outcome,
(b) The list of highest attainment group,
(c) The list of lagging or weak attainment group,
(d) The complete attainment report in excel sheet and
portable document format, and
(e) The record of average attainment in each case of
program outcome based on multiple mapped courses.
In the next section we shall illustrate the details of the
methodology by running input on sample course.
4. ILLUSTRATIVE EXAMPLE
In this section we shall demonstrate the implementation
of the methodology on sample input course. The system
architecture and methodology is tested over 50 courses
in real life with each course containing 3 examinations
and its corresponding laboratories.
FIGURE 3 shows the percentage of average attainment
of each course outcome for the Course Title: Theoretical
Computer Science, Course Code: CT-202, Semester IV,
with 82 students.
FIGURE 3: Percentage of average attainment of each
course outcome
It is clear from the percentage of average course outcomes
attainment that course outcome CO 3 is highest; while as
course outcome CO 1 and CO 5 attainment needs attention,
and accordingly the planning for the lessons to strengthen
further.
TABLE 1 shows the course outcomes attainment with
number of students achieving with 75% and above and the
lagging behind on 35% and below. For example the course
outcome CO 2 has 8 students lagging behind by score 9.88
% or below.
TABLE 1: Course outcomes attainment with number of
students
The course outcomes CO 1 and CO 5 have 5 and 7 students
respectively with attainment of 75% or above. The number
of students with lagging behind in the attainment of course
outcomes CO 1 and CO 5 is more. This cyclic feedback
shall help in planning the lessons, in designing outcomes,
and in adoption of the next assessment tools and methods.
5. DISCUSSION
The algorithmic implementation demonstrates an illustration
of iterative cycle of improving the skills attainment. The
presented methodology quantifies the attainment and
provides the detailed reports. By analyzing the generated
reports it is feasible to address the weaker groups with
respect to the outcomes.
In future there is also need to quantify and map the iterations
for benchmarking excellence in courses. The feedback of the
iterations shall bring the improvements in each step of the
learning process.

_______________________________________________________________________________________
REFERENCES
[1] Learning Analytics, on Wikipedia URL: http://
en.wikipedia.org/ wiki/ Learning_analytics.
[2] Tanya Elias, “Learning Analytics: The Definitions, the
Processes, and the Potential”, January, 2011.
[3] Mark Rukavina, Chris Dragon, Mike Ray, Carman, etal.,
“E-Learning Tool for Dynamically Rendering Course
Content,” Patent US 2002/ 0188583 A1, Dec 12, 2002.
[4] Robin U Sperle and Marcus Philipp, “Delivery Methods
for Remote Learning System Courses,” Patent US 2007/
0111180 A1, May 17, 2007.
[5] Sarah Buchanan and Joseph J Laviola, “ CSTutor: A
Sketch-Based tool for Visualizing Data Structures,” ACM
Transactions on Computing Education, Volume 14 Issue 1,
pp. 3:1- 3:27, March 2014.
[6] Steven Lonn, Andrew E., Krumm, R. Joseph, Stephanie
D., “Bridging the Gap from Knowledge to Action: Putting
Analytics in the Hands of Academic Advisors”, LAK’12,
Vancouver, BC, Canada, April 29–May 2, 2012.
[7] Sabine Graf, Arnold Ferri, “AAT – A Tool for
Accessing and Analyzing Students Behavior Data in
Learning Systems”, LAK’11, Banff, AB, Canada, February
27-March 1, 2011.
[8] Gonzalo M., Xavier O., Katherine C., “Techniques for
Data-Driven Curriculum Analysis”, LAK ’14, Indianapolis,
IN, USA, March 24 - 28 2014.
[9] Angela van Barneveld, Kimberly E. Arnold, and John P.
Campbell, “Analytics in Higher Education: Establishing a
Common Language”, January 2012.
[10] Stuart Palmer, “Modelling Engineering Student
Academic Performance Using Academic Analytics”,
International Journal of Engineering Education Vol. 29, No.
1, pp. 132–138, 2013.
[11] Taeho Yu, II-Hyun Jo, “Educational Technology
Approach toward Learning Analytics: Relationship between
Student Online Behavior and Learning Performance in
Higher Education”, LAK '14, Indianapolis, IN, USA, Mar
24-28 2014.
[12] Dehler, C. (2004). Best Practices for Effective Online
Teaching. In R. Ferdig et al. (Eds.), Proceedings of Society
for Information Technology & Teacher Education
International Conference pp. 407-412, 2004.
[13] Eleni Koulocheri, Alexandros S., Michalis X.,
“Applying Learning Analytics in an Open Personal Learning
Environment”, 16th PanHellenic Conference on Informatics,
2012.
[14] Kimberly E. Arnold, Matthew D. Pistilli, “Course
Signals at Purdue: Using Learning Analytics to Increase
Student Success”, LAK’12, Vancouver, BC, Canada,
April 29 - May 2, 2012.
[15] Philip J., Daniel T., MJ Bishop, “Educational Data
Sciences – Framing Emergent Practices for Analytics of
Learning, Organizations, and Systems”, LAK '14,
Indianapolis, IN, USA, March 24 - 28 2014.
[16] Eitel J.M. Lauria, Joshua D. Baron, Mallika Devireddy,
“Mining academic data to improve college student retention:
An open source perspective”, LAK’12, Vancouver, BC,
Canada, 29 April – 2 May 2012.
[17] Eitel J.M. Lauria, Erik W. Moody et al., “Open
Academic Analytics Initiative: Initial Research Findings”,
LAK '13, Leuven, Belgium, April 08 – 12, 2013
[18] Stephen Aguilar, Steve Lonn, Stephanie D.,
“Perceptions and Use of an Early Warning System During a
Higher Education Transition Program”, LAK '14, ,
Indianapolis, IN, USA, March 24 – 28, 2014.
[19] Campbell, J. P., DeBlois, P. B., Oblinger, D. G.,
“Academic analytics: A new tool for a new era”, Educause
Review, 2007.
[20] Educause, “Academic Analytics in the future of higher
education”, ECAR Research Study 8, 2005.
[21] John P. Campbell and Diana G. Oblinger, “Academic
Analytics”, Educause, October 2007.
[22] Jacqueline Bichsel, “Analytics in Higher Education”,
ECAR Research Study 8, 2012.
[23] Judith A. Pirani, Bob Albrecht, “University of Phoenix:
Driving Decisions through Academic Analytics” ECAR
Case Study 9, 2005.
[24] Philip J. Goldstein, Richard N. Katz, “Academic
Analytics: The Use of Management Information and
Technology in Higher Education”, ECAR Research Study,
Volume 8, 2005.
[25] M.A. Chatti, A.L. Dyckhoff, U. Schroeder, and H.
Thus, “A Reference Model for Learning Analytics”,
International Journal of Technology Enhanced Learning.
[26] Malcolm Brown, “Learning Analytics: The Coming
Third Wave”, April 2011.
[27] Beth Dietz-Uhler and Janet E. Hurn, “Using Learning
Analytics to Predict (and Improve) Student Success: A
Faculty Perspective”, Journal of Interactive Online
Learning, Volume 12, Number 1, spring 2013.
[28] Ravi Shankar, Jonathan Dickson and Carlo Mazoleny,
“A Tool for ABET Accreditation”, 120th ASEE Annual
Conference and Exposition, Atlanta, GA, June 2013.
[29] Eugene Essa, Andrew Dittrich, Sergiu Dascalu,
Frederick C. Harris, Jr., “ACAT: A Web-based Software
Tool to Facilitate Course Assessment for ABET
Accreditation”, Seventh International Conference on
Information Technology, IEEE, 2010.
[30] Eugene O. Essa, “ACAT: ABET Course Assessment
Tool”, Thesis report, University of Nevada, May, 2010.
[31] Dr. Tanya Gorman, “Assessing Student Learning
Outcomes”, COC Summer Institute, 2008.
[32] Rebecca Cartwright, Ken Weiner, and Samantha
Streamer-Veneruso, “Student Learning Outcomes
Assessment Handbook”, revised August, 2009.
[33] Norris, D., Baer, L., Leonard, J., Pugliese, L., Lefrere,
P., “Action analytics”, Educause Review, 2008.
[34] Paul Baepler, Cyntia James Murdoch, “Academic
Analytics and Data Mining in Higher Education”,
International Journal for the Scholarship of Teaching and
Learning (IJSOTL), Volume 4, No. 2, Article 17, 2010.
[35] Bandura, A., “Self-efficacy: The exercise of control”,
New York: Freeman & Co., 1997.
[36] Cross K. P., “Thirty years passed: Trends in general
education.”, in B. L. Johnson (Ed.), General education in
two-year colleges, San Francisco, 1982, pp. 11-20.

_______________________________________________________________________________________
BIOGRAPHIES
Ms Ishwari Aghav has obtained Bachelor
of Information Technology from PICT
Pune (affiliated to University of Pune)
and is the cofounder of start-up Saple
Software Pune India.
Jagannath Aghav has obtained PhD in
Computer Science from Tata Institute of
Fundamental Research (University of
Mumbai) India. He is working as
Professor in Computer and IT Department
of College of Engineering Pune India.

Learning analytics to quantize and improve the skills development and attainment in large classes

More Related Content

What's hot

Viewers also liked

Similar to Learning analytics to quantize and improve the skills development and attainment in large classes

More from eSAT Publishing House

Recently uploaded

Learning analytics to quantize and improve the skills development and attainment in large classes