Data Analytics.01. Data selection and capture

Data Analytics process in
Learning and Academic
Analytics projects
Day 1: Data selection and capture
Alex Rayón Jerez
alex.rayon@deusto.es
DeustoTech Learning – Deusto Institute of Technology – University of Deusto
Avda. Universidades 24, 48007 Bilbao, Spain
www.deusto.es

Objectives
How to tackle an AA/LA project
1. Objectives: what do I want to improve?
2. Data: automated processes for data discovery
and later processing
3. Integration, not substitution
4. Technology
5. KPIs: define and test

Table of contents
● ETL approach
● Data analytics cycle
● Architecture principles
● Requirements
● Components
● Data process
○ Questions
○ Data model
○ Data sources
○ Use cases

Table of contents
● ETL approach
● Requirements
● Components
● Data process

ETL approach
Definition and characteristics
● An ETL tool is a tool that
○ Extracts data from various data sources (usually
legacy data)
○ Transforms data
■ from → being optimized for transaction
■ to → being optimized for reporting and analysis
■ synchronizes the data coming from different
databases
■ data cleanses to remove errors
○ Loads data into a data warehouse

ETL approach
Why do I need it?
● ETL tools save time and money when
developing a data warehouse by removing
the need for hand-coding
● It is very difficult for database administrators
to connect between different brands of
databases without using an external tool
● In the event that databases are altered or new
databases need to be integrated, a lot of hand-
coded work needs to be completely redone

ETL approach
Kettle
Project Kettle
Powerful Extraction, Transformation and
Loading (ETL) capabilities using an
innovative, metadata-driven approach

ETL approach
Kettle (II)
● It uses an innovative meta-driven approach
● It has a very easy-to-use GUI
● Strong community of 13,500 registered
users
● It uses a stand-alone Java engine that
process the tasks for moving data between
many different databases and files

ETL approach
Kettle (IV)
Source: http://download.101com.com/tdwi/research_report/2003ETLReport.pdf

ETL approach
Kettle (V)
Source: Pentaho Corporation

ETL approach
Kettle (VI)
● Datawarehouse and datamart loads
● Data integration
● Data cleansing
● Data migration
● Data export
● etc.

ETL approach
Transformations
● String and Date Manipulation
● Data Validation / Business Rules
● Lookup / Join
● Calculation, Statistics
● Cryptography
● Decisions, Flow control
● Scripting
● etc.

ETL approach
What is good for?
● Mirroring data from master to slave
● Syncing two data sources
● Processing data retrieved from multiple
sources and pushed to multiple
destinations
● Loading data to RDBMS
● Datamart / Datawarehouse
○ Dimension lookup/update step
● Graphical manipulation of data

Data Analytics cycle
Challenges
● Data is everywhere
● Data is inconsistent
○ Records are different in each system
● Performance issues
○ Running queries to summarize data for
stipulated long period takes operating
system for task
○ Brings the OS on max load
● Data is never all in Data Warehouse
○ Excel sheet, acquisition, new application

Challenges (II)
● Data is incomplete
● Certain types of usage data are not logged
● Data are not aggregated following a
didactical perspective
● Users are afraid that they could draw
unsound inferences from some of the data
[Mazza2012]

Academic Analytics Model
1) Capture
2) Report5) Refine
4) Act 3) Predict
Academic Analytics
[CampbellOblinger2007]

Learning Analytics Model
1) Select
2) Capture
3) Aggregate
4) Process
5) Visualize
On the design of collective applications
[DronAnderson2009]

Learning Analytics Model (II)
On the design of collective applications
[DronAnderson2009]
1) Select
2) Capture
3) Aggregate
4) Process
5) Visualize
Day 1
Day 2
Day 3
Day 4

Learning Analytics Model (III)
● As [Clow2012] states, it is
necessary to close the
feedback loop through
appropriate interventions
unmistakable
● It also draws on the wider
educational literature,
seeking to place learning
analytics on an established
theoretical base, and
develops a number of
insights for learning
analytics practice

Learning Analytics Model (IV)

Learning Analytics Model (III)

Architecture principles
A model for adoption, use and improvement of analytics
A framework of characteristics for Analytics
Adam Cooper, 2012 [Cooper2012]

Development of common language for data exchange
The IEEE defines interoperability to be:
“The ability of two or more systems or
components to exchange information and
to use the information that has been
exchanged”

Development of common language for data exchange (II)

Development of common language for data exchange (III)
● The most difficult challenges with achieving
interoperability are typically found in
establishing common meanings to the data
● Sometimes this is a matter of technical
precision
○ But culture – regional, sector-specific, and
institutional – and habitual practices also affect
meaning

Development of common language for data exchange (IV)
● Potential benefits
○ Efficiency and timeliness
■ No need for a persona to intervene to re-enter, re-
format or transform data
○ Independence
■ Resilience
○ Adaptability
■ Faster, cheaper and less disruptive to change
○ Innovation and market growth
■ Interoperability combined with modularity makes
it easier to build IT systems that are better
matched to local culture without needing to create

Development of common language for data exchange (V)
● Potential benefits
○ Durability of data
■ Structures and formats change over time
■ The changes are rarely properly documented
○ Aggregation
■ Data joining might be supported by a common set
of definitions around course structure, combined
with a unified identification scheme
○ Sharing
■ Specially when there are multiple parties involved

Development of common language for data exchange (VI)
[LACE2013]

Development of common language for data exchange (VII)
[LACE2013]
In our case?

Development of common language for data exchange (VIII)
[LACE2013]
In our case?

Requirements
● Usability: prepare an understandable user interface
(UI), appropriate methods for data visualization, and
guide the user through the analytics process.
● Usefulness: provide relevant, meaningful indicators
that help teachers to gain insight in the learning
behavior of their students and support them in
reflecting on their teaching.
● Interoperability: ensure compatibility for any kind
of VLE by allowing for integration of different data
sources.
[Dyckhoff2010]

Requirements (II)
● Extensibility: allow for incremental extension of
analytics functionality after the system has been
deployed without rewriting code.
● Reusability: target for a building-block approach to
make sure that re-using simpler ones can implement
more complex functions.
● Real-time operation: make sure that the toolkit can
return answers within microseconds to allow for an
exploratory user experience
● Data Privacy: preserve confidential user information
and protect the identities of the users at all times
[Dyckhoff2010]

Components
● Process
○ A systematic process of educational data analysis
● Model
○ The definition of a suitable model to represent the
knowledge domain
● Tool/platform
○ The design and implementation of a monitoring
and presentation tool based on the Process and
Model
[Mazza2012]

Table of contents
● ETL approach
● Requirements
● Data process

Data process
Introduction
“Measurement, collection, analysis and
reporting of data about learners and
their contexts, for purposes of
understanding and optimising learning
and the environments in which it
occurs”
First international conference on Learning Analytics and
Knowledge, Alberta, 2011 [LAK2011]

Data process
Introduction (II)
However, the challenge is to determine
which data are of interest
We are now in an era where gaining
access to data is not the problem;
the challenge lies in determining
which data are significant and why

Data process
Introduction (III)
“The basic question
is not what can we
measure? The basic
question is what
does a good
education look like?
Big questions”

Data process
Introduction (IV)
“More data does not mean more knowledge”
[Jenkins2013]
Searching for the evidence in a mass of data
requires knowing what kind of evidence is
needed
Knowledge of the domain and understanding
and interpretation of the patterns we see

Data process
Introduction (VI)
Source: http://www.learningfrontiers.eu/?q=story/will-analytics-transform-education

Data process
Introduction (VII)
A brief comparison of the two fields (George Siemens and Ryan Baker [SiemensBaker2012])

Data process
Introduction (VIII)
First of all, education is a highly collaborative space and it represents a social good. Keeping a valuable secret
that might help students succeed is antithetical to the nature of education. Second, education is a complex
ecosystem of people, processes, policies, content, etc. I would have strong doubts about anyone who claimed to
have a formula that worked for a wide variety of institutions.
Mike Sharkey, 2014

Data process
Questions
Source: http://www.slideshare.net/sbs/learning-analytics-uts-2013

Data process
Questions (II)
The question depends on who is making it ;)

Data process
Questions (III)
Horizon Report 2012 [HR2012]

Data process
Questions (IV)
1) Adaptive testing, tracking and reporting
● Progress summary, daily activity report, class
goals report, progress report, student activity
report, student focus report, etc [Khan2012]
● By using various analytics tools, students can
review their learning progress and teachers
are also supported in how to personalise
learning for students in need for more help in
specific areas

Data process
Questions (V)
2) Analytics tools for early alert, intervention
and collaboration
● Integrating their data collected from a variety
of information management systems
○ Allowing educators to assess the risk, initiate early
interventions and support collaborative learning

Data process
Questions (VI)
and collaboration
● For example, the Signals project at Purdue
University utilizes the data collected from
student information systems, learning
management systems, and the grade book for
a specific course to track students’
performances and identify at-risk students in
real time

Data process
Questions (VII)
and collaboration
● The LOCO-Analyst provides teachers with
charts, graphs, and other data representations
that help them see how their students are
performing and how students interact with
one another in web-based learning
environments to help the teacher determine
how to engage their students online

Data process
Questions (VIII)
and collaboration
● Social Networks Adapting Pedagogical
Practice (SNAPP), a network visualization tool
developed by researchers at the University of
Wollongong, can analyse students’
interactions in a forum and display it in a
visualised diagram which help teachers to
identify the key connections and disconnected
students and support collaborative learning in
a web-based learning environment

Data process
Questions (IX)
3) Analytics projects for institutional
efficiency and effectiveness
● There are a number of institutional analytics
initiatives which enable institutions to
improve the effectiveness of operations,
including admission management and drop-
out prevention, resource management,
financial planning, etc
○ Student Experience Traffic Lighting (SETL)
○ The Enhancing Student Centred Administration
Placement Experience (ESCAPES)

Data process
Questions (X)
Learning Analytics
are not neutral

Data process
Questions (XI)
“Accounting tools… do not simply
aid the measurement of economic
activity, they shape the reality they
measure”
[GayPryke2002]

Data process
Questions (XII)
Fuente: http://mfeldstein.com/harvard-mit-learn-university-phoenix-analytics/

Data process
Questions (XIII)
● The Harvard and MIT data ignores student
goals or any information giving a clue on
whether students desired to complete the
course, get a good grade, get a certificate, or
just sample some material
● Without this information, the actual
aggregate behavior is missing context
○ We don’t know if a certain student intended to just
audit a course, sample it, or attempt to complete it.
○ We don’t know if students started the course intended
to complete but became frustrated

Data process
Questions (XIV)
● The value of learner behavior patterns, which
can only be learned by viewing data patterns
over time
● If you want to “share best practices to improve
teaching and learning”, then you need data
organized around the learner
○ With transactions captured over time – not just in
aggregate
○ What we have now is an honest start, but a very
limited data set

Data process
Data model
So, which is our data
model to answer to
our questions?

Data process
Data model (II)
The data model, or the concept map,
describes the concepts and their
relationships used by the organization
in its daily work, expressed in its own
language
It enables the whole organization to
participate in the maintenance of it

Data process
Data model (III)
Source: http://www.economist.com/news/finance-and-economics/21578041-containers-have-been-more-important-globalisation-freer-trade-
humble
Source: http://www.economist.com/blogs/economist-explains/2013/05/economist-explains-14

Data process
Data model (IV)
The best approach that we have
found for this task is constituted by
the theory of eLearning functions
Reinmann [Reinmann2006]

Data process
Data model (V)
[Reinmann2006]

Data process
Data model (VI)
Example
[Mazza2012]

Data process
Data model (VII)
Example
This model answers the monitoring questions:
● Which way of eLearning enables to reach the
given objectives?
● By which means (functions, tools) does the
LMS enable these ways of learning?
● How is the use of these means traced in the log
files (activity log codes)?
[Mazza2012]

Data process
Data model (VIII)
[Mazza2012]

Data process
Data sources
Today we have so much data
that come in an unstructured
or semi-structured form that
may nonetheless be of value in
understanding more about our
learners

Data process
Data sources (II)
“Learning is a complex social activity”
[Siemens2012]
Lots of data
Lots of tools
Humans to make sense

Data process
Data sources (III)
Traditional data sources:
● Student data: demographics,
qualification aim, modules taken,
results, etc.
● Student feedback data: end of
module survey and others
● Student activity data: delivery data,
completion, pass rates, etc.

Data process
Data sources (IV)
● The world of technology has changed
[Eaton2012]
○ 80% of the world’s information is unstructured
○ Unstructured data are growing at 15 times the rate
of structured information
○ Raw computational power is growing at such an
enormous rate that we almost have a supercomputer
in our hands
○ Access to information is available to all

Data process
Data sources (V)
Source: http://www.bigdata-startups.com/BigData-startup/understanding-sources-big-data-infographic/

Data process
Data sources (VI)
● RDBMS (SQL Server, DB2, Oracle, MySQL,
PostgreSQL, Sybase IQ, etc.)
● NoSQL Data: HBase, Cassandra, MongoDB
● OLAP (Mondrian, Palo, XML/A)
● Web (REST, SOAP, XML, JSON)
● Files (CSV, Fixed, Excel, etc.)
● ERP (SAP, Salesforce, OpenERP)
● Hadoop Data: HDFS, Hive
● Web Data: Twitter, Facebook, Log Files, Web Logs
● Others: LDAP/Active Directory, Google Analytics,
etc.

Data process
Use cases
1) Student data: XML

Data process
Use cases (II)
2) Moodle: MySQL database
mdl_forum
- id
- course
- name
mdl_user
- id
- username
- firstname
- lastname
mdl_forum_discussions
- id
- name
- userid
- timemodified
- usermodified
mdl_forum_posts
- id
- userid
- discussion
- message
- modified
- created

Data process
Use cases (III)
3) MediaWiki: MySQL database
user
- user_real_name
- user_editcount recentchanges
- rc_old_len
- rc_new_len
revision
- rev_timestamp page
- page_counter
- page_len
rev_user = user_id
rev_page = page_id
user_id = rc_user

Data process
Use cases (IV)
4) Google Doc: Google API

Data process
Use cases (V)

Data process
Use cases (VI)

Data process
Use cases (VII)

Data process
Use cases (VIII)

Data process
Use cases (IX)

Data process
Use cases (X)

Data process
Use cases (XI)

Data process
Use cases (VI)
5) Twitter: API

References
[CampbellOblinger2007] Campbell, John P., Peter B. DeBlois, and Diana G. Oblinger. "Academic analytics: A new tool for a new era." Educause
Review 42.4 (2007): 40.
[Clow2012] Clow, Doug. "The learning analytics cycle: closing the loop effectively." Proceedings of the 2nd International Conference on Learning
Analytics and Knowledge. ACM, 2012.
[Cooper2012] Cooper, Adam. "What is analytics? Definition and essential characteristics." CETIS Analytics Series 1.5 (2012): 1-10.
[DronAnderson2009] Dron, J., & Anderson, T. (2009). On the design of collective applications. In Proceedings of the 2009 International Conference
on Computational Science and Engineering, 4, 368–374.
[Dyckhoff2010] Dyckhoff, Anna Lea, et al. "Design and Implementation of a Learning Analytics Toolkit for Teachers." Educational Technology &
Society 15.3 (2012): 58-76.
[Eaton2012] Chris Eaton, Dirk Deroos, Tom Deutsch, George Lapis & Paul Zikopoulos, “Understanding Big Data: Analytics for Enterprise Class
Hadoop and Streaming Data”, p.XV. McGraw-Hill, 2012.
[GayPryke2002] Cultural Economy: Cultural Analysis and Commercial Life (Culture, Representation and Identity series) Paul du Gay (Editor),
Michael Pryke. 2002.
[HR2012] NMC Horizon Report 2012 http://www.nmc.org/publications/horizon-report-2012-higher-ed-edition
[Jenkins2013] BBC Radio 4, Start the Week, Big Data and Analytics, first broadcast 11 February 2013 http://www.bbc.co.
uk/programmes/b01qhqfv
[Khan2012] http://www.emergingedtech.com/2012/04/exploring-the-khan-academys-use-of-learning-data-and-learning-analytics/
[LACE2013] Learning Analytics Community Exchange http://www.laceproject.eu/
[LAK2011] 1st International Conference on Learning Analytics and Knowledge, 27 February - 1 March 2011, Banff, Alberta, Canada https://tekri.
athabascau.ca/analytics/
[Mazza2006] Mazza, Riccardo, et al. "MOCLog–Monitoring Online Courses with log data." Proceedings of the 1st Moodle Research Conference. 2012.
[Reinmann2006] Reinmann, G. (2006). Understanding e-learning: an opportunity for Europe? European Journal of Vocational Training, 38, 27-42.
[SiemensBaker2012] Siemens & Baker (2012). Learning Analytics and Educational Data Mining: Towards Communication and Collaboration.
Learning Analytics and Knowledge 2012. Available in .pdf format at http://users.wpi.edu/~rsbaker/LAKs%20reformatting%20v2.pdf

Data Analytics.01. Data selection and capture

More Related Content

Viewers also liked (20)

Similar to Data Analytics.01. Data selection and capture (20)

More from Alex Rayón Jerez (20)

Recently uploaded (20)

Data Analytics.01. Data selection and capture