Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education

Establishing an Ethics
Framework for Predictive
Analytics in Higher Education
Cyber Summit 2016, Banff
Stephen Childs, Institutional Analyst
October 27, 2016

Disclamer
 The content of this presentation represents my views only.
and not that of my employer, the University of Calgary.
 I am not qualified to accurately describe University of Calgary
policy in the areas discussed in this talk.
 Please contact the University if you have policy questions.

Data Abundance in Higher Education
3

Big Data, Big Problems
 Advancing technology
—Better data collection
—Handle more data
—Apply algorithms to data
 We know more about our students
 Can make predictions about their behavior
 Very few guidelines about this practice

Solutions
 Develop an ethics framework around student data.
 Build on existing guidelines.
 Build on the norms of service to students
 Do this now while these practices are new.

Outline
 Introduction
 Students and Student Data
 Predictive Analytics
 Existing Frameworks
 Next Steps

About My Office
 Office of Institutional Analysis
 https://oia.ucalgary.ca/

Student Data
 Application
 Student Information System
 LMS
 Unicard
 Surveys
 Residence
 Facilities
 Awarding Degrees
 Grades
 USRI
 IT usage
 Others…

Student Data
 Students can opt out of some data collection, but not all
 Student give us their data because they trust us
 We need to deserve that trust!
—Respect student privacy
—Transparency about how data is used
—Accountability
—Consultation
—Consider the Consequences

Transparency and Accountability
 Internalize norms is not enough!
 How Universities use data should be known
—We aren’t corporations with competitive secrets
—We need to set up ways to report and share
 We need to be able describe what happened!
 Log events
 Version control your software
 Develop reporting methods

Consider the Consequences
 Moving from institutional decision
making to acting on individual data
 Lathe of Heaven – a mad social
scientist

Best Practices using Predictive Analytics
 Have to carefully present information to students
—Present a positive outlook
—Don’t personalize it – talk about a group of similar
students.
 The factors in the model may be less deterministic than
unobserved factors.
 Difference between causality and correlation.
 Beware the self-fulfilling prophecy

Cathy O’Neil
 @mathbabe, mathbabe.org
 Mathematician, former hedge-fund
quant

Weapons of Math Destruction
 Three factors make a model a WMD:
—Is the participant aware of the model? Is the model
opaque or invisible?
—Does the model work against the participant’s interest? Is
it unfair? Does it create feedback loops?
—Can the model scale?

Student Data Principles
 http://studentdataprinciples.org/
 Purpose and use of student data
 Timely access to data
 Data should not replace professional judgement.
 Data governance, security, breach notification

Student Data Pledge
 http://www.edtechmagazine.com/k12/article/2015/03/prote
ct-personal-student-information-pair-organizations-
recommends-commitment
 Don’t sell student data, use data to target ads, or profile
students for non-educational purposes
 Don’t collect more information or retain information longer
than necessary.
 Do disclose how, what and why

uCalgary Data Rules
 Freedom of Information and Privacy Act (1999)
—Students must be able to correct own info
—University must provide own info upon confirmation of ID
 Categories of Data Confidentiality
 Research Ethics Boards
—Data collection for University operations does not
generally fall under REB jurisdiction.

Financial Modeler’s Manifesto
 https://www.wilmott.com/financial-modelers-manifesto/
 Emanuel Derman and Paul Wilmott – January 7, 2009
 The Modelers’ Hippocratic Oath
— I will remember that I didn’t make the world, and it doesn’t satisfy my
equations.
— Though I will use models boldly to estimate value, I will not be overly
impressed by mathematics.
— I will never sacrifice reality for elegance without explaining why I have
done so.
— Nor will I give the people who use my model false comfort about its
accuracy. Instead, I will make explicit its assumptions and oversights.
— I understand that my work may have enormous effects on society and
the economy, many of them beyond my comprehension.

Responsible Use of Student Data in Higher Education
 http://gsd.su.domains/
 Opportunity to understand student learning and enhance
educational attainment.
 New questions about the ethical collection, use, and sharing
of information.
 Commitments to honor the integrity, discretion, and humanity
of students.
 Improve practice in light of accumulating information and
knowledge.

Maciej Cegłowski
 https://pinboard.in and @pinboard
 http://idlewords.com/talks/
 Two talks on Data in particular:
—http://idlewords.com/talks/deep_fried_data.htm
—http://idlewords.com/talks/haunted_by_data.htm

Basic Framework
 Safeguard Student Privacy
—Vendors; Monetizing Data
 Strong internal norms around data
 Consider and Measure Outcomes
 Work with Data Owners and Stewards
 Responsibility to Educate
 Consult with Students and Stakeholders
 Data should have a clear purpose

Next Steps
 Write down your norms/expectations for working with
Student data
 Set up a discussion with your co-workers about it.
 Seek out others who perform a similar role and discuss it.
 Discuss with the Student Data Steward at your institution.
 Send me your comments!

Continue the Conversation
 Follow me on twitter: @sechilds
 Stephen.Childs@ucalgary.ca or sechilds@gmail.com
 https://oia.ucalgary.ca/Contact
 https://www.meetup.com/PyData-Calgary/

Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education

More Related Content

What's hot (20)

Similar to Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education (20)

More from Cybera Inc. (20)

Recently uploaded (20)

Cyber Summit 2016: Establishing an Ethics Framework for Predictive Analytics in Higher Education

Editor's Notes