OER Impact at Community
Colleges
Dr. Rob Farrow
The Open University
Una Daly
CCCOER
Background
OER Research Hub
•

Research project at The Open University (UK)

•

Funded by William & Flora Hewlett Foundation for two years

•

Two professors lead four researchers among a team of ten

•

Tasked with building the most comprehensive picture of OER impact

•

Organised by eleven research hypotheses

•

Collaboration model across different educational sectors

•

Global reach but with a USA focus

oerresearchhub.org
Keyword
Performance
Openness
Access

Hypothesis
OER improve student performance/satisfaction
People use OER differently from other online materials
OER widen participation in education

Retention

OER can help at-risk learners to finish their studies

Reflection

OER use leads educators to reflect on their practice

Finance
Indicators
Support

OER adoption brings financial benefits for students/institutions
Informal learners use a variety of indicators when selecting OER
Informal learners develop their own forms of study support

Transition

OER support informal learners in moving to formal study

Policy

OER use encourages institutions to change their policies

Assessment

Informal assessments motivate learners using OER
Collaboration Model

Collaboration Model
Collaboration Model

Collaboration Model
CCCOER Mission & Goals

Promote adoption of OER to enhance teaching and learning
•
•
•

Document impact of OER on teaching and learning
Promote integration of OER into curricula
Share best practices for OER through professional development opportunities.

Funded by the William & Flora Hewlett Foundation
240+ Colleges in 16 states & provinces

General stuff about OERRH & CCCOER, collaboration
Research Collaboration

Focus on impact of OER adoption on teaching practice, institutional policies and
factors of cost and access
Feb-May 2013

May-Dec 2013

Ongoing

Faculty Survey
Development

Survey Deployment

Interviews and focus
groups with faculty;
policymakers; students

IRB Process

Jan-Apr 2014
Analysis &
Dissemination

Incorporation of
institutional evidence
into impact map
Survey Research in Community Colleges

•

Many community colleges require IRB approval for faculty surveys

•

IRBs may meet infrequently particularly during academic breaks

•

IRBs limit approvals to prevent duplication and survey fatigue

•

The process can take longer than expected - good planning is essential!
OER Impact Map
OER Impact Map
http://chaos.open.ac.uk
OER Impact Map
http://chaos.open.ac.uk
Survey of College Educators
Methodology
136 usable survey responses were recorded:
•
•
•
•
•
•

De Anza College, CA (5)
Foothill College, CA (33)
Houston Community College, TX (41)
Northern Virginia Community College (30)
Roane State Community College, TN (13)
South Florida Community College, FL (4)

This is being supplemented with qualitative data gathered from college
visits in Virginia, Maryland, Washington, and California.
Highest Educational Qualification
Bachelors Degree
4%

PhD or Professional
Doctorate
35%
Masters Degree
61%
Years of Teaching Experience
6%

1%

10%

More than 10

11%

7-10
4-6
1-3

Less than 1

72%
Types of Teaching Activity
60

50

36%

34%
No. of resposes

40

29%
30

23%
19%
17%

20

15%
10

0

Part-time blended
Full-time blended
Full-time
Part-time
(face to face Part-time face-toand
(face to face and
Full-time face to
Work-based training
distance/online
distance/online
face teaching
face teaching
distance/online)
distance/online)
teaching
teaching
teaching
teaching
Series1

20

23

26

31

39

46

49
Patterns of OER Use
80
70

52%

60

Of those that create
OER (24%) less than
half publish them on an
open (CC) licence

50
40
30

24%
18%

20

11%

10
0
I have adapted
I have created
open educational open educational
resources to fit my resources for
needs
study or teaching

I have added a
resource to a
repository

9%

7%

I have created
I have added
I have added
resources myself comments to a
comments to a
and published
repository
repository
them on a Creative regarding the suggesting ways of
Commons (CC)
quality of a
using a resource
licence
resource
• Most respondents were
experienced teachers with
postgraduate degrees
• A majority teach full-time and are
involved in online instruction
• Around half have adapted/used
OER but only around half of these
create or upload OER
OER Behaviours
Types of OER Used
Multimedia content is
around twice as popular
as other OER (including
textbooks)

Videos

69.8%

Images

64.7%

Lectures

38.8%

Quizzes

38.1%

Open textbooks

37.4%

Elements of a course (e.g. a module/unit)

34.5%

Tutorials

32.4%

E-books

29.5%

Lesson plans

25.9%

Learning tools, instruments and plugins

25.2%

Audio podcasts

23%

Infographics
Interactive games
Whole course
Data sets

Few reported using a
whole course of
OER, suggesting they
cherry pick resources as
needed

18 %
12.2%
10.8%
7.9%

Very few are using
openly available data to
teach
OER Repositories Used
80%

71%

70%
60%
50%

43%
32% 35%

40%

26% 26%

30%
20%
10%
0%

1%

2%

4%

5%

9%

11% 13%

17% 20%
Factors relevant to choosing OER
The resource being relevant to my particular interests/needs

69%

Created/uploaded by a reputable/trusted institution/person

58%

The resource having an open license allowing adaptation

51%

The resource being easy to download

49%

A detailed description of the resource content

48%

The resource having a Creative Commons license

48%

A description of learning objectives or outcomes being provided

48%

Positive user ratings or comments about the resource

43%

Use of interactive or multimedia content

43%

Having previously used this resource successfully

42%

Personal recommendation

42%

The resource being recently created, uploaded or updated

36%

The length/complexity of the resource

33%

Evidence of interest in that resource (e.g. lots of downloads)

29%

The resource having previously been used with students

21%

The resource featuring a catchy title or attractive image(s)

9%

Being required to use a resource for a project/task

7%
0%

10%

20%

30%

40%

50%

60%

70%

80%
Factors relevant to choosing OER
The resource being relevant to my particular interests/needs

69%

Created/uploaded by a reputable/trusted institution/person

58%

The resource having an open license allowing adaptation

51%

The resource being easy to download

49%

A detailed description of the resource content

48%

The resource having a Creative Commons license

48%

A description of learning objectives or outcomes being provided

48%

Positive user ratings or comments about the resource

43%

Use of interactive or multimedia content

43%

Having previously used this resource successfully

42%

Personal recommendation

42%

The resource being recently created, uploaded or updated

36%

The length/complexity of the resource

33%

Evidence of interest in that resource (e.g. lots of downloads)

29%

The resource having previously been used with students

21%

The resource featuring a catchy title or attractive image(s)

9%

Being required to use a resource for a project/task

7%
0%

10%

20%

30%

40%

50%

60%

70%

80%
Impact on teachers/students
Impact on teaching practice from OER use
I make more use of culturally diverse resources

23%

2%

I collaborate more with colleagues

22%

2%

I use a broader range of teaching and learning methods

21%

2%

I have a more up-to-date knowledge of my subject area

19%

2%

I more frequently compare my own teaching with others

18%

2%

I have improved ICT skills

16%

4%

I now use OER study to develop my teaching

14%

3%

I have broadened my coverage of the curriculum

14%

1%

I reflect more on the way that I teach

13%

5%

I make use of a wider range of multimedia

6%
0%

strongly agree–

agree–

3%
20%

neither agree nor disagree–

40%

disagree–

60%

80%

strongly disagree–

100%
OER: perceptions of impact on learners
... Increases experimentation with ways of learning

17.1%

... leads to interest in a wider range of subjects

15.5%

... develops independence and self-reliance

15.4%

... increases engagement with lesson content

15.2%

... leads to improved student grades

15.2%

... increases satisfaction with the learning experience

14.3%

... increases participation in class discussions

12.3%

... increases enthusiasm for future study

11.8%

... increases collaboration and/or peer-support

11.5%

... allows me to better accommodate learners' needs

8.7%

... builds confidence

8.6%

... increases interest in the subjects taught

8.5%
0%

Strongly agree

10%
Agree

20%
Neutral

30%

40%

Disagree

50%

60%

Strongly disagree

70%

80%

90%

100%
Financial Savings
Has your institution saved money
through OER?
“They don't like losing the
revenue stream from the
bookstore”
“Students return for
additional classes”

don't know
37%

yes
44%

“OER resources are old-school, low-tech
modules that are not peer-reviewed or
nationally normed. There are some
interesting tentative attempts at creativity
but much of OER is cr*p.”

no
19%

“I know that some
instructors are only using
OER which provides
substantial savings for our
students.”
Have your students saved money
through OER?
“I still use publishers'
textbooks in my classes. I
use OER as supplements to
the textbook.”

“Saving money is a big
incentive for students and
institutions.”
don't know
25%

“My students tell me and
enrollment in my classes has
continually increased over
those of my peers.”

no
13%

“I know that some
instructors are only using
OER which provides
substantial savings for our
students.”
yes
62%
“I developed an online
textbook for the personal
health class that I teach. This
saves each of my students
approximately $100.”
Student Retention
“OER use helps at-risk students to continue
their studies”
• More than 1/3 believed
that OER use promotes
student retention

strongly disagree
4%
disagree
8%

strongly agree
12%

• Around half feel it has no
effect
• The remainder (12%)
disagreed or strongly
disagreed with one noting
that “other things are more
important”.

agree
26%
neither agree nor
disagree
50%

“Driving down the cost coupled with the
ability to modify and adapt the material
to meet the needs of my learners are
two major factors in why I like using
OER materials.”
OER and retention of at-risk
students: comments against
“Some at-risk students benefit from
OER because of the obvious release
of financial obligation. Others are
challenged by the technology and
OER actually makes their success
rates drop.”

Attitudes

“My concern is that at-risk students
don't seem to do well in online
environments because it doesn't
always provide them the structure
they need.”

“Many at risk students don't have the
means to access high speed internet or
have limited technological availability. To
assume they do is simply wrong.
Additionally, they have more complicated
extrinsic factors impacting their
lives, which may require more intensive
contact from the instructor to keep them
involved in the course. OER is not going to
be a make or break issue of retention. It is
not a panacea for at-risk students.”
“The biggest factors in physics for student
attrition are time-constraints and
insufficient previous preparation, neither of
which is affected by the class resources.”
OER as promoter of student retention: factors

120
100
80

47%
35%

60
40
20

11%

18%

36%

57%

60%

36%

21%

0

Materials are Availability of Materials
Materials
available in culturally- can be used
can be
different
relevant for improving adapted to
languages
materials
non-native suit student
needs
language
skills

Greater
range of
learning
methods

Use of
Materials
Materials
Reduced
resources for can be used
can be
cost of study
improving
flexibly
accessed at materials
study skills
any time
Importance of Open Licensing
How important is open licensing (e.g. CC)?
not at all important
5%

neither important nor
unimportant
22%

crucial
20%
More than half feel that open
licensing is important, but fewer
actually practice it.
This could indicate either 1) that
educators are not confident about
licensing their work or 2) they feel
it is an avoidable addition to their
workload.

somewhat important
19%

very important
34%
Summary
Summary of Results

•

Most of the respondents have used some sort of OER, though only around a
quarter create OER

•

Most report positive effects on their teaching practice as a result of OER
use, particularly around peer collaboration and improved subject knowledge

•

A smaller proportion (but still in excess of 40%) feel that OER use directly leads
to improved reflection on pedagogical practice

•

Positive effects were also identified for learners, especially around increased
self-reliance, subject interest and experimentation

•

There were similar numbers who thought OER wasn‟t making much of a
difference and a core of what might be termed „anti-OER‟ responses
Summary of Results

•

There were mixed views about whether OER was saving institutions money, but
approximately 2/3 felt that students had saved money

•

Around 1/3 believe that OER is improving student attrition while around 1/2
believe it is not having an effect

•

Only around half of OER creators have used open licensing

•

There is a core of advocates who understand and actively promote OER; they
adopt open educational practices and believe it leads to benefits
Next Steps
Next Steps

•

Further analysis of this cohort, including isolation of the OER advocates and
OER detractors to identify behavioural and attitudinal patterns

•

Mapping the survey data

•

Cross-referencing with other OERRH surveys, e.g. Saylor „informal learners‟
survey (n=3014) to build picture of different stakeholders/sectors

•

Integration of institutional metrics and qualitative data gathered in field work

•

Open dissemination of raw data; openly licensed research instruments

•

Adding your data to OER Impact Map?
Thanks for listening!
oerresearchhub.org
chaos.open.ac.uk
oerconsortium.org
in service of The Open University

Open Educational Resources Impact in Community Colleges

  • 1.
    OER Impact atCommunity Colleges Dr. Rob Farrow The Open University Una Daly CCCOER
  • 2.
  • 3.
    OER Research Hub • Researchproject at The Open University (UK) • Funded by William & Flora Hewlett Foundation for two years • Two professors lead four researchers among a team of ten • Tasked with building the most comprehensive picture of OER impact • Organised by eleven research hypotheses • Collaboration model across different educational sectors • Global reach but with a USA focus oerresearchhub.org
  • 4.
    Keyword Performance Openness Access Hypothesis OER improve studentperformance/satisfaction People use OER differently from other online materials OER widen participation in education Retention OER can help at-risk learners to finish their studies Reflection OER use leads educators to reflect on their practice Finance Indicators Support OER adoption brings financial benefits for students/institutions Informal learners use a variety of indicators when selecting OER Informal learners develop their own forms of study support Transition OER support informal learners in moving to formal study Policy OER use encourages institutions to change their policies Assessment Informal assessments motivate learners using OER
  • 5.
  • 6.
  • 8.
    CCCOER Mission &Goals Promote adoption of OER to enhance teaching and learning • • • Document impact of OER on teaching and learning Promote integration of OER into curricula Share best practices for OER through professional development opportunities. Funded by the William & Flora Hewlett Foundation
  • 9.
    240+ Colleges in16 states & provinces General stuff about OERRH & CCCOER, collaboration
  • 10.
    Research Collaboration Focus onimpact of OER adoption on teaching practice, institutional policies and factors of cost and access Feb-May 2013 May-Dec 2013 Ongoing Faculty Survey Development Survey Deployment Interviews and focus groups with faculty; policymakers; students IRB Process Jan-Apr 2014 Analysis & Dissemination Incorporation of institutional evidence into impact map
  • 11.
    Survey Research inCommunity Colleges • Many community colleges require IRB approval for faculty surveys • IRBs may meet infrequently particularly during academic breaks • IRBs limit approvals to prevent duplication and survey fatigue • The process can take longer than expected - good planning is essential!
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
    136 usable surveyresponses were recorded: • • • • • • De Anza College, CA (5) Foothill College, CA (33) Houston Community College, TX (41) Northern Virginia Community College (30) Roane State Community College, TN (13) South Florida Community College, FL (4) This is being supplemented with qualitative data gathered from college visits in Virginia, Maryland, Washington, and California.
  • 18.
    Highest Educational Qualification BachelorsDegree 4% PhD or Professional Doctorate 35% Masters Degree 61%
  • 19.
    Years of TeachingExperience 6% 1% 10% More than 10 11% 7-10 4-6 1-3 Less than 1 72%
  • 20.
    Types of TeachingActivity 60 50 36% 34% No. of resposes 40 29% 30 23% 19% 17% 20 15% 10 0 Part-time blended Full-time blended Full-time Part-time (face to face Part-time face-toand (face to face and Full-time face to Work-based training distance/online distance/online face teaching face teaching distance/online) distance/online) teaching teaching teaching teaching Series1 20 23 26 31 39 46 49
  • 21.
    Patterns of OERUse 80 70 52% 60 Of those that create OER (24%) less than half publish them on an open (CC) licence 50 40 30 24% 18% 20 11% 10 0 I have adapted I have created open educational open educational resources to fit my resources for needs study or teaching I have added a resource to a repository 9% 7% I have created I have added I have added resources myself comments to a comments to a and published repository repository them on a Creative regarding the suggesting ways of Commons (CC) quality of a using a resource licence resource
  • 22.
    • Most respondentswere experienced teachers with postgraduate degrees • A majority teach full-time and are involved in online instruction • Around half have adapted/used OER but only around half of these create or upload OER
  • 23.
  • 24.
    Types of OERUsed Multimedia content is around twice as popular as other OER (including textbooks) Videos 69.8% Images 64.7% Lectures 38.8% Quizzes 38.1% Open textbooks 37.4% Elements of a course (e.g. a module/unit) 34.5% Tutorials 32.4% E-books 29.5% Lesson plans 25.9% Learning tools, instruments and plugins 25.2% Audio podcasts 23% Infographics Interactive games Whole course Data sets Few reported using a whole course of OER, suggesting they cherry pick resources as needed 18 % 12.2% 10.8% 7.9% Very few are using openly available data to teach
  • 25.
    OER Repositories Used 80% 71% 70% 60% 50% 43% 32%35% 40% 26% 26% 30% 20% 10% 0% 1% 2% 4% 5% 9% 11% 13% 17% 20%
  • 26.
    Factors relevant tochoosing OER The resource being relevant to my particular interests/needs 69% Created/uploaded by a reputable/trusted institution/person 58% The resource having an open license allowing adaptation 51% The resource being easy to download 49% A detailed description of the resource content 48% The resource having a Creative Commons license 48% A description of learning objectives or outcomes being provided 48% Positive user ratings or comments about the resource 43% Use of interactive or multimedia content 43% Having previously used this resource successfully 42% Personal recommendation 42% The resource being recently created, uploaded or updated 36% The length/complexity of the resource 33% Evidence of interest in that resource (e.g. lots of downloads) 29% The resource having previously been used with students 21% The resource featuring a catchy title or attractive image(s) 9% Being required to use a resource for a project/task 7% 0% 10% 20% 30% 40% 50% 60% 70% 80%
  • 27.
    Factors relevant tochoosing OER The resource being relevant to my particular interests/needs 69% Created/uploaded by a reputable/trusted institution/person 58% The resource having an open license allowing adaptation 51% The resource being easy to download 49% A detailed description of the resource content 48% The resource having a Creative Commons license 48% A description of learning objectives or outcomes being provided 48% Positive user ratings or comments about the resource 43% Use of interactive or multimedia content 43% Having previously used this resource successfully 42% Personal recommendation 42% The resource being recently created, uploaded or updated 36% The length/complexity of the resource 33% Evidence of interest in that resource (e.g. lots of downloads) 29% The resource having previously been used with students 21% The resource featuring a catchy title or attractive image(s) 9% Being required to use a resource for a project/task 7% 0% 10% 20% 30% 40% 50% 60% 70% 80%
  • 28.
  • 29.
    Impact on teachingpractice from OER use I make more use of culturally diverse resources 23% 2% I collaborate more with colleagues 22% 2% I use a broader range of teaching and learning methods 21% 2% I have a more up-to-date knowledge of my subject area 19% 2% I more frequently compare my own teaching with others 18% 2% I have improved ICT skills 16% 4% I now use OER study to develop my teaching 14% 3% I have broadened my coverage of the curriculum 14% 1% I reflect more on the way that I teach 13% 5% I make use of a wider range of multimedia 6% 0% strongly agree– agree– 3% 20% neither agree nor disagree– 40% disagree– 60% 80% strongly disagree– 100%
  • 30.
    OER: perceptions ofimpact on learners ... Increases experimentation with ways of learning 17.1% ... leads to interest in a wider range of subjects 15.5% ... develops independence and self-reliance 15.4% ... increases engagement with lesson content 15.2% ... leads to improved student grades 15.2% ... increases satisfaction with the learning experience 14.3% ... increases participation in class discussions 12.3% ... increases enthusiasm for future study 11.8% ... increases collaboration and/or peer-support 11.5% ... allows me to better accommodate learners' needs 8.7% ... builds confidence 8.6% ... increases interest in the subjects taught 8.5% 0% Strongly agree 10% Agree 20% Neutral 30% 40% Disagree 50% 60% Strongly disagree 70% 80% 90% 100%
  • 31.
  • 32.
    Has your institutionsaved money through OER? “They don't like losing the revenue stream from the bookstore” “Students return for additional classes” don't know 37% yes 44% “OER resources are old-school, low-tech modules that are not peer-reviewed or nationally normed. There are some interesting tentative attempts at creativity but much of OER is cr*p.” no 19% “I know that some instructors are only using OER which provides substantial savings for our students.”
  • 33.
    Have your studentssaved money through OER? “I still use publishers' textbooks in my classes. I use OER as supplements to the textbook.” “Saving money is a big incentive for students and institutions.” don't know 25% “My students tell me and enrollment in my classes has continually increased over those of my peers.” no 13% “I know that some instructors are only using OER which provides substantial savings for our students.” yes 62% “I developed an online textbook for the personal health class that I teach. This saves each of my students approximately $100.”
  • 34.
  • 35.
    “OER use helpsat-risk students to continue their studies” • More than 1/3 believed that OER use promotes student retention strongly disagree 4% disagree 8% strongly agree 12% • Around half feel it has no effect • The remainder (12%) disagreed or strongly disagreed with one noting that “other things are more important”. agree 26% neither agree nor disagree 50% “Driving down the cost coupled with the ability to modify and adapt the material to meet the needs of my learners are two major factors in why I like using OER materials.”
  • 36.
    OER and retentionof at-risk students: comments against “Some at-risk students benefit from OER because of the obvious release of financial obligation. Others are challenged by the technology and OER actually makes their success rates drop.” Attitudes “My concern is that at-risk students don't seem to do well in online environments because it doesn't always provide them the structure they need.” “Many at risk students don't have the means to access high speed internet or have limited technological availability. To assume they do is simply wrong. Additionally, they have more complicated extrinsic factors impacting their lives, which may require more intensive contact from the instructor to keep them involved in the course. OER is not going to be a make or break issue of retention. It is not a panacea for at-risk students.” “The biggest factors in physics for student attrition are time-constraints and insufficient previous preparation, neither of which is affected by the class resources.”
  • 37.
    OER as promoterof student retention: factors 120 100 80 47% 35% 60 40 20 11% 18% 36% 57% 60% 36% 21% 0 Materials are Availability of Materials Materials available in culturally- can be used can be different relevant for improving adapted to languages materials non-native suit student needs language skills Greater range of learning methods Use of Materials Materials Reduced resources for can be used can be cost of study improving flexibly accessed at materials study skills any time
  • 38.
  • 39.
    How important isopen licensing (e.g. CC)? not at all important 5% neither important nor unimportant 22% crucial 20% More than half feel that open licensing is important, but fewer actually practice it. This could indicate either 1) that educators are not confident about licensing their work or 2) they feel it is an avoidable addition to their workload. somewhat important 19% very important 34%
  • 40.
  • 41.
    Summary of Results • Mostof the respondents have used some sort of OER, though only around a quarter create OER • Most report positive effects on their teaching practice as a result of OER use, particularly around peer collaboration and improved subject knowledge • A smaller proportion (but still in excess of 40%) feel that OER use directly leads to improved reflection on pedagogical practice • Positive effects were also identified for learners, especially around increased self-reliance, subject interest and experimentation • There were similar numbers who thought OER wasn‟t making much of a difference and a core of what might be termed „anti-OER‟ responses
  • 42.
    Summary of Results • Therewere mixed views about whether OER was saving institutions money, but approximately 2/3 felt that students had saved money • Around 1/3 believe that OER is improving student attrition while around 1/2 believe it is not having an effect • Only around half of OER creators have used open licensing • There is a core of advocates who understand and actively promote OER; they adopt open educational practices and believe it leads to benefits
  • 43.
  • 44.
    Next Steps • Further analysisof this cohort, including isolation of the OER advocates and OER detractors to identify behavioural and attitudinal patterns • Mapping the survey data • Cross-referencing with other OERRH surveys, e.g. Saylor „informal learners‟ survey (n=3014) to build picture of different stakeholders/sectors • Integration of institutional metrics and qualitative data gathered in field work • Open dissemination of raw data; openly licensed research instruments • Adding your data to OER Impact Map?
  • 45.
  • 46.
    in service ofThe Open University