Guidelines to Promote Best
Practices, Advice and Information to
an Online Faculty Audience
       HE527 Final Project
       HE527: Assessing Student Learning
       Online
       Sharifah Ali
Introduction
   Below is a set of guidelines for an online faculty audience. With increased
    demand for online learning as well as more institutions of higher learning
    striving to provide diverse educational opportunities, online learning
    continues to grow as a viable means of providing increased access to a
    greater number of students (Keengwe & Kidd, 2010).
   No matter how strong the business case is for creating an online education
    program, every college should be ready for faculty apprehension, or even
    resistance. Just 31 percent of chief academic officers who responded to the
    Babson/College Board survey believe their faculty members accept the
    value and legitimacy of online education (Blackboard, 2010). Certain
    characteristics found within some faculty cultures can act to impede the
    development of an effective assessment system. Some faculty members
    are indifferent about the need to have their course outcomes evaluated in
    new ways. Some may feel satisfied that the quality of the department’s
    program is adequate and that no new assessment system is needed. On the
    other hand, some faculty characteristics supportive of a new assessment
    system are valuing quality, empowered faculty, desire for certainty, and
    respect for industry involvement (Tener, 1999).
Introduction
   If program assessment is ever to be taken seriously and used effectively, it
    must be systematic and built into the very structure of the program or
    department by the faculty themselves (Gigliotti, 2009). The use of program
    assessment helps ensure that regular program reviews occur, keeps the focus
    on outcomes and competency development, and keeps the process learner
    focused. It is a comprehensive means by which courses and programs can be
    evaluated on an ongoing basis. The results can be analyzed to determine
    areas of the curriculum that are working well, need improvement, or might
    be missing (Palloff & Pratt, 2009).
Development of Learning Objectives
   Given that a well-designed online course should be learner focused and
    centered, it follows that student assessment within that course should be the
    same. The reflective process that should be included in an online course
    provides the basis for learner-centered assessment. Students should be given
    credit for self-reflection, and it should be incorporated into the design and
    expectations for the online course. Each collaborative activity should
    contain a reflective component. At the very least, students should be asked
    to reflect on their participation in the activity and their contributions to the
    group (Palloff & Pratt, 2009).
Alignment
    Edvantia (2005) defines alignment as “the degree to which expectations
    [standards] and assessments are in agreement and serve in conjunction
    with one another to guide the system towards students learning what they
    are expected to know and do” (p. 3). ”There need to be strong links among
    assessment of student performance at the course outcome level, the
    competencies developed for the online program of study, and the mission
    of the university to create alignment.
   Curriculum alignment includes alignment between and among several
    education variables, including state standards, state-mandated
    assessments, resources such as textbooks, content of instruction, and
    instructional strategies. The extent to which textbooks are aligned with
    standards and assessments is important due to the widespread use of
    textbooks to guide instruction (Edvantia, 2005). Given that a well
    designed online course should be learner focused and centered, it follows
    that student assessment within that course should be the same. When
    instructors use multiple measures and authentic assessments are based in
    real life and not just classroom learning, there is more likelihood that a
    true measurement of student competency and performance will result
    (Palloff & Pratt, 2009).
Alignment
   The process of instructional alignment is labeled as the process between
    instruction and assessments. Students can demonstrate mastery, when
    instruction is designed to ensure that students perform well on competency
    tests (Edvantia, 2005).
Assessment Planning
    Before an instructor can embark on the development of good online
    activities and assessments, they must have a solid understanding of how
    assessment fits into the scheme of course development as well as the
    components it comprises. Instructors need to understand learning
    outcomes, their importance in the learning process, their development, and
    how to achieve them. The components that make up course design are
    often referred to as competencies, outcomes, and objectives. There is
    different meaning for each term and assessment for each term differs as
    well.
   In a learner-centered teaching design, students construct knowledge by
    gathering together and synthesizing information by using inquiry,
    communication, critical thinking, and problem solving. The instructor’s
    role is to facilitate the process, and instructors and students together assess
    learning. By this method, teaching and learning are intertwined and the
    results are best assessed through papers, projects, performance of
    authentic application activities, and portfolios (Palloff & Pratt, 2009).
Assessment Planning
   Online assessment is a very contemporary phenomenon that uses the most
    current electronic technologies. It involves instructors and instructional
    designers who are innovative and who wish to go beyond conventional
    approaches to instruction and learning (Oosterhof, Conrad, & Ely, 2008). In
    order to conduct effective assessments at the course and program level,
    faculty also need be trained on how to create effective assessments by
    simply designing their own courses. Many instructors do not write or design
    the courses they teach (Palloff & Pratt, 2009). If a faculty member has
    created their own course, then an assessment that demonstrates the
    instructor’s own objective to test their student’s knowledge will be more
    effective.
Collect Assessment Data
   Providing feedback directly after an assessment makes it more likely that
    students will recall the logic they used when answering items and will take
    advantage of that information when reading the feedback.
   The feedback provided at the end of an assessment should match as
    closely as possible the reasoning students used during the assessment.
    Instead of creating feedback tied to how the student responds, it is better
    to prepare just one feedback message for each item that explains the
    general logic of the answer and, in the case of multiple- choice items, or
    fixed-response items, why the wrong responses are incorrect. This allows
    the student to evaluate the logic he or she used to answer the item
    (Oosterhof et al., 2008).
Collect Assessment Data
   With today’s information technology it is possible to improve both the
    construction of tests and the quality of feedback while saving instructor
    time. Immediate performance feedback positively affects students, and a
    fast and accurate grading process should enhance student learning. By using
    information technology to automate the preparation of exam questions,
    answer keys, and student feedback, student learning is enhanced by
    providing typo-free exam modifications, faster and more accurate grading,
    and detailed written explanations for all exam questions. The instructor does
    not have to allocate a significant amount of time for exam and answer key
    preparation, having to come up with different stories and numbers for each
    question version, and making sure that each version is equally challenging
    (Ozkul, 2009).
Collect Assessment Data
   Consistency in scoring is essential with all item formats. To facilitate
    reliable scoring, the scoring plan should incorporate three characteristics in
    constructed-response items; the total number of points assigned to the item
    based on its importance relative to other items, specific attributes to be
    evaluated in students’ responses, and for each attribute, criteria for
    rewarding points, including partial credit. Fixed –response assessment
    format lends itself to an online administration. Responses can be scored
    immediately as each student completes the test, and the student’s records
    can be updated automatically without the instructor’s involvement
    (Oosterhof et al., 2008).
Using Results
   Rubrics have become a useful tool in the assessment of student work online.
    They provide clarity in expectations for student performance and assist in
    turning subjective assessments of assignments, into more objective
    measures of student work (Palloff & Pratt, 2009). Providing examples of
    superior work help amplify instructions to students and provide information
    about the scoring plan and help to set standards. Checklists are another
    option to interpret assessment data. A checklist can structure observations of
    a learner in a performance assessment. Checklists can be used in a variety of
    settings to establish the presence or absence of a series of conditions. There
    are also four references that are widely used for interpreting the
    performance of learners. The four references are; ability-referenced, growth-
    referenced, norm-referenced, and criterion-referenced. Of these, the norm-
    referenced and criterion-referenced interpretations tend to be the most
    useful (Oosterhof et al., 2008).
Continuous Improvement
   External assessors and their roles vary widely, but they are often most
    valued for providing oversight on the complete assessment process. They
    comment on the exam questions, they provide feedback to the teachers and
    their institution on links between teaching and assessment, they interview
    the students and the teachers, and they compare processes and outcomes in
    one institution with those in others. External assessors provides impartial
    advice to institutions to ensure that students are treated fairly, to enable
    teaching teams to understand their standards better, and to improve the
    quality of the education which they provide. Evaluation can be achieved on
    a group-wide basis and the tools and techniques involve the use of surveys,
    questionnaires and semantic differential techniques (Shephard, 2009).
References
   Keengwe, J., & Kidd, T. T. (2010). Towards best practices in online
    learning and teaching in higher education. Merlot Journal of Online
    Learning and Teaching, 6(2), 1-9. Retrieved from
    http://jolt.merlot.org/vol6no2/keengwe_0610.pdf
   Oosterhof, A., Conrad, R.M., & Ely, D. (2008). Assessing Learners Online.
    New Jersey: Pearson Merrill Prentice Hall
   Ozkul, A. (2009). Using information technology to enhance assessment of
    learning: Automating preparation of course exam materials and student
    feedback. Information Technology and Performance Journal, 25(1).
    Retrieved from http://www.osra.org/itlpj/ozkulspring2009.pdf
   Palloff, R.M., & Pratt, K. (2009). Assessing the Online Learner: Resources
    and Strategies for Faculty. San Francisco: Jossey-Bass.
   Shephard, K. (2009). E is for exploration: Assessing hard-to-measure
    learning outcomes. British Journal of Educational Technology, 40(2), 386-
    398. Retrieved from
    http://ehis.ebscohost.com.lib.kaplan.edu/ehost/pdfviewer/pdfviewer?sid
    =50f5205a-f447-4546-8e7d-120de776f5ae%40sessionmgr111&vid=6&hid=120
References
   Tener, R. K. (1999, January). Outcomes assessment and the faculty
    culture: Conflict or congruence. Jornal of Engineering Education.
    Retrieved from http://www.jee.org/1999/january/614.pdf
   Educational assessment: designing a system for a more meaningful
    approach. (2011, June). Academic Leader. Retrieved from
    http://www.facultyfocus.com/free-reports/educational-assessment-
    designing-a-system-for-more-meaningful-results/
   Smart Steps to Online Learning. (2012). In Blackboard Career
    Colleges. Retrieved from
    http://www.blackboard.com/CMSPages/GetFile.aspx?
    guid=b4d4d142-7530-4290-a2c2-510a82b7f127
   U.S. Department of Education, Institute of Educational Sciences,
    Edvantia. (2006). Aligned curriculum and student achievement
    (contract number EDO-01-CO-0016). Retrieved from
    http://www.edvantia.org/pdta/pdf/Aligned.pdf

He547 unit 7 tech intergration

  • 1.
    Guidelines to PromoteBest Practices, Advice and Information to an Online Faculty Audience HE527 Final Project HE527: Assessing Student Learning Online Sharifah Ali
  • 2.
    Introduction  Below is a set of guidelines for an online faculty audience. With increased demand for online learning as well as more institutions of higher learning striving to provide diverse educational opportunities, online learning continues to grow as a viable means of providing increased access to a greater number of students (Keengwe & Kidd, 2010).  No matter how strong the business case is for creating an online education program, every college should be ready for faculty apprehension, or even resistance. Just 31 percent of chief academic officers who responded to the Babson/College Board survey believe their faculty members accept the value and legitimacy of online education (Blackboard, 2010). Certain characteristics found within some faculty cultures can act to impede the development of an effective assessment system. Some faculty members are indifferent about the need to have their course outcomes evaluated in new ways. Some may feel satisfied that the quality of the department’s program is adequate and that no new assessment system is needed. On the other hand, some faculty characteristics supportive of a new assessment system are valuing quality, empowered faculty, desire for certainty, and respect for industry involvement (Tener, 1999).
  • 3.
    Introduction  If program assessment is ever to be taken seriously and used effectively, it must be systematic and built into the very structure of the program or department by the faculty themselves (Gigliotti, 2009). The use of program assessment helps ensure that regular program reviews occur, keeps the focus on outcomes and competency development, and keeps the process learner focused. It is a comprehensive means by which courses and programs can be evaluated on an ongoing basis. The results can be analyzed to determine areas of the curriculum that are working well, need improvement, or might be missing (Palloff & Pratt, 2009).
  • 4.
    Development of LearningObjectives  Given that a well-designed online course should be learner focused and centered, it follows that student assessment within that course should be the same. The reflective process that should be included in an online course provides the basis for learner-centered assessment. Students should be given credit for self-reflection, and it should be incorporated into the design and expectations for the online course. Each collaborative activity should contain a reflective component. At the very least, students should be asked to reflect on their participation in the activity and their contributions to the group (Palloff & Pratt, 2009).
  • 5.
    Alignment  Edvantia (2005) defines alignment as “the degree to which expectations [standards] and assessments are in agreement and serve in conjunction with one another to guide the system towards students learning what they are expected to know and do” (p. 3). ”There need to be strong links among assessment of student performance at the course outcome level, the competencies developed for the online program of study, and the mission of the university to create alignment.  Curriculum alignment includes alignment between and among several education variables, including state standards, state-mandated assessments, resources such as textbooks, content of instruction, and instructional strategies. The extent to which textbooks are aligned with standards and assessments is important due to the widespread use of textbooks to guide instruction (Edvantia, 2005). Given that a well designed online course should be learner focused and centered, it follows that student assessment within that course should be the same. When instructors use multiple measures and authentic assessments are based in real life and not just classroom learning, there is more likelihood that a true measurement of student competency and performance will result (Palloff & Pratt, 2009).
  • 6.
    Alignment  The process of instructional alignment is labeled as the process between instruction and assessments. Students can demonstrate mastery, when instruction is designed to ensure that students perform well on competency tests (Edvantia, 2005).
  • 7.
    Assessment Planning  Before an instructor can embark on the development of good online activities and assessments, they must have a solid understanding of how assessment fits into the scheme of course development as well as the components it comprises. Instructors need to understand learning outcomes, their importance in the learning process, their development, and how to achieve them. The components that make up course design are often referred to as competencies, outcomes, and objectives. There is different meaning for each term and assessment for each term differs as well.  In a learner-centered teaching design, students construct knowledge by gathering together and synthesizing information by using inquiry, communication, critical thinking, and problem solving. The instructor’s role is to facilitate the process, and instructors and students together assess learning. By this method, teaching and learning are intertwined and the results are best assessed through papers, projects, performance of authentic application activities, and portfolios (Palloff & Pratt, 2009).
  • 8.
    Assessment Planning  Online assessment is a very contemporary phenomenon that uses the most current electronic technologies. It involves instructors and instructional designers who are innovative and who wish to go beyond conventional approaches to instruction and learning (Oosterhof, Conrad, & Ely, 2008). In order to conduct effective assessments at the course and program level, faculty also need be trained on how to create effective assessments by simply designing their own courses. Many instructors do not write or design the courses they teach (Palloff & Pratt, 2009). If a faculty member has created their own course, then an assessment that demonstrates the instructor’s own objective to test their student’s knowledge will be more effective.
  • 9.
    Collect Assessment Data  Providing feedback directly after an assessment makes it more likely that students will recall the logic they used when answering items and will take advantage of that information when reading the feedback.  The feedback provided at the end of an assessment should match as closely as possible the reasoning students used during the assessment. Instead of creating feedback tied to how the student responds, it is better to prepare just one feedback message for each item that explains the general logic of the answer and, in the case of multiple- choice items, or fixed-response items, why the wrong responses are incorrect. This allows the student to evaluate the logic he or she used to answer the item (Oosterhof et al., 2008).
  • 10.
    Collect Assessment Data  With today’s information technology it is possible to improve both the construction of tests and the quality of feedback while saving instructor time. Immediate performance feedback positively affects students, and a fast and accurate grading process should enhance student learning. By using information technology to automate the preparation of exam questions, answer keys, and student feedback, student learning is enhanced by providing typo-free exam modifications, faster and more accurate grading, and detailed written explanations for all exam questions. The instructor does not have to allocate a significant amount of time for exam and answer key preparation, having to come up with different stories and numbers for each question version, and making sure that each version is equally challenging (Ozkul, 2009).
  • 11.
    Collect Assessment Data  Consistency in scoring is essential with all item formats. To facilitate reliable scoring, the scoring plan should incorporate three characteristics in constructed-response items; the total number of points assigned to the item based on its importance relative to other items, specific attributes to be evaluated in students’ responses, and for each attribute, criteria for rewarding points, including partial credit. Fixed –response assessment format lends itself to an online administration. Responses can be scored immediately as each student completes the test, and the student’s records can be updated automatically without the instructor’s involvement (Oosterhof et al., 2008).
  • 12.
    Using Results  Rubrics have become a useful tool in the assessment of student work online. They provide clarity in expectations for student performance and assist in turning subjective assessments of assignments, into more objective measures of student work (Palloff & Pratt, 2009). Providing examples of superior work help amplify instructions to students and provide information about the scoring plan and help to set standards. Checklists are another option to interpret assessment data. A checklist can structure observations of a learner in a performance assessment. Checklists can be used in a variety of settings to establish the presence or absence of a series of conditions. There are also four references that are widely used for interpreting the performance of learners. The four references are; ability-referenced, growth- referenced, norm-referenced, and criterion-referenced. Of these, the norm- referenced and criterion-referenced interpretations tend to be the most useful (Oosterhof et al., 2008).
  • 13.
    Continuous Improvement  External assessors and their roles vary widely, but they are often most valued for providing oversight on the complete assessment process. They comment on the exam questions, they provide feedback to the teachers and their institution on links between teaching and assessment, they interview the students and the teachers, and they compare processes and outcomes in one institution with those in others. External assessors provides impartial advice to institutions to ensure that students are treated fairly, to enable teaching teams to understand their standards better, and to improve the quality of the education which they provide. Evaluation can be achieved on a group-wide basis and the tools and techniques involve the use of surveys, questionnaires and semantic differential techniques (Shephard, 2009).
  • 14.
    References  Keengwe, J., & Kidd, T. T. (2010). Towards best practices in online learning and teaching in higher education. Merlot Journal of Online Learning and Teaching, 6(2), 1-9. Retrieved from http://jolt.merlot.org/vol6no2/keengwe_0610.pdf  Oosterhof, A., Conrad, R.M., & Ely, D. (2008). Assessing Learners Online. New Jersey: Pearson Merrill Prentice Hall  Ozkul, A. (2009). Using information technology to enhance assessment of learning: Automating preparation of course exam materials and student feedback. Information Technology and Performance Journal, 25(1). Retrieved from http://www.osra.org/itlpj/ozkulspring2009.pdf  Palloff, R.M., & Pratt, K. (2009). Assessing the Online Learner: Resources and Strategies for Faculty. San Francisco: Jossey-Bass.  Shephard, K. (2009). E is for exploration: Assessing hard-to-measure learning outcomes. British Journal of Educational Technology, 40(2), 386- 398. Retrieved from http://ehis.ebscohost.com.lib.kaplan.edu/ehost/pdfviewer/pdfviewer?sid =50f5205a-f447-4546-8e7d-120de776f5ae%40sessionmgr111&vid=6&hid=120
  • 15.
    References  Tener, R. K. (1999, January). Outcomes assessment and the faculty culture: Conflict or congruence. Jornal of Engineering Education. Retrieved from http://www.jee.org/1999/january/614.pdf  Educational assessment: designing a system for a more meaningful approach. (2011, June). Academic Leader. Retrieved from http://www.facultyfocus.com/free-reports/educational-assessment- designing-a-system-for-more-meaningful-results/  Smart Steps to Online Learning. (2012). In Blackboard Career Colleges. Retrieved from http://www.blackboard.com/CMSPages/GetFile.aspx? guid=b4d4d142-7530-4290-a2c2-510a82b7f127  U.S. Department of Education, Institute of Educational Sciences, Edvantia. (2006). Aligned curriculum and student achievement (contract number EDO-01-CO-0016). Retrieved from http://www.edvantia.org/pdta/pdf/Aligned.pdf