TECH4102 Evaluation in Educational Technology   Evaluation of Computer-Based Instruction   Dr. Alaa Sadik Department of Instructional & Learning Technologies www.alaasadik.net [email_address]
Dimensions of Evaluation in Computer-Based Instruction Pedagogical Dimension Technical Dimension Evaluation of CBI
Evaluation of Computer-Based Instruction Pedagogical aspects goal;  underlying psychology;  learning strategy; accuracy of content; student role;  teacher role;  flexibility of integration;  motivation and reinforcement;  meeting individual differences;  interactivity;  assessment; cultural sensitivity. Technical aspects navigation; standardization; accessibility;  user-friendliness; presentation;  documentation; performance; costs.
Perspectives on Teaching and Learning with Computers 1. Learning theories Behaviorism  Constructivism
Perspectives on Teaching and Learning with Computers Behaviorism: Assumptions A teacher-directed approach ( controlled by the designer ). Learning is manipulated by the teacher ( the courseware ). Learning is described as a stimulus and response relationship. Students learn new concepts via the observation of the teacher ( courseware ) and content.
Perspectives on Teaching and Learning with Computers Behaviorism: Assumptions Learning content is provided in small chunks. Learning content is provided in linear sequence. The student can’t study a new topic before achieving the prior topic. Learning processes can be studied most objectively when the focus of study is on stimulus and responses. Learning involves a behavior change .
Perspectives on Teaching and Learning with Computers Behaviorism: Elements of a good courseware Gaining attention Informing learner of the objective Recall of prerequisite learning Presenting the stimulus material Providing learning guidance Electing the performance Providing feedback Assessing the performance Enhancing retention and transfer
Perspectives on Teaching and Learning with Computers Behaviorism: Example of software - Drill and practice software - Tutorials e.g.,  Math Blaster ,  Math Munchers Deluxe   Word Munchers   Early Music Skills Organic Chemistry
Perspectives on Teaching and Learning with Computers Constructivism: Assumptions   Learning is an active and individualized process. The learner must construct new knowledge based on his/her own individualized experience (learner background). The student is the producer of information rather than the consumer. The teacher is a member of learning community rather than the only source of information (facilitator). Learning emphasizes the application of knowledge in real life situations .
Perspectives on Teaching and Learning with Computers Constructivism: Assumptions   Teaching must fosters critical thinking and creates active and motivated learners.  Learning is a cooperative process constructed within the social context of the classroom. Assessment is a continuous and interactive process that measures the achievement of the learner and the quality of the learning experience.
Perspectives on Teaching and Learning with Computers 2. Cooperative learning Ways of learning: Individual  (alone) Competitive  (against each other) Cooperative  (together)
Perspectives on Teaching and Learning with Computers 2. Cooperative learning Is a  productive strategy that uses small  teams of students  through which they work together  to develop communication skills, higher-order thinking skills , and social awareness, and maximize learning. The computer is a powerful tool for facilitating cooperation can serve an important role in cooperative learning environments.   Suitable for computer-poor schools
Perspectives on Teaching and Learning with Computers 3. Types of intelligence Professor Howard Gardner
Perspectives on Teaching and Learning with Computers 3. Types of intelligence 1. Linguistic  enjoy writing, reading, telling stories or doing crossword puzzles. 2. Logical-Mathematical   interested in patterns, categories and relationships. They are drawn to arithmetic problems, strategy games and experiments. 3. Bodily-Kinesthetic   process knowledge through bodily sensations, often athletic, dancers or good at crafts such as sewing or woodworking. 4. Spatial  think in images and pictures, fascinated with mazes or puzzles, or spend free time drawing, building with Leggos.
Perspectives on Teaching and Learning with Computers 3. Types of intelligence 5. Musical   always singing or drumming to themselves. They are usually quite aware of sounds others may miss.  6. Interpersonal   leaders among their peers, good at communicating and who seem to understand others' feelings and motives possess interpersonal intelligence.  7. Intrapersonal   shy, aware of their own feelings and are self-motivated.
Perspectives on Teaching and Learning with Computers 3. Types of intelligence  Using Technology to Teach to the Multiple Intelligences Example: Linguistic Intelligence These kids are most likely to use  word processing  programs on the computer than anything else. They will know how to manipulate the text, do crazy things like create columns and outlines that actually work and look good.
Perspectives on Teaching and Learning with Computers 4. Perception Perception is the process of acquiring, interpreting, selecting and organizing  sensory information . e.g., visual perception, auditory perception
Perspectives on Teaching and Learning with Computers 4. Perception Because perceptions are organized into understandings:  The quality of the visual and aural stimuli embodied in software is very important. Software design must limit distraction and guide the learner’s attention to the essential information.
Perspectives on Teaching and Learning with Computers 5. Assessment All those activities undertaken by teachers and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. Learners learn best when they ... are given feedback are given advice  are fully involved
Perspectives on Teaching and Learning with Computers 5. Assessment Assessment should be performance-based (authentic) Authentic assessment refers to assessment tasks that  resemble real-world situations promote higher-order thinking solve problems
Workshop Applications (Workshops:  week 7: Tuesday ):   Suggest items that best describe the pedagogical and technical dimensions and their aspects of evaluation.  Design an instrument ( questionnaire, checklist, interview, observation …) to evaluate these dimensions and aspects. Apply the above instrument to evaluate a computer-based learning environment/courseware.
Workshop Examples Accuracy   Error-free information.  Updated information.  Objective, balanced presentation of information. Bias-free viewpoints and images.  Balanced representations of cultural, ethnic, and racial groups.  Correct use of grammar, spelling, and sentence structure.
Workshop Examples Navigation Rapid retrieval of information and screen transitions. Intuitive icons, menus, and directional symbols that foster independent use. Controllable pace, including options for stop/pause/exit  Controllable elements.
Workshop Examples Presentation  Information presented in a manner to stimulate imagination.  Use of appropriate and supportive feedback.  Options for help, tutorial segments.  Captions, labels, or legends for visuals.  Legible text and print size that is appropriate for the intended audience.
Workshop Examples Teacher's Guide Description of target audience.  Summary of the contents of the application.  Instructional and/or behavioral objectives.  Suggestions for classroom use, lesson plans, related activities. Support materials for student use, such as camera-ready worksheets and activity pages.
Workshop Applications   Suggest items that best describe the pedagogical and technical dimensions and their aspects of evaluation.  Design a strategy ( questionnaire, checklist, interview, observation, class assessment, portfolio, logs …) to evaluate these dimensions and aspects. Apply the above strategy to evaluate a computer-based learning environment/courseware.

Evaluation of Computer-Based Instruction

  • 1.
    TECH4102 Evaluation inEducational Technology Evaluation of Computer-Based Instruction Dr. Alaa Sadik Department of Instructional & Learning Technologies www.alaasadik.net [email_address]
  • 2.
    Dimensions of Evaluationin Computer-Based Instruction Pedagogical Dimension Technical Dimension Evaluation of CBI
  • 3.
    Evaluation of Computer-BasedInstruction Pedagogical aspects goal; underlying psychology; learning strategy; accuracy of content; student role; teacher role; flexibility of integration; motivation and reinforcement; meeting individual differences; interactivity; assessment; cultural sensitivity. Technical aspects navigation; standardization; accessibility; user-friendliness; presentation; documentation; performance; costs.
  • 4.
    Perspectives on Teachingand Learning with Computers 1. Learning theories Behaviorism Constructivism
  • 5.
    Perspectives on Teachingand Learning with Computers Behaviorism: Assumptions A teacher-directed approach ( controlled by the designer ). Learning is manipulated by the teacher ( the courseware ). Learning is described as a stimulus and response relationship. Students learn new concepts via the observation of the teacher ( courseware ) and content.
  • 6.
    Perspectives on Teachingand Learning with Computers Behaviorism: Assumptions Learning content is provided in small chunks. Learning content is provided in linear sequence. The student can’t study a new topic before achieving the prior topic. Learning processes can be studied most objectively when the focus of study is on stimulus and responses. Learning involves a behavior change .
  • 7.
    Perspectives on Teachingand Learning with Computers Behaviorism: Elements of a good courseware Gaining attention Informing learner of the objective Recall of prerequisite learning Presenting the stimulus material Providing learning guidance Electing the performance Providing feedback Assessing the performance Enhancing retention and transfer
  • 8.
    Perspectives on Teachingand Learning with Computers Behaviorism: Example of software - Drill and practice software - Tutorials e.g., Math Blaster , Math Munchers Deluxe Word Munchers Early Music Skills Organic Chemistry
  • 9.
    Perspectives on Teachingand Learning with Computers Constructivism: Assumptions Learning is an active and individualized process. The learner must construct new knowledge based on his/her own individualized experience (learner background). The student is the producer of information rather than the consumer. The teacher is a member of learning community rather than the only source of information (facilitator). Learning emphasizes the application of knowledge in real life situations .
  • 10.
    Perspectives on Teachingand Learning with Computers Constructivism: Assumptions Teaching must fosters critical thinking and creates active and motivated learners. Learning is a cooperative process constructed within the social context of the classroom. Assessment is a continuous and interactive process that measures the achievement of the learner and the quality of the learning experience.
  • 11.
    Perspectives on Teachingand Learning with Computers 2. Cooperative learning Ways of learning: Individual (alone) Competitive (against each other) Cooperative (together)
  • 12.
    Perspectives on Teachingand Learning with Computers 2. Cooperative learning Is a productive strategy that uses small teams of students through which they work together to develop communication skills, higher-order thinking skills , and social awareness, and maximize learning. The computer is a powerful tool for facilitating cooperation can serve an important role in cooperative learning environments. Suitable for computer-poor schools
  • 13.
    Perspectives on Teachingand Learning with Computers 3. Types of intelligence Professor Howard Gardner
  • 14.
    Perspectives on Teachingand Learning with Computers 3. Types of intelligence 1. Linguistic enjoy writing, reading, telling stories or doing crossword puzzles. 2. Logical-Mathematical interested in patterns, categories and relationships. They are drawn to arithmetic problems, strategy games and experiments. 3. Bodily-Kinesthetic process knowledge through bodily sensations, often athletic, dancers or good at crafts such as sewing or woodworking. 4. Spatial think in images and pictures, fascinated with mazes or puzzles, or spend free time drawing, building with Leggos.
  • 15.
    Perspectives on Teachingand Learning with Computers 3. Types of intelligence 5. Musical always singing or drumming to themselves. They are usually quite aware of sounds others may miss. 6. Interpersonal leaders among their peers, good at communicating and who seem to understand others' feelings and motives possess interpersonal intelligence. 7. Intrapersonal shy, aware of their own feelings and are self-motivated.
  • 16.
    Perspectives on Teachingand Learning with Computers 3. Types of intelligence Using Technology to Teach to the Multiple Intelligences Example: Linguistic Intelligence These kids are most likely to use word processing programs on the computer than anything else. They will know how to manipulate the text, do crazy things like create columns and outlines that actually work and look good.
  • 17.
    Perspectives on Teachingand Learning with Computers 4. Perception Perception is the process of acquiring, interpreting, selecting and organizing sensory information . e.g., visual perception, auditory perception
  • 18.
    Perspectives on Teachingand Learning with Computers 4. Perception Because perceptions are organized into understandings: The quality of the visual and aural stimuli embodied in software is very important. Software design must limit distraction and guide the learner’s attention to the essential information.
  • 19.
    Perspectives on Teachingand Learning with Computers 5. Assessment All those activities undertaken by teachers and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. Learners learn best when they ... are given feedback are given advice are fully involved
  • 20.
    Perspectives on Teachingand Learning with Computers 5. Assessment Assessment should be performance-based (authentic) Authentic assessment refers to assessment tasks that resemble real-world situations promote higher-order thinking solve problems
  • 21.
    Workshop Applications (Workshops: week 7: Tuesday ): Suggest items that best describe the pedagogical and technical dimensions and their aspects of evaluation. Design an instrument ( questionnaire, checklist, interview, observation …) to evaluate these dimensions and aspects. Apply the above instrument to evaluate a computer-based learning environment/courseware.
  • 22.
    Workshop Examples Accuracy Error-free information. Updated information. Objective, balanced presentation of information. Bias-free viewpoints and images. Balanced representations of cultural, ethnic, and racial groups. Correct use of grammar, spelling, and sentence structure.
  • 23.
    Workshop Examples NavigationRapid retrieval of information and screen transitions. Intuitive icons, menus, and directional symbols that foster independent use. Controllable pace, including options for stop/pause/exit Controllable elements.
  • 24.
    Workshop Examples Presentation Information presented in a manner to stimulate imagination. Use of appropriate and supportive feedback. Options for help, tutorial segments. Captions, labels, or legends for visuals. Legible text and print size that is appropriate for the intended audience.
  • 25.
    Workshop Examples Teacher'sGuide Description of target audience. Summary of the contents of the application. Instructional and/or behavioral objectives. Suggestions for classroom use, lesson plans, related activities. Support materials for student use, such as camera-ready worksheets and activity pages.
  • 26.
    Workshop Applications Suggest items that best describe the pedagogical and technical dimensions and their aspects of evaluation. Design a strategy ( questionnaire, checklist, interview, observation, class assessment, portfolio, logs …) to evaluate these dimensions and aspects. Apply the above strategy to evaluate a computer-based learning environment/courseware.