Download to read offline
![18
RECOMMENDATION
Student Name: [Name]
Student ID: [ID#]
Score Date: [Date]
The Student’s Background
• Gender: [Male/Female]
• Race: [Race]
• Age: [Age]
• Is English Best Language? [Yes/No]
Academic Skills – Tools and strategies to succeed in the classroom
Skills
• Sometimes uses strategies to effectively manage time and
assignments
• Occasionally misses class or comes unprepared
Tools/
Tips
Refer the student to the [Tutoring Center] to provide goal-
setting strategies and organizational tools. For additional ways of
working with the student, click here.
Commitment – Active pursuit toward an academic goal
Skills
• Sees great value in a college degree
• Is very attached to the college
Tools/
Tips
The [Career Center] can help the student align degree goals with
career goals. For additional ways of working with the student,
click here.
Social Support – Connections with people and resources for student success
Skills
• Holds strong connections to people and resources
• Effectively balances the demands of college and personal life
Tools/
Tips
The [Office of Student Life] can connect the student with valuable
student groups on campus. For helpful tools and tips about
student connections, click here.
Self-Management – Reactions to academic and other stressors
Skills
• Has difficulty managing stress in a positive, productive
manner
• Doubts personal skills and abilities
Tools/
Tips
Refer the student to the [Wellness Center] on campus to help
manage pressures and succeed. For effective ways of helping the
student manage everyday academic stress, click here.
MODERATE
ACADEMIC
SUCCESS INDEX*
HIGH
RETENTION
SUCCESS INDEX*
* Course Acceleration: Please see Technical User’s Guide to fully understand how to make an informed course placement.
* Academic Success Index: Weighted composite of student’s SuccessNavigator™ profile and other academic indicators of student preparedness, such as self- or institution-
reported high school GPA, SAT/ACT, etc.
* Retention Success Index: Projected likelihood that student will return for a second semester or year at the institution.
Advisor Report
SKILL REPORT
HIGH
HIGH
COURSE
ACCELERATION*
MODERATE
LOW
COURSE
ACCELERATION*
GLISH
MATH:CAUTION
ENGLISH: YES
Domain Scores: Four general
areas of student strengths
and weaknesses. Scores are
presented normatively.
Feedback: Determine by
more specific “facet” scores
(see next page).
Action Plans: Suggested
interaction with programs
and services on campus.
Background Information:
Communicate key student
information from both
SuccessNavigator™ and SIS
to faculty/advisor.
Success Indices: Separate
indices for both
classroom and
enrollment success.
Based on background,
cognitive and psycho-
social information and
supported by statistical
relationships with
success.](https://image.slidesharecdn.com/fa9f1149-a40b-496a-8db1-4db5359db9e7-151102160623-lva1-app6892/75/NASPA-AnP-2014-18-2048.jpg)
More Related Content
Similar to NASPA AnP 2014
NASPA AnP 2014
- 1. Holistic Assessment: From Researchto Application Copyright © 2013 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SUCCESSNAVIGATOR is a trademark of ETS. All other trademarks are the property of their respective owners. Ross Markle, Ph.D., Senior Research and Assessment Advisor Educational Testing Service Renée Delgado-Riley, Ph.D., Program Planning Officer The University of New Mexico Saturday, June 20, 2014
- 2. ATTAINED 35% ENROLLED 19% NO DEGREE, NOT ENROLLED 46% Students Starting at Two-YearInstitutions 1National Center for Educational Statistics’ Beginning Postsecondary Students Survey (BPS: 04-09) 2National Center for Education Statistics, Integrated Postsecondary Education Data System (IPEDS), Spring 2004. Degree Attainment for Beginning College Students: Starting in 2003-04 Academic Year, followed through 20091 ATTAINED 64% ENROLLED 12% NO DEGREE, NOT ENROLLED 24% Students Starting at Four-Year Institutions Of the 1.4 million students2 who enrolled in four- year schools in the fall of 2003, an estimated 525,686 STUDENTS had not received a degree as of 2009. Of the 1.1 million students2 who enrolled in two- year schools in the fall of 2003, an estimated 743,909 STUDENTS had not received a degree as of 2009. 2
- 3. “Underserved Populations” Cohort starting in: White African American Hispanic/ Latino White African American Hispanic/ Latino 200060.2 42.1 49.1 31.5 26.1 30.1 2002 60.2 40.1 48.9 30.4 24.2 30.7 2004 61.5 39.5 50.1 29.0 22.9 26.3 Percentage of first-time, full-time undergraduate students receiving a degree/certificate within 150% of normal FTE. (Source: : U.S. Department of Education, National Center for Education Statistics, Integrated Postsecondary Education Data System.) 4-Year Institutions 2-Year Institutions
- 4. 4 “Underserved Populations” - We’vedone a very good job of identifying who succeeds and who fails, but not a very good job of understanding why (Bean & Eaton, 1995). - Demographic and background characteristics predict success, but they don’t suggest the underlying causes of attrition or what we can do to improve persistence and graduation. Carnevale & Strohl, (2010)
- 5. Course Placement: • Studentsface a long, complex sequence of developmental courses. Early Academic Success • Students lack the effective behaviors (organization, study skills) that are needed to succeed in college- level courses. Persistence Over Time • Without well- developed and aligned goals, self- management skills, and social connections, students may fail to persist to a degree. Where we lose students ... 5
- 6. 6 Challenges to supportingstudent success Course Placement Problem: High remediation rates lead to the “Bridge to Nowhere” (CCA, 2012) Need: Identify students who might succeed on an accelerated path Advising Problem: Traditional indicators of student success are insufficient Need: Better understanding of students’ likelihood of success, tied to course of action Institutional Planning Problem: A host of programs and services based on “best practices” Need: Data to target and structure programs and services to the population- specific strengths and weaknesses
- 7. So what’s thesolution?
- 8. What are noncogntiveskills? Conscientiousness Teamwork Institutional Commitment Motivation Metacognition Factors outside of “academic ability” or “academic intelligence” that contribute to or are part of student learning Study Skills Goal Setting Self-Efficacy Social Support Response to Stress Test-Taking Strategies 8
- 9. 9 K-8 High SchoolCollege Conscientiousness 0.28 0.21 0.23 Agreeableness 0.3 0.05 0.06 Cognitive ability 0.58 0.24 0.23 Socioeconomic Status 0.32 0.32 0.32 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Correlationw/StudentSuccess Poropat (2009) Meta-analysis
- 10. Construct rGPA Academic Skills.129 Academic Goals .155 Academic Self-Efficacy .378 Institutional Commitment .108 Social Support .096 SES .155 ACT/SAT Scores .376 Construct rGPA rretention Academic Skills .129 .298 Academic Goals .155 .210 Academic Self-Efficacy .378 .257 Institutional Commitment .108 .204 Social Support .096 .199 SES .155 .173 ACT®/SAT® Scores .376 .079 Robbins, et al. (2004) meta-analytic correlations with retention, GPA 10
- 11. 11 Effort Level (participation, attendance,complete assignments) High Medium Low Math Readiness High 96% 91% -- Medium 94% 77% 82% Low 86% 61% 38% n=411 Developmental Education Class Success Rates Based on Math Readiness and In-course Behavior Levels Success rate not calculated due to insufficient sample size Success rates for students with low readiness but high effort more than twice as high as for low effort, low readiness Research shows that behavioral monitoring and intervention are key to student success Source: ETS RR- 13-12, Examining Evidence of Reliability, Validity, and Fairness for the SuccessNavigator™ Assessment , July 2013
- 12. 12 Working with Subpopulations •Many view noncognitive factors as key to improving success, particularly for traditionally disadvantaged populations (e.g., academically underprepared, first-generation or low-SES students) • However, there are several possible ways in which noncognitive variables might relate to these populations, each of which has different implications for guiding intervention Group Status Success Academic Achievement Noncognitive factors Group Status Success Academic Achievement Noncognitive factors
- 13. A holistic assessmentsolution • Targets students between pre- and early enrollment (post- admissions) • Inclusion of both cognitive and psychosocial (“noncognitive”) factors • Alignment between assessment and institutional practices • Added value: – Respects the whole student (examines cognitive and noncognitive factors) – Focuses on the unique characteristics of each student – Focuses on factors educators can control: such as motivation, social connectedness, as opposed to socioeconomic, situational factors 13
- 14.
- 15. Use of theSuccessNavigator™ Assessment • Low stakes: diagnostic and developmental: A 30-minute online, nonproctored noncognitive assessment for incoming college students. • Can be used with or without academic markers (SAT® scores, HS GPA, course placement score) • Three primary intended uses: – Identifying likelihood of persistence to degree and academic success – Informing course placement acceleration decisions – Feedback for advising, FYE, etc. • Assessments are scored immediately and will be sent to the candidates and academic advisors. Administrators have access to appropriate reports within the online system. 1515
- 16. 16 Domain Facet DefinitionExample Items Academic Skills Tools and strategies for academic success Organization Strategies for organizing work and time. I make a schedule for getting my school work done. I take due dates seriously. Meeting Class Expectations Doing what’s expected to meet the requirements of your course including assignments and in-class behaviors. I attend almost all of my classes. I complete the reading that is assigned to me. Commitment Active pursuit toward an academic goal. Commitment to College Goals Perceived value and determination to succeed in and complete college. One of my life goals is to graduate college. The benefit of a college education outweighs the cost. Institutional Commitment Attachment to and positive evaluations of the school. This is the right school for me. I’m proud to say I attend this school. Self-Management Reactions to academic and daily stress Sensitivity to Stress Tendency to feel frustrated, discouraged or upset when under pressure or burdened by demands. I get stressed out easily when things don't go my way. I am easily frustrated. Academic Self- Efficacy Belief in one’s ability to perform and achieve in an academic setting. I'm confident that I will succeed in my courses this semester. I can do well in college if I apply myself. Test Anxiety General reactions to test-taking experiences, including negative thoughts and feelings (e.g., worry, dread). When taking a test, I think about what happens if I don't do well. Before a test, my stomach gets upset. Social Support Connecting with people and students resources for success Connectedness A general sense of belonging and engagement. I feel connected to my peers. People understand me. Institutional Support Attitudes about and tendency to seek help from established resources. If I don't understand something in class, I ask the instructor for help. I know how to find out what's expected of me in classes. Barriers to Success Financial pressures, family responsibilities, conflicting work schedules and limited institutional knowledge. Family pressures make it hard for me to commit to school. People close to me support me going to college. 16
- 17. SuccessNavigator™ Score Reports Advisor/FacultyReport • General Skill scores, Success Indices around academic and enrollment success • Granular feedback report based on facet scores • Used to foster conversations and interventions with student Student Report • Feedback at domain level, NO SUCCESS INDICES • Action plans around programs and services on campus Institutional Report (Institution Aggregate Report and Custom Institution Report) • Population and subpopulation statistics (based on normative samples) at domain and facet level • Used for strategic planning, tailoring co-curricular programs and services 1717
- 18. 18 RECOMMENDATION Student Name: [Name] StudentID: [ID#] Score Date: [Date] The Student’s Background • Gender: [Male/Female] • Race: [Race] • Age: [Age] • Is English Best Language? [Yes/No] Academic Skills – Tools and strategies to succeed in the classroom Skills • Sometimes uses strategies to effectively manage time and assignments • Occasionally misses class or comes unprepared Tools/ Tips Refer the student to the [Tutoring Center] to provide goal- setting strategies and organizational tools. For additional ways of working with the student, click here. Commitment – Active pursuit toward an academic goal Skills • Sees great value in a college degree • Is very attached to the college Tools/ Tips The [Career Center] can help the student align degree goals with career goals. For additional ways of working with the student, click here. Social Support – Connections with people and resources for student success Skills • Holds strong connections to people and resources • Effectively balances the demands of college and personal life Tools/ Tips The [Office of Student Life] can connect the student with valuable student groups on campus. For helpful tools and tips about student connections, click here. Self-Management – Reactions to academic and other stressors Skills • Has difficulty managing stress in a positive, productive manner • Doubts personal skills and abilities Tools/ Tips Refer the student to the [Wellness Center] on campus to help manage pressures and succeed. For effective ways of helping the student manage everyday academic stress, click here. MODERATE ACADEMIC SUCCESS INDEX* HIGH RETENTION SUCCESS INDEX* * Course Acceleration: Please see Technical User’s Guide to fully understand how to make an informed course placement. * Academic Success Index: Weighted composite of student’s SuccessNavigator™ profile and other academic indicators of student preparedness, such as self- or institution- reported high school GPA, SAT/ACT, etc. * Retention Success Index: Projected likelihood that student will return for a second semester or year at the institution. Advisor Report SKILL REPORT HIGH HIGH COURSE ACCELERATION* MODERATE LOW COURSE ACCELERATION* GLISH MATH:CAUTION ENGLISH: YES Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively. Feedback: Determine by more specific “facet” scores (see next page). Action Plans: Suggested interaction with programs and services on campus. Background Information: Communicate key student information from both SuccessNavigator™ and SIS to faculty/advisor. Success Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive and psycho- social information and supported by statistical relationships with success.
- 19.
- 20. 20 Advising • Problem: Traditionalindicators of student success do not capture many of the critical factors • Need: To better understand students’ likelihood of success so that faculty and staff can determine the appropriate level of engagement • Solution: Integrated holistic assessments with accurate and intuitive score reporting
- 21.
- 22.
- 23. From the ResourceLibrary: Supports for Academic Skills and Commitment 23
- 24. 24 Course Placement • Problem:Too many students are placed into developmental courses, effectively placing them on a “Bridge to Nowhere” • Need: Identify students who are likely to succeed if placed in higher-level courses (i.e., shorter paths to success) • Solution: Holistic assessments that indicate students who will put forth effort, attend class, study and succeed (even when their cognitive test scores might not suggest such)
- 25. Placement Test Score SuccessNavigator™Course Acceleration Indicator Yellow – Caution Green - Accelerate 0-30 Low Developmental Course Low Developmental Course 31-40 Low Developmental Course* High Developmental Course 41-70 High Developmental Course High Developmental Course 71-80 High Developmental Course* College-Level Course Above 80 College-Level Course College-Level Course 25
- 26. TESTS + The SuccessNavigator™ Assessment PLACEMENT BANDS(?) ACCELERATE ACCELERATE WITH SUPPORTS? ACCELERATION DECISION 1st Level Support Intrusive Advising 2nd Level Support Supplemental Instruction DO NOT ACCELERATE Instructional interventions at the class/program level Interventions at the individual student level 26
- 27. 27 Institutional Planning • Problem:Institutions develop a panoply of services based on best practices that may or may not fit their population • Need: Data to target programs and services to the specific strengths and weaknesses of students at a particular institution • Solution: Aggregate reports of relevant psychosocial skills that are tied to institutional practice
- 28. • Tutoring • Advising •FYE (Study Skills) • Advising • Career Counseling • FYE (Goal Setting) • Counseling • Testing Center • FYE (Test-Taking Skills) • Student Life • Clubs and Organizations • Orientation, FYE • (Help seeking) GENERAL SKILL SCORES AT COMMUNITY COLLEGE of NJ 28
- 29. 29 New Mexico ata Glance Population: 2.1 million • 1 of 4 minority-majority States, 57.9% minority population • 46.7% of total population is Hispanic New Mexico United States Living in Poverty 20.4% 15.3% Bachelor’s Degree 14.2% 17.6% Some College 24.9% 20.6% Speak More Than One Language 35.8% 19.6% (U.S. Census Bureau, 2010)
- 30. 30 The University ofNew Mexico is a public, four-year degree-granting institution located on Old Route 66, in the heart of Albuquerque, New Mexico. Current enrollment is approximately 36,510 students. UNM is a Hispanic-Serving Institution and is classified by the Carnegie Foundation as a Research One, Very High Research University. UNM offers a total of 210 degree and certificate programs, including 93 baccalaureate level degree programs, 68 master’s programs, and 37 doctoral programs. History of UNM Copyright © 2014 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SUCCESSNAVIGATOR is a trademark of ETS. All other trademarks are the property of their respective owners. 25841
- 31.
- 32. 32 UNM at aGlance UNM Hispanic Student Challenges Hispanic White Hispanic Postsecondary Completion (6 years) 42% 49% Postsecondary Drop Out Rates 29.5% 15.1% Student: Advisor Ratio 335:1 (UNM Office of Institutional Research, 2011; UNM Provost Report, 2007; U.S. Census Bureau, 2010) Copyright © 2014 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SUCCESSNAVIGATOR is a trademark of ETS. All other trademarks are the property of their respective owners. 25841
- 33. 33 Driving Student SuccessThrough the SuccessNavigator™ Assessment Early Alert Coaching & Advising Target Student Populations • Identify early risk • Develop strategies for intervention • Provide programming to increase persistence • 1-on-1 support and guidance • Resources to navigate through college • Connecting students to on- campus resources and programming inside and outside the classroom • Develop profiles of noncognitive scores based on race/ethnicity and gender • Develop interventions that specifically align to those skills and profiles
- 34. At UNM: Actionfor Every Student High Skill: Leadership Moderate Skill: Refinement Low Skill: Readjustment Copyright © 2014 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SUCCESSNAVIGATOR is a trademark of ETS. All other trademarks are the property of their respective owners. 25841 34
- 35. 35 Academic Skills CommitmentSelf-Management Social Support CAPS Accessibility Resource Center College Enrichment & Outreach Programs (e.g., CEP, SSS) Advisement (University, Fine Arts, Engineering) Academic Coaching Academic Level Student Support (e.g., Engineering Student Services, Student Services, etc.) Faculty Office Hours (Noted on course syllabi) Career Services Academic Departments Student Activities (i.e., Greeks, Student Organizations, etc.) Advisement (University, Fine Arts, Engineering) Alumni Support Mentoring Program Student Employment Residential Life Student Health & Counseling Center (SHAC) CAPS Targeted Resource Centers (African American Student Services, American Indian Student Services, El Centro, LGBTQ Resource Center, Men of Color Initiative, Women’s Resource Center, Veteran’s Resource Center) Religious Organizations (e.g., Newman Center, etc.) College Enrichment & Outreach Programs (e.g., CEP, SSS) Academic Coaching Recreational Services_Johnson Center Targeted Resource Centers (African American Student Services, American Indian Student Services, El Centro, LGBTQ Resource Center, Men of Color Initiative, Women’s Resource Center, Veteran’s Resource Center) Student Activities (i.e., Greeks, Student Organizations, etc.) College Enrichment & Outreach Programs (e.g., CEP, SSS) Dean of Students Student Employment Recreational Services_Johnson Center Early Alert http://success.unm.edu/success-navigator/index.html
- 36. 36 Coaching & Advising Copyright© 2014 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SUCCESSNAVIGATOR is a trademark of ETS. All other trademarks are the property of their respective owners. 25841
- 37. 37 Comparing Men ofColor to the General Population General Mean (n=800) M.O.C. Mean (n=209) Total SD Cohen's d Background ACT® Composite 22.9 22.3 4.1 -0.15 H.S. GPA 3.4 3.3 0.5 -0.30 Expected Family Contribution 14,710 10,344 19,933 -0.22 SuccesNavigator™ Subskills Organization 102.2 94.7 15.2 -0.49 Sensitivity to Stress 102.1 107.3 13.6 0.38 Test Anxiety 100.6 105.9 14.3 0.37 Barriers to Success 101.9 100.8 12.9 -0.09 Campus Engagement Visits to Tutoring, Advising 0.0 0.3 0.8 0.42 Faculty Mentors, Study Groups, Clubs/Organizations 0.0 0.0 0.7 -0.04 Showing up, Completing Assignments 0.0 0.2 0.8 0.37 Outcomes Completing 1st Math Course 84.7% 83.9% 0.4 -0.02 Completing 1st English Course 93.8% 90.2% 0.3 -0.14 First Semester GPA 3.19 2.93 0.9 -0.30 Persistence to Spring 93.1% 89.0% 0.3 -0.15 Targeting Student Populations
- 38. 38 • Campuswide involvementin the process • Thorough exploration and dissemination of the data • External and internal analyses and presentations of data • Sharing and offering data with programs • Upcoming Initiatives: First Year Required Course Student Success Center Restructure New Student Orientation Gamification of Student Engagement Holistic Student Success Copyright © 2014 by Educational Testing Service. All rights reserved. ETS, the ETS logo and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SUCCESSNAVIGATOR is a trademark of ETS. All other trademarks are the property of their respective owners. 25841
- 39. 39 Embracing a HolisticViewpoint • Improving student success will require us to consider a broader set of skills. • We must transform data to information, and information to action. • We will also have to integrate student success efforts into a holistic plan of action. Recognize you can begin to increase retention and success rates as early as your next incoming class.