MR
Uploaded byMebane Rash
439 views

Ensuring Opportunity Summary

This document provides an overview of efforts to evaluate North Carolina's Opportunity Scholarship Program, which provides state-funded vouchers for low-income students to attend private schools. Key findings include: 1) Recipients tend to be from low-income families and are more likely to be African American or in elementary school compared to public school students. 2) Access to information varies geographically, with social networks being the primary source of information. 3) Private schools that participate tend to have a religious orientation and emphasize character building over standardized testing. 4) While the program has grown, limitations in data prevent a full understanding of impacts on student achievement and outcomes. Improved data collection and a more robust evaluation are recommended

Embed presentation

Download to read offline
Ensuring Opportunity in North Carolina’s Opportunity Scholarship Program A Summary of Work to Date June 2018 D.T. Stallings The William and Ida Friday Institute for Educational Innovation Anna Egalite College of Education Steven Porter College of Education North Carolina State University
Page 1 Ensuring Opportunity in the OS Program Acknowledgements The early demographic analyses and evaluation results presented in this document are the product of several years of work—all completed (to date) without state support. Contributions from private funders were critical in getting the evaluation work off the ground in its earlier years. Since those first investments, the project has benefitted from additional informational and participatory support from the North Carolina State Education Assistance Authority, the North Carolina Division of Non-Public Education, stakeholder organizations, graduate student researchers, and, most importantly, the families and practitioners who have made our data collection possible: superintendents, principals, parents, and students in several North Carolina public school districts; and headmasters, parents, and students in private schools from across the state. Our work is not finished. We continue to pressure the state to support a balanced, comprehensive evaluation of the complex impacts of the Opportunity Scholarship program—the only such program in the country that previously has not been nor currently is implemented in a way that supports rigorous quantitative evaluation of the program’s impacts on student achievement due to statutorily-prescribed structural limitations.1 We continue to push for the programmatic structures that will allow for an evaluation because we believe in the importance of sharing learnings from all state education policy implementations with all education stakeholders, and because we believe in the value of thoughtful stewardship of state resources. 1 The Florida Tax Credit Scholarship also no longer is able to be rigorously quantitatively evaluated but did undergo such an evaluation before recent program design changes.
Page 2 Ensuring Opportunity in the OS Program Table of Contents Executive Summary 4 Introduction 8 The Opportunity Scholarship Program 8 The Opportunity Scholarship in the Context of Other State Voucher Programs 9 Data Collection and Evaluation Progress to Date 10 Questions Guiding the Foundational and Background Portion of the Evaluation 11 Gauging the Extent to which Access to the Opportunity Scholarship is Equitable across Potential Participants 11 Assuring Access to a Quality Education for Participating Students from Low- Income Families 11 Assuring Fair and Equitable Use of State Funding 12 A Note on Gaining a Comprehensive Understanding of the Opportunity Scholarship 12 Data and Methods 13 Demographics and Descriptive Data 13 Student Demographic Data 13 School Descriptive Data 13 A Note on the Quality and Availability of Non-Public School Data 13 Surveys 13 Instrument 13 Response Rates 14 Focus Groups 14 Testing 15 Gauging Equity of Access: Students and their Families 16 Characteristics of Participating Students 16 Eligibility 16 Grade Level 17 Ethnicity 17 Differences between Participating Students and Other Eligible Students 18 Access to Information about the Opportunity Scholarship 18 The Application Process and Equity of Access 21 Perceptions of Opportunity Scholarship Students at Participating Schools 24 Assuring Access to a Quality Education for Participating Students: Participating Schools 25
Page 3 Ensuring Opportunity in the OS Program Characteristics of Participating Private Schools 25 Geographic Distribution 25 Access to the Opportunity Scholarship Statewide 25 Comparisons to Public and Non-Participating Private Schools 28 Expenditure Patterns of Participating Private and Nearby Public Schools 33 Tuition Support 34 Participation and Enrollment Restrictions Related to Finance 34 The Opportunity Scholarship and Changes in the Supply of Private Schools 35 Understanding the Current Academic Outcomes Evaluation Landscape 37 Testing in Participating Schools 37 Performance of Participating Students 38 Participating Schools with more then Twenty-Five Opportunity Scholarship Students 38 Recommended Steps for Implementing a More Robust Evaluation, and Initial Evaluation Team Efforts to Analyze Student Performance (2016-17) 41 Other Evaluation Options 43 Next Steps in Our Evaluation Efforts 45 Securing Continuing Support for Evaluation 45 Expanding Analysis of the Program’s Impact on Student Outcomes 45 Steps for Improving Program-Related Data Quality 45 Determining Impacts on Cognitive Outcomes 45 Determining Impacts on Non-Cognitive Outcomes 46 Appendix. Communities with Opportunity Scholarship-Enrolling Schools, 2014-2017 47 Cover image courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action.
Page 4 Ensuring Opportunity in the OS Program Executive Summary Overview The North Carolina Opportunity Scholarship Program, in operation since the beginning of the 2014-15 school year, provides state-funded financial support for eligible children in Kindergarten through twelfth grade that can be used to cover or reduce tuition and fees for non-public school education. This report provides an overview of efforts to date to evaluate multiple aspects of the program. Gauging Equity of Access: Students and their Families How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? Like many other programs nationwide, the Opportunity Scholarship is means-tested. The program treats all applicants relatively equally, with only a few exceptions (for example, the lottery does preference siblings and members of military families). The Opportunity Scholarship requires family co-payment of tuition and fees if those costs exceed the value of the voucher; both the Indiana Choice Scholarship Program and the Florida McKay Scholarship Program have similar guidelines, but the voucher program in Louisiana requires participating schools to cover or waive any differences. What are the characteristics of student applicants? The adjusted median per-household-member income for a participating family (ca. $16,000 per member) is low, and the overall household income for most participating families falls well below the threshold for eligibility (e.g., ca. $30,000 for a family of two; ca. $61,000 for a household of six). Compared to public school students, voucher recipients are more likely to be in elementary school and they are less likely to be in high school. A larger proportion of recipients are African-American, relative to the population of public school students, and a smaller proportion of recipients are Hispanic or White. Are there systematic differences between students who apply and eligible non-applicants? Reflecting the diversity of the private schools they attended, the population of Opportunity Scholarship recipients is quite diverse along a number of demographic axes, as supported by the random nature of the lottery used to identify recipients. However, there may be some differences between applicants and eligible non-applicants as a result of three non-systematic variables: a) differences in access to information about the program (via social information networks and general understanding of how private schooling works); b) differences in ability to navigate the application process; and c) differences in family motivation to apply. 2 Adjusted per-household-member income = household income / square root of household size; this approach is commonly used by economists to account for household savings from economies of scale and resource-sharing.
Page 5 Ensuring Opportunity in the OS Program How do families receive information about the Opportunity Scholarship and participating nonpublic schools? Are there differences in information access by sub-groups (by ethnicity, by geographic location, etc.?) Social information networks appeared to be the most influential means by which families learned about the program during its first two years, followed by direct marketing on the part of participating private schools. The state agency that oversees the program advertises across the state using several media platforms. Any differences in access to information appear to be geographically based, but not in a systematic way; instead, the differences have more to do with variations in the strengths of local and regional information networks, as well as the degree to which local private schools share information about the program. Because of their faith- based community connections, some private schools are able to share information with language minority groups through non-English language sister church circles. For the first two years of the program, data suggest that most families apply to only one private school, but that outcome likely is tied in part to the fact that many rural communities have access to only one participating school, and in part to a phenomenon among programs nationally of families becoming accustomed to exercising greater choice only once a program is better established and families gain more experience with school choice options. Does the timing of the application (early February) impact who applies? Survey responses3 indicate that most families (87 percent) believed that the application process was simple to complete, but the timing of the application process (with the first families learning of their awards in mid- Spring) is not well aligned with many private schools’ enrollment calendars (which can begin as early as January). The challenge increases for wait-list families, who often are notified of awards sometimes well after their schools of choice have completed enrollment, and sometimes even into the start of the school year. Assuring Access to a Quality Education for Participating Students: Participating Schools Are there differences in access across counties? For example, do rural students have proportionally more or less opportunity to attend a participating nonpublic school than do urban and suburban students, in terms of both availability and cost? Even though more than 20 rural counties do not have participating schools, students in 97 of North Carolina’s 100 counties applied for and received scholarships for 2016-17, and some schools even provide limited transportation support. Program participation on the part of private schools has grown proportionally in urban and rural areas alike. Sometimes vast differences in cost across private schools may be a more important factor in access, since the standard voucher amount (no more than $4,200) typically covers less than the average private school tuition in North Carolina (between $7,600 and $9,400). 3 2016 parent survey response rate: 2,425 out of 8,739 households (28%) responded; households included new and renewal applicants; all applicant households were surveyed, regardless of whether the household either was awarded or accepted a voucher.
Page 6 Ensuring Opportunity in the OS Program What are the characteristics of nonpublic schools that choose to participate in the voucher program, and how do they compare to public and non-participating nonpublic schools? Are there discernible patterns in the types of private schools that participate (e.g., by location, school type, school size, etc.), and, if so, what are they? A higher proportion of participating schools (76 percent) indicated in survey responses4 that their school has a religious orientation than did non-participating schools (42 percent). Participating and non-participating schools indicated that their primary admissions screening components were interviews and evidence of prior academic achievement. By their own assessment, participating schools believe they place a stronger emphasis on college preparation and character building, relative to public schools, while non-participating schools tended to emphasize less reliance on standardized testing than their public school neighbors. The most frequently cited factors for a school’s decision not to participate in the program were concerns about future changes to state laws or policies regarding participation requirements (57 percent of survey respondents) and about the amount of required paperwork (47 percent). Though anecdotal only, some participating schools posited that non-participating schools also were wary of changes to the make-up of their student populations if they enrolled Opportunity Scholarship students. To what extent do participating nonpublic and nearby public school expenditure patterns differ? Because of limitations in the reporting required of non-public schools, the level of fiscal transparency common among public schools does not exist to a similar degree among Opportunity Scholarship schools. As a result, this question may never be answerable unless the state requires reporting of financial data by participating schools. What impact has the availability of the program appeared to have on the supply of private schools? Overall, available data do not suggest that the advent of the Opportunity Scholarship on its own has led directly to a meaningful uptick in the number of private schools in the state; the number was growing before the program and continued to grow at about the same pace afterwards, with only a slightly larger than usual spike in the first year after passage of the enacting legislation. Much of the growth in terms of the gross number of school openings has been offset by a smaller but still substantial number of school closings. Understanding the Current Academic Outcomes Evaluation Landscape Testing in Participating Schools Per legislation, nonpublic schools that receive Opportunity Scholarship funds are required to administer a nationally standardized test or equivalent at least once a year to all Opportunity Scholarship recipients. Results of these tests are reported to NCSEAA, but there are no target performance levels, nor are there any actions NCSEAA is required to take related to the scores. 4 2016 survey response rate for non-public schools with active email contact information: 40% (266 out of 673 schools). Schools were surveyed regardless of whether they participated in the Opportunity Scholarship program or not.
Page 7 Ensuring Opportunity in the OS Program Performance of Participating Students Participating nonpublic schools with more than 25 Opportunity Scholarship recipients also must report aggregated standardized test performance results for eligible students, and those results are published annually. The performance of students at these schools differed widely, but without prior achievement data or reliable comparison groups, these data alone are insufficient for understanding the impact of schooling on achievement and growth. This report also describes the evaluation team’s efforts to collect and analyze comparable test data for the 2016-17 school year. Recommended Steps for Implementing a More Robust Evaluation 1. Identify a common measure of achievement; 2. Limit program enrollment to create natural experimental treatment and control groups; 3. Make program application contingent upon willingness to participate in evaluation; and 4. Secure adequate funding to support the evaluation effort. Next Steps in Our Evaluation Efforts Some of the questions that will guide the next components of our work include: Steps for Improving Program-Related Data Quality 1. What is the nature and quality of the nationally-normed tests used by participating schools to meet the legislative requirement of annual testing of Opportunity Scholarship students? 2. What improvements should the state make in its annual collection of operational descriptive data on non- public schools? Impacts on Cognitive Outcomes 3. What is the impact of a voucher5 on students’ mathematics and reading achievement? 4. Are the effects evenly distributed for students from different subgroups and at different points along ability and income distributions? Impacts on Non-Cognitive Outcomes 5. What is the impact of a voucher on students’ non-cognitive outcomes, including grit, conscientiousness, and delay of gratification? 6. How do these impacts vary for specific student subgroups, defined by race, locale, and socio-economic background? 5 Most voucher research differentiates between the impact of a voucher offer and the impact of voucher use. Data from tests administered by the research team in spring 2017 to voucher recipients and voucher-eligible public school students allowed for an early estimation of the impact of voucher use (https://ced.ncsu.edu/elphd/research/the-impact-of-the-north-carolina-opportunity- scholarship-program/); important limitations of the extent to which the team could draw firm conclusions from the data collected are outlined in the Understanding the Current Academic Outcomes Evaluation Landscape section of this report. The opportunity to conduct more rigorous estimations of the impact of voucher use and/or of a voucher offer will require statutory changes to the program (as summarized in the Recommended Steps list in this summary and as described in greater detail in the main report).
Page 8 Ensuring Opportunity in the OS Program Introduction The Opportunity Scholarship Program The North Carolina Opportunity Scholarship Program provides state-funded financial support for eligible children in Kindergarten through twelfth grade that can be used to cover or reduce tuition and fees for non- public school education. Established by North Carolina General Statute 115C-562 in 2013 and administered by the North Carolina State Education Assistance Authority (NCSEAA), the Opportunity Scholarship Program provides funding of up to $4,200 per year for eligible students to attend participating private schools. Awards are distributed one semester at a time and can be used for tuition and required fees at registered private schools. Opportunity Scholarships were first awarded in school year 2014-15, providing scholarships for 1,216 students to attend 224 private schools. The program has experienced significant growth every year since then, with 3,460 recipients attending 313 private schools in 2015-16 and 5,432 recipients in 349 private schools in 2016-17. Eligibility for the Opportunity Scholarship program is determined by two criterion categories. First, students must be North Carolina residents living in a household that does not exceed a statutorily-defined income cap, must not have graduated from high school or attended college, and must have turned five years old on or before August 31. The household income eligibility threshold is set at 133 percent of the eligibility threshold for the federal free and reduced-price lunch program for a partial scholarship, and it is set at 100 percent of the federal free and reduced-price lunch program eligibility threshold for a full scholarship. For a family of four, for example, maximum household gross income for 2016 could not exceed $45,510 if a student applied for a full scholarship or $60,528 for a partial scholarship. Second, students must be enrolled in a public school in North Carolina in the application year, be entering Kindergarten or first grade, or have received an Opportunity Scholarship in a previous academic year. Students who do not meet any of these second criteria still may be eligible if they are in foster care or were adopted within the last year, or if they have a parent on fulltime active duty with the military. 6 Although the scholarship award has a maximum value of $4,200, it cannot exceed the cost of tuition and fees, including books, equipment, transportation or other items required by the private school 7 These data were published in NCSEAA’s annual reports to the North Carolina legislature on the Opportunity Scholarship Program for the 2014-15 and 2015-16 academic years. Retrieved from http://www.ncleg.net/documentsites/committees/JLEOC/Reports%20 Received/2015%20Reports%20Received/Opportunity%20Scholarship%20Program.pdf and http://ncga.state.nc.us/documentsites/ committees/JLEOC/Reports%20Received/2016%20Reports%20Received/Opportunity%20Scholarship%20Program%20Report%20 2016.pdf 8 Because Opportunity Scholarships are awarded throughout the academic year until all resources are depleted, numbers for the 2017-18 school year will not be final until summer 2018. 9 See http://www.ncseaa.edu/pdf/OPS_Overview.pdf 10 One interesting aspect of this requirement is that it may incentivize parents to change a student’s school enrollment from private to public for one year in order to become eligible. While we did not find evidence of widespread adoption of this strategy, at least one parent did bring it up during a focus group: “I do know parents that have taken the chance—even though they didn’t want to, but they thought [of] the benefit in the long run—they put their kids back in public school because they knew that they would qualify.” – Fayetteville Parent (2016)
Page 9 Ensuring Opportunity in the OS Program Part of the reason for the growth of the program has been growth in the state’s investment in the program, along with projections for continued growth in the investment over the next decade. Total scholarship funding is set by the state budget. In 2014-15, $4,635,320 was disbursed in scholarship funds; in 2015-16, that total rose to $13,149,842, and in 2016-17 climbed even higher, to $21,760,837. Effective July 1 2017, the 2016 Appropriations Act approved an increase of funds by at least ten million dollars every year for ten years.11 Relative to the state’s overall annual funding for K-12 education, however (over $9 billion annually), and even accounting for continued growth in the annual program allotment, the state investment is small (about 0.2% of the amount allotted to public K-12 education). After the renewal scholarships have been awarded, at least fifty percent of the remaining funds must be used to award grants to students who qualify for the free and reduced-price lunch program. Any remaining funds are distributed among the remaining eligible applicants, with no more than forty percent of the remaining funds directed to students entering Kindergarten or First Grade. To participate in the Opportunity Scholarship program, private schools must satisfy the requirements established by the state’s Division of Non-Public Education for all private schools, they must register with NCSEAA, and they must conduct a criminal background check for the employee with the highest decision- making authority. Eligible private schools that receive more than $300,000 in Opportunity Scholarship funds must submit an annual financial review report that has been prepared by a certified public accountant. Finally, all participating private schools are required to administer a nationally standardized test of their choosing to all scholarship students annually and to report these results to NCSEAA. The ramifications of this last guideline on the quality of any current or future evaluation is discussed at greater length in the Understanding the Current Academic Outcomes Evaluation Landscape section below. The Opportunity Scholarship program has faced two legal challenges, both of which alleged that the program violates the North Carolina Constitution: Hart v. North Carolina and Richardson v. North Carolina. Although the trial court ruled in favor of the plaintiffs, the North Carolina Supreme Court declared the program constitutional in a 4-3 decision in July 2015. The Opportunity Scholarship in the Context of Other State Voucher Programs An important contextual consideration to keep in mind before reviewing the results of the evaluation efforts described in this report is the degree to which these results are comparable to results from other states. While the Opportunity Scholarship program was based in part on voucher programs in other states and municipalities, its combination of features is unique (Table 1, following page), which limits our ability to make comparisons to results from evaluations of other voucher programs. 11 See NCSEAA Opportunity Scholarship Program Summary of Data as of 2/1/17. Retrieved from http://www.ncseaa.edu/documents/ OPS_Summary_Data.pdf
Page 10 Ensuring Opportunity in the OS Program Table 1. Comparison of Selected North Carolina Opportunity Scholarship Components with Components of Other Voucher Programs Key Characteristics NC FL IN LA Program Design Voucher Tax Credit Voucher Voucher Max Voucher Value (2016-17) $4,200 $5,886 $6,473 $10,090 Avg Voucher Value (2015-16) $4,116 $5,476 $4,520 (1-8) $5,496 (9-12) $5,856 Participating Schools Can Charge More than Voucher Value? Yes Yes Yes No School-Specific Admission Standards Yes Yes Yes No Private Students Eligible? No Yes No No Accountability Norm-referenced test of choice Norm-referenced test of choice State Test State Test Non-Public School Participation Rate (2016-17) ~60% ~69% ~45% ~34% The structure of North Carolina’s program relative to the structures of programs in states like Florida, Indiana, and Louisiana—especially in terms of program design, voucher amount, and admissions standards—is at least partially and (in the case of Louisiana) notably different. As a result, the measured impacts of these programs as reported in other studies may provide good context for the impact of voucher programs overall but do not necessarily inform what we should expect to find in our examination of the North Carolina program. Data Collection and Evaluation Progress to Date In September of 2013, a research team from North Carolina State University began what has now become a five-year effort to conduct a formal, unbiased, and rigorous evaluation of the North Carolina Opportunity Scholarship program. With support from a private foundation, in 2014 the team conducted a limited number of focus groups with a handful of parents of Scholarship recipients and with headmasters at two participating schools; the team also compiled a comparative overview of all voucher programs nationwide. After securing support from a second private foundation, the team expanded its efforts in 2015 and 2016, traveling across North Carolina to learn more about key stakeholder perceptions of various aspects of the program. The team distributed online surveys to every private school in the state (both those that participate in the program and those that do not) and to all applicant families. In addition, the team conducted focus
Page 11 Ensuring Opportunity in the OS Program groups and interviews with private school leaders and applicant parents in five representative locations to better understand how the program works and how it has impacted participating schools, students, and their families. Next, the team worked with public school administrators, private school headmasters, a school choice advocacy group, and state officials to administer a common standardized test to 698 students in grades four through eight—401 public and 297 private school students in 14 public schools and 24 private schools— from four representative regions across the state. More details about the survey, focus group, and testing components are included in the Data and Methods section, below. Finally, two members of the team were appointed to a legislatively-mandated Task Force that, in March 2018, recommended options for a formal statewide evaluation. This summary report includes data and analyses from the four standalone reports generated to date12 —all of which are part of the team’s efforts to establish a formal statewide evaluation of the program. It addresses each of the questions originally proposed in an initial research plan developed in 2014 to the extent allowable with current data. In many cases, the data necessary to completely address a question are not available; these deficiencies are noted in the text, along with anticipated timelines for when such questions can be answered, or, in the case of questions answerable only with data not collectable under current statute, recommendations for statutory or policy changes that would make such data collection possible. Questions Guiding the Foundational and Background Portion of the Evaluation The first phase of the evaluation was designed to address several short-term and longer-term quantitative and qualitative outcomes that provide foundational and background context for future research and evaluation. The components of this phase of the evaluation are listed here: Gauging the Extent to which Access to the Opportunity Scholarship is Equitable across Potential Participants 1. How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? 2. Are there differences in access across counties? For example, do rural students have proportionally more or less opportunity to attend a participating nonpublic school than do urban and suburban students, in terms of both availability and cost? 3. How do families receive information about the Opportunity Scholarship and participating nonpublic schools? Are there differences in information access by sub-groups (by ethnicity, by geographic location, etc.?) Does the timing of the application (early February) impact who applies? Assuring Access to a Quality Education for Participating Students from Low-Income Families 4. What are the characteristics (including student performance results) of nonpublic schools that choose to 12 All three completed reports are available at: https://ced.ncsu.edu/elphd/research/the-impact-of-the-north-carolina-opportunity- scholarship-program/; all future reports will be posted to the same website.
Page 12 Ensuring Opportunity in the OS Program participate in the voucher program, and how do they compare to public and non-participating nonpublic schools? 5. Are there discernible patterns in the types of non-public schools that participate (e.g., by location, school type, school size, etc.), and, if so, what are they? What impact has the availability of the scholarship had on the opening of new nonpublic schools? Assuring Fair and Equitable Use of State Funding 6. What are the characteristics of student applicants? Are there systematic differences between students who apply and eligible non-applicants? 7. To what extent do participating nonpublic and nearby public school expenditure patterns differ? As noted above, in some cases, the evaluation team has determined that some of these questions are not yet answerable (and in one case may not ever be answerable), given available data and statutory limitations. These limitations are noted throughout the sections below. We also have included information at the end of this report about our progress toward conducting a rigorous and fair quantitative assessment of the academic impact of the Opportunity Scholarship. A Note on Gaining a Comprehensive Understanding of the Opportunity Scholarship Because this report is a summary of several standalone reports, it does not include all of the important insights and learnings presented in those reports,13 nor does it reflect the information we hope to learn through additional study in the coming years (as outlined in the final section of the report). The authors encourage readers of this summary report to read the other available analyses as well, in order to gain a better overall sense of the ways in which the Opportunity Scholarship is impacting the education of lower-income students in North Carolina. 13 https://ced.ncsu.edu/elphd/research/the-impact-of-the-north-carolina-opportunity-scholarship-program/
Page 13 Ensuring Opportunity in the OS Program Data and Methods Demographic and Descriptive Data Student Demographic Data Much of the student demographic data included in this report were drawn from a dataset of the 2016-17 applicants for the Opportunity Scholarship. This dataset, provided by NCSEAA, consists of 11,459 applications and includes information on students’ demographic information, household income, and final voucher award status. School Descriptive Data Data about non-public schools were derived from a number of sources, including data collected by NCSEAA as part of its process for verifying applicant non-public school eligibility to participate in the Opportunity Scholarship program, as well as data collected annually by the North Carolina Department of Administration’s Division of Non-Public Education as part of its public reporting responsibilities.14 A Note on the Quality and Availability of Non-Public School Data The research team identified multiple inconsistencies and cases of missing values in data provided by the Division of Non-Public Education; the team has been working with the Division to resolve these data issues both for past years and for future data collection efforts. Surveys The team administered surveys online to parents and guardians of Opportunity Scholarship applicants, as well as to people in leadership positions in non-public schools (whether those schools participated in the Opportunity Scholarship program or not). Instrument The process for developing specific questions for each survey consisted of reviewing relevant research to identify existing applicable questions from other studies, as well as writing original questions that were specific to the North Carolina context. The final survey instruments consisted of approximately twenty questions and took approximately fifteen minutes to complete. Questions were a mixture of multiple-choice with responses on a Likert scale (e.g., Very Dissatisfied, Dissatisfied, Neither Satisfied nor Dissatisfied, Satisfied, Very Satisfied) and open-response questions. The surveys were piloted at North Carolina State University prior to distribution. 14 Some of these data are posted on the Division’s website (https://ncadmin.nc.gov/citizens/private-school-information), but other data were provided after a public records request.
Page 14 Ensuring Opportunity in the OS Program Response Rates Parents. In coordination with NCSEAA, the research team emailed an invitation to the parents/ guardians of all applicants to the Opportunity Scholarship Program in 2015-16 and 2016-17 to participate in the online survey. Accounting for inactive email addresses, the target sample was 12,894 students, represented by 8,739 parent email addresses, and we received 2,425 parent responses, representing a response rate of 28%—somewhat low, but not unexpected, given the challenges NCSEAA has had in establishing follow-up connections with many applicants. Parent/guardian responses were split between first-time applicants (59 percent) and renewal applicants (41 percent). The majority of parents we surveyed indicated that they were applying for an Opportunity Scholarship for just one child (63 percent). Twenty-six percent indicated that they were applying for Opportunity Scholarships for two children, with the remainder applying for three or more children. Schools. In coordination with the Division of Non-Public Education, the team gathered contact information for all private schools in the state (n=724) and distributed the survey via email. Respondents occupied various school leadership roles, including principal, administrator, school founder, school president, school director, business manager, and Director of Admissions. Accounting for inactive email addresses, the target sample was 673 schools and we received 266 responses for a school response rate of 40 percent—again, somewhat low, but also not unexpected, given the lack of incentives for a school to respond. Respondent schools were categorized as participating in the program in one of four ways: Schools that currently participate in the program (n=177), schools that have never participated in the program and do not plan to do so (n=58), schools that are planning to participate in the program in the future (n=25), and schools that participated in the program in a previous year, but withdrew (n=6). When analyzing the data, the research team broke out responses across categories where appropriate (remaining mindful of the differences in sample size across sub-groups), to identify differences in responses by school type. Focus Groups Focus groups and interviews with parents of Opportunity Scholarship students and with private school leaders were conducted in two waves, starting in the summer of 2014 and concluding in the summer of 2016. To recruit participants for the first wave of data collection, the research team identified a stratified random sample of schools in order to solicit feedback from stakeholders in different regions of the state, serving student populations of varying sizes, and representing both religious and non-religious schools. Parents and leaders from just two schools agreed to participate for this first wave of data collection, so, to boost participation in the second wave of data collection, the research team worked with a third-party school choice non-profit organization that was well-known to many of the participating schools. This organization helped the research team to identify and recruit schools and parents to participate in the second round of focus groups, which took place in the summer of 2016. Focus groups were conducted in five locations: Wilson (East), High Point (Piedmont), Raleigh (Central), Charlotte (West), and Fayetteville (South). As a result of the recruitment 15 NCSEAA reports that one of the key reasons why nearly every qualified family that applies for a Scholarship is offered one is because the agency is not able to re-establish contact with many of the applicant families after the initial application.
Page 15 Ensuring Opportunity in the OS Program approach taken by the partnering organization, the sample is less representative than it would have been if randomly chosen, but participation numbers were much higher than they had been before the organization’s involvement. Overall, the final sample for focus groups and interviews consisted of 49 school leaders and 13 parents, representing 34 schools across 15 counties. Testing A key challenge to conducting a meaningful evaluation of the overall impact of the Opportunity Scholarship program is the statutory language that allows participating non-public schools to fulfill the annual testing requirement via any nationally-normed test. Because results from different tests are not easily comparable (for example, establishing what is called a concordance across two different tests requires at least a sample of students to take both tests—and in the case of the North Carolina program, literally dozens of different kinds of tests are administered by participating schools in order to fulfill the statutory testing requirement), the testing data collected by the state cannot be used in rigorous estimations of the overall academic impact of the program. Limiting analyses to only those schools that offer a certain test (say, for instance, all participating non-public schools that administer the Iowa Test of Basic Skills in order to fulfill the requirement) introduces another set of biases, as those schools are not necessarily representative of all of the participating schools in the state. These and other limitations are described in greater detail in the Understanding the Current Academic Outcomes Evaluation Landscape section below. To begin to address this challenge, the research team secured private funding and the cooperation of a sample of public and non-public schools to conduct testing of Opportunity Scholarship-awarded and -eligible students in spring 2017 on a common measure. While providing better information and analytical opportunities than would have been possible with extant, state-mandated testing alone, there remain several limitations to what can be learned from this approach to measuring the impact of the voucher on student academic achievement. The section on academic outcomes included at the end of this report highlights these challenges and offers possible solutions.
Page 16 Ensuring Opportunity in the OS Program Gauging Equity of Access: Students and their Families Characteristics of Participating Students How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? What are the characteristics of student applicants? For the 2016-17 school year (the third year of the program),16 69 percent of all applicants were first-time applicants, with the remaining 31 percent applying to renew a scholarship received in a previous year. Eligibility Families that receive vouchers are among the lowest-income households in the state, and many are well below the eligibility threshold for each household size (e.g., below $30,044 for a family of two; below $60,976 for a household of six). The median adjusted per-household-member income17 is $16,213 for new voucher recipients and $15,000 for renewal recipients. In each year of the program, many applicant families are deemed ineligible for the program. In total, 1,976 new applicants and 110 renewal applicants were classified as ineligible in 2016-17. For new applicants, the two most common reasons for ineligibility were that the student did not currently attend a North Carolina public school (51 percent) or that family income exceeded the statutorily-defined income cap for a given household size (42 percent). For renewal applicants who were determined to be ineligible, the most common reason was an increase in family income, such that it exceeded the cap (88 percent of cases). As had been the case in previous years, over one quarter of new applicants in 2016-17 were deemed eligible for the program but were unresponsive to NCSEAA’s attempts at communication. As a result, these students were not awarded an Opportunity Scholarship. Potential explanations for this phenomenon include: the parent’s email address used at the time of application may have become inactive, or the cell phone number provided on the application form was no longer in service; applicants may have moved out of state; applicants may not have received admission to their desired private school; or applicants may have determined that private school tuition and fees would not be affordable, even with the assistance of the Opportunity Scholarship. In 2014, the evaluation team worked with Master’s in Public Policy candidates at the Sanford School at Duke University to draft a comparison of the Opportunity Scholarship guidelines with those of other programs across the country. That comparison will be available soon.18 16 All demographic data reported in this section of the report are for the 2016-17 school year; some geographic data are for the 2015- 16 school year. 17 Adjusted per-household-member income = household income / square root of household size; this approach is commonly used by economists to account for household savings from economies of scale and resource-sharing.
Page 17 Ensuring Opportunity in the OS Program Grade Level Compared to public school students, voucher recipients are more likely to be in elementary school and they are less likely to be in high school (Figure 1). Though speculative only, possible reasons for this discrepancy may have to do with differences in the relative number of seats available across grades at participating private schools, or with a decrease in interest among potential eligible families to move students into a new school as they grow older, or both. Figure 1. Grade Levels of All Opportunity Scholarship Recipients, 2016-17 Ethnicity A larger proportion of 2016-17 Opportunity Scholarship recipients were African-American, relative to the population of public school students (35% versus 25.5%), and a smaller proportion of recipients were Hispanic (9.5% versus 17.3%) or White (41.3% versus 48.6%). In addition, a much higher proportion of Opportunity Scholarship recipients identified as being either biracial or of an ethnic background not common in North Carolina than did public school students as a whole (12.2% versus 4%). Because of the close association in North Carolina between poverty and race, these differences mostly are reflective of differences between the statewide population of students eligible for the Opportunity Scholarship and the ineligible population. 18 The draft report is entitled 50-State Voucher Scan: An Overview of United States Voucher Programs; it is currently being updated by one of the authors of the report. An updated list of comparable programs nationwide also can be found at this address (though the list does not include direct comparisons to the Opportunity Scholarship): https://www.edchoice.org/research/the-abcs-of-school- choice/
Page 18 Ensuring Opportunity in the OS Program Differences between Participating Students and Other Eligible Students Are there systematic differences between students who apply and eligible non-applicants? Our conversations with and survey responses from participating private school headmasters and parents of Opportunity Scholarship recipients in 2014 and 2016 revealed additional details about the recipient population that are not captured in traditional demographic categories. Reflecting the diversity of the private schools they attended, the population of Opportunity Scholarship recipients is quite diverse along a number of axes, as suggested by the data reported above and as supported by the random nature of the lottery used to identify recipients. In terms of fairness and equity in the abstract, the program treats all applicants relatively equally, with only a few exceptions (for example, the lottery does preference siblings and members of military families). Worth considering, however, is whether certain eligible populations are more likely to attempt to access the Scholarships at a greater rate than are others. We consider this question along three axes: differences in access to information about the program; the equity of the application process itself; and finally, participating school perceptions of the characteristic of their Opportunity Scholarship students.19 Access to Information about the Opportunity Scholarship How do families receive information about the Opportunity Scholarship and participating nonpublic schools? Are there differences in information access by sub-groups (by ethnicity, by geographic location, etc.?) Information Networks. The viability of a school choice program like the Opportunity Scholarship program depends on use of the program by the intended population. One of the primary concerns, therefore, is the possibility of inequities in parent awareness of the program’s existence and of their eligibility for it. To learn more about these possible inequities, via survey we asked parents to share how they learned about the Opportunity Scholarship program (Figure 2, following page). Their responses revealed the importance of social networks in the information diffusion process. The majority of parents (40 percent) indicated that they had learned about the program through informal means, primarily from conversations with friends and relatives. Thirty-three percent indicated that they had been contacted about the program by a private school, and 15 percent learned about the program from an Internet search. Such searches likely pointed the user either to news articles related to the program or to the website maintained by the state agency that administers the program (NCSEAA), which offers detailed application information and up-to-date program usage statistics on its website.20 Several state and national school choice advocacy organizations also offer information about North Carolina’s program on their websites. 19 We explore a fourth, related aspect—location of participating schools—in a later section. 20 www.ncseaa.edu
Page 19 Ensuring Opportunity in the OS Program Figure 2. Pathways through which Parents Learned about the Opportunity Scholarship Program Note: n = 2,425 respondents; TPS = Traditional Public School Focus group data revealed similar learning channels. The majority of participants indicated that they initially became aware of the program through word of mouth: We did not know anything about the scholarship. I think a friend of ours told us. . . . – Fayetteville Parent (2014) My daughter’s basketball coach told me about it, I think before I saw the billboards and stuff. He was like, “Do not forget to get on the computer and do the application on such-and-such date. . . .” – Fayetteville Parent (2014) I didn’t even know this program existed until I spoke with [advocacy group staff member]. Somebody had given me her number and that’s how I found out about this Opportunity Scholarship. – Fayetteville Parent (2016) In some cases, the word-of-mouth network included the private schools themselves. Of particular interest in terms of equity of access to information, their dissemination of information may be most important for
language minority populations. [Y]ou get a lot of phone calls through the admission process: “What kind of tuition assistance do you offer? What do you have?” We can’t do a whole lot because of our school size and where we’re located so I very quickly state that there is a program through the state of North Carolina and I direct them to the website. – High Point Headmaster (2016) [W]e do a radio ad that lets families know they can get online and check their eligibility. . . . – High Point Headmaster [B]ecause of connection with our Spanish church, our Spanish pastors . . . will . . . give the information out to their Spanish members of the church . . . . – High Point Headmaster (2016) [O]ur Spanish church has quite a few of the Opportunity students coming. The problem I find [is that] most of the parents are speaking Spanish, they do not speak English, so that all the [information about OS is] coming in English and they can’t read them. – High Point Headmaster (2016) Our survey asked parents to share how easy it was for them to access information about the Opportunity Scholarship program, once they began their search. Just over 80% of survey respondents agreed or strongly agreed that it was easy to get the information they needed about the program. Selection of Private Schools. Our survey also asked parents about their procedure for choosing private schools. School choice research suggests that parents become more discerning in their school “shopping” behavior as experience with participating in a school choice program grows.21 Those who are inexperienced with exercising school choice may not consider a broad range of schools initially, but grow more sophisticated in their search for a private school over time. It is possible that these national trends are reflected in our data, as the majority of parents surveyed so far (that is, during the early years of the program) indicated that they only considered a single school when choosing which private school their child would attend with the Opportunity Scholarship (Figure 3, following page). This also may be an artifact of the rural nature of many of North Carolina’s counties, in which there are often very few private schools (and sometimes only one) from which to choose (discussed in greater detail below). Some participating private schools also make referrals, as indicated by participants in our focus groups: [I]f there’s a school that would better fit that child’s needs and so forth we’ll do a referral knowing that the Opportunity Scholarship could be taken by that school, so there is that discourse. Also with administrators I’ll call and let them know we’re sending someone [their] way that has the Opportunity Scholarship that could benefit from [their] school. – Raleigh Headmaster (2016) Page 20 Ensuring Opportunity in the OS Program 21 See Stewart, T., & Wolf, P. J. (2014). The School Choice Journey: School Vouchers and the Empowerment of Urban Families. New York, NY: Palgrave MacMillan.
Page 21 Ensuring Opportunity in the OS Program Figure 3. Number of Private Schools Considered Note: n = 2,277 respondents The Application Process and Equity of Access Does the timing of the application (early February) impact who applies? Standards for admission as a possible barrier to access will be discussed below, but the greater access challenge with respect to admissions may have more to do with differences in families’ understanding of the admissions process. Especially during the first year, some Opportunity Scholarship applicant families were unaware that the application process for an Opportunity Scholarship was not also the application process for their top school choices, and that receipt of an Opportunity Scholarship was not a guarantee of enrollment in a given school.22 In a survey administered by the research team to Opportunity Scholarship applicant families, this disconnect was the second most frequently selected reason for non-acceptance of a Scholarship among applicants who declined an offered Scholarship. Headmasters noted as much across multiple locations and across years: 22 Based on interview responses from state-level administrators of the program; the research team did not ask this question directly of Opportunity Scholarship parents who responded to our survey.
Page 22 Ensuring Opportunity in the OS Program [W]e have had a bunch of people in [our community] . . . put [our school down as their first choice] but not really understand when they put that down that when they were accepted into the program, that did not mean that they were accepted into the school as well. They still had to apply and do everything else. – Wilson Headmaster (2014) [Y]ou have to explain that to parents when they come in. They’ll say, “I have a scholarship, I’ve been awarded a scholarship,” and then you have to explain your process in terms of admissions. – Raleigh Headmaster (2016) They’ve heard of it, they know there’s something available but . . . the ones that I talk to are rarely very learned as to . . . how to go about doing it [completing the application process]; they need to have their hand held through it. . . . – High Point Headmaster (2016) Focus groups with parents shed more light on those struggles: Navigating print was pretty intensive. It was a lot of reading. I was able to understand everything pretty easily, but I could see for people that are not well-versed in how schools work and things like that it could have been frustrating. – Wilson Parent (2014) This type of application confusion is one that likely will diminish as the program grows and the informational materials and webinars offered by NCSEAA continue to stress this program requirement. Indeed, data from our 2016 survey indicated that 87 percent of responding families agreed or strongly agreed that the application process was simple to complete, and some private schools indicated that they provided application support for families in need of it. [D]uring our admissions process when parents come out to find out about our school, we actually let the parents know, we give them information about the Opportunity Scholarship and show them . . . how to apply. If they need help applying, we actually set them up on a computer at our campus and allow them to apply there. – Raleigh Headmasters Some parents indicated that having a point of contact to guide them through the application process was particularly useful: It really helps to have an advocate, you know, someone . . . who’s very experienced to kind of help people along. – Charlotte Parent (2016) Impact of the timing of the application. One particular potential access challenge for programs like the Opportunity Scholarship is the administrative calendar—when applications for participation in the program are due, when the funds are actually awarded, and when parents have to make decisions about their children’s schools. While some focus group parents indicated that the timing was not consequential, other parents and several headmasters noted challenges related to the program’s timing. [The calendar] hasn’t [worked] for us. . . . January 1, we start taking applications for those children who would be starting with us in August. So the challenge is that if the Opportunity Scholarship folks are
Page 23 Ensuring Opportunity in the OS Program applying in February and then getting notified in March that they’ve received a scholarship . . . our seats have been filling up when we do a first notification in the first of February. – Raleigh Headmaster (2016) [W]e actually start enrollment for the next year in January. And so, when they start putting information out to us—we’re talking February-March—by that time sometimes we’ve already filled up and we would like not to have to give sort of the advantage to the tuition-paying parents when the other students should have a right to come and be accepted as well. – Fayetteville Headmaster (2016) I had access and luck that other people probably did not have. I wonder how many people were told that they had the scholarship but did not have enough time to get all of that stuff turned around. – Wilson Parent (2014) Impact of the timing of the award. Another recurring timing-related theme is the challenge faced by private schools when they try to integrate recently-awarded Opportunity Scholarship students in the middle of the school year. This challenge is not new to most public schools, some of which constantly enroll new students throughout the year, but focus group data suggest that it may not be as common among private schools. That’s the biggest challenge, for us at least, is if you have a waiting list and not everybody responds to the email when they’re awarded the scholarship and so those families get moved back to a waiting list and other people who are on the waiting list get moved up and then, it’s February or March and you have these families calling me saying “I got the Opportunity Scholarship.” And I’m thinking, I can’t do this to my teachers, they’re going to kill me. – High Point Headmaster (2016) [T]he parent does not want to go ahead and enroll [a student] in our school because they know that they cannot afford the tuition so they put it off as long as they can and here we are, you know, trying to figure out numbers, trying to figure out staffing and the parents are, “Well, I’m still waiting to hear from the scholarship.” – Fayetteville Headmaster (2016) From the parents’ perspective, the challenge is as much related to how it impacts their school choices as it is about changing schools in the middle of the year. Parent 1: [O]ne or two [schools had] closed [admissions] at the time that we needed to get in and they couldn’t break it up, so [the school we eventually chose] allowed us to come in and [my child] would have to make [missed work] up in the summer time. Parent 2: Same thing for me, [my first choice] wouldn’t accept him mid-year, so I went to somewhere that was like more flexible. . . . – Fayetteville Parents (2016) This enrollment timing challenge may dissipate as the program evolves and all awards are received early enough for families to make decisions in the summer before school begins.
Page 24 Ensuring Opportunity in the OS Program Perceptions of Opportunity Scholarship Students at Participating Schools What are the characteristics of student applicants? Finally, we consider how staff at participating schools characterize their Opportunity Scholarship students— not as an exercise in critiquing participating school perceptions but instead to investigate whether their descriptions reveal any additional characteristics of the students who apply for the Scholarship that are not discernible in the more quantifiable data reported above. Our survey asked school leaders to describe their satisfaction with students’ preparedness upon arrival at their schools: 91% were satisfied with the preparation of their typical students, and 80% were satisfied with the preparation of their Opportunity Scholarship students. We had hypothesized that the perception would be that Opportunity Scholarship students are perceived as being less prepared than their fee-paying counterparts, which is borne out by the data; with a high proportion reporting satisfaction with both groups, however, the disparity is less notable than it might have been, had the overall satisfaction rate been lower. Focus group participants also noted that some of their Opportunity Scholarship students are not always ready for the transition to their schools; incorporating these students into their schools often involves helping them transition not only to a new academic environment but also to a different cultural and social environment: [L]ast year there was a lot of kids coming in with B’s on report cards in math classes that came to our math classes and not have a clue of what was going on. – Fayetteville School Leader (2016) [T]ranscending even the testing is the work that’s being done teaching these kids how to be good students, . . . teaching them how to learn respect and how to treat people. . . . [W]e get kids that are coming ready to fight, who are disrespectful, who have no concept of how to behave in a classroom, unlike kids that come up through our system. – Fayetteville School Leader (2016) Focus group participants also noted, however, that the Opportunity Scholarship gave them an opportunity as a school to extend their services to students they wanted to reach but for whom tuition was a barrier. For example, one participant indicated that he appreciated that the Opportunity Scholarship made his schools more accessible to special population students: [W]e work with students that have learning differences; it’s a school that works with students with average to above average IQ, bright students but they either have an attention deficit diagnosis or another diagnosed learning disorder. . . . [The Opportunity Scholarship] really opens doors that [families] never thought would ever be open to them before. So that’s the benefit that we’re reaping from it; it’s not just rear-ends in the seats, it’s kids . . . that really need what we do because it’s very specialized what we do. They could have never afforded it, never entertained the idea of coming to our school. Now they can, and they can make it. The $4,200 makes it work in some budgets where it could never have worked before. – High Point Headmaster (2016)
Page 25 Ensuring Opportunity in the OS Program Assuring Access to a Quality Education for Participating Students: Participating Schools Characteristics of Participating Private Schools How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? Are there differences in access across counties? For example, do rural students have proportionally more or less opportunity to attend a participating nonpublic school than do urban and suburban students, in terms of both availability and cost? What are the characteristics of nonpublic schools that choose to participate in the voucher program, and how do they compare to public and non-participating nonpublic schools? Are there discernible patterns in the types of private schools that participate (e.g., by location, school type, school size, etc.), and, if so, what are they? Geographic Distribution Students in 97 of North Carolina’s 100 counties received scholarships for 2016-17.23 Relative to overall county population, participation for that year was much higher than expected in Hoke, Mitchell, Northampton, Bertie, Perquimans, and Washington counties; by the same measure, participation was much lower than expected in Orange, Burke, Chatham, Haywood, Moore, Lincoln, and Watauga counties. In some cases (e.g., Lincoln and Watauga), these differences may be related to the somewhat uneven distribution of participating private schools across the state (Figure 4, following page) and to the relatively low overall number of recipients, but other factors likely contributed as well, possibly including variations in: the perception of public school quality among eligible families in different counties; varying outreach efforts by Opportunity Scholarship advocates across the state; and tuition of nearby participating private schools relative to scholarship amount. Access to the Opportunity Scholarship Statewide Access is determined by a number of factors, including accessibility (determined by such factors as location and personal transportation options), cost after application of the Opportunity Scholarship (including financial aid support), and admissions standards. General Accessibility. As the program has grown, so, too has the number of participating schools—from 224 schools with Opportunity Scholarship recipients enrolled for the 2014-15 school year to 359 schools with enrollments for the 2016-17 school year (see Appendix for a full list). In addition, up to 100 more schools each year have been willing to enroll students (an additional 109 in 2014-15, an additional 101 in 2015-16, and an additional 78 in 2016-17). 23 In 2016-17, Tyrrell (size rank by population: 100), Hyde (99), and Dare (66) counties had no students who were awarded Opportunity Scholarships.
Page 26 Ensuring Opportunity in the OS Program Figure 4. Participating North Carolina Private Schools, 2015-1624 Note: Purple diamonds represent participating private schools (2015-16); grey diamonds represent non-participating private schools (2015-16) 24 This map and others included in this report were generated by Emily Antoszyk for the Friday Institute’s Consider It Mapped series on EducationNC. 25 For example, in 2015-16, those counties included: Alexander, Anson, Ashe, Bladen, Camden, Caswell, Cherokee, Chowan, Davie, Duplin, Franklin, Gates, Graham, Hyde, Jones, Martin, Pamlico, Pender, Perquimans, Polk, Swain, Tyrrell, Warren, Washington, Watauga, Yadkin, and Yancey. 26 All counts are based on schools that actually enrolled students for a given year. Readers are asked to keep in mind that the number of communities with schools enrolling Opportunity Scholarship students is lower than the number of communities with schools eligible to enroll these students; several schools indicate a willingness to participate in the program each year but either do not end up receiving applications from these students or decline admission based on admission standards. We use enrolling school numbers rather than eligible school numbers, even though they underrepresent the total number of potentially participating schools, because the list of eligible schools includes an unknown number that did not actively make their involvement in the Opportunity Scholarship program public. A challenge that we’ve experienced in our school is just the growth that we’ve experienced through the Opportunity Scholarship. So just student population has increased which has provided its own challenges; which is a good challenge to have. – High Point Headmaster Several rural counties do not have participating schools (Figure 4, above).25 However, school participation growth has been relatively equally distributed geographically. Schools in 92 communities enrolled Opportunity Scholarship students in 2014-15,26 and the number of communities served increased to 128 (328 schools) in 2015-16 and 137 (358 schools) in 2016-17. Data on the number of available enrollment slots are not available, so it is not possible to estimate geographic differences in accessibility at the individual student level (i.e., whether there are proportionally more or fewer slots available in one community relative to another).
Page 27 Ensuring Opportunity in the OS Program Rural versus Urban Accessibility. Unsurprisingly, growth in terms of the number of schools has been strongest in the largest urban areas—with the number of schools in Charlotte, Raleigh, Fayetteville, Winston-Salem, Greensboro, and Durham that enrolled Opportunity Scholarship students growing from 79 schools in 2014-15 to 111 in 2015-16 and 120 in 2016-17—but that growth did not necessarily represent an imbalance, in the sense that the proportion of enrolling schools that were located in those communities remained at around 34% of all enrolling schools each year. In other words, as the program grew, the number of schools available for students in urban areas grew proportional to the number of schools available in less urbanized areas. In fact, the total number of participating schools grew in 33 communities in Year 2 and by an additional 17 communities in Year 3, with only 6 communities and 11 communities, respectively, seeing declines in the number of enrolling schools. In Year 2, 38 communities with no enrolling school in Year 1 were home to at least one enrolling school, and in Year 3, 14 more communities without an enrolling school in Year 2 were home to at least one enrolling school. Transportation. Opportunity Scholarship recipients in most counties appear to have access to at least one participating school within a reasonable driving distance of most communities (Figure 4, above), as indicated by the number of counties with Opportunity Scholarship recipients (97 of 100 in 2016-17), even though in most years there have been more than 20 counties without a participating school. Most private schools in the state do not provide transportation, but some do offer neighborhood-based shuttle services, and many work with private transportation providers in their area. As noted above in the Data and Methods section, publicly-available data on non-public schools are limited; without conducting a school-by-school investigation of transportation options, it is not possible to quantify accurately the provision of non-public school transportation statewide. Cost and Financial Assistance. Unlike the statewide voucher program in Louisiana, the Opportunity Scholarship requires family co-payment of tuition and fees if those costs exceed the value of the voucher.27 This program feature increases the likelihood that parents will need to maintain direct financial responsibility for their child’s education, but it has been criticized by some for pricing low-income parents out of the market. The Opportunity Scholarship provides up to $4,200 a year for each participating student, which is about $800 less than the state invests in low-income students in grades 4 through 12, and more than $1,500 less than the state invests in low-income early-grade students.28 Including average local and federal supplements provided to public schools for low-income students (about $2,200 in local funding and about $2,000 in federal funding), the amount is less than half of the total average amount per public school student. The voucher amount also is less than the estimated average of one year’s tuition in a typical private school in North Carolina (about $7,600 for elementary; about $9,400 for high school),29 and well below the cost of one year’s tuition in the most exclusive 27 Two other statewide school choice programs that share this design feature are the Indiana Choice Scholarship Program and Florida’s McKay Scholarship Program for students with special needs. The Louisiana Scholarship Program, on the other hand, requires participating private schools to accept the voucher as full payment. 28 Includes state per-pupil expenditures only (not local or federal), plus adjustment for low-income family earnings; does not include adjustments for low-wealth counties, special needs status, Disadvantaged Students Supplemental Fund, or any other district- or school-level adjustments; see Highlights of the North Carolina School Budget (February 2017): http://www.ncpublicschools.org/docs/ fbs/resources/data/highlights/2017highlights.pdf 29 Third-party estimates of means: https://www.privateschoolreview.com/north-carolina; estimated median costs are somewhat lower, but still higher than the amount of the Opportunity Scholarship. The research team cannot confirm the precision of The Private School Review’s figures and uses them here for general descriptive purposes only.
Page 28 Ensuring Opportunity in the OS Program schools (many of which can exceed $20,000 a year). Our tuition is probably maybe a little higher than some. . . . The Opportunity education grant covers a smaller percentage of what [students’] tuition is, [and] we have some kids that get both [state-funded scholarships—the Opportunity Scholarship and a scholarship for students with special needs], but still, that only pays about 60% if they get both grants. – High Point Headmaster Some parents acknowledged that the difference between tuition and Opportunity Scholarship amount influenced the list of private schools they considered, suggesting that cost may have led some parents to choose schools not based on fit alone: I’m a single parent of three kids, so obviously I’m not rich, so I needed [a school] that . . . [the] scholarship could cover as much as possible. So I went with that, that’s why I went with [specific private school]. . . . – Fayetteville Parent (2016) Many schools also have additional fees—expenses that some Opportunity Scholarship families reported initially came as a surprise to them, again often because of their inexperience with the private school process. The Opportunity Scholarship has covered our tuition but there are other costs. . . . There is some added stuff each month, but public school has the same thing, if not more. So it covered our tuition, and everything after that we sacrificed as parents. . . . – Fayetteville Parent (2014) In many cases, however, participating schools are able to provide tuition support, and some even reported determining their available Opportunity Scholarship slots based on the amount of aid they would be able to provide. Admissions Standards. Perhaps the greatest potential accessibility barrier is not geography but individual school admissions and financial assistance policies. By definition, private schools are not obligated to offer enrollment to all applicants, and schools that participate in the Opportunity Scholarship program are not obligated to offer enrollment to all Opportunity Scholarship recipients; however, per state statute (115C-562.5), private schools are not allowed to discriminate on the basis of race, ethnicity, or national origin. In addition, statewide, acceptance rates are high.30 Comparisons to Public and Non-Participating Private Schools In all, 266 non-public schools responded to our survey of headmasters, and 67 percent of those respondents were participants in the Opportunity Scholarship program in school year 2015-16. This variation is large enough to allow us to compare across school categories to estimate the characteristics of all participating and non- participating schools, and to identify potential patterns by school type. 30 One recent estimate places the overall statewide acceptance rate at 86% (https://www.privateschoolreview.com/north-carolina)
Page 29 Ensuring Opportunity in the OS Program Religious Orientation and Affiliation A higher proportion of participating schools (76 percent) indicated that their school has a religious orientation than did non-participating schools (42 percent).31 The most common religious denomination represented among participating schools is Baptist (29 percent), followed by Christian (27 percent), then Catholic (18 percent; Figure 5). Figure 5. Religious Affiliation of Participating Private Schools One focus group participant noted that having a religious orientation can mean more than just having a religiously-oriented curriculum: [We’re] a Christian based school, and that world view is taught in every aspect of our program. – (High Point School Leader, 2016) Admissions As noted above, participating non-public schools in North Carolina’s Opportunity Scholarship Program are permitted to screen all applicants when deciding who to admit, even if a student already has been awarded an Opportunity Scholarship by the state. This feature is common to many special needs scholarship programs, but is not as common among less-targeted voucher programs.32 Voucher programs with alternative designs include the Louisiana Scholarship Program (formerly known as the Student Scholarships for Educational Excellence Program), which requires schools to use an open admissions process for enrolling scholarship recipients, and the Milwaukee Parental Choice Program, which requires schools to admit eligible students on a random basis. 31 This difference is statistically significant, p = 0.000 32 See, for example, the Arkansas Succeed Scholarship Program, Oklahoma’s Lindsey Nicole Henry Scholarship for Students with Disabilities Program, Louisiana’s School Choice Program for Certain Students with Exceptionalities, and North Carolina’s Special Education Scholarship Grants for Children with Disabilities. Baptist Christian Catholic 7th Day Pentecost. Lutheran Other
Page 30 Ensuring Opportunity in the OS Program To learn more about admissions criteria in North Carolina, we asked schools to share the criteria they routinely use when admitting students. The vast majority of schools in our sample that participate in the Opportunity Scholarship program reported rely upon an interview with prospective students (81 percent). The second most frequently selected option was information about students’ academic ability, as measured by test scores (64 percent), and the third most frequently selected option was a review of the student’s disciplinary record (59 percent; Figure 6). Figure 6. Admissions Criteria Notes: n = 178 participating schools, 88 non-participating schools; OSP = Opportunity Scholarship Program; * indicates that a difference is statistically significant at p < .05 Data for the non-public schools that do not participate in the Opportunity Scholarship Program and that responded to the survey (n=88) revealed a somewhat similar pattern, though criteria were more evenly distributed. The majority of these schools also relied upon an interview with prospective students, but at a lower rate (66 percent). The second and third most frequently selected options were a review of academic achievement, either measured by a test (41 percent) or some other indicator of academic achievement (44 percent).
Distinguishing Characteristics We also asked participating schools to choose the characteristics that they believe set them apart from nearby public schools (Figure 7).33 The most frequently selected perceived distinguishing characteristic was a stronger emphasis on college preparation, chosen by 19 percent of participating schools (n=34). The second most frequently selected option was a stronger emphasis on character building, also chosen by 19 percent of participating schools (n=33), and the third most frequently selected option was religious education, chosen by 16 percent of participating schools (n=28). Figure 7. Distinguishing Characteristics of Participating and Non-Participating Schools 33 As with all such choice lists in the survey, the response options were presented in a random order so as not to bias the frequency with which certain response categories were selected. Notes: n = 178 participating schools, 88 non-participating schools; OSP = Opportunity Scholarship Program; * indicates that a difference is statistically significant at p < .0 Page 31 Ensuring Opportunity in the OS Program
Page 32 Ensuring Opportunity in the OS Program In focus groups, participating school leaders provided more detail about what they believe makes their schools different from their public school neighbors. Some noted their focus on special populations; for instance, one school leader noted that his school “work[s] with students that have learning differences” (High Point School Leader, 2016)—but most emphasized their ability to provide a religiously-minded curriculum and culture: [O]ur mission is to impact the world for Christ – Concord School Leader (2016) We posed the same question to non-public schools that elected not to participate in the Opportunity Scholarship Program, which revealed key differences in how these schools perceive themselves. When asked to choose the top characteristics that they believe distinguishes their school from nearby public schools, non- public schools that elected not to participate in the Opportunity Scholarship Program were most likely to say that their school requires less standardized testing, with 24 percent choosing this option (n=21)—statistically significantly higher than the proportion of participating schools that selected this option (10 percent). The second most frequently selected option was a stronger emphasis on character-building, chosen by 18 percent of schools (n=16), and the third most frequently selected options were religious education and a stronger emphasis on college preparation, each chosen by 10 percent of non-participating schools (n=9). Non-Participating Schools We also were interested in learning more about the factors influencing the participation decision of schools that opted out of the program, so we posed this question to the leaders of schools that have never participated in the Opportunity Scholarship Program and schools that indicated they have no plans to do so in the near future (n=58). The most frequently selected factor explaining a school’s non-participation decision was a concern that future state laws or policies might change requirements for participation, with 57 percent of non-participating respondents indicating that this concern played a role (minor or major) in their decision. Another common concern was the amount of paperwork and reports that are required of participating schools, selected by 47 percent of non-participants. Interestingly, 54 percent of schools in this category chose “Other Concerns” as playing a major or minor role, implying that their primary reason for non-participation was not listed in the options we provided. We offered an open-response text box for schools to list their concerns if they selected this option. The two most commonly cited reasons provided were that their school was not aware the program existed or that their school makes an effort to avoid accepting government funds: [We] did not know about the scholarship program. The board leadership of the school is concerned about political entanglements that comes [sic] from receiving state and/or federal funding. We question the long-term sustainability of the program. Also as a matter of policy and principle, we do not participate in government funded programs, directly or indirectly. Concerns that opportunity scholarship students may require more resources than we have to offer. We did not have enough financial aid money left when the applicant applied to fund the year’s tuition for them. They wouldn’t have had enough.
Page 33 Ensuring Opportunity in the OS Program Focus group responses indicated a few other reasons why participating school officials believed that some schools chose not to participate. While some believed the problem was the gap between their tuition and the coverage provided by the Opportunity Scholarship, others thought it had more to do with the types of students who were eligible for the funding. [S]ome of the schools in the area that have more expensive tuition, the Opportunity Scholarship program doesn’t offer enough aid to make it worth it. So they choose not to participate. – High Point Headmaster (2016) R: [U]unfortunately, I think it’s the socio-economic question, and . . . I was, like, “Well, you know, everybody can have their opinions.” But I just kind of have gone over with them that we have not experienced any problems whatsoever. I: So they’re concerned about a change in their student population? R: Yes. . . . – Raleigh Headmaster (2016) We have a lot of private Christian schools in [nearby community], they don’t want low-income students in their schools, so they don’t want the Opportunity Scholarship, that’s the bottom line. – Raleigh Headmaster (2016) It is important to keep in mind that this last group of perceptions of non-participating schools come from participating private schools only and has not been verified in any way, but the difference it highlights with respect to their own willingness to enroll lower-income students versus their perceptions of other schools’ resistance to do so may merit additional study. Expenditure Patterns of Participating Private and Nearby Public Schools To what extent do participating nonpublic and nearby public school expenditure patterns differ? Expenditure patterns for public schools are relatively easy to trace across years via analysis of state object and purpose codes. The quality and validity of the data associated with those codes varies from school district to school district, as does each district’s interpretations of some of those codes, but, overall, it is possible to determine general patterns in spending of local, state, and federal funds. Because of limitations in the reporting required of non-public schools,34 this level of fiscal transparency does not exist to a similar degree among Opportunity Scholarship schools. As a result, the question may never be answerable unless the state requires reporting of such financial data by participating schools. Even so, the evaluation team was able to piece together—albeit primarily anecdotally via focus group 34 See, for instance, the publicly-reported data available on the website of the North Carolina Department of Education’s Division of Non-Public Education (DNPE; https://ncadmin.nc.gov/about-doa/divisions/division-non-public-education)—the most comprehensive source for non-public education data in the state. Two of the authors of this report have worked with DNPE to improve data availability for private schools, but the results of that effort will not include additional financial information.
Page 34 Ensuring Opportunity in the OS Program responses—some information about financial decision-making at participating private schools that is directly connected to the participation of those schools in the Opportunity Scholarship program. Tuition Support Some schools have begun a practice of personalizing tuition based on what a family can afford to pay, or some other related metric: What we’ve been doing is we’ve been making accommodations for our students who participate in the OS program; but . . . even though . . . the Opportunity Scholarships have been extremely helpful for those parents who wanted to continue educating their students through our school, it’s still a lot of extra costs where we have to try to work something out for them to attend. – Raleigh School Leader (2016) One common theme across focus groups was a belief that requiring some parent participation in covering tuition costs—even if nominal—was an important part of generating parent commitment to the school: [W]e . . . feel like the parent needs to have a vested interest, so we still are requiring those parents pay the registration fee. . . [W]e feel like they ought to have some ownership. – Fayetteville School Leader (2016) [W]e want those parents to have that little bit of commitment. . . [I]f you’re just sending your kid and you know, not participating in the education of the child at all, what are you gaining, what is the child gaining from it? – Fayetteville School Leader (2016) [W]e do have tuition assistance, however, we have a rule of thumb that everyone has to pay something for school so no one has a full ride, but we do try to work with families and we have an outside evaluator who helps us to see exactly what they can pay and we work with them individually on that. – Concord School Leader (2016) Participation and Enrollment Restrictions Related to Finance Though we have very little data on this particular issue, one finance-related consideration with respect to private school participation in the program is whether a school can enroll Opportunity Scholarship-eligible students without state support. For some schools, even though they already could support some tuition reductions on their own, the Opportunity Scholarship eased their ability to do so. Other schools, however, appear to have weighed the value of that state support against the regulations that come with it and decided that the extra support was not worth participation in the program. [S]ome [private schools] have their own tuition reduction program. They don’t accept a[n Opportunity] Scholarship but they have their own kind of thing. – Fayetteville Parent (2016) As a result, there may be some value in exploring further whether schools of a certain financial standing are less likely to accept the Opportunity Scholarship.
Page 35 Ensuring Opportunity in the OS Program There also appears to be a related debate at several private schools about how much of their available funding they are willing to reallocate to scholarship support for Opportunity Scholarship families whose vouchers only cover part of their tuition. These schools appear to be wary of the program’s stability and remain concerned that they may run the risk of taking on a significant financial burden, if the program ends and they want to keep their Opportunity Scholarship families in their schools: We had conversations about whether we should apply for the Opportunity Scholarship, and understood that there would be a cap on the number of students we would accept using the scholarship, because if the money goes away, when the money goes away, what will we choose to use? Once we accept a family into our [school] family . . . we did not want to be in a position to have to say, “OK, 25 families that we love, you are part of the community, you volunteer in our kitchen, you come and help out and do all of these things, but now we have to say goodbye to you because that money is gone.” We recognized that we probably would not do that as a school, so we had to limit the number of people that we would accept using these scholarships to a number that we could afford to—if that money went away—supplement financially. – Wilson School Leader, 2014 The Opportunity Scholarship and Changes in the Supply of Private Schools What impact has the availability of the program appeared to have on the supply of private schools? Overall, available data make it hard to argue that the advent of the Opportunity Scholarship has led directly to a meaningful uptick in the number of private schools in the state (Table 2), even though there was a larger-than- usual spike in the year the legislation passed (2013-14). Also, while the number of Opportunity Scholarship students has increased over the past three years as the program grew, the overall number of private school students statewide decreased, which likely would have offset at least some of the potential growth in the number of schools possibly attributable to the availability of the voucher. Table 2. Private School Openings byYear SchoolYear Number of New Private Schools 2010-11 40 2011-12 52 2012-13 35 2013-14 (legislation passage year) 56 2014-15 (initial scholarship availability year) 39 2015-16 46 2016-17 50
Page 36 Ensuring Opportunity in the OS Program In addition, spikes in private school openings have not been restricted to recent history only; there have been several over the years, most notably during the entirety of the Clinton Administration (1992-2000) and just before the Recession (2005-2008). Finally, these growth numbers do not reflect changes at the other end of the spectrum: Schools that closed in the same year. Thus, overall net growth in the number of private schools in the state is significantly lower across the same years (a total increase of less than 50 schools across those years) than was gross growth (318 school openings; Figure 8). Figure 8. Net Private School Growth in North Carolina, 1991-2017 Though anecdotal only, one school official posited that, rather than encouraging the opening of new schools, the larger impact of the Opportunity Scholarship may have been to stabilize existing schools with financial challenges: I wouldn’t say that any new ones have opened, but there’s some that might have folded or might be closer to folding without it. – High Point Headmaster (2016)
Page 37 Ensuring Opportunity in the OS Program Understanding the Current Academic Outcomes Evaluation Landscape Testing in Participating Schools Per legislation,35 nonpublic schools that receive Opportunity Scholarship funds are required to administer a nationally standardized test or equivalent at least once a year to all Opportunity Scholarship recipients. Results of these tests are reported to NCSEAA, but there are no target performance levels, nor are there any actions NCSEAA is required to take related to the scores. The testing requirement is above and beyond a pre- existing statewide testing requirement for all nonpublic schools (regardless of their Opportunity Scholarship participation), which directs all nonpublic schools to administer a nationally standardized test annually, but only for students in grades 3, 6, 9, and 11. 36 Participating nonpublic schools with more than 25 Opportunity Scholarship recipients also must report aggregated standardized test performance results for eligible students. Unlike the individual test results for all Opportunity Scholarship students, these aggregated results are public record and are reported by NCSEAA annually. While the statutory language, “nationally standardized test,” technically allows participating schools to choose from hundreds of tests, most appear to meet this criterion by administering the same tests they administer to all of their students in the required grades. A full list of tests administered each year, like so many other aspects of the Opportunity Scholarship program, is not available, largely due to administrative constraints at NCSEAA: Without adequate funding to provide a full support staff to oversee the Opportunity Scholarship program, the manpower necessary to tally all tests administered each year currently is not available. Only data for school year 2014-15 is relatively complete, primarily because of the smaller number of Opportunity Scholarship participants that year. Tests from the first year of the program (2014-15), in descending order of frequency, are listed in Table 3. Table 3. Nationally StandardizedTests Administered to Opportunity Scholarship Recipients by Participating Nonpublic Schools, 2014-15 Test Proportion of OS StudentsTested TerraNova 35.7% Stanford 32.1% Iowa Assessments 24.1% California Achievement Test 3.9% Comprehensive Testing Program 2.2% Woodcock Johnson 1.1% BASI 0.4% Otis-Lennon School Ability Test 0.4% BJU Press Testing 0.2% 35 Section 115C-562 ; https://www.ncleg.net/EnactedLegislation/Statutes/PDF/BySection/Chapter_115C/GS_115C-562.5.pdf 36 Section 115C-549; Section 115C-550; http://ncleg.net/EnactedLegislation/Statutes/HTML/ByArticle/Chapter_115C/Article_39.html
Page 38 Ensuring Opportunity in the OS Program Performance of Participating Students How do participating students perform, relative to national norms? This question is perhaps the most challenging to answer in a comprehensive and objective manner, given the restrictions in the enacting legislation. This report includes two cuts, neither of which is sufficient, but both of which help to underscore the need for more transparent data rules related to Opportunity Scholarship evaluation. Participating Schools with more thanTwenty-Five Opportunity Scholarship Students The first cut includes the aggregate reports for 2014-15 (Table 4) and 2015-16 (Table 5) provided by participating schools with more than 25 students. Table 4. Publicly-Reported Results, 2014-15 School Test Number of Students Tested Reading Language Mathematics % below 50th % at or above 50th % below 50th % at or above 50th % below 50th % at or above 50th Greensboro Islamic Academy ITBS 51 59 41 61 39 53 47 Word of God Christian Academy TerraNova 30 70 30 * * 83 17 Victory Christian Center School TerraNova 28 46 54 50 50 50 50 Concord First Assembly Academy TerraNova 23 52 48 61 39 44 56 Freedom Christian Academy TerraNova 20 45 55 20 80 40 60 Fayetteville Christian School ITBS 2 ** ** ** ** ** ** Verbal Reasoning Nonverbal Reasoning % below 50th % at or above 50th % below 50th % at or above 50th Fayetteville Christian School TerraNova 20 50 50 45 55 Note: Data provided by NCSEAA *Word of God Christian Academy provided scores for Reading and Mathematics only ** Data not provided; less than 5 students. Other students tested by alternate-format Terra Nova; see below.
Page 39 Ensuring Opportunity in the OS Program School Test Number of Students Tested Reading Language Mathematics % below 50th % at or above 50th % below 50th % at or above 50th % below 50th % at or above 50th Alamance Christian School TerraNova 18 28 72 39 61 33 67 Al-Iman School N’west Evol. Assn 23 9 91 9 91 17 83 Bal-Perazim Christian Acad. TerraNova 19 47 53 53 47 47 53 Berean Baptist Academy ITBS 14 57 43 43 57 50 50 Bethel Christian Academy ITBS 21 58 42 67 33 60 40 Bethel Christian Academy ITBS 15 60 40 47 53 73 27 Bible Baptist Christian School ITBS 10 60 40 50 50 80 20 Brookstone Schools ITBS 3 ** ** ** ** ** ** Brookstone Schools TerraNova 2 ** ** ** ** ** ** Community Baptist School TerraNova 20 80 20 80 20 70 30 Concord First Assembly Acad. TerraNova 33 70 30 68 32 52 48 Cornerstone Christian Acad. Stanford 3 ** ** ** ** ** ** Cornerstone Christian Acad. TerraNova 8 86 14 71 29 88 12 Dream Big Christian Acad. ITBS 16 69 31 86 14 69 31 Fayetteville Christian School TerraNova 51 33 67 33 67 42 57 First Wesleyan Christian School TerraNova 8 38 63 25 75 38 63 Freedom Christian Acad. TerraNova 22 32 68 50 50 41 59 Gospel Light Christian School ITBS 28 75 25 79 21 79 21 Greensboro Islamic Academy ITBS 9 33 67 33 67 22 78 Table 5. Publicly-Reported Results, 2015-16
Page 40 Ensuring Opportunity in the OS Program School Test Number of Students Tested Reading Language Mathematics % below 50th % at or above 50th % below 50th % at or above 50th % below 50th % at or above 50th Greensboro Islamic Academy ITBS 9 33 67 33 67 22 78 High Point Christian Acad. ACT 4 ** ** ** ** ** ** High Point Christian Acad. TerraNova 22 36 64 36 64 27 73 Liberty Christian Academy TerraNova 30 43 57 53 47 53 47 Living Water Christian School Stanford 10 50 50 30 70 20 80 Mount Zion Christian Acad. SAT 15 87 13 53 47 47 53 Raleigh Christian Academy ITBS 44 67 33 52 48 64 36 Rockwell Christian School ITBS 20 35 65 40 60 45 55 Salem Baptist Christian School TerraNova 23 35 65 65 35 48 52 St. Raphaels Catholic School ITBS 13 38 62 46 54 46 54 Star Christian Academy California Ach. Test 29 79 21 93 7 69 31 Tabernacle Christian School ITBS 44 80 20 75 25 57 43 Tri-City Christian Academy ITBS 40 70 30 80 20 88 13 Trinity Christian School ITBS 79 27 73 29 71 16 84 Victory Christian Academy Stanford 15 80 20 87 13 60 40 Victory Christian Center School TerraNova 35 63 37 57 43 60 40 Winston-Salem Christian School TerraNova 17 71 29 53 47 65 35 Word of God Christian Acad. TerraNova 54 67 33 * * 78 22 Note: Data provided by NCSEAA *Word of God Christian Academy provided scores for Reading and Mathematics only ** Data not provided; less than 5 students
Page 41 Ensuring Opportunity in the OS Program As is evident from the tables, the performance of students at these schools differed widely, but these data alone are insufficient for understanding the impact of schooling on achievement and growth. To understand that kind of programmatic impact requires at least a baseline of information about how students were performing before entering the school, as well as information about how students from similar backgrounds performed in the public schools from which the Opportunity Scholarship students came. The next section— our second cut—details what such an evaluation might look like; it also critiques the evaluation team’s efforts to begin to approach this more comprehensive level of estimation using volunteer private and public school students in spring 2017. Recommended Steps for Implementing a More Robust Evaluation, and Initial EvaluationTeam Efforts to Analyze Student Performance (2016-17) For our second cut, we conducted an analysis of test scores with the aid of privately-supported administration of a common test to Opportunity Scholarship participants and eligible non-participants in spring 2017. We outline the limitations of our efforts in this section, along with what an optimal approach might look like. Without significant changes to the statutes that currently govern the program, it will not be possible to conduct a reliable and robust causal evaluation of the impact of the program on student academic achievement and academic growth. As noted above, members of the research team were appointed to a legislatively- mandated Task Force that, in March 2018, recommended options for a formal statewide evaluation.37 Those team members shared the steps below with the larger Task Force with the hope that at least some will be adopted by the General Assembly; in all likelihood, full adoption will require persistent engagement from a number of stakeholder groups. Step One: Identify a Common Measure of Achievement. Currently, participating schools are required to administer a nationally normed test to Opportunity Scholarship students each year, but results from these tests are only comparable if students take all of the tests whose results a researcher wishes to compare. For example, if School A administers the Iowa Test of Basic Skills, but School B administers the California Achievement Test, results from those tests can only be compared if at least a sample of the same students have taken both tests, to establish a concordance. Similarly, comparable public school students (details about what constitutes comparability are included below) would need to take the same test. To resolve this problem, the optimal solution from an evaluation perspective is for the state to require participating schools to administer the same test to all Opportunity Scholarship students. Such a policy will have several negative impacts, however, the most significant of which likely will be withdrawal of several schools from the program. Private schools choose tests that best meet their educational needs and cannot easily switch out one test for another. One viable compromise solution could be to require only a (sufficiently large) random sample of all Opportunity Scholarship applicants to take the common test, allowing participating private schools the right to administer the tests of their choice to their full complement of Opportunity Scholarship students. 37 https://www.ncleg.net/documentsites/committees/JLEOC/Reports%20Received/2018%20Reports%20Received/NCSEAA%20 Opportunity%20Scholarship%20Task%20Force%20Report.pdf
Page 42 Ensuring Opportunity in the OS Program Other states with similar programs require participating schools to administer the state test, but that is not a requirement in this state. Of note, the North Carolina test is standards-based, not national norm-based, and therefore only best reflects the public school curriculum. For the 2016-17 testing pilot: The evaluation team administered a short version of the Iowa Test of Basic Skills to Opportunity Scholarship student volunteers and public school student volunteers. Step Two: Limit Program Enrollment. Even if students take a common test, the evaluation still will need to identify a group of students with whom to compare results. For our first attempt in spring 2017, the research team was only able to identify this group via a matching process that paired Opportunity Scholarship students with statistically-similar non-participating students based on a number of common variables, including prior test scores and demographic data. Not only does this approach require a significantly larger group of comparison students than Opportunity Scholarship students in order to obtain the best matches, but it also excludes a key variable: the desire to participate in the Opportunity Scholarship, a characteristic that is only identifiable among Opportunity Scholarship applicants and that may be a significant representation of a host of other unmeasurable characteristics. The optimal solution from an evaluation perspective is to limit the size of the program such that there are more students who apply for the scholarship than there are scholarships available, with the unsuccessful applicants then making up the comparison population. To date, oversubscription has not been the case (every student who has applied for an Opportunity Scholarship in the past has at least been offered the scholarship), and with the General Assembly’s plan to expand funding for the program over each of the next ten years is not likely to be the case going forward. Without a cap on enrollment, however, a truly random and truly representative group of Opportunity Scholarship recipients and Opportunity Scholarship unsuccessful applicants cannot be established. For the 2016-17 testing pilot: The evaluation team had to rely on student volunteers. Step Three: Make Program Application Contingent upon Willingness to Participate in Evaluation. Assuming there is an enrollment cap that restricts the number of recipients enough to create viable awarded and non- awarded pools of students, there is no compulsion for non-winners to participate in the evaluation by taking the additional common test. A possible solution here is to make submission of an application contingent upon willingness to participate in the evaluation. After all Opportunity Scholarships have been awarded, random samples of both awarded and non-awarded students could be identified, with non-awarded students obliged (as a condition of their application) to take the common test. Making the state test the mandatory common test would eliminate this problem, since non-awarded students who remain in public school will take the state test anyway, but, as noted above, mandating use of the state test by participating private schools comes not only at a high cost in terms of private school participation, both in the evaluation and in the program itself, but also in terms of the relative fairness of the testing instrument designed for the public school curriculum as a measure of private school and public school student performance.
Page 43 Ensuring Opportunity in the OS Program For the 2016-17 testing pilot: Not applicable. Step Four: Secure Adequate Funding to Support the Evaluation Effort. In all scenarios, the ability to conduct a rigorous, valid, and reliable evaluation is contingent upon adequate funding: funding for the tests themselves, funding for test scoring, funding for test administration (which, based on our experiences in spring 2017, will require a significant travel budget), and funding for analysis. Contributions from two third-party foundations made limited testing possible in spring 2017, but in order to conduct enough tests while minimizing the burden on public and private schools alike, the state will need to commit adequate funding. Our estimates suggest that a budget of about $190,000 for each year of a three-year study would be required to conduct a rigorous evaluation that minimizes disruptions to participating schools, though that figure could be reduced with increased participation from those schools (for example, in the form of handling test administration duties locally). For the 2016-17 testing pilot: Contributions from third-party funders allowed the evaluation team to conduct, score, and analyze about 700 student tests (about 300 Opportunity Scholarship students and about 400 public school students from similar backgrounds).38 Other Evaluation Options Even with full support from the Task Force and leading public and private schools, it is highly unlikely that a gold standard, rigorous, causal estimate evaluation plan like the one outlined above will be fully adopted by the state. It is still possible to conduct rigorous correlational analyses that can provide good and useful information, even if some of the steps above are not accomplished, though it is important to emphasize that any such studies— even with the best data available—will fall short of a gold standard causal analysis. Some strong correlational analysis options include (but are not limited to): 1. Require Opportunity Scholarship students to take the state test. This is a near-gold standard approach, but, as noted above, the state test aligns with all public school curricula, but very few private school curricula (though a handful of private schools already administer the state test). Thus, motivation, test preparation, and curriculum alignment will be different across the two types of schools, which could bias results in favor of non-awarded students. 2. Administer the most commonly-used private school test to comparable public school students. For example, if TerraNova is the most commonly used test among private schools, a comparable sample of public non-awarded applicants could take the same test. The obvious benefit is that it requires almost no additional effort on the part of potentially resistant private schools. Concerns about this approach include possible inconsistencies in the tests private schools choose to administer each year, as well as the possibility that private school test selection reflects important differences in the private schools themselves 38 The report detailing the results of these analyses is available at: https://ced.ncsu.edu/elphd/research/the-impact-of-the-north- carolina-opportunity-scholarship-program/
Page 44 Ensuring Opportunity in the OS Program (for example, if TerraNova is predominantly administered by private schools in the western region of the state). In addition, private schools would have to share individual student scores with evaluators, which currently is not required by statute. One partial remedy might be to require participating private schools to administer tests from a shorter list of approved tests, thus retaining some private school choice but also improving the likelihood that students who took a particular test are more representative of the entire population of Opportunity Scholarship students.
Page 45 Ensuring Opportunity in the OS Program Next Steps in Our Evaluation Efforts Securing Continuing Support for Evaluation The analyses summarized in this report would not have been possible without support from multiple funders, but it represents only a fraction of the analyses necessary to better understand the impact of the Opportunity Scholarship program on North Carolina students, educators, and schools. The enacting legislation requires an evaluation of the program, with initial results due in 2018, but current state funding for program administration is not sufficient for supporting a meaningful evaluation. Even with state funding, however, the resulting evaluation is likely to be limited—not only by the strictures imposed by current statutes but also by the state’s level of financial commitment to the evaluation. Direct support from third-party groups likely will continue to be an important ingredient in ensuring a comprehensive, fair, and meaningful evaluation of the program as it continues to grow. Expanding Analysis of the Program’s Impact on Student Outcomes In addition to securing ongoing evaluation support—both in terms of financial support and of willingness to participate—the evaluation team will continue to develop plans for a deeper evaluation of the impacts of the program on student academic and non-academic outcomes. Academic outcomes alone are important, but they will not fully capture the impact of the program on all aspects of a student’s schooling. We outline here some of the major evaluation questions that will frame our thinking about future outcomes analyses. Steps for Improving Program-Related Data Quality As noted above, our work to date remains limited in scope as a result of existing statutory barriers, but even without formal statutory changes there is much that can be done to improve the data available to the public and to researchers. In particular, there is great potential for improving information about the tests administered to Opportunity Scholarship students and the data collected about the schools they attend. 1. What is the nature and quality of the nationally-normed tests used by participating schools to meet the legislative requirement of annual testing of Opportunity Scholarship students? 2. What improvements should the state make in its annual collection of operational descriptive data on non- public schools? Determining Impacts on Cognitive Outcomes Much of our thinking about the best approaches for estimating cognitive outcomes is outlined in the preceding section. The two specific questions that we think should guide this component of a complete evaluation are: 3. What is the impact of a voucher on students’ mathematics and reading achievement? 4. Are the effects evenly distributed for students from different subgroups and at different points along ability and income distributions?
Page 46 Ensuring Opportunity in the OS Program Determining Impacts on Non-Cognitive Outcomes An important finding from our focus group sessions in 2014 and 2016 not reported above was the belief among parents and headmasters alike that participation in the Opportunity Scholarship program helped students in more ways than just improvement in their testable achievement. He enjoys reading more than he did before. He was not turned off to it because he liked when we would read to him, but now he is really comfortable taking the reins and trying to go through the skills that they are teaching him in school to read aloud with confidence. Mathematically and in all of the other aspects of education, I would say that he is being given challenges, meeting challenges, and feeling successful in working through frustration. Last year, he never got too frustrated which told me he was not being challenged. This year he is learning what I think is a very important life skill which is to work through your frustrations and know what success on the other end feels like. – Wilson Parent (2014) The greatest success is that [my children] are doing much better academically as far as their grades go and actually enjoying school. – Fayetteville Parent (2014) Transcending even the testing is the work that’s being done teaching these kids how to be good students, . . . teaching them how to learn respect and how to treat people. . . . [G]od bless you guys who were in public schools . . . because it’s a zoo out there and we get kids that are coming ready to fight, who are disrespectful, who have no concept of how to behave in a classroom, unlike kids that come up through our system. . . . – Fayetteville Headmaster (2016) Therefore, in addition to increasing our focus on academic outcomes, we hope to be able to explore in greater depth possible non-cognitive areas of impact, with the following questions as guides: 5. What is the impact of a voucher on students’ non-cognitive outcomes, including grit, conscientiousness, and delay of gratification? 6. How do these impacts vary for specific student subgroups, defined by race, locale, and socio-economic background? 39 Most voucher research differentiates between the impact of a voucher offer and the impact of voucher use. Data from the spring 2017 pilot testing program described in the previous section allowed for an early estimation of the impact of voucher use, albeit with the important limitations noted in that section. Also as noted in the previous section, the opportunity to conduct more rigorous estimations of the impact of voucher use and/or of a voucher offer will require statutory changes to the program.
Page 47 Ensuring Opportunity in the OS Program Appendix. Communities with Opportunity Scholarship- Enrolling Schools, 2014-2017 2014-15 Original Communities Total 2015-16 NewCommunities # NewTotal 2016-17 NewCommunities # NewTotal Raleigh 22 Albemarle 1 1 Ahoskie 2 2 Charlotte 17 Bryson City 1 1 Cary 2 2 Fayetteville 17 Burnsville 1 1 Huntersville 2 2 Greensboro 9 ElmCity 1 1 Morganton 2 2 High Point 8 Grantsboro 1 1 Archdale 1 1 Winston-Salem 8 Harrells 1 1 Belhaven 1 1 Durham 6 Knightdale 1 1 Black Mountain 1 1 Greenville 5 Louisburg 1 1 Brevard 1 1 Wilmington 5 Marshall 1 1 Candler 1 1 Asheville 4 Mills River 1 1 Clayton 1 1 Gastonia 4 Newport 1 1 Clyde 1 1 Rocky Mount 4 Pantego 1 1 Cornelius 1 1 Asheboro 3 Sneads Ferry 1 1 Dover 1 1 Concord 3 Windsor 1 1 Falcon 1 1 Goldsboro 3 Fallston 1 1 Matthews 3 AddedSchools (net) Franklin 1 1 Monroe 3 Charlotte 13 30 Fuquay Varina 1 1 Southern Pines 3 Raleigh 5 27 Glade Valley 1 1 Statesville 3 Fayetteville 3 20 Hillsborough 1 1 Wake Forest 3 Winston-Salem 6 14 Holly Springs 1 1 Wilson 3 Greensboro 2 11 Hookerton 1 1 Bessemer City 2 Asheville 6 10 Kannapolis 1 1 Burlington 2 Durham 3 9 Locust 1 1 Henderson 2 Wilmington 2 7 Lumberton 1 1 Jacksonville 2 Goldsboro 2 5 Marion 1 1 Kernersville 2 Matthews 2 5 Mars Hill 1 1 Lexington 2 Statesville 2 5 McLeansville 1 1 New Bern 2 Concord 1 4 Merry Hill 1 1 North Wilkesboro 2 Elizabeth City 3 4 Mount Airy 1 1 Reidsville 2 Jacksonville 2 4 New London 1 1 Roanoke Rapids 2 Reidsville 2 4 Oak Ridge 1 1 Rockwell 2 Wilson 1 4 Pikeville 1 1 Salisbury 2 Burlington 1 3 Ramseur 1 1 Thomasville 2 New Bern 1 3 Rolesville 1 1 Whiteville 2 Apex 1 2 Shallotte 1 1 Winterville 2 Chapel Hill 1 2 Siler City 1 1 Apex 1 Clinton 1 2 Stokesdale 1 1 Boiling Springs 1 Conover 1 2 Swannanoa 1 1 Chapel Hill 1 Dunn 1 2 Chocowinity 1 Hamlet 1 2 AddedSchools (net) Clinton 1 Hickory 1 2 Charlotte 2 32 Conover 1 Hudson 1 2 Greensboro 4 15 Cramerton 1 Indian Trail 1 2 Durham 4 13 Creedmoor 1 Laurinburg 1 2 Matthews 1 6 Davidson 1 Mooresville 1 2 Gastonia 1 5 Dunn 1 Red Springs 1 2 Asheboro 1 4 Elizabeth City 1 Roseboro 1 2 Burlington 1 4 Ellenboro/Forest City 1 Spruce Pine 1 2 Southern Pines 1 4 Erwin 1 Troy 1 2 Cary 1 3 Fletcher 1 Henderson 1 3 Fuquay 1 Lost Schools Hudson 1 3 Gibsonville 1 Rocky Mount -2 2 Morganton 1 3 Graham 1 Bessemer City -1 1 Wake Forest 1 3 Hamlet 1 Lexington -1 1 Clayton 1 2 Harrisburg 1 Wake Forest -1 2 Harrisburg 1 2 Havelock 1 Hendersonville 1 2 Hayesville 1 Stedman -1 0 Sanford 1 2 Hendersonville 1 Weaverville -1 0 Hickory 1 Lost Schools Hobgood 1 Raleigh -1 26 Hudson 1 Asheville -1 9 Indian Trail 1 Elizabeth City -1 3 Jarvisburg 1 Hickory -1 1 King 1 Laurinburg -1 1 Kings Mountain 1 Red Springs -1 1 Kinston 1 Lasker 1 Black Mountain -1 0 Laurinburg 1 Boiling Springs -1 0 Lemon Springs 1 Candler -1 0 Millers Creek 1 Cornelius -1 0 Mooresville 1 Wanchese -1 0 Morehead City 1 Nebo 1 Pineville 1 Pisgah Forest 1 Pittsboro 1 Raeford 1 Red Springs 1 Richlands 1 Rockingham 1 Roseboro 1 Roxboro 1 Rutherfordton 1 Sanford 1 Smithfield 1 Southport 1 Spruce Pine 1 Stedman 1 Supply 1 Troy 1 Wanchese 1 Weaverville 1
Page 48 Ensuring Opportunity in the OS Program Contact Information: Please direct all inquiries to Trip Stallings dtstalli@ncsu.edu © 2018 The William and Ida Friday Institute for Educational Innovation

More Related Content

PDF
An Impact Analysis of North Carolina’s Opportunity Scholarship Program on Stu...
byMebane Rash
 
PDF
DHHS_NCREACHEdProgramFosterYouth_Review_2012
byDonnie Charleston
 
PPS
Assessing the Impact of Academic Preparation, Finances and Social Ca...
byIria Puyosa
 
PDF
Migrant Student Day Assessment Report 2015-2016
byNoe Valdovinos M.Ed
 
PPT
Washington dc presentation
bySlacko23
 
PPT
Washington dc presentation
bySlacko23
 
PDF
Fast Track FAFSA Completion
byAnalisa Sorrells
 
PDF
Complimentary Research Resource: Fear of Student Loan Debt & Enrollment Impac...
byArdeo Education Solutions
 
An Impact Analysis of North Carolina’s Opportunity Scholarship Program on Stu...
byMebane Rash
 
DHHS_NCREACHEdProgramFosterYouth_Review_2012
byDonnie Charleston
 
Assessing the Impact of Academic Preparation, Finances and Social Ca...
byIria Puyosa
 
Migrant Student Day Assessment Report 2015-2016
byNoe Valdovinos M.Ed
 
Washington dc presentation
bySlacko23
 
Washington dc presentation
bySlacko23
 
Fast Track FAFSA Completion
byAnalisa Sorrells
 
Complimentary Research Resource: Fear of Student Loan Debt & Enrollment Impac...
byArdeo Education Solutions
 

What's hot

PPTX
20-21 Title I Parent and Family Engagement Orientation
byBFC
 
PDF
Lasting benefits of early college high schools
byMebane Rash
 
PDF
Aid Policy: Lessons from Research
byInstituto Nacional de Evaluación Educativa
 
PDF
AttendaNCe Counts: What North Carolina School Districts are Doing to Reduce C...
byMolly Osborne
 
PDF
SES Report 07-08
byStephanie Amedeo Marquez
 
PDF
China delegation 06.12.17 v1
byLouis Soares
 
PPTX
Student Debt: Myths and Facts
byCouncil of Independent Colleges
 
PDF
20110830 Alger Safety Opportunity Scholarships (SOS) How States Can Fulfill t...
byVicki Alger
 
PDF
Foundation Blueprint: Broadening our approach and expanding our impact
byAndy Pino
 
PPT
Research, Policy & Evaluation: Complex Intersections: Navigating the Waters o...
byguestd83a57
 
DOCX
Paper
byNina Englund
 
PDF
EEC_School Funding Memo
bySharonne Navas
 
PPTX
Education Savings Accounts Primary Users using Area Weighted Averages
byBrian Fogarty
 
PPTX
2016 Building a Grad Nation webinar slides
byAmerica's Promise Alliance
 
PPTX
Private Colleges and STEM: Myths and Facts
byCouncil of Independent Colleges
 
PPTX
Commissioner Buhler - Higher Ed Appropriations, Jan. 29, 2014
byUtah System of Higher Education
 
PDF
Benchmarks square
byWestern Interstate Commission for Higher Education
 
PPT
Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborativ...
byMarissa Lowman
 
PPTX
Supporting Student Choice and Opportunity in Higher Education
byTennessee Independent Colleges and Universities Association
 
20-21 Title I Parent and Family Engagement Orientation
byBFC
 
Lasting benefits of early college high schools
byMebane Rash
 
Aid Policy: Lessons from Research
byInstituto Nacional de Evaluación Educativa
 
AttendaNCe Counts: What North Carolina School Districts are Doing to Reduce C...
byMolly Osborne
 
SES Report 07-08
byStephanie Amedeo Marquez
 
China delegation 06.12.17 v1
byLouis Soares
 
Student Debt: Myths and Facts
byCouncil of Independent Colleges
 
20110830 Alger Safety Opportunity Scholarships (SOS) How States Can Fulfill t...
byVicki Alger
 
Foundation Blueprint: Broadening our approach and expanding our impact
byAndy Pino
 
Research, Policy & Evaluation: Complex Intersections: Navigating the Waters o...
byguestd83a57
 
Paper
byNina Englund
 
EEC_School Funding Memo
bySharonne Navas
 
Education Savings Accounts Primary Users using Area Weighted Averages
byBrian Fogarty
 
2016 Building a Grad Nation webinar slides
byAmerica's Promise Alliance
 
Private Colleges and STEM: Myths and Facts
byCouncil of Independent Colleges
 
Commissioner Buhler - Higher Ed Appropriations, Jan. 29, 2014
byUtah System of Higher Education
 
Benchmarks square
byWestern Interstate Commission for Higher Education
 
Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborativ...
byMarissa Lowman
 
Supporting Student Choice and Opportunity in Higher Education
byTennessee Independent Colleges and Universities Association
 

Similar to Ensuring Opportunity Summary

PDF
An Analysis of North Carolina's Private School Landscape
byAnalisa Sorrells
 
PPT
Making A Difference Through College Counseling
bydaveshafron
 
PDF
Access To Higher Education Exploring The Variation In Pell Grant Prevalence ...
byGina Rizzo
 
PDF
Creating Opportunities and Access for Every Student in North Carolina
byMebane Rash
 
PDF
123s of School Choice 2024 Slideshare.pdf
byEdChoice
 
PPTX
Florida scholarship programs
byDanielBreuer1
 
PDF
20090813 Down but Not Out in DC Bi-Partisan, Bi-Cameral Efforts to Continue t...
byVicki Alger
 
PDF
123s of School Choice
byEdChoice
 
PDF
20100630 Fostering Opportunity and Improving Achievement The Benefits of a Fo...
byVicki Alger
 
PDF
The school choice journey: Parents experiencing more than improved test scores
byFLE Liberdade de Educação
 
PPTX
Senior exit
bybpryor
 
PDF
College Financial Aid 101
byDiscover Student Loans
 
PDF
20100811 Murray (Alger) Foster-Care Opportunity Scholarships
byVicki Alger
 
PPTX
Financial aid presentation 2012 13
byjharrell
 
DOCX
Vouchers
byJOrndoff
 
PPTX
Bettinger Keynote: The Difficulty of Knowing and The "E" Word
byNational Partnership for Educational Access
 
PDF
Opportunity-Scholarships-and-the-Schools-that-Receive-Them-1.pdf
byAnnaPogarcic
 
PDF
Opportunity scholarships and the schools that receive them
byEducationNC
 
PDF
20140428 Alger Ending Nebraska's Achievement Gap
byVicki Alger
 
PPT
Federal Student Aid presentation 091812
bySusan Thares
 
An Analysis of North Carolina's Private School Landscape
byAnalisa Sorrells
 
Making A Difference Through College Counseling
bydaveshafron
 
Access To Higher Education Exploring The Variation In Pell Grant Prevalence ...
byGina Rizzo
 
Creating Opportunities and Access for Every Student in North Carolina
byMebane Rash
 
123s of School Choice 2024 Slideshare.pdf
byEdChoice
 
Florida scholarship programs
byDanielBreuer1
 
20090813 Down but Not Out in DC Bi-Partisan, Bi-Cameral Efforts to Continue t...
byVicki Alger
 
123s of School Choice
byEdChoice
 
20100630 Fostering Opportunity and Improving Achievement The Benefits of a Fo...
byVicki Alger
 
The school choice journey: Parents experiencing more than improved test scores
byFLE Liberdade de Educação
 
Senior exit
bybpryor
 
College Financial Aid 101
byDiscover Student Loans
 
20100811 Murray (Alger) Foster-Care Opportunity Scholarships
byVicki Alger
 
Financial aid presentation 2012 13
byjharrell
 
Vouchers
byJOrndoff
 
Bettinger Keynote: The Difficulty of Knowing and The "E" Word
byNational Partnership for Educational Access
 
Opportunity-Scholarships-and-the-Schools-that-Receive-Them-1.pdf
byAnnaPogarcic
 
Opportunity scholarships and the schools that receive them
byEducationNC
 
20140428 Alger Ending Nebraska's Achievement Gap
byVicki Alger
 
Federal Student Aid presentation 091812
bySusan Thares
 

More from Mebane Rash

PDF
How to (Re)Build Your District's Enrollment Numbers
byMebane Rash
 
PDF
Letter to Secretary Linda McMahon from U.S. Senators
byMebane Rash
 
PDF
Statement by Linda McMahon on May 21, 2025
byMebane Rash
 
PDF
A report on the county distress rankings in NC
byMebane Rash
 
PDF
CMS letter to U.S. Representatives about House budget
byMebane Rash
 
PDF
Stein's WNC request of Trump and Congress
byMebane Rash
 
PDF
Stein, Hunt, Green letter to Congress April 2025
byMebane Rash
 
PDF
Case study of Henderson Collegiate by Relay/GSE
byMebane Rash
 
PDF
The 2024 Survey of Community College Outcomes
byMebane Rash
 
PDF
NC Advisory Council on Student Safety and Well-Being
byMebane Rash
 
PDF
Trump Administration's workforce development strategy
byMebane Rash
 
PDF
2024 Summer Nutrition North Carolina Profile
byMebane Rash
 
PDF
N.C. DPI's 2023 Language Diversity Briefing
byMebane Rash
 
PDF
Governor Josh Stein letter to NC delegation of U.S. House
byMebane Rash
 
PDF
Forsyth Tech's new mission, vision, values, and North Star
byMebane Rash
 
PDF
Hurricane Helene Application Documents Checklists
byMebane Rash
 
PDF
U.S. Department of Education certification
byMebane Rash
 
PDF
Federal dollars withheld by district, charter, grant recipient
byMebane Rash
 
PDF
Fiscal Research Division handout on voucher by county and income tier
byMebane Rash
 
PDF
NC DHHS Information about Measles and Vaccination
byMebane Rash
 
How to (Re)Build Your District's Enrollment Numbers
byMebane Rash
 
Letter to Secretary Linda McMahon from U.S. Senators
byMebane Rash
 
Statement by Linda McMahon on May 21, 2025
byMebane Rash
 
A report on the county distress rankings in NC
byMebane Rash
 
CMS letter to U.S. Representatives about House budget
byMebane Rash
 
Stein's WNC request of Trump and Congress
byMebane Rash
 
Stein, Hunt, Green letter to Congress April 2025
byMebane Rash
 
Case study of Henderson Collegiate by Relay/GSE
byMebane Rash
 
The 2024 Survey of Community College Outcomes
byMebane Rash
 
NC Advisory Council on Student Safety and Well-Being
byMebane Rash
 
Trump Administration's workforce development strategy
byMebane Rash
 
2024 Summer Nutrition North Carolina Profile
byMebane Rash
 
N.C. DPI's 2023 Language Diversity Briefing
byMebane Rash
 
Governor Josh Stein letter to NC delegation of U.S. House
byMebane Rash
 
Forsyth Tech's new mission, vision, values, and North Star
byMebane Rash
 
Hurricane Helene Application Documents Checklists
byMebane Rash
 
U.S. Department of Education certification
byMebane Rash
 
Federal dollars withheld by district, charter, grant recipient
byMebane Rash
 
Fiscal Research Division handout on voucher by county and income tier
byMebane Rash
 
NC DHHS Information about Measles and Vaccination
byMebane Rash
 

Recently uploaded

PPTX
unit 1 Introduction and Theoretical Foundations of educational technology.pptx
byKavitha Krishnan
 
PPTX
ENGLISH 8 PRESENTATION FOR WEEK 1 LESSONS
byPrincessJaneSantaAna1
 
PDF
Optomery MCQ pdf/exam/pyq/joinquiz/..pdf
byAnmol Singh
 
PPTX
The Role of Metadata for Archival Discovery (by Getaneh Alemu & James Clark,...
bygetanehmetadata
 
PPTX
Health Care Agencies & Hospital .pptx.Slideshare
byPompi Begum
 
PDF
Introductions Introductions Introductions
byaidanattema
 
PPTX
FAMILY AND CONSUMER SERVICES week2-day1-4.pptx
byEdgarDenaga1
 
PDF
Fundamental of Horticulture _HORTICULTURAL PLANT CLASSIFICATION.pdf
bybisensharad
 
PPTX
TEACHING OF POETRY.pptx. The PPT helps and provides guidance on Lesson Planni...
byDr. Meeta Agrawal
 
PDF
Graphics Graphics Graphics Graphics Graphics
byaidanattema
 
PPTX
How to Create Certifications in Odoo 18 Employees
byCeline George
 
PPTX
Structure and Functions of Cell (Unit I)
byPranaliChandurkar2
 
PDF
Quiz.exe (Coding Quiz) 2025- 09th October 2025, Quiz Club NITW
byQuiz Club NITW
 
PDF
CMOS VLSI Fabrication Technology: From Lithography to Advanced CMOS Processes
byGS Virdi
 
PDF
Keratometer /ppt/pdf/notespdf/kerato.pdf
byAnmol Singh
 
PPTX
How to Manage Performance in Odoo 18 e-Commerce
byCeline George
 
PPTX
Unit 8- Immunochemistry.pptx............
byAkanksha Kotangale
 
PPTX
The Ides Of March Throughout History-- not just Caesar but also . . .
byBob Mayer
 
PPTX
Overview of AI integration in Chatter in Odoo 19
byCeline George
 
PPT
COCCYDYNIA.ppt for physiotherapist students
byDrNidhiAgarwal
 
unit 1 Introduction and Theoretical Foundations of educational technology.pptx
byKavitha Krishnan
 
ENGLISH 8 PRESENTATION FOR WEEK 1 LESSONS
byPrincessJaneSantaAna1
 
Optomery MCQ pdf/exam/pyq/joinquiz/..pdf
byAnmol Singh
 
The Role of Metadata for Archival Discovery (by Getaneh Alemu & James Clark,...
bygetanehmetadata
 
Health Care Agencies & Hospital .pptx.Slideshare
byPompi Begum
 
Introductions Introductions Introductions
byaidanattema
 
FAMILY AND CONSUMER SERVICES week2-day1-4.pptx
byEdgarDenaga1
 
Fundamental of Horticulture _HORTICULTURAL PLANT CLASSIFICATION.pdf
bybisensharad
 
TEACHING OF POETRY.pptx. The PPT helps and provides guidance on Lesson Planni...
byDr. Meeta Agrawal
 
Graphics Graphics Graphics Graphics Graphics
byaidanattema
 
How to Create Certifications in Odoo 18 Employees
byCeline George
 
Structure and Functions of Cell (Unit I)
byPranaliChandurkar2
 
Quiz.exe (Coding Quiz) 2025- 09th October 2025, Quiz Club NITW
byQuiz Club NITW
 
CMOS VLSI Fabrication Technology: From Lithography to Advanced CMOS Processes
byGS Virdi
 
Keratometer /ppt/pdf/notespdf/kerato.pdf
byAnmol Singh
 
How to Manage Performance in Odoo 18 e-Commerce
byCeline George
 
Unit 8- Immunochemistry.pptx............
byAkanksha Kotangale
 
The Ides Of March Throughout History-- not just Caesar but also . . .
byBob Mayer
 
Overview of AI integration in Chatter in Odoo 19
byCeline George
 
COCCYDYNIA.ppt for physiotherapist students
byDrNidhiAgarwal
 

Ensuring Opportunity Summary

  • 1.
    Ensuring Opportunity inNorth Carolina’s Opportunity Scholarship Program A Summary of Work to Date June 2018 D.T. Stallings The William and Ida Friday Institute for Educational Innovation Anna Egalite College of Education Steven Porter College of Education North Carolina State University
  • 2.
    Page 1 EnsuringOpportunity in the OS Program Acknowledgements The early demographic analyses and evaluation results presented in this document are the product of several years of work—all completed (to date) without state support. Contributions from private funders were critical in getting the evaluation work off the ground in its earlier years. Since those first investments, the project has benefitted from additional informational and participatory support from the North Carolina State Education Assistance Authority, the North Carolina Division of Non-Public Education, stakeholder organizations, graduate student researchers, and, most importantly, the families and practitioners who have made our data collection possible: superintendents, principals, parents, and students in several North Carolina public school districts; and headmasters, parents, and students in private schools from across the state. Our work is not finished. We continue to pressure the state to support a balanced, comprehensive evaluation of the complex impacts of the Opportunity Scholarship program—the only such program in the country that previously has not been nor currently is implemented in a way that supports rigorous quantitative evaluation of the program’s impacts on student achievement due to statutorily-prescribed structural limitations.1 We continue to push for the programmatic structures that will allow for an evaluation because we believe in the importance of sharing learnings from all state education policy implementations with all education stakeholders, and because we believe in the value of thoughtful stewardship of state resources. 1 The Florida Tax Credit Scholarship also no longer is able to be rigorously quantitatively evaluated but did undergo such an evaluation before recent program design changes.
  • 3.
    Page 2 EnsuringOpportunity in the OS Program Table of Contents Executive Summary 4 Introduction 8 The Opportunity Scholarship Program 8 The Opportunity Scholarship in the Context of Other State Voucher Programs 9 Data Collection and Evaluation Progress to Date 10 Questions Guiding the Foundational and Background Portion of the Evaluation 11 Gauging the Extent to which Access to the Opportunity Scholarship is Equitable across Potential Participants 11 Assuring Access to a Quality Education for Participating Students from Low- Income Families 11 Assuring Fair and Equitable Use of State Funding 12 A Note on Gaining a Comprehensive Understanding of the Opportunity Scholarship 12 Data and Methods 13 Demographics and Descriptive Data 13 Student Demographic Data 13 School Descriptive Data 13 A Note on the Quality and Availability of Non-Public School Data 13 Surveys 13 Instrument 13 Response Rates 14 Focus Groups 14 Testing 15 Gauging Equity of Access: Students and their Families 16 Characteristics of Participating Students 16 Eligibility 16 Grade Level 17 Ethnicity 17 Differences between Participating Students and Other Eligible Students 18 Access to Information about the Opportunity Scholarship 18 The Application Process and Equity of Access 21 Perceptions of Opportunity Scholarship Students at Participating Schools 24 Assuring Access to a Quality Education for Participating Students: Participating Schools 25
  • 4.
    Page 3 EnsuringOpportunity in the OS Program Characteristics of Participating Private Schools 25 Geographic Distribution 25 Access to the Opportunity Scholarship Statewide 25 Comparisons to Public and Non-Participating Private Schools 28 Expenditure Patterns of Participating Private and Nearby Public Schools 33 Tuition Support 34 Participation and Enrollment Restrictions Related to Finance 34 The Opportunity Scholarship and Changes in the Supply of Private Schools 35 Understanding the Current Academic Outcomes Evaluation Landscape 37 Testing in Participating Schools 37 Performance of Participating Students 38 Participating Schools with more then Twenty-Five Opportunity Scholarship Students 38 Recommended Steps for Implementing a More Robust Evaluation, and Initial Evaluation Team Efforts to Analyze Student Performance (2016-17) 41 Other Evaluation Options 43 Next Steps in Our Evaluation Efforts 45 Securing Continuing Support for Evaluation 45 Expanding Analysis of the Program’s Impact on Student Outcomes 45 Steps for Improving Program-Related Data Quality 45 Determining Impacts on Cognitive Outcomes 45 Determining Impacts on Non-Cognitive Outcomes 46 Appendix. Communities with Opportunity Scholarship-Enrolling Schools, 2014-2017 47 Cover image courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action.
  • 5.
    Page 4 EnsuringOpportunity in the OS Program Executive Summary Overview The North Carolina Opportunity Scholarship Program, in operation since the beginning of the 2014-15 school year, provides state-funded financial support for eligible children in Kindergarten through twelfth grade that can be used to cover or reduce tuition and fees for non-public school education. This report provides an overview of efforts to date to evaluate multiple aspects of the program. Gauging Equity of Access: Students and their Families How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? Like many other programs nationwide, the Opportunity Scholarship is means-tested. The program treats all applicants relatively equally, with only a few exceptions (for example, the lottery does preference siblings and members of military families). The Opportunity Scholarship requires family co-payment of tuition and fees if those costs exceed the value of the voucher; both the Indiana Choice Scholarship Program and the Florida McKay Scholarship Program have similar guidelines, but the voucher program in Louisiana requires participating schools to cover or waive any differences. What are the characteristics of student applicants? The adjusted median per-household-member income for a participating family (ca. $16,000 per member) is low, and the overall household income for most participating families falls well below the threshold for eligibility (e.g., ca. $30,000 for a family of two; ca. $61,000 for a household of six). Compared to public school students, voucher recipients are more likely to be in elementary school and they are less likely to be in high school. A larger proportion of recipients are African-American, relative to the population of public school students, and a smaller proportion of recipients are Hispanic or White. Are there systematic differences between students who apply and eligible non-applicants? Reflecting the diversity of the private schools they attended, the population of Opportunity Scholarship recipients is quite diverse along a number of demographic axes, as supported by the random nature of the lottery used to identify recipients. However, there may be some differences between applicants and eligible non-applicants as a result of three non-systematic variables: a) differences in access to information about the program (via social information networks and general understanding of how private schooling works); b) differences in ability to navigate the application process; and c) differences in family motivation to apply. 2 Adjusted per-household-member income = household income / square root of household size; this approach is commonly used by economists to account for household savings from economies of scale and resource-sharing.
  • 6.
    Page 5 EnsuringOpportunity in the OS Program How do families receive information about the Opportunity Scholarship and participating nonpublic schools? Are there differences in information access by sub-groups (by ethnicity, by geographic location, etc.?) Social information networks appeared to be the most influential means by which families learned about the program during its first two years, followed by direct marketing on the part of participating private schools. The state agency that oversees the program advertises across the state using several media platforms. Any differences in access to information appear to be geographically based, but not in a systematic way; instead, the differences have more to do with variations in the strengths of local and regional information networks, as well as the degree to which local private schools share information about the program. Because of their faith- based community connections, some private schools are able to share information with language minority groups through non-English language sister church circles. For the first two years of the program, data suggest that most families apply to only one private school, but that outcome likely is tied in part to the fact that many rural communities have access to only one participating school, and in part to a phenomenon among programs nationally of families becoming accustomed to exercising greater choice only once a program is better established and families gain more experience with school choice options. Does the timing of the application (early February) impact who applies? Survey responses3 indicate that most families (87 percent) believed that the application process was simple to complete, but the timing of the application process (with the first families learning of their awards in mid- Spring) is not well aligned with many private schools’ enrollment calendars (which can begin as early as January). The challenge increases for wait-list families, who often are notified of awards sometimes well after their schools of choice have completed enrollment, and sometimes even into the start of the school year. Assuring Access to a Quality Education for Participating Students: Participating Schools Are there differences in access across counties? For example, do rural students have proportionally more or less opportunity to attend a participating nonpublic school than do urban and suburban students, in terms of both availability and cost? Even though more than 20 rural counties do not have participating schools, students in 97 of North Carolina’s 100 counties applied for and received scholarships for 2016-17, and some schools even provide limited transportation support. Program participation on the part of private schools has grown proportionally in urban and rural areas alike. Sometimes vast differences in cost across private schools may be a more important factor in access, since the standard voucher amount (no more than $4,200) typically covers less than the average private school tuition in North Carolina (between $7,600 and $9,400). 3 2016 parent survey response rate: 2,425 out of 8,739 households (28%) responded; households included new and renewal applicants; all applicant households were surveyed, regardless of whether the household either was awarded or accepted a voucher.
  • 7.
    Page 6 EnsuringOpportunity in the OS Program What are the characteristics of nonpublic schools that choose to participate in the voucher program, and how do they compare to public and non-participating nonpublic schools? Are there discernible patterns in the types of private schools that participate (e.g., by location, school type, school size, etc.), and, if so, what are they? A higher proportion of participating schools (76 percent) indicated in survey responses4 that their school has a religious orientation than did non-participating schools (42 percent). Participating and non-participating schools indicated that their primary admissions screening components were interviews and evidence of prior academic achievement. By their own assessment, participating schools believe they place a stronger emphasis on college preparation and character building, relative to public schools, while non-participating schools tended to emphasize less reliance on standardized testing than their public school neighbors. The most frequently cited factors for a school’s decision not to participate in the program were concerns about future changes to state laws or policies regarding participation requirements (57 percent of survey respondents) and about the amount of required paperwork (47 percent). Though anecdotal only, some participating schools posited that non-participating schools also were wary of changes to the make-up of their student populations if they enrolled Opportunity Scholarship students. To what extent do participating nonpublic and nearby public school expenditure patterns differ? Because of limitations in the reporting required of non-public schools, the level of fiscal transparency common among public schools does not exist to a similar degree among Opportunity Scholarship schools. As a result, this question may never be answerable unless the state requires reporting of financial data by participating schools. What impact has the availability of the program appeared to have on the supply of private schools? Overall, available data do not suggest that the advent of the Opportunity Scholarship on its own has led directly to a meaningful uptick in the number of private schools in the state; the number was growing before the program and continued to grow at about the same pace afterwards, with only a slightly larger than usual spike in the first year after passage of the enacting legislation. Much of the growth in terms of the gross number of school openings has been offset by a smaller but still substantial number of school closings. Understanding the Current Academic Outcomes Evaluation Landscape Testing in Participating Schools Per legislation, nonpublic schools that receive Opportunity Scholarship funds are required to administer a nationally standardized test or equivalent at least once a year to all Opportunity Scholarship recipients. Results of these tests are reported to NCSEAA, but there are no target performance levels, nor are there any actions NCSEAA is required to take related to the scores. 4 2016 survey response rate for non-public schools with active email contact information: 40% (266 out of 673 schools). Schools were surveyed regardless of whether they participated in the Opportunity Scholarship program or not.
  • 8.
    Page 7 EnsuringOpportunity in the OS Program Performance of Participating Students Participating nonpublic schools with more than 25 Opportunity Scholarship recipients also must report aggregated standardized test performance results for eligible students, and those results are published annually. The performance of students at these schools differed widely, but without prior achievement data or reliable comparison groups, these data alone are insufficient for understanding the impact of schooling on achievement and growth. This report also describes the evaluation team’s efforts to collect and analyze comparable test data for the 2016-17 school year. Recommended Steps for Implementing a More Robust Evaluation 1. Identify a common measure of achievement; 2. Limit program enrollment to create natural experimental treatment and control groups; 3. Make program application contingent upon willingness to participate in evaluation; and 4. Secure adequate funding to support the evaluation effort. Next Steps in Our Evaluation Efforts Some of the questions that will guide the next components of our work include: Steps for Improving Program-Related Data Quality 1. What is the nature and quality of the nationally-normed tests used by participating schools to meet the legislative requirement of annual testing of Opportunity Scholarship students? 2. What improvements should the state make in its annual collection of operational descriptive data on non- public schools? Impacts on Cognitive Outcomes 3. What is the impact of a voucher5 on students’ mathematics and reading achievement? 4. Are the effects evenly distributed for students from different subgroups and at different points along ability and income distributions? Impacts on Non-Cognitive Outcomes 5. What is the impact of a voucher on students’ non-cognitive outcomes, including grit, conscientiousness, and delay of gratification? 6. How do these impacts vary for specific student subgroups, defined by race, locale, and socio-economic background? 5 Most voucher research differentiates between the impact of a voucher offer and the impact of voucher use. Data from tests administered by the research team in spring 2017 to voucher recipients and voucher-eligible public school students allowed for an early estimation of the impact of voucher use (https://ced.ncsu.edu/elphd/research/the-impact-of-the-north-carolina-opportunity- scholarship-program/); important limitations of the extent to which the team could draw firm conclusions from the data collected are outlined in the Understanding the Current Academic Outcomes Evaluation Landscape section of this report. The opportunity to conduct more rigorous estimations of the impact of voucher use and/or of a voucher offer will require statutory changes to the program (as summarized in the Recommended Steps list in this summary and as described in greater detail in the main report).
  • 9.
    Page 8 EnsuringOpportunity in the OS Program Introduction The Opportunity Scholarship Program The North Carolina Opportunity Scholarship Program provides state-funded financial support for eligible children in Kindergarten through twelfth grade that can be used to cover or reduce tuition and fees for non- public school education. Established by North Carolina General Statute 115C-562 in 2013 and administered by the North Carolina State Education Assistance Authority (NCSEAA), the Opportunity Scholarship Program provides funding of up to $4,200 per year for eligible students to attend participating private schools. Awards are distributed one semester at a time and can be used for tuition and required fees at registered private schools. Opportunity Scholarships were first awarded in school year 2014-15, providing scholarships for 1,216 students to attend 224 private schools. The program has experienced significant growth every year since then, with 3,460 recipients attending 313 private schools in 2015-16 and 5,432 recipients in 349 private schools in 2016-17. Eligibility for the Opportunity Scholarship program is determined by two criterion categories. First, students must be North Carolina residents living in a household that does not exceed a statutorily-defined income cap, must not have graduated from high school or attended college, and must have turned five years old on or before August 31. The household income eligibility threshold is set at 133 percent of the eligibility threshold for the federal free and reduced-price lunch program for a partial scholarship, and it is set at 100 percent of the federal free and reduced-price lunch program eligibility threshold for a full scholarship. For a family of four, for example, maximum household gross income for 2016 could not exceed $45,510 if a student applied for a full scholarship or $60,528 for a partial scholarship. Second, students must be enrolled in a public school in North Carolina in the application year, be entering Kindergarten or first grade, or have received an Opportunity Scholarship in a previous academic year. Students who do not meet any of these second criteria still may be eligible if they are in foster care or were adopted within the last year, or if they have a parent on fulltime active duty with the military. 6 Although the scholarship award has a maximum value of $4,200, it cannot exceed the cost of tuition and fees, including books, equipment, transportation or other items required by the private school 7 These data were published in NCSEAA’s annual reports to the North Carolina legislature on the Opportunity Scholarship Program for the 2014-15 and 2015-16 academic years. Retrieved from http://www.ncleg.net/documentsites/committees/JLEOC/Reports%20 Received/2015%20Reports%20Received/Opportunity%20Scholarship%20Program.pdf and http://ncga.state.nc.us/documentsites/ committees/JLEOC/Reports%20Received/2016%20Reports%20Received/Opportunity%20Scholarship%20Program%20Report%20 2016.pdf 8 Because Opportunity Scholarships are awarded throughout the academic year until all resources are depleted, numbers for the 2017-18 school year will not be final until summer 2018. 9 See http://www.ncseaa.edu/pdf/OPS_Overview.pdf 10 One interesting aspect of this requirement is that it may incentivize parents to change a student’s school enrollment from private to public for one year in order to become eligible. While we did not find evidence of widespread adoption of this strategy, at least one parent did bring it up during a focus group: “I do know parents that have taken the chance—even though they didn’t want to, but they thought [of] the benefit in the long run—they put their kids back in public school because they knew that they would qualify.” – Fayetteville Parent (2016)
  • 10.
    Page 9 EnsuringOpportunity in the OS Program Part of the reason for the growth of the program has been growth in the state’s investment in the program, along with projections for continued growth in the investment over the next decade. Total scholarship funding is set by the state budget. In 2014-15, $4,635,320 was disbursed in scholarship funds; in 2015-16, that total rose to $13,149,842, and in 2016-17 climbed even higher, to $21,760,837. Effective July 1 2017, the 2016 Appropriations Act approved an increase of funds by at least ten million dollars every year for ten years.11 Relative to the state’s overall annual funding for K-12 education, however (over $9 billion annually), and even accounting for continued growth in the annual program allotment, the state investment is small (about 0.2% of the amount allotted to public K-12 education). After the renewal scholarships have been awarded, at least fifty percent of the remaining funds must be used to award grants to students who qualify for the free and reduced-price lunch program. Any remaining funds are distributed among the remaining eligible applicants, with no more than forty percent of the remaining funds directed to students entering Kindergarten or First Grade. To participate in the Opportunity Scholarship program, private schools must satisfy the requirements established by the state’s Division of Non-Public Education for all private schools, they must register with NCSEAA, and they must conduct a criminal background check for the employee with the highest decision- making authority. Eligible private schools that receive more than $300,000 in Opportunity Scholarship funds must submit an annual financial review report that has been prepared by a certified public accountant. Finally, all participating private schools are required to administer a nationally standardized test of their choosing to all scholarship students annually and to report these results to NCSEAA. The ramifications of this last guideline on the quality of any current or future evaluation is discussed at greater length in the Understanding the Current Academic Outcomes Evaluation Landscape section below. The Opportunity Scholarship program has faced two legal challenges, both of which alleged that the program violates the North Carolina Constitution: Hart v. North Carolina and Richardson v. North Carolina. Although the trial court ruled in favor of the plaintiffs, the North Carolina Supreme Court declared the program constitutional in a 4-3 decision in July 2015. The Opportunity Scholarship in the Context of Other State Voucher Programs An important contextual consideration to keep in mind before reviewing the results of the evaluation efforts described in this report is the degree to which these results are comparable to results from other states. While the Opportunity Scholarship program was based in part on voucher programs in other states and municipalities, its combination of features is unique (Table 1, following page), which limits our ability to make comparisons to results from evaluations of other voucher programs. 11 See NCSEAA Opportunity Scholarship Program Summary of Data as of 2/1/17. Retrieved from http://www.ncseaa.edu/documents/ OPS_Summary_Data.pdf
  • 11.
    Page 10 EnsuringOpportunity in the OS Program Table 1. Comparison of Selected North Carolina Opportunity Scholarship Components with Components of Other Voucher Programs Key Characteristics NC FL IN LA Program Design Voucher Tax Credit Voucher Voucher Max Voucher Value (2016-17) $4,200 $5,886 $6,473 $10,090 Avg Voucher Value (2015-16) $4,116 $5,476 $4,520 (1-8) $5,496 (9-12) $5,856 Participating Schools Can Charge More than Voucher Value? Yes Yes Yes No School-Specific Admission Standards Yes Yes Yes No Private Students Eligible? No Yes No No Accountability Norm-referenced test of choice Norm-referenced test of choice State Test State Test Non-Public School Participation Rate (2016-17) ~60% ~69% ~45% ~34% The structure of North Carolina’s program relative to the structures of programs in states like Florida, Indiana, and Louisiana—especially in terms of program design, voucher amount, and admissions standards—is at least partially and (in the case of Louisiana) notably different. As a result, the measured impacts of these programs as reported in other studies may provide good context for the impact of voucher programs overall but do not necessarily inform what we should expect to find in our examination of the North Carolina program. Data Collection and Evaluation Progress to Date In September of 2013, a research team from North Carolina State University began what has now become a five-year effort to conduct a formal, unbiased, and rigorous evaluation of the North Carolina Opportunity Scholarship program. With support from a private foundation, in 2014 the team conducted a limited number of focus groups with a handful of parents of Scholarship recipients and with headmasters at two participating schools; the team also compiled a comparative overview of all voucher programs nationwide. After securing support from a second private foundation, the team expanded its efforts in 2015 and 2016, traveling across North Carolina to learn more about key stakeholder perceptions of various aspects of the program. The team distributed online surveys to every private school in the state (both those that participate in the program and those that do not) and to all applicant families. In addition, the team conducted focus
  • 12.
    Page 11 EnsuringOpportunity in the OS Program groups and interviews with private school leaders and applicant parents in five representative locations to better understand how the program works and how it has impacted participating schools, students, and their families. Next, the team worked with public school administrators, private school headmasters, a school choice advocacy group, and state officials to administer a common standardized test to 698 students in grades four through eight—401 public and 297 private school students in 14 public schools and 24 private schools— from four representative regions across the state. More details about the survey, focus group, and testing components are included in the Data and Methods section, below. Finally, two members of the team were appointed to a legislatively-mandated Task Force that, in March 2018, recommended options for a formal statewide evaluation. This summary report includes data and analyses from the four standalone reports generated to date12 —all of which are part of the team’s efforts to establish a formal statewide evaluation of the program. It addresses each of the questions originally proposed in an initial research plan developed in 2014 to the extent allowable with current data. In many cases, the data necessary to completely address a question are not available; these deficiencies are noted in the text, along with anticipated timelines for when such questions can be answered, or, in the case of questions answerable only with data not collectable under current statute, recommendations for statutory or policy changes that would make such data collection possible. Questions Guiding the Foundational and Background Portion of the Evaluation The first phase of the evaluation was designed to address several short-term and longer-term quantitative and qualitative outcomes that provide foundational and background context for future research and evaluation. The components of this phase of the evaluation are listed here: Gauging the Extent to which Access to the Opportunity Scholarship is Equitable across Potential Participants 1. How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? 2. Are there differences in access across counties? For example, do rural students have proportionally more or less opportunity to attend a participating nonpublic school than do urban and suburban students, in terms of both availability and cost? 3. How do families receive information about the Opportunity Scholarship and participating nonpublic schools? Are there differences in information access by sub-groups (by ethnicity, by geographic location, etc.?) Does the timing of the application (early February) impact who applies? Assuring Access to a Quality Education for Participating Students from Low-Income Families 4. What are the characteristics (including student performance results) of nonpublic schools that choose to 12 All three completed reports are available at: https://ced.ncsu.edu/elphd/research/the-impact-of-the-north-carolina-opportunity- scholarship-program/; all future reports will be posted to the same website.
  • 13.
    Page 12 EnsuringOpportunity in the OS Program participate in the voucher program, and how do they compare to public and non-participating nonpublic schools? 5. Are there discernible patterns in the types of non-public schools that participate (e.g., by location, school type, school size, etc.), and, if so, what are they? What impact has the availability of the scholarship had on the opening of new nonpublic schools? Assuring Fair and Equitable Use of State Funding 6. What are the characteristics of student applicants? Are there systematic differences between students who apply and eligible non-applicants? 7. To what extent do participating nonpublic and nearby public school expenditure patterns differ? As noted above, in some cases, the evaluation team has determined that some of these questions are not yet answerable (and in one case may not ever be answerable), given available data and statutory limitations. These limitations are noted throughout the sections below. We also have included information at the end of this report about our progress toward conducting a rigorous and fair quantitative assessment of the academic impact of the Opportunity Scholarship. A Note on Gaining a Comprehensive Understanding of the Opportunity Scholarship Because this report is a summary of several standalone reports, it does not include all of the important insights and learnings presented in those reports,13 nor does it reflect the information we hope to learn through additional study in the coming years (as outlined in the final section of the report). The authors encourage readers of this summary report to read the other available analyses as well, in order to gain a better overall sense of the ways in which the Opportunity Scholarship is impacting the education of lower-income students in North Carolina. 13 https://ced.ncsu.edu/elphd/research/the-impact-of-the-north-carolina-opportunity-scholarship-program/
  • 14.
    Page 13 EnsuringOpportunity in the OS Program Data and Methods Demographic and Descriptive Data Student Demographic Data Much of the student demographic data included in this report were drawn from a dataset of the 2016-17 applicants for the Opportunity Scholarship. This dataset, provided by NCSEAA, consists of 11,459 applications and includes information on students’ demographic information, household income, and final voucher award status. School Descriptive Data Data about non-public schools were derived from a number of sources, including data collected by NCSEAA as part of its process for verifying applicant non-public school eligibility to participate in the Opportunity Scholarship program, as well as data collected annually by the North Carolina Department of Administration’s Division of Non-Public Education as part of its public reporting responsibilities.14 A Note on the Quality and Availability of Non-Public School Data The research team identified multiple inconsistencies and cases of missing values in data provided by the Division of Non-Public Education; the team has been working with the Division to resolve these data issues both for past years and for future data collection efforts. Surveys The team administered surveys online to parents and guardians of Opportunity Scholarship applicants, as well as to people in leadership positions in non-public schools (whether those schools participated in the Opportunity Scholarship program or not). Instrument The process for developing specific questions for each survey consisted of reviewing relevant research to identify existing applicable questions from other studies, as well as writing original questions that were specific to the North Carolina context. The final survey instruments consisted of approximately twenty questions and took approximately fifteen minutes to complete. Questions were a mixture of multiple-choice with responses on a Likert scale (e.g., Very Dissatisfied, Dissatisfied, Neither Satisfied nor Dissatisfied, Satisfied, Very Satisfied) and open-response questions. The surveys were piloted at North Carolina State University prior to distribution. 14 Some of these data are posted on the Division’s website (https://ncadmin.nc.gov/citizens/private-school-information), but other data were provided after a public records request.
  • 15.
    Page 14 EnsuringOpportunity in the OS Program Response Rates Parents. In coordination with NCSEAA, the research team emailed an invitation to the parents/ guardians of all applicants to the Opportunity Scholarship Program in 2015-16 and 2016-17 to participate in the online survey. Accounting for inactive email addresses, the target sample was 12,894 students, represented by 8,739 parent email addresses, and we received 2,425 parent responses, representing a response rate of 28%—somewhat low, but not unexpected, given the challenges NCSEAA has had in establishing follow-up connections with many applicants. Parent/guardian responses were split between first-time applicants (59 percent) and renewal applicants (41 percent). The majority of parents we surveyed indicated that they were applying for an Opportunity Scholarship for just one child (63 percent). Twenty-six percent indicated that they were applying for Opportunity Scholarships for two children, with the remainder applying for three or more children. Schools. In coordination with the Division of Non-Public Education, the team gathered contact information for all private schools in the state (n=724) and distributed the survey via email. Respondents occupied various school leadership roles, including principal, administrator, school founder, school president, school director, business manager, and Director of Admissions. Accounting for inactive email addresses, the target sample was 673 schools and we received 266 responses for a school response rate of 40 percent—again, somewhat low, but also not unexpected, given the lack of incentives for a school to respond. Respondent schools were categorized as participating in the program in one of four ways: Schools that currently participate in the program (n=177), schools that have never participated in the program and do not plan to do so (n=58), schools that are planning to participate in the program in the future (n=25), and schools that participated in the program in a previous year, but withdrew (n=6). When analyzing the data, the research team broke out responses across categories where appropriate (remaining mindful of the differences in sample size across sub-groups), to identify differences in responses by school type. Focus Groups Focus groups and interviews with parents of Opportunity Scholarship students and with private school leaders were conducted in two waves, starting in the summer of 2014 and concluding in the summer of 2016. To recruit participants for the first wave of data collection, the research team identified a stratified random sample of schools in order to solicit feedback from stakeholders in different regions of the state, serving student populations of varying sizes, and representing both religious and non-religious schools. Parents and leaders from just two schools agreed to participate for this first wave of data collection, so, to boost participation in the second wave of data collection, the research team worked with a third-party school choice non-profit organization that was well-known to many of the participating schools. This organization helped the research team to identify and recruit schools and parents to participate in the second round of focus groups, which took place in the summer of 2016. Focus groups were conducted in five locations: Wilson (East), High Point (Piedmont), Raleigh (Central), Charlotte (West), and Fayetteville (South). As a result of the recruitment 15 NCSEAA reports that one of the key reasons why nearly every qualified family that applies for a Scholarship is offered one is because the agency is not able to re-establish contact with many of the applicant families after the initial application.
  • 16.
    Page 15 EnsuringOpportunity in the OS Program approach taken by the partnering organization, the sample is less representative than it would have been if randomly chosen, but participation numbers were much higher than they had been before the organization’s involvement. Overall, the final sample for focus groups and interviews consisted of 49 school leaders and 13 parents, representing 34 schools across 15 counties. Testing A key challenge to conducting a meaningful evaluation of the overall impact of the Opportunity Scholarship program is the statutory language that allows participating non-public schools to fulfill the annual testing requirement via any nationally-normed test. Because results from different tests are not easily comparable (for example, establishing what is called a concordance across two different tests requires at least a sample of students to take both tests—and in the case of the North Carolina program, literally dozens of different kinds of tests are administered by participating schools in order to fulfill the statutory testing requirement), the testing data collected by the state cannot be used in rigorous estimations of the overall academic impact of the program. Limiting analyses to only those schools that offer a certain test (say, for instance, all participating non-public schools that administer the Iowa Test of Basic Skills in order to fulfill the requirement) introduces another set of biases, as those schools are not necessarily representative of all of the participating schools in the state. These and other limitations are described in greater detail in the Understanding the Current Academic Outcomes Evaluation Landscape section below. To begin to address this challenge, the research team secured private funding and the cooperation of a sample of public and non-public schools to conduct testing of Opportunity Scholarship-awarded and -eligible students in spring 2017 on a common measure. While providing better information and analytical opportunities than would have been possible with extant, state-mandated testing alone, there remain several limitations to what can be learned from this approach to measuring the impact of the voucher on student academic achievement. The section on academic outcomes included at the end of this report highlights these challenges and offers possible solutions.
  • 17.
    Page 16 EnsuringOpportunity in the OS Program Gauging Equity of Access: Students and their Families Characteristics of Participating Students How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? What are the characteristics of student applicants? For the 2016-17 school year (the third year of the program),16 69 percent of all applicants were first-time applicants, with the remaining 31 percent applying to renew a scholarship received in a previous year. Eligibility Families that receive vouchers are among the lowest-income households in the state, and many are well below the eligibility threshold for each household size (e.g., below $30,044 for a family of two; below $60,976 for a household of six). The median adjusted per-household-member income17 is $16,213 for new voucher recipients and $15,000 for renewal recipients. In each year of the program, many applicant families are deemed ineligible for the program. In total, 1,976 new applicants and 110 renewal applicants were classified as ineligible in 2016-17. For new applicants, the two most common reasons for ineligibility were that the student did not currently attend a North Carolina public school (51 percent) or that family income exceeded the statutorily-defined income cap for a given household size (42 percent). For renewal applicants who were determined to be ineligible, the most common reason was an increase in family income, such that it exceeded the cap (88 percent of cases). As had been the case in previous years, over one quarter of new applicants in 2016-17 were deemed eligible for the program but were unresponsive to NCSEAA’s attempts at communication. As a result, these students were not awarded an Opportunity Scholarship. Potential explanations for this phenomenon include: the parent’s email address used at the time of application may have become inactive, or the cell phone number provided on the application form was no longer in service; applicants may have moved out of state; applicants may not have received admission to their desired private school; or applicants may have determined that private school tuition and fees would not be affordable, even with the assistance of the Opportunity Scholarship. In 2014, the evaluation team worked with Master’s in Public Policy candidates at the Sanford School at Duke University to draft a comparison of the Opportunity Scholarship guidelines with those of other programs across the country. That comparison will be available soon.18 16 All demographic data reported in this section of the report are for the 2016-17 school year; some geographic data are for the 2015- 16 school year. 17 Adjusted per-household-member income = household income / square root of household size; this approach is commonly used by economists to account for household savings from economies of scale and resource-sharing.
  • 18.
    Page 17 EnsuringOpportunity in the OS Program Grade Level Compared to public school students, voucher recipients are more likely to be in elementary school and they are less likely to be in high school (Figure 1). Though speculative only, possible reasons for this discrepancy may have to do with differences in the relative number of seats available across grades at participating private schools, or with a decrease in interest among potential eligible families to move students into a new school as they grow older, or both. Figure 1. Grade Levels of All Opportunity Scholarship Recipients, 2016-17 Ethnicity A larger proportion of 2016-17 Opportunity Scholarship recipients were African-American, relative to the population of public school students (35% versus 25.5%), and a smaller proportion of recipients were Hispanic (9.5% versus 17.3%) or White (41.3% versus 48.6%). In addition, a much higher proportion of Opportunity Scholarship recipients identified as being either biracial or of an ethnic background not common in North Carolina than did public school students as a whole (12.2% versus 4%). Because of the close association in North Carolina between poverty and race, these differences mostly are reflective of differences between the statewide population of students eligible for the Opportunity Scholarship and the ineligible population. 18 The draft report is entitled 50-State Voucher Scan: An Overview of United States Voucher Programs; it is currently being updated by one of the authors of the report. An updated list of comparable programs nationwide also can be found at this address (though the list does not include direct comparisons to the Opportunity Scholarship): https://www.edchoice.org/research/the-abcs-of-school- choice/
  • 19.
    Page 18 EnsuringOpportunity in the OS Program Differences between Participating Students and Other Eligible Students Are there systematic differences between students who apply and eligible non-applicants? Our conversations with and survey responses from participating private school headmasters and parents of Opportunity Scholarship recipients in 2014 and 2016 revealed additional details about the recipient population that are not captured in traditional demographic categories. Reflecting the diversity of the private schools they attended, the population of Opportunity Scholarship recipients is quite diverse along a number of axes, as suggested by the data reported above and as supported by the random nature of the lottery used to identify recipients. In terms of fairness and equity in the abstract, the program treats all applicants relatively equally, with only a few exceptions (for example, the lottery does preference siblings and members of military families). Worth considering, however, is whether certain eligible populations are more likely to attempt to access the Scholarships at a greater rate than are others. We consider this question along three axes: differences in access to information about the program; the equity of the application process itself; and finally, participating school perceptions of the characteristic of their Opportunity Scholarship students.19 Access to Information about the Opportunity Scholarship How do families receive information about the Opportunity Scholarship and participating nonpublic schools? Are there differences in information access by sub-groups (by ethnicity, by geographic location, etc.?) Information Networks. The viability of a school choice program like the Opportunity Scholarship program depends on use of the program by the intended population. One of the primary concerns, therefore, is the possibility of inequities in parent awareness of the program’s existence and of their eligibility for it. To learn more about these possible inequities, via survey we asked parents to share how they learned about the Opportunity Scholarship program (Figure 2, following page). Their responses revealed the importance of social networks in the information diffusion process. The majority of parents (40 percent) indicated that they had learned about the program through informal means, primarily from conversations with friends and relatives. Thirty-three percent indicated that they had been contacted about the program by a private school, and 15 percent learned about the program from an Internet search. Such searches likely pointed the user either to news articles related to the program or to the website maintained by the state agency that administers the program (NCSEAA), which offers detailed application information and up-to-date program usage statistics on its website.20 Several state and national school choice advocacy organizations also offer information about North Carolina’s program on their websites. 19 We explore a fourth, related aspect—location of participating schools—in a later section. 20 www.ncseaa.edu
  • 20.
    Page 19 EnsuringOpportunity in the OS Program Figure 2. Pathways through which Parents Learned about the Opportunity Scholarship Program Note: n = 2,425 respondents; TPS = Traditional Public School Focus group data revealed similar learning channels. The majority of participants indicated that they initially became aware of the program through word of mouth: We did not know anything about the scholarship. I think a friend of ours told us. . . . – Fayetteville Parent (2014) My daughter’s basketball coach told me about it, I think before I saw the billboards and stuff. He was like, “Do not forget to get on the computer and do the application on such-and-such date. . . .” – Fayetteville Parent (2014) I didn’t even know this program existed until I spoke with [advocacy group staff member]. Somebody had given me her number and that’s how I found out about this Opportunity Scholarship. – Fayetteville Parent (2016) In some cases, the word-of-mouth network included the private schools themselves. Of particular interest in terms of equity of access to information, their dissemination of information may be most important for
  • 21.
    language minority populations. [Y]ouget a lot of phone calls through the admission process: “What kind of tuition assistance do you offer? What do you have?” We can’t do a whole lot because of our school size and where we’re located so I very quickly state that there is a program through the state of North Carolina and I direct them to the website. – High Point Headmaster (2016) [W]e do a radio ad that lets families know they can get online and check their eligibility. . . . – High Point Headmaster [B]ecause of connection with our Spanish church, our Spanish pastors . . . will . . . give the information out to their Spanish members of the church . . . . – High Point Headmaster (2016) [O]ur Spanish church has quite a few of the Opportunity students coming. The problem I find [is that] most of the parents are speaking Spanish, they do not speak English, so that all the [information about OS is] coming in English and they can’t read them. – High Point Headmaster (2016) Our survey asked parents to share how easy it was for them to access information about the Opportunity Scholarship program, once they began their search. Just over 80% of survey respondents agreed or strongly agreed that it was easy to get the information they needed about the program. Selection of Private Schools. Our survey also asked parents about their procedure for choosing private schools. School choice research suggests that parents become more discerning in their school “shopping” behavior as experience with participating in a school choice program grows.21 Those who are inexperienced with exercising school choice may not consider a broad range of schools initially, but grow more sophisticated in their search for a private school over time. It is possible that these national trends are reflected in our data, as the majority of parents surveyed so far (that is, during the early years of the program) indicated that they only considered a single school when choosing which private school their child would attend with the Opportunity Scholarship (Figure 3, following page). This also may be an artifact of the rural nature of many of North Carolina’s counties, in which there are often very few private schools (and sometimes only one) from which to choose (discussed in greater detail below). Some participating private schools also make referrals, as indicated by participants in our focus groups: [I]f there’s a school that would better fit that child’s needs and so forth we’ll do a referral knowing that the Opportunity Scholarship could be taken by that school, so there is that discourse. Also with administrators I’ll call and let them know we’re sending someone [their] way that has the Opportunity Scholarship that could benefit from [their] school. – Raleigh Headmaster (2016) Page 20 Ensuring Opportunity in the OS Program 21 See Stewart, T., & Wolf, P. J. (2014). The School Choice Journey: School Vouchers and the Empowerment of Urban Families. New York, NY: Palgrave MacMillan.
  • 22.
    Page 21 EnsuringOpportunity in the OS Program Figure 3. Number of Private Schools Considered Note: n = 2,277 respondents The Application Process and Equity of Access Does the timing of the application (early February) impact who applies? Standards for admission as a possible barrier to access will be discussed below, but the greater access challenge with respect to admissions may have more to do with differences in families’ understanding of the admissions process. Especially during the first year, some Opportunity Scholarship applicant families were unaware that the application process for an Opportunity Scholarship was not also the application process for their top school choices, and that receipt of an Opportunity Scholarship was not a guarantee of enrollment in a given school.22 In a survey administered by the research team to Opportunity Scholarship applicant families, this disconnect was the second most frequently selected reason for non-acceptance of a Scholarship among applicants who declined an offered Scholarship. Headmasters noted as much across multiple locations and across years: 22 Based on interview responses from state-level administrators of the program; the research team did not ask this question directly of Opportunity Scholarship parents who responded to our survey.
  • 23.
    Page 22 EnsuringOpportunity in the OS Program [W]e have had a bunch of people in [our community] . . . put [our school down as their first choice] but not really understand when they put that down that when they were accepted into the program, that did not mean that they were accepted into the school as well. They still had to apply and do everything else. – Wilson Headmaster (2014) [Y]ou have to explain that to parents when they come in. They’ll say, “I have a scholarship, I’ve been awarded a scholarship,” and then you have to explain your process in terms of admissions. – Raleigh Headmaster (2016) They’ve heard of it, they know there’s something available but . . . the ones that I talk to are rarely very learned as to . . . how to go about doing it [completing the application process]; they need to have their hand held through it. . . . – High Point Headmaster (2016) Focus groups with parents shed more light on those struggles: Navigating print was pretty intensive. It was a lot of reading. I was able to understand everything pretty easily, but I could see for people that are not well-versed in how schools work and things like that it could have been frustrating. – Wilson Parent (2014) This type of application confusion is one that likely will diminish as the program grows and the informational materials and webinars offered by NCSEAA continue to stress this program requirement. Indeed, data from our 2016 survey indicated that 87 percent of responding families agreed or strongly agreed that the application process was simple to complete, and some private schools indicated that they provided application support for families in need of it. [D]uring our admissions process when parents come out to find out about our school, we actually let the parents know, we give them information about the Opportunity Scholarship and show them . . . how to apply. If they need help applying, we actually set them up on a computer at our campus and allow them to apply there. – Raleigh Headmasters Some parents indicated that having a point of contact to guide them through the application process was particularly useful: It really helps to have an advocate, you know, someone . . . who’s very experienced to kind of help people along. – Charlotte Parent (2016) Impact of the timing of the application. One particular potential access challenge for programs like the Opportunity Scholarship is the administrative calendar—when applications for participation in the program are due, when the funds are actually awarded, and when parents have to make decisions about their children’s schools. While some focus group parents indicated that the timing was not consequential, other parents and several headmasters noted challenges related to the program’s timing. [The calendar] hasn’t [worked] for us. . . . January 1, we start taking applications for those children who would be starting with us in August. So the challenge is that if the Opportunity Scholarship folks are
  • 24.
    Page 23 EnsuringOpportunity in the OS Program applying in February and then getting notified in March that they’ve received a scholarship . . . our seats have been filling up when we do a first notification in the first of February. – Raleigh Headmaster (2016) [W]e actually start enrollment for the next year in January. And so, when they start putting information out to us—we’re talking February-March—by that time sometimes we’ve already filled up and we would like not to have to give sort of the advantage to the tuition-paying parents when the other students should have a right to come and be accepted as well. – Fayetteville Headmaster (2016) I had access and luck that other people probably did not have. I wonder how many people were told that they had the scholarship but did not have enough time to get all of that stuff turned around. – Wilson Parent (2014) Impact of the timing of the award. Another recurring timing-related theme is the challenge faced by private schools when they try to integrate recently-awarded Opportunity Scholarship students in the middle of the school year. This challenge is not new to most public schools, some of which constantly enroll new students throughout the year, but focus group data suggest that it may not be as common among private schools. That’s the biggest challenge, for us at least, is if you have a waiting list and not everybody responds to the email when they’re awarded the scholarship and so those families get moved back to a waiting list and other people who are on the waiting list get moved up and then, it’s February or March and you have these families calling me saying “I got the Opportunity Scholarship.” And I’m thinking, I can’t do this to my teachers, they’re going to kill me. – High Point Headmaster (2016) [T]he parent does not want to go ahead and enroll [a student] in our school because they know that they cannot afford the tuition so they put it off as long as they can and here we are, you know, trying to figure out numbers, trying to figure out staffing and the parents are, “Well, I’m still waiting to hear from the scholarship.” – Fayetteville Headmaster (2016) From the parents’ perspective, the challenge is as much related to how it impacts their school choices as it is about changing schools in the middle of the year. Parent 1: [O]ne or two [schools had] closed [admissions] at the time that we needed to get in and they couldn’t break it up, so [the school we eventually chose] allowed us to come in and [my child] would have to make [missed work] up in the summer time. Parent 2: Same thing for me, [my first choice] wouldn’t accept him mid-year, so I went to somewhere that was like more flexible. . . . – Fayetteville Parents (2016) This enrollment timing challenge may dissipate as the program evolves and all awards are received early enough for families to make decisions in the summer before school begins.
  • 25.
    Page 24 EnsuringOpportunity in the OS Program Perceptions of Opportunity Scholarship Students at Participating Schools What are the characteristics of student applicants? Finally, we consider how staff at participating schools characterize their Opportunity Scholarship students— not as an exercise in critiquing participating school perceptions but instead to investigate whether their descriptions reveal any additional characteristics of the students who apply for the Scholarship that are not discernible in the more quantifiable data reported above. Our survey asked school leaders to describe their satisfaction with students’ preparedness upon arrival at their schools: 91% were satisfied with the preparation of their typical students, and 80% were satisfied with the preparation of their Opportunity Scholarship students. We had hypothesized that the perception would be that Opportunity Scholarship students are perceived as being less prepared than their fee-paying counterparts, which is borne out by the data; with a high proportion reporting satisfaction with both groups, however, the disparity is less notable than it might have been, had the overall satisfaction rate been lower. Focus group participants also noted that some of their Opportunity Scholarship students are not always ready for the transition to their schools; incorporating these students into their schools often involves helping them transition not only to a new academic environment but also to a different cultural and social environment: [L]ast year there was a lot of kids coming in with B’s on report cards in math classes that came to our math classes and not have a clue of what was going on. – Fayetteville School Leader (2016) [T]ranscending even the testing is the work that’s being done teaching these kids how to be good students, . . . teaching them how to learn respect and how to treat people. . . . [W]e get kids that are coming ready to fight, who are disrespectful, who have no concept of how to behave in a classroom, unlike kids that come up through our system. – Fayetteville School Leader (2016) Focus group participants also noted, however, that the Opportunity Scholarship gave them an opportunity as a school to extend their services to students they wanted to reach but for whom tuition was a barrier. For example, one participant indicated that he appreciated that the Opportunity Scholarship made his schools more accessible to special population students: [W]e work with students that have learning differences; it’s a school that works with students with average to above average IQ, bright students but they either have an attention deficit diagnosis or another diagnosed learning disorder. . . . [The Opportunity Scholarship] really opens doors that [families] never thought would ever be open to them before. So that’s the benefit that we’re reaping from it; it’s not just rear-ends in the seats, it’s kids . . . that really need what we do because it’s very specialized what we do. They could have never afforded it, never entertained the idea of coming to our school. Now they can, and they can make it. The $4,200 makes it work in some budgets where it could never have worked before. – High Point Headmaster (2016)
  • 26.
    Page 25 EnsuringOpportunity in the OS Program Assuring Access to a Quality Education for Participating Students: Participating Schools Characteristics of Participating Private Schools How do the Opportunity Scholarship family eligibility guidelines compare to guidelines for similar programs nationwide? Are there differences in access across counties? For example, do rural students have proportionally more or less opportunity to attend a participating nonpublic school than do urban and suburban students, in terms of both availability and cost? What are the characteristics of nonpublic schools that choose to participate in the voucher program, and how do they compare to public and non-participating nonpublic schools? Are there discernible patterns in the types of private schools that participate (e.g., by location, school type, school size, etc.), and, if so, what are they? Geographic Distribution Students in 97 of North Carolina’s 100 counties received scholarships for 2016-17.23 Relative to overall county population, participation for that year was much higher than expected in Hoke, Mitchell, Northampton, Bertie, Perquimans, and Washington counties; by the same measure, participation was much lower than expected in Orange, Burke, Chatham, Haywood, Moore, Lincoln, and Watauga counties. In some cases (e.g., Lincoln and Watauga), these differences may be related to the somewhat uneven distribution of participating private schools across the state (Figure 4, following page) and to the relatively low overall number of recipients, but other factors likely contributed as well, possibly including variations in: the perception of public school quality among eligible families in different counties; varying outreach efforts by Opportunity Scholarship advocates across the state; and tuition of nearby participating private schools relative to scholarship amount. Access to the Opportunity Scholarship Statewide Access is determined by a number of factors, including accessibility (determined by such factors as location and personal transportation options), cost after application of the Opportunity Scholarship (including financial aid support), and admissions standards. General Accessibility. As the program has grown, so, too has the number of participating schools—from 224 schools with Opportunity Scholarship recipients enrolled for the 2014-15 school year to 359 schools with enrollments for the 2016-17 school year (see Appendix for a full list). In addition, up to 100 more schools each year have been willing to enroll students (an additional 109 in 2014-15, an additional 101 in 2015-16, and an additional 78 in 2016-17). 23 In 2016-17, Tyrrell (size rank by population: 100), Hyde (99), and Dare (66) counties had no students who were awarded Opportunity Scholarships.
  • 27.
    Page 26 EnsuringOpportunity in the OS Program Figure 4. Participating North Carolina Private Schools, 2015-1624 Note: Purple diamonds represent participating private schools (2015-16); grey diamonds represent non-participating private schools (2015-16) 24 This map and others included in this report were generated by Emily Antoszyk for the Friday Institute’s Consider It Mapped series on EducationNC. 25 For example, in 2015-16, those counties included: Alexander, Anson, Ashe, Bladen, Camden, Caswell, Cherokee, Chowan, Davie, Duplin, Franklin, Gates, Graham, Hyde, Jones, Martin, Pamlico, Pender, Perquimans, Polk, Swain, Tyrrell, Warren, Washington, Watauga, Yadkin, and Yancey. 26 All counts are based on schools that actually enrolled students for a given year. Readers are asked to keep in mind that the number of communities with schools enrolling Opportunity Scholarship students is lower than the number of communities with schools eligible to enroll these students; several schools indicate a willingness to participate in the program each year but either do not end up receiving applications from these students or decline admission based on admission standards. We use enrolling school numbers rather than eligible school numbers, even though they underrepresent the total number of potentially participating schools, because the list of eligible schools includes an unknown number that did not actively make their involvement in the Opportunity Scholarship program public. A challenge that we’ve experienced in our school is just the growth that we’ve experienced through the Opportunity Scholarship. So just student population has increased which has provided its own challenges; which is a good challenge to have. – High Point Headmaster Several rural counties do not have participating schools (Figure 4, above).25 However, school participation growth has been relatively equally distributed geographically. Schools in 92 communities enrolled Opportunity Scholarship students in 2014-15,26 and the number of communities served increased to 128 (328 schools) in 2015-16 and 137 (358 schools) in 2016-17. Data on the number of available enrollment slots are not available, so it is not possible to estimate geographic differences in accessibility at the individual student level (i.e., whether there are proportionally more or fewer slots available in one community relative to another).
  • 28.
    Page 27 EnsuringOpportunity in the OS Program Rural versus Urban Accessibility. Unsurprisingly, growth in terms of the number of schools has been strongest in the largest urban areas—with the number of schools in Charlotte, Raleigh, Fayetteville, Winston-Salem, Greensboro, and Durham that enrolled Opportunity Scholarship students growing from 79 schools in 2014-15 to 111 in 2015-16 and 120 in 2016-17—but that growth did not necessarily represent an imbalance, in the sense that the proportion of enrolling schools that were located in those communities remained at around 34% of all enrolling schools each year. In other words, as the program grew, the number of schools available for students in urban areas grew proportional to the number of schools available in less urbanized areas. In fact, the total number of participating schools grew in 33 communities in Year 2 and by an additional 17 communities in Year 3, with only 6 communities and 11 communities, respectively, seeing declines in the number of enrolling schools. In Year 2, 38 communities with no enrolling school in Year 1 were home to at least one enrolling school, and in Year 3, 14 more communities without an enrolling school in Year 2 were home to at least one enrolling school. Transportation. Opportunity Scholarship recipients in most counties appear to have access to at least one participating school within a reasonable driving distance of most communities (Figure 4, above), as indicated by the number of counties with Opportunity Scholarship recipients (97 of 100 in 2016-17), even though in most years there have been more than 20 counties without a participating school. Most private schools in the state do not provide transportation, but some do offer neighborhood-based shuttle services, and many work with private transportation providers in their area. As noted above in the Data and Methods section, publicly-available data on non-public schools are limited; without conducting a school-by-school investigation of transportation options, it is not possible to quantify accurately the provision of non-public school transportation statewide. Cost and Financial Assistance. Unlike the statewide voucher program in Louisiana, the Opportunity Scholarship requires family co-payment of tuition and fees if those costs exceed the value of the voucher.27 This program feature increases the likelihood that parents will need to maintain direct financial responsibility for their child’s education, but it has been criticized by some for pricing low-income parents out of the market. The Opportunity Scholarship provides up to $4,200 a year for each participating student, which is about $800 less than the state invests in low-income students in grades 4 through 12, and more than $1,500 less than the state invests in low-income early-grade students.28 Including average local and federal supplements provided to public schools for low-income students (about $2,200 in local funding and about $2,000 in federal funding), the amount is less than half of the total average amount per public school student. The voucher amount also is less than the estimated average of one year’s tuition in a typical private school in North Carolina (about $7,600 for elementary; about $9,400 for high school),29 and well below the cost of one year’s tuition in the most exclusive 27 Two other statewide school choice programs that share this design feature are the Indiana Choice Scholarship Program and Florida’s McKay Scholarship Program for students with special needs. The Louisiana Scholarship Program, on the other hand, requires participating private schools to accept the voucher as full payment. 28 Includes state per-pupil expenditures only (not local or federal), plus adjustment for low-income family earnings; does not include adjustments for low-wealth counties, special needs status, Disadvantaged Students Supplemental Fund, or any other district- or school-level adjustments; see Highlights of the North Carolina School Budget (February 2017): http://www.ncpublicschools.org/docs/ fbs/resources/data/highlights/2017highlights.pdf 29 Third-party estimates of means: https://www.privateschoolreview.com/north-carolina; estimated median costs are somewhat lower, but still higher than the amount of the Opportunity Scholarship. The research team cannot confirm the precision of The Private School Review’s figures and uses them here for general descriptive purposes only.
  • 29.
    Page 28 EnsuringOpportunity in the OS Program schools (many of which can exceed $20,000 a year). Our tuition is probably maybe a little higher than some. . . . The Opportunity education grant covers a smaller percentage of what [students’] tuition is, [and] we have some kids that get both [state-funded scholarships—the Opportunity Scholarship and a scholarship for students with special needs], but still, that only pays about 60% if they get both grants. – High Point Headmaster Some parents acknowledged that the difference between tuition and Opportunity Scholarship amount influenced the list of private schools they considered, suggesting that cost may have led some parents to choose schools not based on fit alone: I’m a single parent of three kids, so obviously I’m not rich, so I needed [a school] that . . . [the] scholarship could cover as much as possible. So I went with that, that’s why I went with [specific private school]. . . . – Fayetteville Parent (2016) Many schools also have additional fees—expenses that some Opportunity Scholarship families reported initially came as a surprise to them, again often because of their inexperience with the private school process. The Opportunity Scholarship has covered our tuition but there are other costs. . . . There is some added stuff each month, but public school has the same thing, if not more. So it covered our tuition, and everything after that we sacrificed as parents. . . . – Fayetteville Parent (2014) In many cases, however, participating schools are able to provide tuition support, and some even reported determining their available Opportunity Scholarship slots based on the amount of aid they would be able to provide. Admissions Standards. Perhaps the greatest potential accessibility barrier is not geography but individual school admissions and financial assistance policies. By definition, private schools are not obligated to offer enrollment to all applicants, and schools that participate in the Opportunity Scholarship program are not obligated to offer enrollment to all Opportunity Scholarship recipients; however, per state statute (115C-562.5), private schools are not allowed to discriminate on the basis of race, ethnicity, or national origin. In addition, statewide, acceptance rates are high.30 Comparisons to Public and Non-Participating Private Schools In all, 266 non-public schools responded to our survey of headmasters, and 67 percent of those respondents were participants in the Opportunity Scholarship program in school year 2015-16. This variation is large enough to allow us to compare across school categories to estimate the characteristics of all participating and non- participating schools, and to identify potential patterns by school type. 30 One recent estimate places the overall statewide acceptance rate at 86% (https://www.privateschoolreview.com/north-carolina)
  • 30.
    Page 29 EnsuringOpportunity in the OS Program Religious Orientation and Affiliation A higher proportion of participating schools (76 percent) indicated that their school has a religious orientation than did non-participating schools (42 percent).31 The most common religious denomination represented among participating schools is Baptist (29 percent), followed by Christian (27 percent), then Catholic (18 percent; Figure 5). Figure 5. Religious Affiliation of Participating Private Schools One focus group participant noted that having a religious orientation can mean more than just having a religiously-oriented curriculum: [We’re] a Christian based school, and that world view is taught in every aspect of our program. – (High Point School Leader, 2016) Admissions As noted above, participating non-public schools in North Carolina’s Opportunity Scholarship Program are permitted to screen all applicants when deciding who to admit, even if a student already has been awarded an Opportunity Scholarship by the state. This feature is common to many special needs scholarship programs, but is not as common among less-targeted voucher programs.32 Voucher programs with alternative designs include the Louisiana Scholarship Program (formerly known as the Student Scholarships for Educational Excellence Program), which requires schools to use an open admissions process for enrolling scholarship recipients, and the Milwaukee Parental Choice Program, which requires schools to admit eligible students on a random basis. 31 This difference is statistically significant, p = 0.000 32 See, for example, the Arkansas Succeed Scholarship Program, Oklahoma’s Lindsey Nicole Henry Scholarship for Students with Disabilities Program, Louisiana’s School Choice Program for Certain Students with Exceptionalities, and North Carolina’s Special Education Scholarship Grants for Children with Disabilities. Baptist Christian Catholic 7th Day Pentecost. Lutheran Other
  • 31.
    Page 30 EnsuringOpportunity in the OS Program To learn more about admissions criteria in North Carolina, we asked schools to share the criteria they routinely use when admitting students. The vast majority of schools in our sample that participate in the Opportunity Scholarship program reported rely upon an interview with prospective students (81 percent). The second most frequently selected option was information about students’ academic ability, as measured by test scores (64 percent), and the third most frequently selected option was a review of the student’s disciplinary record (59 percent; Figure 6). Figure 6. Admissions Criteria Notes: n = 178 participating schools, 88 non-participating schools; OSP = Opportunity Scholarship Program; * indicates that a difference is statistically significant at p < .05 Data for the non-public schools that do not participate in the Opportunity Scholarship Program and that responded to the survey (n=88) revealed a somewhat similar pattern, though criteria were more evenly distributed. The majority of these schools also relied upon an interview with prospective students, but at a lower rate (66 percent). The second and third most frequently selected options were a review of academic achievement, either measured by a test (41 percent) or some other indicator of academic achievement (44 percent).
  • 32.
    Distinguishing Characteristics We alsoasked participating schools to choose the characteristics that they believe set them apart from nearby public schools (Figure 7).33 The most frequently selected perceived distinguishing characteristic was a stronger emphasis on college preparation, chosen by 19 percent of participating schools (n=34). The second most frequently selected option was a stronger emphasis on character building, also chosen by 19 percent of participating schools (n=33), and the third most frequently selected option was religious education, chosen by 16 percent of participating schools (n=28). Figure 7. Distinguishing Characteristics of Participating and Non-Participating Schools 33 As with all such choice lists in the survey, the response options were presented in a random order so as not to bias the frequency with which certain response categories were selected. Notes: n = 178 participating schools, 88 non-participating schools; OSP = Opportunity Scholarship Program; * indicates that a difference is statistically significant at p < .0 Page 31 Ensuring Opportunity in the OS Program
  • 33.
    Page 32 EnsuringOpportunity in the OS Program In focus groups, participating school leaders provided more detail about what they believe makes their schools different from their public school neighbors. Some noted their focus on special populations; for instance, one school leader noted that his school “work[s] with students that have learning differences” (High Point School Leader, 2016)—but most emphasized their ability to provide a religiously-minded curriculum and culture: [O]ur mission is to impact the world for Christ – Concord School Leader (2016) We posed the same question to non-public schools that elected not to participate in the Opportunity Scholarship Program, which revealed key differences in how these schools perceive themselves. When asked to choose the top characteristics that they believe distinguishes their school from nearby public schools, non- public schools that elected not to participate in the Opportunity Scholarship Program were most likely to say that their school requires less standardized testing, with 24 percent choosing this option (n=21)—statistically significantly higher than the proportion of participating schools that selected this option (10 percent). The second most frequently selected option was a stronger emphasis on character-building, chosen by 18 percent of schools (n=16), and the third most frequently selected options were religious education and a stronger emphasis on college preparation, each chosen by 10 percent of non-participating schools (n=9). Non-Participating Schools We also were interested in learning more about the factors influencing the participation decision of schools that opted out of the program, so we posed this question to the leaders of schools that have never participated in the Opportunity Scholarship Program and schools that indicated they have no plans to do so in the near future (n=58). The most frequently selected factor explaining a school’s non-participation decision was a concern that future state laws or policies might change requirements for participation, with 57 percent of non-participating respondents indicating that this concern played a role (minor or major) in their decision. Another common concern was the amount of paperwork and reports that are required of participating schools, selected by 47 percent of non-participants. Interestingly, 54 percent of schools in this category chose “Other Concerns” as playing a major or minor role, implying that their primary reason for non-participation was not listed in the options we provided. We offered an open-response text box for schools to list their concerns if they selected this option. The two most commonly cited reasons provided were that their school was not aware the program existed or that their school makes an effort to avoid accepting government funds: [We] did not know about the scholarship program. The board leadership of the school is concerned about political entanglements that comes [sic] from receiving state and/or federal funding. We question the long-term sustainability of the program. Also as a matter of policy and principle, we do not participate in government funded programs, directly or indirectly. Concerns that opportunity scholarship students may require more resources than we have to offer. We did not have enough financial aid money left when the applicant applied to fund the year’s tuition for them. They wouldn’t have had enough.
  • 34.
    Page 33 EnsuringOpportunity in the OS Program Focus group responses indicated a few other reasons why participating school officials believed that some schools chose not to participate. While some believed the problem was the gap between their tuition and the coverage provided by the Opportunity Scholarship, others thought it had more to do with the types of students who were eligible for the funding. [S]ome of the schools in the area that have more expensive tuition, the Opportunity Scholarship program doesn’t offer enough aid to make it worth it. So they choose not to participate. – High Point Headmaster (2016) R: [U]unfortunately, I think it’s the socio-economic question, and . . . I was, like, “Well, you know, everybody can have their opinions.” But I just kind of have gone over with them that we have not experienced any problems whatsoever. I: So they’re concerned about a change in their student population? R: Yes. . . . – Raleigh Headmaster (2016) We have a lot of private Christian schools in [nearby community], they don’t want low-income students in their schools, so they don’t want the Opportunity Scholarship, that’s the bottom line. – Raleigh Headmaster (2016) It is important to keep in mind that this last group of perceptions of non-participating schools come from participating private schools only and has not been verified in any way, but the difference it highlights with respect to their own willingness to enroll lower-income students versus their perceptions of other schools’ resistance to do so may merit additional study. Expenditure Patterns of Participating Private and Nearby Public Schools To what extent do participating nonpublic and nearby public school expenditure patterns differ? Expenditure patterns for public schools are relatively easy to trace across years via analysis of state object and purpose codes. The quality and validity of the data associated with those codes varies from school district to school district, as does each district’s interpretations of some of those codes, but, overall, it is possible to determine general patterns in spending of local, state, and federal funds. Because of limitations in the reporting required of non-public schools,34 this level of fiscal transparency does not exist to a similar degree among Opportunity Scholarship schools. As a result, the question may never be answerable unless the state requires reporting of such financial data by participating schools. Even so, the evaluation team was able to piece together—albeit primarily anecdotally via focus group 34 See, for instance, the publicly-reported data available on the website of the North Carolina Department of Education’s Division of Non-Public Education (DNPE; https://ncadmin.nc.gov/about-doa/divisions/division-non-public-education)—the most comprehensive source for non-public education data in the state. Two of the authors of this report have worked with DNPE to improve data availability for private schools, but the results of that effort will not include additional financial information.
  • 35.
    Page 34 EnsuringOpportunity in the OS Program responses—some information about financial decision-making at participating private schools that is directly connected to the participation of those schools in the Opportunity Scholarship program. Tuition Support Some schools have begun a practice of personalizing tuition based on what a family can afford to pay, or some other related metric: What we’ve been doing is we’ve been making accommodations for our students who participate in the OS program; but . . . even though . . . the Opportunity Scholarships have been extremely helpful for those parents who wanted to continue educating their students through our school, it’s still a lot of extra costs where we have to try to work something out for them to attend. – Raleigh School Leader (2016) One common theme across focus groups was a belief that requiring some parent participation in covering tuition costs—even if nominal—was an important part of generating parent commitment to the school: [W]e . . . feel like the parent needs to have a vested interest, so we still are requiring those parents pay the registration fee. . . [W]e feel like they ought to have some ownership. – Fayetteville School Leader (2016) [W]e want those parents to have that little bit of commitment. . . [I]f you’re just sending your kid and you know, not participating in the education of the child at all, what are you gaining, what is the child gaining from it? – Fayetteville School Leader (2016) [W]e do have tuition assistance, however, we have a rule of thumb that everyone has to pay something for school so no one has a full ride, but we do try to work with families and we have an outside evaluator who helps us to see exactly what they can pay and we work with them individually on that. – Concord School Leader (2016) Participation and Enrollment Restrictions Related to Finance Though we have very little data on this particular issue, one finance-related consideration with respect to private school participation in the program is whether a school can enroll Opportunity Scholarship-eligible students without state support. For some schools, even though they already could support some tuition reductions on their own, the Opportunity Scholarship eased their ability to do so. Other schools, however, appear to have weighed the value of that state support against the regulations that come with it and decided that the extra support was not worth participation in the program. [S]ome [private schools] have their own tuition reduction program. They don’t accept a[n Opportunity] Scholarship but they have their own kind of thing. – Fayetteville Parent (2016) As a result, there may be some value in exploring further whether schools of a certain financial standing are less likely to accept the Opportunity Scholarship.
  • 36.
    Page 35 EnsuringOpportunity in the OS Program There also appears to be a related debate at several private schools about how much of their available funding they are willing to reallocate to scholarship support for Opportunity Scholarship families whose vouchers only cover part of their tuition. These schools appear to be wary of the program’s stability and remain concerned that they may run the risk of taking on a significant financial burden, if the program ends and they want to keep their Opportunity Scholarship families in their schools: We had conversations about whether we should apply for the Opportunity Scholarship, and understood that there would be a cap on the number of students we would accept using the scholarship, because if the money goes away, when the money goes away, what will we choose to use? Once we accept a family into our [school] family . . . we did not want to be in a position to have to say, “OK, 25 families that we love, you are part of the community, you volunteer in our kitchen, you come and help out and do all of these things, but now we have to say goodbye to you because that money is gone.” We recognized that we probably would not do that as a school, so we had to limit the number of people that we would accept using these scholarships to a number that we could afford to—if that money went away—supplement financially. – Wilson School Leader, 2014 The Opportunity Scholarship and Changes in the Supply of Private Schools What impact has the availability of the program appeared to have on the supply of private schools? Overall, available data make it hard to argue that the advent of the Opportunity Scholarship has led directly to a meaningful uptick in the number of private schools in the state (Table 2), even though there was a larger-than- usual spike in the year the legislation passed (2013-14). Also, while the number of Opportunity Scholarship students has increased over the past three years as the program grew, the overall number of private school students statewide decreased, which likely would have offset at least some of the potential growth in the number of schools possibly attributable to the availability of the voucher. Table 2. Private School Openings byYear SchoolYear Number of New Private Schools 2010-11 40 2011-12 52 2012-13 35 2013-14 (legislation passage year) 56 2014-15 (initial scholarship availability year) 39 2015-16 46 2016-17 50
  • 37.
    Page 36 EnsuringOpportunity in the OS Program In addition, spikes in private school openings have not been restricted to recent history only; there have been several over the years, most notably during the entirety of the Clinton Administration (1992-2000) and just before the Recession (2005-2008). Finally, these growth numbers do not reflect changes at the other end of the spectrum: Schools that closed in the same year. Thus, overall net growth in the number of private schools in the state is significantly lower across the same years (a total increase of less than 50 schools across those years) than was gross growth (318 school openings; Figure 8). Figure 8. Net Private School Growth in North Carolina, 1991-2017 Though anecdotal only, one school official posited that, rather than encouraging the opening of new schools, the larger impact of the Opportunity Scholarship may have been to stabilize existing schools with financial challenges: I wouldn’t say that any new ones have opened, but there’s some that might have folded or might be closer to folding without it. – High Point Headmaster (2016)
  • 38.
    Page 37 EnsuringOpportunity in the OS Program Understanding the Current Academic Outcomes Evaluation Landscape Testing in Participating Schools Per legislation,35 nonpublic schools that receive Opportunity Scholarship funds are required to administer a nationally standardized test or equivalent at least once a year to all Opportunity Scholarship recipients. Results of these tests are reported to NCSEAA, but there are no target performance levels, nor are there any actions NCSEAA is required to take related to the scores. The testing requirement is above and beyond a pre- existing statewide testing requirement for all nonpublic schools (regardless of their Opportunity Scholarship participation), which directs all nonpublic schools to administer a nationally standardized test annually, but only for students in grades 3, 6, 9, and 11. 36 Participating nonpublic schools with more than 25 Opportunity Scholarship recipients also must report aggregated standardized test performance results for eligible students. Unlike the individual test results for all Opportunity Scholarship students, these aggregated results are public record and are reported by NCSEAA annually. While the statutory language, “nationally standardized test,” technically allows participating schools to choose from hundreds of tests, most appear to meet this criterion by administering the same tests they administer to all of their students in the required grades. A full list of tests administered each year, like so many other aspects of the Opportunity Scholarship program, is not available, largely due to administrative constraints at NCSEAA: Without adequate funding to provide a full support staff to oversee the Opportunity Scholarship program, the manpower necessary to tally all tests administered each year currently is not available. Only data for school year 2014-15 is relatively complete, primarily because of the smaller number of Opportunity Scholarship participants that year. Tests from the first year of the program (2014-15), in descending order of frequency, are listed in Table 3. Table 3. Nationally StandardizedTests Administered to Opportunity Scholarship Recipients by Participating Nonpublic Schools, 2014-15 Test Proportion of OS StudentsTested TerraNova 35.7% Stanford 32.1% Iowa Assessments 24.1% California Achievement Test 3.9% Comprehensive Testing Program 2.2% Woodcock Johnson 1.1% BASI 0.4% Otis-Lennon School Ability Test 0.4% BJU Press Testing 0.2% 35 Section 115C-562 ; https://www.ncleg.net/EnactedLegislation/Statutes/PDF/BySection/Chapter_115C/GS_115C-562.5.pdf 36 Section 115C-549; Section 115C-550; http://ncleg.net/EnactedLegislation/Statutes/HTML/ByArticle/Chapter_115C/Article_39.html
  • 39.
    Page 38 EnsuringOpportunity in the OS Program Performance of Participating Students How do participating students perform, relative to national norms? This question is perhaps the most challenging to answer in a comprehensive and objective manner, given the restrictions in the enacting legislation. This report includes two cuts, neither of which is sufficient, but both of which help to underscore the need for more transparent data rules related to Opportunity Scholarship evaluation. Participating Schools with more thanTwenty-Five Opportunity Scholarship Students The first cut includes the aggregate reports for 2014-15 (Table 4) and 2015-16 (Table 5) provided by participating schools with more than 25 students. Table 4. Publicly-Reported Results, 2014-15 School Test Number of Students Tested Reading Language Mathematics % below 50th % at or above 50th % below 50th % at or above 50th % below 50th % at or above 50th Greensboro Islamic Academy ITBS 51 59 41 61 39 53 47 Word of God Christian Academy TerraNova 30 70 30 * * 83 17 Victory Christian Center School TerraNova 28 46 54 50 50 50 50 Concord First Assembly Academy TerraNova 23 52 48 61 39 44 56 Freedom Christian Academy TerraNova 20 45 55 20 80 40 60 Fayetteville Christian School ITBS 2 ** ** ** ** ** ** Verbal Reasoning Nonverbal Reasoning % below 50th % at or above 50th % below 50th % at or above 50th Fayetteville Christian School TerraNova 20 50 50 45 55 Note: Data provided by NCSEAA *Word of God Christian Academy provided scores for Reading and Mathematics only ** Data not provided; less than 5 students. Other students tested by alternate-format Terra Nova; see below.
  • 40.
    Page 39 EnsuringOpportunity in the OS Program School Test Number of Students Tested Reading Language Mathematics % below 50th % at or above 50th % below 50th % at or above 50th % below 50th % at or above 50th Alamance Christian School TerraNova 18 28 72 39 61 33 67 Al-Iman School N’west Evol. Assn 23 9 91 9 91 17 83 Bal-Perazim Christian Acad. TerraNova 19 47 53 53 47 47 53 Berean Baptist Academy ITBS 14 57 43 43 57 50 50 Bethel Christian Academy ITBS 21 58 42 67 33 60 40 Bethel Christian Academy ITBS 15 60 40 47 53 73 27 Bible Baptist Christian School ITBS 10 60 40 50 50 80 20 Brookstone Schools ITBS 3 ** ** ** ** ** ** Brookstone Schools TerraNova 2 ** ** ** ** ** ** Community Baptist School TerraNova 20 80 20 80 20 70 30 Concord First Assembly Acad. TerraNova 33 70 30 68 32 52 48 Cornerstone Christian Acad. Stanford 3 ** ** ** ** ** ** Cornerstone Christian Acad. TerraNova 8 86 14 71 29 88 12 Dream Big Christian Acad. ITBS 16 69 31 86 14 69 31 Fayetteville Christian School TerraNova 51 33 67 33 67 42 57 First Wesleyan Christian School TerraNova 8 38 63 25 75 38 63 Freedom Christian Acad. TerraNova 22 32 68 50 50 41 59 Gospel Light Christian School ITBS 28 75 25 79 21 79 21 Greensboro Islamic Academy ITBS 9 33 67 33 67 22 78 Table 5. Publicly-Reported Results, 2015-16
  • 41.
    Page 40 EnsuringOpportunity in the OS Program School Test Number of Students Tested Reading Language Mathematics % below 50th % at or above 50th % below 50th % at or above 50th % below 50th % at or above 50th Greensboro Islamic Academy ITBS 9 33 67 33 67 22 78 High Point Christian Acad. ACT 4 ** ** ** ** ** ** High Point Christian Acad. TerraNova 22 36 64 36 64 27 73 Liberty Christian Academy TerraNova 30 43 57 53 47 53 47 Living Water Christian School Stanford 10 50 50 30 70 20 80 Mount Zion Christian Acad. SAT 15 87 13 53 47 47 53 Raleigh Christian Academy ITBS 44 67 33 52 48 64 36 Rockwell Christian School ITBS 20 35 65 40 60 45 55 Salem Baptist Christian School TerraNova 23 35 65 65 35 48 52 St. Raphaels Catholic School ITBS 13 38 62 46 54 46 54 Star Christian Academy California Ach. Test 29 79 21 93 7 69 31 Tabernacle Christian School ITBS 44 80 20 75 25 57 43 Tri-City Christian Academy ITBS 40 70 30 80 20 88 13 Trinity Christian School ITBS 79 27 73 29 71 16 84 Victory Christian Academy Stanford 15 80 20 87 13 60 40 Victory Christian Center School TerraNova 35 63 37 57 43 60 40 Winston-Salem Christian School TerraNova 17 71 29 53 47 65 35 Word of God Christian Acad. TerraNova 54 67 33 * * 78 22 Note: Data provided by NCSEAA *Word of God Christian Academy provided scores for Reading and Mathematics only ** Data not provided; less than 5 students
  • 42.
    Page 41 EnsuringOpportunity in the OS Program As is evident from the tables, the performance of students at these schools differed widely, but these data alone are insufficient for understanding the impact of schooling on achievement and growth. To understand that kind of programmatic impact requires at least a baseline of information about how students were performing before entering the school, as well as information about how students from similar backgrounds performed in the public schools from which the Opportunity Scholarship students came. The next section— our second cut—details what such an evaluation might look like; it also critiques the evaluation team’s efforts to begin to approach this more comprehensive level of estimation using volunteer private and public school students in spring 2017. Recommended Steps for Implementing a More Robust Evaluation, and Initial EvaluationTeam Efforts to Analyze Student Performance (2016-17) For our second cut, we conducted an analysis of test scores with the aid of privately-supported administration of a common test to Opportunity Scholarship participants and eligible non-participants in spring 2017. We outline the limitations of our efforts in this section, along with what an optimal approach might look like. Without significant changes to the statutes that currently govern the program, it will not be possible to conduct a reliable and robust causal evaluation of the impact of the program on student academic achievement and academic growth. As noted above, members of the research team were appointed to a legislatively- mandated Task Force that, in March 2018, recommended options for a formal statewide evaluation.37 Those team members shared the steps below with the larger Task Force with the hope that at least some will be adopted by the General Assembly; in all likelihood, full adoption will require persistent engagement from a number of stakeholder groups. Step One: Identify a Common Measure of Achievement. Currently, participating schools are required to administer a nationally normed test to Opportunity Scholarship students each year, but results from these tests are only comparable if students take all of the tests whose results a researcher wishes to compare. For example, if School A administers the Iowa Test of Basic Skills, but School B administers the California Achievement Test, results from those tests can only be compared if at least a sample of the same students have taken both tests, to establish a concordance. Similarly, comparable public school students (details about what constitutes comparability are included below) would need to take the same test. To resolve this problem, the optimal solution from an evaluation perspective is for the state to require participating schools to administer the same test to all Opportunity Scholarship students. Such a policy will have several negative impacts, however, the most significant of which likely will be withdrawal of several schools from the program. Private schools choose tests that best meet their educational needs and cannot easily switch out one test for another. One viable compromise solution could be to require only a (sufficiently large) random sample of all Opportunity Scholarship applicants to take the common test, allowing participating private schools the right to administer the tests of their choice to their full complement of Opportunity Scholarship students. 37 https://www.ncleg.net/documentsites/committees/JLEOC/Reports%20Received/2018%20Reports%20Received/NCSEAA%20 Opportunity%20Scholarship%20Task%20Force%20Report.pdf
  • 43.
    Page 42 EnsuringOpportunity in the OS Program Other states with similar programs require participating schools to administer the state test, but that is not a requirement in this state. Of note, the North Carolina test is standards-based, not national norm-based, and therefore only best reflects the public school curriculum. For the 2016-17 testing pilot: The evaluation team administered a short version of the Iowa Test of Basic Skills to Opportunity Scholarship student volunteers and public school student volunteers. Step Two: Limit Program Enrollment. Even if students take a common test, the evaluation still will need to identify a group of students with whom to compare results. For our first attempt in spring 2017, the research team was only able to identify this group via a matching process that paired Opportunity Scholarship students with statistically-similar non-participating students based on a number of common variables, including prior test scores and demographic data. Not only does this approach require a significantly larger group of comparison students than Opportunity Scholarship students in order to obtain the best matches, but it also excludes a key variable: the desire to participate in the Opportunity Scholarship, a characteristic that is only identifiable among Opportunity Scholarship applicants and that may be a significant representation of a host of other unmeasurable characteristics. The optimal solution from an evaluation perspective is to limit the size of the program such that there are more students who apply for the scholarship than there are scholarships available, with the unsuccessful applicants then making up the comparison population. To date, oversubscription has not been the case (every student who has applied for an Opportunity Scholarship in the past has at least been offered the scholarship), and with the General Assembly’s plan to expand funding for the program over each of the next ten years is not likely to be the case going forward. Without a cap on enrollment, however, a truly random and truly representative group of Opportunity Scholarship recipients and Opportunity Scholarship unsuccessful applicants cannot be established. For the 2016-17 testing pilot: The evaluation team had to rely on student volunteers. Step Three: Make Program Application Contingent upon Willingness to Participate in Evaluation. Assuming there is an enrollment cap that restricts the number of recipients enough to create viable awarded and non- awarded pools of students, there is no compulsion for non-winners to participate in the evaluation by taking the additional common test. A possible solution here is to make submission of an application contingent upon willingness to participate in the evaluation. After all Opportunity Scholarships have been awarded, random samples of both awarded and non-awarded students could be identified, with non-awarded students obliged (as a condition of their application) to take the common test. Making the state test the mandatory common test would eliminate this problem, since non-awarded students who remain in public school will take the state test anyway, but, as noted above, mandating use of the state test by participating private schools comes not only at a high cost in terms of private school participation, both in the evaluation and in the program itself, but also in terms of the relative fairness of the testing instrument designed for the public school curriculum as a measure of private school and public school student performance.
  • 44.
    Page 43 EnsuringOpportunity in the OS Program For the 2016-17 testing pilot: Not applicable. Step Four: Secure Adequate Funding to Support the Evaluation Effort. In all scenarios, the ability to conduct a rigorous, valid, and reliable evaluation is contingent upon adequate funding: funding for the tests themselves, funding for test scoring, funding for test administration (which, based on our experiences in spring 2017, will require a significant travel budget), and funding for analysis. Contributions from two third-party foundations made limited testing possible in spring 2017, but in order to conduct enough tests while minimizing the burden on public and private schools alike, the state will need to commit adequate funding. Our estimates suggest that a budget of about $190,000 for each year of a three-year study would be required to conduct a rigorous evaluation that minimizes disruptions to participating schools, though that figure could be reduced with increased participation from those schools (for example, in the form of handling test administration duties locally). For the 2016-17 testing pilot: Contributions from third-party funders allowed the evaluation team to conduct, score, and analyze about 700 student tests (about 300 Opportunity Scholarship students and about 400 public school students from similar backgrounds).38 Other Evaluation Options Even with full support from the Task Force and leading public and private schools, it is highly unlikely that a gold standard, rigorous, causal estimate evaluation plan like the one outlined above will be fully adopted by the state. It is still possible to conduct rigorous correlational analyses that can provide good and useful information, even if some of the steps above are not accomplished, though it is important to emphasize that any such studies— even with the best data available—will fall short of a gold standard causal analysis. Some strong correlational analysis options include (but are not limited to): 1. Require Opportunity Scholarship students to take the state test. This is a near-gold standard approach, but, as noted above, the state test aligns with all public school curricula, but very few private school curricula (though a handful of private schools already administer the state test). Thus, motivation, test preparation, and curriculum alignment will be different across the two types of schools, which could bias results in favor of non-awarded students. 2. Administer the most commonly-used private school test to comparable public school students. For example, if TerraNova is the most commonly used test among private schools, a comparable sample of public non-awarded applicants could take the same test. The obvious benefit is that it requires almost no additional effort on the part of potentially resistant private schools. Concerns about this approach include possible inconsistencies in the tests private schools choose to administer each year, as well as the possibility that private school test selection reflects important differences in the private schools themselves 38 The report detailing the results of these analyses is available at: https://ced.ncsu.edu/elphd/research/the-impact-of-the-north- carolina-opportunity-scholarship-program/
  • 45.
    Page 44 EnsuringOpportunity in the OS Program (for example, if TerraNova is predominantly administered by private schools in the western region of the state). In addition, private schools would have to share individual student scores with evaluators, which currently is not required by statute. One partial remedy might be to require participating private schools to administer tests from a shorter list of approved tests, thus retaining some private school choice but also improving the likelihood that students who took a particular test are more representative of the entire population of Opportunity Scholarship students.
  • 46.
    Page 45 EnsuringOpportunity in the OS Program Next Steps in Our Evaluation Efforts Securing Continuing Support for Evaluation The analyses summarized in this report would not have been possible without support from multiple funders, but it represents only a fraction of the analyses necessary to better understand the impact of the Opportunity Scholarship program on North Carolina students, educators, and schools. The enacting legislation requires an evaluation of the program, with initial results due in 2018, but current state funding for program administration is not sufficient for supporting a meaningful evaluation. Even with state funding, however, the resulting evaluation is likely to be limited—not only by the strictures imposed by current statutes but also by the state’s level of financial commitment to the evaluation. Direct support from third-party groups likely will continue to be an important ingredient in ensuring a comprehensive, fair, and meaningful evaluation of the program as it continues to grow. Expanding Analysis of the Program’s Impact on Student Outcomes In addition to securing ongoing evaluation support—both in terms of financial support and of willingness to participate—the evaluation team will continue to develop plans for a deeper evaluation of the impacts of the program on student academic and non-academic outcomes. Academic outcomes alone are important, but they will not fully capture the impact of the program on all aspects of a student’s schooling. We outline here some of the major evaluation questions that will frame our thinking about future outcomes analyses. Steps for Improving Program-Related Data Quality As noted above, our work to date remains limited in scope as a result of existing statutory barriers, but even without formal statutory changes there is much that can be done to improve the data available to the public and to researchers. In particular, there is great potential for improving information about the tests administered to Opportunity Scholarship students and the data collected about the schools they attend. 1. What is the nature and quality of the nationally-normed tests used by participating schools to meet the legislative requirement of annual testing of Opportunity Scholarship students? 2. What improvements should the state make in its annual collection of operational descriptive data on non- public schools? Determining Impacts on Cognitive Outcomes Much of our thinking about the best approaches for estimating cognitive outcomes is outlined in the preceding section. The two specific questions that we think should guide this component of a complete evaluation are: 3. What is the impact of a voucher on students’ mathematics and reading achievement? 4. Are the effects evenly distributed for students from different subgroups and at different points along ability and income distributions?
  • 47.
    Page 46 EnsuringOpportunity in the OS Program Determining Impacts on Non-Cognitive Outcomes An important finding from our focus group sessions in 2014 and 2016 not reported above was the belief among parents and headmasters alike that participation in the Opportunity Scholarship program helped students in more ways than just improvement in their testable achievement. He enjoys reading more than he did before. He was not turned off to it because he liked when we would read to him, but now he is really comfortable taking the reins and trying to go through the skills that they are teaching him in school to read aloud with confidence. Mathematically and in all of the other aspects of education, I would say that he is being given challenges, meeting challenges, and feeling successful in working through frustration. Last year, he never got too frustrated which told me he was not being challenged. This year he is learning what I think is a very important life skill which is to work through your frustrations and know what success on the other end feels like. – Wilson Parent (2014) The greatest success is that [my children] are doing much better academically as far as their grades go and actually enjoying school. – Fayetteville Parent (2014) Transcending even the testing is the work that’s being done teaching these kids how to be good students, . . . teaching them how to learn respect and how to treat people. . . . [G]od bless you guys who were in public schools . . . because it’s a zoo out there and we get kids that are coming ready to fight, who are disrespectful, who have no concept of how to behave in a classroom, unlike kids that come up through our system. . . . – Fayetteville Headmaster (2016) Therefore, in addition to increasing our focus on academic outcomes, we hope to be able to explore in greater depth possible non-cognitive areas of impact, with the following questions as guides: 5. What is the impact of a voucher on students’ non-cognitive outcomes, including grit, conscientiousness, and delay of gratification? 6. How do these impacts vary for specific student subgroups, defined by race, locale, and socio-economic background? 39 Most voucher research differentiates between the impact of a voucher offer and the impact of voucher use. Data from the spring 2017 pilot testing program described in the previous section allowed for an early estimation of the impact of voucher use, albeit with the important limitations noted in that section. Also as noted in the previous section, the opportunity to conduct more rigorous estimations of the impact of voucher use and/or of a voucher offer will require statutory changes to the program.
  • 48.
    Page 47 EnsuringOpportunity in the OS Program Appendix. Communities with Opportunity Scholarship- Enrolling Schools, 2014-2017 2014-15 Original Communities Total 2015-16 NewCommunities # NewTotal 2016-17 NewCommunities # NewTotal Raleigh 22 Albemarle 1 1 Ahoskie 2 2 Charlotte 17 Bryson City 1 1 Cary 2 2 Fayetteville 17 Burnsville 1 1 Huntersville 2 2 Greensboro 9 ElmCity 1 1 Morganton 2 2 High Point 8 Grantsboro 1 1 Archdale 1 1 Winston-Salem 8 Harrells 1 1 Belhaven 1 1 Durham 6 Knightdale 1 1 Black Mountain 1 1 Greenville 5 Louisburg 1 1 Brevard 1 1 Wilmington 5 Marshall 1 1 Candler 1 1 Asheville 4 Mills River 1 1 Clayton 1 1 Gastonia 4 Newport 1 1 Clyde 1 1 Rocky Mount 4 Pantego 1 1 Cornelius 1 1 Asheboro 3 Sneads Ferry 1 1 Dover 1 1 Concord 3 Windsor 1 1 Falcon 1 1 Goldsboro 3 Fallston 1 1 Matthews 3 AddedSchools (net) Franklin 1 1 Monroe 3 Charlotte 13 30 Fuquay Varina 1 1 Southern Pines 3 Raleigh 5 27 Glade Valley 1 1 Statesville 3 Fayetteville 3 20 Hillsborough 1 1 Wake Forest 3 Winston-Salem 6 14 Holly Springs 1 1 Wilson 3 Greensboro 2 11 Hookerton 1 1 Bessemer City 2 Asheville 6 10 Kannapolis 1 1 Burlington 2 Durham 3 9 Locust 1 1 Henderson 2 Wilmington 2 7 Lumberton 1 1 Jacksonville 2 Goldsboro 2 5 Marion 1 1 Kernersville 2 Matthews 2 5 Mars Hill 1 1 Lexington 2 Statesville 2 5 McLeansville 1 1 New Bern 2 Concord 1 4 Merry Hill 1 1 North Wilkesboro 2 Elizabeth City 3 4 Mount Airy 1 1 Reidsville 2 Jacksonville 2 4 New London 1 1 Roanoke Rapids 2 Reidsville 2 4 Oak Ridge 1 1 Rockwell 2 Wilson 1 4 Pikeville 1 1 Salisbury 2 Burlington 1 3 Ramseur 1 1 Thomasville 2 New Bern 1 3 Rolesville 1 1 Whiteville 2 Apex 1 2 Shallotte 1 1 Winterville 2 Chapel Hill 1 2 Siler City 1 1 Apex 1 Clinton 1 2 Stokesdale 1 1 Boiling Springs 1 Conover 1 2 Swannanoa 1 1 Chapel Hill 1 Dunn 1 2 Chocowinity 1 Hamlet 1 2 AddedSchools (net) Clinton 1 Hickory 1 2 Charlotte 2 32 Conover 1 Hudson 1 2 Greensboro 4 15 Cramerton 1 Indian Trail 1 2 Durham 4 13 Creedmoor 1 Laurinburg 1 2 Matthews 1 6 Davidson 1 Mooresville 1 2 Gastonia 1 5 Dunn 1 Red Springs 1 2 Asheboro 1 4 Elizabeth City 1 Roseboro 1 2 Burlington 1 4 Ellenboro/Forest City 1 Spruce Pine 1 2 Southern Pines 1 4 Erwin 1 Troy 1 2 Cary 1 3 Fletcher 1 Henderson 1 3 Fuquay 1 Lost Schools Hudson 1 3 Gibsonville 1 Rocky Mount -2 2 Morganton 1 3 Graham 1 Bessemer City -1 1 Wake Forest 1 3 Hamlet 1 Lexington -1 1 Clayton 1 2 Harrisburg 1 Wake Forest -1 2 Harrisburg 1 2 Havelock 1 Hendersonville 1 2 Hayesville 1 Stedman -1 0 Sanford 1 2 Hendersonville 1 Weaverville -1 0 Hickory 1 Lost Schools Hobgood 1 Raleigh -1 26 Hudson 1 Asheville -1 9 Indian Trail 1 Elizabeth City -1 3 Jarvisburg 1 Hickory -1 1 King 1 Laurinburg -1 1 Kings Mountain 1 Red Springs -1 1 Kinston 1 Lasker 1 Black Mountain -1 0 Laurinburg 1 Boiling Springs -1 0 Lemon Springs 1 Candler -1 0 Millers Creek 1 Cornelius -1 0 Mooresville 1 Wanchese -1 0 Morehead City 1 Nebo 1 Pineville 1 Pisgah Forest 1 Pittsboro 1 Raeford 1 Red Springs 1 Richlands 1 Rockingham 1 Roseboro 1 Roxboro 1 Rutherfordton 1 Sanford 1 Smithfield 1 Southport 1 Spruce Pine 1 Stedman 1 Supply 1 Troy 1 Wanchese 1 Weaverville 1
  • 49.
    Page 48 EnsuringOpportunity in the OS Program Contact Information: Please direct all inquiries to Trip Stallings [email protected] © 2018 The William and Ida Friday Institute for Educational Innovation