Downloaded 77 times
Uploaded byJohn Cronin
Tasa presentation version 2
This document provides guidance on developing effective community-based accountability reports. It defines community-based accountability and outlines key conditions needed for it to work. It also recommends including a "management letter" to discuss successes, failures, goals and strategies for improvement. Additionally, it suggests focusing reports on a few key metrics that demonstrate progress toward community values and priorities, rather than overwhelming data. Observations from sample reports note opportunities to improve focus, transparency on performance and improvement, addressing equity, and including meaningful narrative.
More Related Content
Similar to Tasa presentation version 2
Tasa presentation version 2
- 1.
- 2. What research saysabout community-based accountability. John Cronin, Ph.D. Senior Director of Education Research NWEA
- 3. Damn little Thank youvery much.
- 4. What I hopeto accomplish. • Define community-based accountability. • Suggest that a “management letter” accompany your accountability report. • Describe three types of metrics your report should always include. • Offer observations based on my review of available community-based accountability reports.
- 5. What is accountability? Tobe “answerable” means that accountability is a dialogue between the stakeholders and the leaders of their schools. Part of that dialogue is understanding the goals and objectives of the community. Part of that dialogue is discussing how your schools are doing in reaching those goals.
- 6. What is accountability? Tobe “answerable” means that accountability is a dialogue between the stakeholders and the leaders of their schools. One part of that dialogue is understanding the goals and objectives of the community. Another part of that dialogue is discussing how your schools are doing in reaching those goals. The most important part of the dialogue is what you’re doing to improve performance based on this information, which is leading.
- 7. What conditions arerequired for community-based accountability to work? • Agreement on who the community is and your constituencies. • Agreement on what the school is answerable for. • Transparency in regard to how the school reports performance on these objectives. • Strategy to improve performance, and corrective action when strategy fails.
- 8. The management letter… Isan opportunity to show leadership. It is a conversation with your stakeholders about the performance of the school system. In it you explain your successes and failures, and discuss your key goals and strategies for improving the educational experience of students. Warren Buffet’s annual report to the Berkshire Hathaway shareholders provides a great model for content and tone.
- 9. The management letteris… Your narrative. It is the centerpiece of the accountability report The report on indicators is… Like financials in an annual report. They demonstrate transparency and, if well presented, allow readers to make their own judgments
- 10. The management letter… Isnot simply for the community… It requires the district’s leader(s) to think seriously about your work and communicate coherently about the state of the district and your strategy. It forces you to reflect and clarify your thinking.
- 11. Our bad newsfrom 2014 comes from (Burlington-Northern Santa Fe) and is unrelated to earnings. During the year, BNSF disappointed many of its customers. These shippers depend on us, and service failures can badly hurt their businesses. (transparency about failure) BNSF is, by far, Berkshire’s most important non-insurance subsidiary and, to improve its performance, we will spend $6 billion on plant and equipment in 2015. That sum is nearly 50% more than any other railroad has spent in a single year and is a truly extraordinary amount, whether compared to revenues, earnings or depreciation charges. (our strategy to address failure) An illustration of good tone from the Berkshire Hathaway Management Letter
- 12. Though weather, whichwas particularly severe last year, will always cause railroads a variety of operating problems, our responsibility is to do whatever it takes to restore our service to industry-leading levels. That can’t be done overnight: The extensive work required to increase system capacity sometimes disrupts operations while it is underway. (recognizing responsibility and tempering expectations) Recently, however, our outsized expenditures are beginning to show results. During the last three months, BNSF’s performance metrics have materially improved from last year’s figures. (reporting progress) From the Berkshire Hathaway Management Letter
- 13. In a yearin which most equity managers found it impossible to outperform the S&P 500, both Todd Combs and Ted Weschler (Berkshire Managers) handily did so. Each now runs a portfolio exceeding $7 billion. They’ve earned it. I must again confess that their investments outperformed mine. (Charlie says I should add “by a lot.”) If such humiliating comparisons continue, I’ll have no choice but to cease talking about them. Todd and Ted have also created significant value for you in several matters unrelated to their portfolio activities. Their contributions are just beginning: Both men have Berkshire blood in their veins. From the Berkshire Hathaway Management Letter
- 14. You know fromnews reports that the costs of poor performance on the state pension fund have passed to schools. This has stressed our budget and one consequence was that we froze hiring at the high schools. Because of retirements and transfers at some of the high schools, we did not replace AP teachers who retired or transferred and those courses were dropped. That was necessary for a year, but unacceptable for the long-term. This year our principals reviewed teacher assignments and are reinstituting courses in areas where we have a teacher with appropriate credentials. Because those teachers will be dealing with new content and larger class sizes, we’re funding summer training for them in both the AP content and in teaching strategies that better enable adaption of instruction with larger student groups. We can’t promise that this will entirely solve the problem, and our pass rate on AP exams may take a hit while our new AP teachers master their assignment. I’ll keep you apprised in my monthly newsletter as to how this is going. How this would read in a community-based accountability management letter
- 15. Observations from thereview of community- based accountability reports
- 16. Accomplishments! •The reports aredata rich! •They are reflect a broader picture of schools than standardized tests and school report cards. •They report on aspects of education that the community values. •They provide measurements of performance.
- 17. Comments from reviewof four Community-Based Accountability Reports 1.They are very detailed and somewhat overwhelming to process. 2.They put more focus on “achievements” and not much on “improvement”. 1. Is the business “profitable”? Is the school “performing”? 2. Is the business “improving”? Are the metrics headed getting better year over year? 3. Is the business “accelerating”? Is the rate of improvement increasing or decreasing? 3.Some focus more on “scoring schools” (which is what you were trying to avoid) then informing. 4.They generally don’t address questions of equity. 5.They generally don’t identify or address weaknesses in schools. 6.They generally don’t contain meaningful narrative. 7.They rationale underlying indicators is sometimes unclear (why did the district choose to measure the number of classrooms which use non-traditional delivery methods and why should the community care) 8.They often lack “leading indicators” that predict possible outcomes at the high school level. (e.g. – what are the signals of drop-out risk in fifth grade, how many students show those signals, and what is being done about them?) 9.They lack any mention of accountability. This business underperformed and here’s what we’ve done.
- 18. The number ofindicators in the reports is overwhelming Suggestion – The management letter should bring focus. Make sure every indicator reflects an outcome that the community values. The scorecard section of the report can be an appendix.
- 19. The reports focusmore on “scoring” schools than “informing” stakeholders. Suggestion – Establish goals rather than categories and report whether schools have “achieved the goal”, and whether they are “improving”, and “accelerating”.
- 20. Exemplary (3 points) Recognized (2 points) Acceptable(1 point) Unacceptable (0 points) Percentage of students reading at or above grade level in third grade 100-90% 80-89% 70-79% X Less than 70% From a community-based accountability report
- 21. Our first gradereading rate continues to be below goal and we were alarmed by the declines in reading that we were seeing in 2009 through 2013. Two years ago, we brought in a part-time specialist to help at-risk readers and teachers added more guided reading practice to the schedule. The last two years show we’ve improved, and that improvement is accelerating thanks to the hard work kindergarten and first grade teachers. To get closer to goal this year, we’re paying for teachers to take additional staff development and offering release time for teachers to observe and coach each other as they implement new practices. How this could be communicated in a management letter?
- 22. The reports areheavy on reporting status and light on reporting trends Suggestion – Report your key indicators longitudinally and focus on improvement and acceleration/deceleration in those indicators.
- 26. Grade Students Athletes% Fine Arts % 9 3276 1487 50% 2256 76% 10 3190 1215 41% 1656 56% 11 2967 932 31% 1233 42% 12 2795 8703 24% 1028 35% Percent of students participating in athletics and fine arts programs
- 27. The reports generallydo not address issues related to equity. Suggestion – Identify and include metrics in your report that show how you are addressing the particular needs of at-risk and minority populations and how you are addressing achievement gaps.
- 28. Calculus AB 20122013 Enroll ed % Taking % 3+ Enrolled % Taking % 3+ All students 347 257 74% 389 300 77% Enrollment and student performance in AP Calculus AB
- 29. Calculus AB 20122013 Enroll ed % Taking % 3+ Enrolled % Taking % 3+ All students 347 257 74% 389 300 77% Minority students 100 40 70% 130 50 70% Comment – Our goal has to improve both enrollment and minority participation in our AP program and our schools have accomplished that. We’re concerned that while minority participation increased, the proportion of minority students actually testing declined this past year. Our principals reported that cost of the exams has been a barrier for some students, so we’ve committed to paying the cost of the exam for all students identified as eligible for free and reduced lunch. Enrollment and student performance in AP Calculus AB
- 30. The reports reportsatisfaction but show little interest in the dissatisfied. Suggestion – Include follow-up questions to illuminate who and what may contribute to dissatisfaction and discuss how you are addressing those issues.
- 31. Very Satisfied Somewhat Satisfied Dissatisfied Unsure 2014 30%55% 9% 3% Survey Question – How satisfied are you with the education that the school system provides your child?
- 32. Very Satisfied Somewhat Satisfied Dissatisfied Unsure 2014 30%55% 9% 3% Under 30k 12% 60% 22% 6% 30-70k 20% 62% 15% 3% Above 70k 40% 50% 6% 4% Survey Question – How satisfied are you with the education that the school system provides your child?
- 33. Top five sourcesof dissatisfaction – high schools parents who responded somewhat or dissatisfied 1.Fees for athletic and extracurricular participation (15%) 2.Lack of communication/responsiveness by some teachers (12%) 3.Cutbacks in AP course availability (8%) 4.Poor teaching (6%) 5.Elimination of jazz choir program (4%)
- 34. The reports lackleading indicators that are predictive of success at the next level. Example – The report will include dropout rates, but lacks reporting on elementary and middle school indicators that are empirically tied to this metric and would be key to prevention.
- 35. The reports showlittle evidence that a poor result leads to some decision to change it.
- 36. Florida District Highly Effective Effective Needs Improvement Developing UnsatisfactoryVA Score Florida Ranking 1 44.4% 55.6% 0.0% 0.0% 0.0% 0.39 109 2 25.0% 75.0% 0.0% 0.0% 0.0% 0.37 121 3 90.9% 9.1% 0.0% 0.0% 0.0% -0.14 2802 4 60.7% 39.3% 0.0% 0.0% 0.0% -0.14 2797 5 81.2% 18.8% 0.0% 0.0% 0.0% -0.16 2831 6 37.3% 54.2% 1.7% 0.0% 6.8% 0.12 880 7 81.3% 18.8% 0.0% 0.0% 0.0% 0.22 402 8 41.7% 55.6% 1.4% 1.4% 0.0% -0.34 3274 9 52.2% 47.8% 0.0% 0.0% 0.0% 0.16 664 10 27.0% 66.2% 1.4% 0.0% 5.4% 0 1764 11 7.1% 72.6% 9.5% 10.7% 0.0% -0.08 2445 Teacher Evaluation Ratings in Eleven Florida Schools 2013
- 37. Thank you! E-mail: [email protected] OrganizationWebsite: www.nwea.org Contact: