Uploaded byCongressional Budget Office
CBO's Use of Evidence in Analysis of Budget and Economic Policies
The document summarizes CBO's use of evidence in analyzing budget and economic policies. CBO provides objective analysis to Congress on various topics, including federal spending, revenues, and the effects of legislation. CBO uses a range of evidence from research studies, data, and expert consultation to make projections and cost estimates. CBO aims to characterize the uncertainty around estimates and clearly explain its analytical methods and use of evidence.
More Related Content
Similar to CBO's Use of Evidence in Analysis of Budget and Economic Policies
CBO's Use of Evidence in Analysis of Budget and Economic Policies
- 1. Congressional Budget Office Presentationat the Annual Fall Research Conference of the Association for Public Policy Analysis & Management CBO’s Use of Evidence in Analysis of Budget and Economic Policies November 3, 2011 Jeffrey R. Kling Associate Director for Economic Analysis
- 2. Parts of thePresentation 1 - An Introduction: What CBO Does 2 - A Detailed Example: CBO’s Use of Evidence in Analyzing Proposals to Limit Costs Related to Medical Malpractice 3 - A General Overview: How CBO Uses Evidence 2
- 3. Part 1 An Introduction: What CBO Does 3
- 4. What is CBO’smandate? To provide the Congress with: ■ Objective, nonpartisan, and timely analyses to aid in economic and budgetary decisions on the wide array of programs covered by the federal budget; and ■ The information and estimates required for the Congressional budget process. CBO's reports contain no policy recommendations. 4
- 5. What are examplesof questions CBO is asked to address? CBO has given recent testimony on: The Budget and Economic Outlook: Fiscal Years 2011 to 2021 The Highway Trust Fund and Paying for Highways Potential Costs of Health Care for Veterans of Recent and Ongoing U.S. Military Operations Confronting the Nation’s Fiscal Policy Challenges Options for Changing the Tax Treatment of Charitable Giving 5
- 6. What types ofprojections does CBO make? CBO projects: ■ Federal spending and revenues under current law; ■ Effects on the federal budget of legislation under consideration; and ■ Economic and budgetary effects of policy options. 6
- 7. What does CBOproduce? CBO produces, in an average year: ■ About 100 reports, studies, testimonies, substantive letters, and issue briefs; ■ 500 to 600 formal cost estimates and related mandate statements posted on its Web site; and ■ Thousands of informal estimates delivered by phone or e-mail to Congressional staff. 7
- 8. What are formalCBO cost estimates? For every bill reported by a Congressional committee, CBO’s cost estimate assesses its impact on: ■ Federal spending subject to appropriation and mandatory federal spending, and ■ Federal revenues (incorporating estimates by the staff of the Joint Committee on Taxation for most revenue legislation). 8
- 9. What is CBO’scurrent-law baseline? Each winter, CBO prepares projections that: ■ Are the starting point for a year of official scorekeeping; and ■ Serve as a neutral benchmark so that policymakers can evaluate what budgetary changes would occur from enacting a given proposal. CBO’s cost estimates for revenues and mandatory spending are projected increments above or below that benchmark. 9
- 10. What time periodsdo CBO’s cost estimates usually cover? ■ CBO’s formal cost estimates focus on the next 5 or 10 years. ■ CBO notes whether proposals would increase deficits by more than $5 billion in any of the four subsequent 10-year periods. ■ Sometimes CBO characterizes budgetary effects more than 10 years in the future, such as for last year’s health legislation. 10
- 11. What additional informationdoes CBO provide on other topics? Information not directly related to the federal budget includes: ■ State & local budgetary effects, such as on Medicaid spending; ■ Economic effects, such as on output and employment; ■ Distributional effects, such as on after-tax income; ■ Private-sector effects, such as on health insurance coverage and premiums; and ■ Assessments of other policy goals, such as effectiveness of weapons systems. 11
- 12. Part 2 A DetailedExample: CBO’s Use of Evidence in Analyzing Proposals to Limit Costs Related to Medical Malpractice 12
- 13. What medical tortreform proposals did CBO analyze? CBO analyzed a package of proposals that would impose limits on medical malpractice litigation in state and federal courts by: ■ Capping noneconomic damages (also known as pain and suffering); ■ Capping punitive damages; ■ Modifying the statute of limitations; ■ Changing rules regarding collateral sources of income; and ■ Eliminating joint-and-several liability. 13
- 14. What was CBO’sestimate of the budgetary impact of tort reform? The proposals would reduce the deficit by about $57 billion from 2012 to 2021. Affected programs would include: ■ Medicare, Medicaid, the Children’s Health Insurance Program; ■ The Federal Employees Health Benefits program, TRICARE for Life; and ■ Subsidies for enrollees in health insurance exchanges. Reductions in the cost of health insurance for workers are projected to increase the share of their compensation that is taxed, yielding higher revenues. 14
- 15. How has CBO’sanalysis of tort reform proposals changed over time? Before 2009, CBO: ■ Focused on impacts of tort reform on malpractice premiums; and ■ Interpreted evidence about the effects of tort reform on the use of health care services as mixed. In October 2009, CBO: ■ Updated its estimate of the budgetary effects of tort reform; and ■ Drew upon newer research indicating that lowering medical malpractice costs tends to reduce use of services. 15
- 16. How did CBOestimate the effects of tort reform? CBO’s analysis had two main steps, estimating reductions in: ■ Spending from lower medical liability premiums; and ■ Use of health care services from lower malpractice costs. In CBO’s estimate: ■ Many states had already implemented proposed changes. ■ Therefore, a significant fraction of the potential cost savings had already been realized. 16
- 17. How did evidenceabout the effects of tort reform affect CBO’s cost estimate? CBO conducted its own research on the effects on malpractice premiums of changes over time in state laws. For estimates of the relationship between changes in malpractice premiums and changes in health care spending, CBO reviewed the research literature and concluded that: ■ Nationally, use of health care services would decrease by a small amount; and ■ In Medicare, use would decrease by a slightly larger amount. 17
- 18. What other evidenceabout tort reform did CBO report? ■ Some researchers who have observed a reduction in the use of health care services as the risk of litigation falls also found that reducing that risk produces a small increase in the mortality rate. ■ Another study found that reform of joint-and-several liability had positive impacts on health, but caps on noneconomic damages had negative impacts. ■ Other studies have concluded that tort limits have no effect on mortality or other measures of health. 18
- 19. Part 3 A GeneralOverview: How CBO Uses Evidence 19
- 20. How does CBOmake projections? CBO’s simulation models project forward using evidence from the past. In those models, CBO: ■ Does not predict future actions of federal legislators; and ■ Incorporates the most likely behavioral responses, when feasible, of: – Households and businesses, – Federal regulators, and – State, local, and foreign governments. 20
- 21. How does CBOexplain its methods for making those projections? CBO’s analyses include: ■ A description of the logic behind the estimates; and ■ Comparisons with other estimates, when available. Background papers describe modeling approaches, such as for: ■ Tax policy and income distribution; ■ Health insurance coverage; ■ Social Security’s lifetime benefits and taxes; and ■ Economic growth. 21
- 22. What types ofevidence does CBO use? CBO uses a broad range of evidence, depending on availability. The agency: ■ Reviews studies by others; ■ Reviews historical data for federal programs, as well as any data available from states for many key grant programs; and ■ Conducts original research using administrative records and survey data. 22
- 23. Whom does CBOconsult with in gathering evidence? Among others, CBO consults with people in: ■ Research organizations; ■ Relevant federal, state, and local government agencies; and ■ Businesses and national associations representing various groups. CBO welcomes and carefully evaluates information, especially primary data, provided by entities that will be affected by policies. 23
- 24. How does CBOassimilate evidence from various sources? CBO’s authors receive input from: ■ Teams of analysts within CBO; ■ The agency’s Panels of Economic Advisors and Health Advisors; and ■ External experts on particular topics, including program area experts within executive branch agencies. CBO’s teams meet frequently to reach consensus on interpretation of evidence and seek expert external reviews for studies and reports. 24
- 25. How does CBOevaluate evidence? CBO considers the current research literature in the relevant fields and asks: ■ How generalizable are research findings to a policy under consideration? ■ What sorts of biases might be affecting results of studies? ■ What is the middle of the distribution of possible outcomes? ■ How can uncertainty about the results be characterized? 25
- 26. What does CBOmean by “the middle of the distribution of possible outcomes”? ■ Formal cost estimates provide point estimates for budgetary effects to support the budget process. ■ When the range of evidence includes several point estimates, CBO generally uses a weighted average. ■ For those point estimates, CBO does not assume “no effect unless a null hypothesis of zero effect is rejected.” 26
- 27. How does CBOcharacterize uncertainty? Evidence is often insufficient to rigorously characterize uncertainty of an estimate. CBO frequently indicates its assessment of an estimate’s uncertainty qualitatively. For economic effects, CBO often: ■ Reviews modeling, sampling, and sources of uncertainty; ■ Judgmentally determines a plausible range for estimates; and ■ Reports sensitivity of results to sources of uncertainty. 27
- 28. Besides writing, howelse does CBO communicate about its projections and use of evidence? CBO’s analysts meet and talk with: ■ Congressional staff, ■ Federal agency staff, ■ State and local government representatives (generally through their national associations), ■ Panels of Economic Advisors and Health Advisors, ■ External experts, and ■ Members of the media. 28
- 29. What types ofinformation would CBO like to use more of in its cost estimates? Three examples are: ■ Critical reviews, ■ Experiments and demonstrations, and ■ Linked administrative-survey data. 29
- 30. What role docritical reviews play for CBO? When available, CBO gives greater weight to critical reviews that: ■ Weigh the strength of all forms of evidence in an area; and ■ Include studies that were not prominently published because of statistically insignificant results. 30
- 31. What role doexperiments and demonstrations play for CBO? When available, CBO gives greater weight to demonstrations and experiments that: ■ Use random assignment into different groups; ■ Shed light on the mechanisms causing responses; and ■ Address the difficulty of implementing new approaches. 31
- 32. What role dolinked data play for CBO? Linkages between administrative and survey data facilitate analysis of: ■ Behavioral responses to changes in policies; and ■ Changes in program enrollment and use based on sociodemographic characteristics. 32