Giving Green's Approach to Policy Change Recommendations
Overview: In this report, we detail our reasoning for investigating US federal policy, how we determined organizations to analyze in-depth, and the criteria we use to evaluate organizations. We also describe why we do not rank organizations against one another. Namely, the causal change between activism and/or policy advocacy and CO2 removal is long and complex and it is unclear what theory of change is most effective in driving policy change.
In 2020 and 2021, our team allocated our research resources on US federal policy. Our decision reflects two considerations: expertise and scale. Overall, our team is most familiar with climate policy in the United States, so we wished to put our own expertise to use in an area familiar to us before moving on to other areas. This decision was in part driven by the small size of our research team and our limited resources as a new organization. In addition, we believe that US national-level climate policy is important for considerations of scale given that the US is among the world’s largest greenhouse gas emitters in total volume and also has one of the world’s highest rates of greenhouse gas emissions per capita. Furthermore, to date the US has not adopted strong, nation-wide climate policy under a comprehensive framework. Finally, the US is the world’s largest funder of research and development and could therefore play a crucial role in developing key technologies needed to fight climate change. For these reasons, we believe that any successful efforts in shifting US climate policy could result in high levels of avoided greenhouse gas emissions.
In future years, we hope to expand our scope to look at both sub-national organizations, as well as those outside the US. (We look forward to releasing work on Australian policy in late 2021.)
How we determined which organizations to analyze in-depth
To create recommendations of organizations working to change US national climate policy, we took the following steps:
Step 1: Identifying key “approaches” to policy-change
We first created a list of “approaches” most commonly used by organizations working towards policy change. These included activism (grass-roots mobilization focused on political change), insider policy advocacy (lobbying and other forms of insider influence), influencing elections (direct involvement in campaigns and election efforts), litigation (bringing cases to court with potential for positive environmental outcomes), and communications (educating the public and advertising climate issues).
Step 2: Use the Importance, Tractability, and Neglectedness Framework to determine specific approaches to focus on for addressing US national climate policy
We believe that each of the approaches listed above is an important part of working towards policy change because policy change rarely happens through only one approach alone. At the same time, some approaches may be in less need of additional funds than others or have a higher potential marginal impact of additional funds. For this reason, we adapted and applied the Importance, Tractability, and Neglectedness framework to determine which approaches we thought would be the most useful for us to focus our work on. The full methodology and results of this exercise are reported in our document How We Determined Our Research Priorities for Policy Change. Through this exercise, we determined that activism and insider policy advocacy were likely to have the largest marginal effect, and we focused on these areas for our 2020 and 2021 research. We hope to explore organizations in the other domains in future years.
Step 3: Desk research and expert interviews to create a “long list” of organizations
To identify organizations that use activism and insider advocacy to impact climate policy, we conducted our own search for organizations and also reached out to experts in academia, policy, and philanthropy to seek their recommendations. Through this process, we created a longlist of organizations that we then filtered based on whether they emphasized the use of activist or insider advocacy methods in their work. Our longlist is not necessarily comprehensive, and we plan to add to it as we learn about new organizations doing great work.
Step 4: Narrow down the long list to a shortlist of high potential organizations that we want to research further
After creating our longlist of organizations, we created a shortlist of organizations on which we wanted to conduct further research. This was done by first rapidly scanning each organizations’ websites to understand if they were indeed working to affect US national policy through activism or insider policy advocacy and to have a better sense of the scope of their work. At this step, we excluded organizations that only conduct their work via 501(c)(4) funds because Giving Green is only able to provide recommendations for non-political organizations.
During this step, we also decided not to conduct further research on the so-called “Big Greens”: Sierra Club, National Resources Defence Council, Environmental Defense Fund, and the World Resources Institute. Although these large environmental organizations fit the definition of the organizations we are looking to recommend and many experts we consulted spoke highly of their activities, we decided not to pursue further research on them for a couple of reasons. First, all the Big Greens have a wide environmental agenda, and generally it is not possible to donate specifically for climate change activities (see endnote 1). Second, the Big Greens tend to be very well-funded, which suggests that donor dollars may have higher leverage in smaller organizations. (Though we admit that just because an organization is well-funded does not necessarily mean it has low marginal effectiveness of additional donations.)
Step 5: Conduct shallow-dive analysis of each of the short-listed organizations
For each of the organizations included on our shortlist, we undertook brief desk research to assess whether we thought each organization would be a promising candidate for further in-depth analysis by our team. In particular, we assessed each organization on the list using the following questions:
In some cases, we were unable to satisfactorily answer all of the above questions for each of the organizations we reviewed. In these cases, we noted which questions were outstanding and used this to think about whether a deep-dive analysis would be useful in order to seek answers to outstanding questions.
Step 6: Select organizations for deep-dive analyses
Based on the findings of our shallow-dive analyses, we identified what we thought were the most promising recommendations based on our framework. These organizations were those that we believed are doing relevant work, had demonstrated meaningful accomplishments, and showed evidence of a strong organizational structure and potential financial need.
For these organizations, we conducted a comprehensive “deep dive” to gather more information with which to make a recommendation.
How we analyze selected organizations (Deep Dive)
To begin analyzing our selected organizations, we first reviewed the literature on what determines the effectiveness of activism or insider advocacy work, specifically around climate. The results of this work are presented in our sector overview documents (Policy Advocacy and Activism).
After this, we turned our attention to the organization(s) selected for deep dives. To assess organizations on our shortlist, we read a variety of sources on each organization both from organizations themselves and from external authors writing about them, reviewed literature relevant to the organization’s work, and spoke to internal and external experts to hear their take on the organization. We employ seven criteria to assess organizations, as detailed in the table below:
History and accomplishments to date
For each deep dive, we first analyze the organization’s history and what it has accomplished to date. In particular, we pay attention to the reasons why the group was founded, how its work has evolved over time, and what verifiable accomplishments it has had to date. During this review, we also seek evidence on whether the organization has successfully adapted to changing circumstances that may affect its work. We also speak with multiple people who work in the climate policy sector to validate claims of accomplishment from each organization.
To assess organizational strength, we primarily rely on discussions with leadership as well as volunteers or employees in entry level roles to identify how the organization is structured, why it is structured like it is, and whether this structure seems to be effective or ineffective.
Activities, theory of change, and evidence
We review each organization’s primary activities that make up its work. These can include activities like conducting seminars, organizing demonstrations, writing policy briefs, or hiring lobbyists among many others.
After noting each of the key activities organizations do in practice, our team constructs a theory of change that describes how we believe the organization seeks to impact climate policy through its activities. Although we sometimes rely on organizations’ own theories of change during this stage, we often seek to build our own that represents a more fine-grained understanding of how their specific activities impact our target outcome of reduced greenhouse gases in the atmosphere rather than other target outcomes that organizations may create themselves. After constructing the theory of change, we note each of the assumptions in the theory and assess how likely it is to hold based on the organization’s past record of success and any available academic or policy literature.
We further assess each organization’s activities by conducting a cost-effectiveness analysis that estimates how much would need to be donated to the organization to remove one metric ton of CO2 from the atmosphere. We develop these models using established research on the potential impact of the organization’s activities and/or conducting our own back-of-the-envelope calculations. Because our models contain a considerable amount of uncertainty, we only use their outcomes as rough indicative estimates and are just one part of our selection process. For example, we would not select an organization solely because a model suggested that they were highly cost-effective; the organization would need to be strong in other areas as well in order to indicate its ability to carry out its proposed work.
For each organization we evaluate, our research team discusses what we think are the greatest risks to the organization achieving its goals among other concerns. We then check in with individuals within the organization to better understand how the organization understands and accounts for its risks.
Room for additional funding
Because we are interested in each organization’s marginal impact, we assess whether it is likely that additional funds would be used productively by the organization and would increase its ability to conduct useful work. Often, we rely on interviews with leadership and finance team members at the organization to assess the organization’s room for additional funding.
Our final recommendation balances all of our findings across these criteria. Our team assess the benefits we believe additional donations of money or time to an organization are likely to have, and then we determine a final recommendation through team consensus.
Why we don’t rank our recommendations against each other
Giving Green does not currently rank our policy change recommendations against each other.
The first reason we do not rank organizations against each other is that all organizations that work to change US national-level policy are engaging in work that is fundamentally uncertain. The causal chain from activist or insider advocacy work to final policy change to reductions in atmospheric greenhouse gas concentrations is long and complex in all cases. This does not prohibit us from identifying promising organizations doing work that we believe has a high likelihood of effectiveness. This does, however, make it difficult to rank different highly-promising organizations and their work against one another with a high degree of precision. We believe that exercises which claim that a certain amount of money donated to an organization has a specific impact on averted CO2 emissions are unlikely to be certain or rigorous enough to be truly informative. Such approaches may unhelpfully misrepresent the uncertainty in policy processes, where one approach or program may quickly become more or less promising as political alignments shift and public opinion changes.
Multiple, contested theories of change
Our second reason for not comparing organizations engaged in disparate approaches to changing policy is that there is no consensus among either the public or experts on what the most feasible approaches to changing US national climate policy are. Overall, we have noted three broad theories of change about the best channels for influencing public policy that have emerged in the last decade:
Bipartisan market-based consensus: According to this theory of change, successful climate policies are policies that appeal to both Democrats and Republicans. Most adherents to this theory advocate for carbon taxes and other carbon pricing policies. Critics of this theory point to the failure of cap-and-trade to pass in Congress in 2009-10 despite seemingly favorable political circumstances.
Mobilization on the political left: Under this theory of change, successful climate policy only emerge on the left, and so climate policies must adopt framings that appeal to the left such as regulatory standards, environmental justice, and investment in public programs. Many adherents of this theory support the idea of a Green New Deal. Critics worry that this approach may undermine bipartisan legislation.
Technology innovation and regulation: According to this theory of change, absent large-scale policy frameworks, the most effective approach to climate policy is through seeking bipartisan consensus to promote frontier technology (such as carbon capture and advanced nuclear), along with tweaking regulations. Adherents of this approach are pessimistic about large-scale policies like carbon pricing schemes or a Green New Deal and believe that technological change is the only feasible way to fight the climate crisis. Critics believe that this approach is too marginal in its impact, and that it does not more quickly enough to decrease greenhouse gases.
Each of these theories of change are accompanied by good, evidence-based arguments both for and against their assumptions. Which of the above theories of change is best is partly a matter of evidence but also partly a matter of political and moral commitments as well as beliefs about aspects of contemporary politics for which little definitive evidence exists, such as whether bipartisan consensus is likely to emerge in the near future or not. In addition, there is little evidence on whether and when the different theories may be compatible with each other (they are not necessarily in competition). In any case, decisions about these different approaches are a matter of both judgment and evidence. Rather than taking a stand on which of these theories of change is most likely to hold, we at Giving Green wish to provide well-researched evidence on organizations that could productively employ additional funds and are working in an evidence-based manner across these different theories of change (see endnote 3).
There are some counterexamples to this. For instance, it’s possible to donate directly to the Sierra Club’s “Beyond Coal” campaign.
Overall, revenue is a poor-quality indicator of room for more funding, however our team chose to include it in our analysis at this stage and consider it along with number of staff and diversity of focus areas to assess neglectedness. No organizations were dropped from consideration due to revenue level alone.
At present, we do not have any recommendations for organizations working to advance bipartisan market-based consensus policies, as there is currently little support for climate policy among the political right. However, we hope that this will change in the future, which will allow a more promising pathway to success for organizations advocating for bipartisan solutions.