.png){kind=small}
This page was last updated on October 26, 2023. The previous version of this page was published on December 09, 2022.
This page summarizes mistakes we believe Giving Green has made over time, and is inspired by several other organizations in the Effective Altruism movement.[1] Sharing these mistakes helps us embody our values of humility and transparency, reflect openly about ways we can continue to improve, and show readers how we have changed and developed over time. Questions, comments, or additions? Please contact us.
Table of Contents
Publication of “shallow dive” write-ups of organizations that are not part of our top recommendations
What did we do? From 2019-2022, it was our policy to publish on our website a short write-up of many organizations that we assessed but did not ultimately recommend as part of our list of “top nonprofits”. We thought this was important to show users that we had assessed a wide array of nonprofits, and to transparently show why we picked some organizations in the space and not others. We also hoped that our shallow dives could introduce users to excellent organizations that were working on strategies we thought were important beyond our top nonprofits.
Why was this a mistake? The publication of shallow dives caused a number of complications. A number of organizations disagreed with things we had written in our shallow dives, specifically any justification for not selecting them as part of our list of top nonprofits. Others provided feedback that they dislike the idea of having a write-up of their organization on our website at all if they were not among the top nonprofits. From their perspective, these write-offs were likely to dissuade potential donors. We also realized that very few users were actually reading our shallow dives, and heard feedback from written-up organizations that our shallow dives never drove donations.
What have we done about it? Starting in 2023, we decided to stop publishing shallow dives or any write-up or organizations we assessed but did not recommend. We also removed any previously-published shallow dives from our website. There is a trade-off associated with this decision, as removing this information goes against our value of transparency, meaning we provide less information for users of our Giving Green. However, we thought that on balance, it was better to adhere to the wishes of the organizations we assessed. We did not want to be in a position where a Giving Green assessment could hurt specific organizations, since we believe that many organizations are doing important work to fight climate change. We also wanted to make sure that organizations we were assessing could be open and honest with us without any downside risk. For users who would like more information about organizations we considered but did not ultimately recommend, we may be able to provide additional information 1:1.
Publication of information shared in confidence
What did we do? We post write-ups of organizations that include specific information about their plans and funding situation, including specific fundraising numbers and goals. This information is contained in our organizational deep and shallow dives. Generally, this information is gathered through 1:1 phone conversations or emails with the organizations we assess.
Why was this a mistake? A couple of times we have received (sometimes sharp) feedback from the organizations that specific information that we made public were told to us in confidence, and were not meant to be shared publicly. Fundraising numbers have been called out as especially sensitive. In all of these cases, we were not aware that any of this information had been told in confidence. But also, we were not as clear as we should have been in stating to organizations that we were planning to write public-facing reports about them, so we ultimately take responsibility for any non-public information.
What have we done about it? In cases where organizations had flagged confidential information, we immediately took down the write-ups and removed confidential information. Starting in 2023, we have instituted a policy in which we ensure that organizations have an opportunity to review any write-up that we post publicly, and that we remove any information told to us in confidence. We specifically flag any fundraising numbers to ensure that organizations are comfortable with anything we share. We also attempt to be much more clear with organizations under assessment that we plan to publish public information with our research findings.
Focus of Initial Carbon Offset Recommendations
What did we do? For our beta launch in 2019, our initial research focused on the offsets market, and looked for carbon offsets that appeared especially additional and cost-effective. We did this because we thought we were more able to analyze the effectiveness of offset projects, since they had relatively straightforward and verifiable theories of change.
Why was it a mistake? While we think our initial recommendation (of BURN stoves) was likely one of the most cost-effective carbon offsets, at that point we hadn’t assessed the broader landscape of climate philanthropy opportunities. In retrospect, we wish we had done this earlier, which we think would have caused us to instead prioritize investigating policy and technology change funding opportunities. We believe that our current recommended policy organizations are an order of magnitude more cost-effective than the best carbon offsets.
What have we done about it? While we still offer carbon offset recommendations, we do so as just one (non-preferred) option in our comprehensive business strategy. We think offset recommendations are still valuable because many businesses are committed to net-net accounting goals, which necessitate carbon offsets. We redesigned our website to explicitly separate our top recommendations from our offset/removals recommendations (which are targeted at businesses), and also include top-line text on all offset/removals recommendations directing unrestricted donors to our top recommendations.
As noted in our 2022 updates to Giving Green's approach and recommendations, starting in 2022 we have taken a more systematic approach to determining our research priorities. We also have plans to further refine this approach in 2023.[2] As we continue to improve our process for assessing the broader climate change philanthropy landscape, we believe this will result in research and recommendations focused on the most promising opportunities.
Differentiating Between Offset and Policy Donors
What did we do? In 2020 we started making recommendations for policy organizations, as well as offsets. At the time, we labeled offset recommendations as “verifiable, high certainty reductions in emissions,” and the policy recommendations as “big and riskier bets.” However, we didn’t take a stance on which approach had the highest expected value, and therefore would be relatively more cost-effective. We thought that some individuals or organizations would have a strong preference for the high-certainty philanthropic option offered by offsets, and wanted to give options in this area to maximize overall giving.
Why was it a mistake? In 2021 we more closely investigated the relative effectiveness of offsets versus policy advocacy, including developing cost-effectiveness models. This additional research led us to strongly believe that policy organizations can offer a much higher expected return (~10x) over even the most cost-effective carbon offset giving opportunities. We therefore think it was a mistake to not encourage donors to donate to our policy recommendations.
What have we done about it? In late 2021 we overhauled our website, explicitly stating that we thought that donating to our top nonprofits was our primary recommendation for individuals or organizations without constraints. In 2022, we’re further segmenting our audience with the release of our business strategy. Our offset and removal recommendations are explicitly targeted at businesses, and are one part of a more comprehensive suite of business recommendations. We include top-line text on all offset/removals recommendations directing unrestricted donors to our top recommendations.
Sunrise Recommendation
What did we do? In 2020, we recommended the Sunrise Movement Education Fund as one of our top policy nonprofits. We did this because we assessed that Sunrise had a strong track record of ensuring climate change was high on Democratic politicians’ agenda, and we thought it was well-positioned to push for pro-climate legislation given a Biden presidency and a Democratic congress. At the time of our recommendation, we were explicit about our uncertainty, and labeled Sunrise a “High-Potential” organization. In our recommendation, we stated “High-potential organizations are those which have not yet fully demonstrated their effectiveness, but which are attempting new, innovative, and promising approaches to policy change. Such organizations show important intermediate successes and effective organization and practices, and they make use of an approach backed by rigorous research and evidence.”[3]
Why was it a mistake? There are a couple of ways to reflect about a “mistake” for a recommendation, especially one made under recognized uncertainty. If we believe that the organization was not as effective as we expected, it could be that the recommendation was still “correct” in expectation, but unknowable changes in the state of the world resulted in it not being effective. It could also be that our assessment of Sunrise’s potential effectiveness was incorrect from the beginning. It’s difficult to disentangle these different interpretations with any certainty. However, we’ll explore both below.
First, we do believe that donations to Sunrise in late 2020 and 2021 were less effective than we had expected. While there were ongoing, tenuous discussions over large pieces of climate legislation (initially the Build Back Better act), we did not see Sunrise as a major player in pressuring on politicians to pass the bill, or ensuring that climate remained centered in the bill. We heard from staff at Sunrise that the pandemic had affected their ability to organize, and they also spent a lot of time redeveloping their strategy. Given this, we ultimately don’t think that donations we directed to Sunrise in 2020 and 2021 were among the most impactful options.
The more difficult question (and the one we think is most worth classifying as a mistake) is whether we made a bad judgment call in making the recommendation, given the information we had at the time. We believe there are arguments in either direction. First, subsequent events in US politics (specifically passing the Inflation Reduction Act with only Democratic support) provided support for Sunrise’s core theory of change. We believe it’s unlikely that Democrats would have spent as much political capital passing a climate-focused bill without the prior work of Sunrise. We therefore think that much of the assessments we made on the effectiveness of Sunrise pre-2020 were valid, and the cost-effectiveness estimations we made from that period were plausible.
On the other hand, when we made the recommendation, there was criticism suggesting that we had made a judgment error. One critique was this comment on the Effective Altruism forum, which included an argument on why the commenter believed our recommendation was incorrect. There are parts of this argument that we still disagree with, such as the argument that Sunrise had an unacceptably large chance of having a negative impact. (We discuss and model this possibility in our current deep dive of Sunrise.) However, with the benefit of hindsight, there are other criticisms that resonate strongly with us. For instance, consider the following quote from the forum comment:
“I would probably agree with the argument that the rise of progressive climate activism over the past four years has been net-positive…However, this does not at all mean that we should donate to TSM [Sunrise] at this point. I agree TSM could have been a great philanthropic bet 4 years ago.”[4]
This comment tracks closely with our current view of the situation, which is that Sunrise’s work before and leading up to the 2020 US elections was highly effective, ensuring that climate became a democratic priority and eventually leading to the passage of the IRA. However, post-2020, Sunrise did not play a key role in crafting and pushing approval for subsequent climate bills.[5] Therefore, we think marginal donations to Sunrise in the period after our recommendation likely had little impact. Given that others made the judgment call (that we ultimately believe was correct) that Sunrise would be far less effective post-2020, it would be reasonable to view our judgment call in 2020 as a mistake.
What have we done about it? In 2021, we removed the Sunrise Movement Education Fund from our list of recommendations, and produced an updated deep dive explaining our reasoning.
In subsequent years, we’ve had to make similar difficult judgment calls with recommendations as we did with the Sunrise Movement Education Fund. With a larger research team, we’ve been able to gather more opinions and more data about approaches, sectors, and specific organizations. We hope that these process improvements have allowed us to continually improve these judgment calls, though we don’t believe it’s possible to entirely remove the risk of more mistakes in judgment with our recommendations.
Confusion among donors as to whether they donated to Giving Green’s operations or recommendations
What did we do? Giving Green had a donation option on our website for which all funds were used for Giving Green’s operations (research and communications). In early versions of the website, this was accessed with a link titled “Give”.
Why was it a mistake? For some early versions of our webpage, we received feedback that donors were confused if our “give” button resulted in donations to Giving Green’s recommended organizations or to Giving Green, itself. Although the text on our original donation page was meant to be clear that donations coming from this page would support Giving Green, we don’t think all donors understood this.[6] We are concerned that there could have been donations coming through this portal that were meant for our recommended organizations and instead went to us.
What have we done about it? We have since made multiple changes to ensure that users understand where their donations are directed. Currently, the donation button on our website states “Support Us” instead of “Give”, and the text on our donate page is much more explicit that this refers to donations to support Giving Green’s research. In November 2022, we launched a new giving portal that allows donors to give donations explicitly to our recommended nonprofits or to Giving Green itself, which we believe has eliminated any potential remaining confusion. We no longer receive feedback that donors are confused.
Endnotes
[1] For example, see Our Mistakes | GiveWell and Our mistakes - 80,000 Hours.
[2] See “Our plans for 2023” section of 2022 updates to Giving Green's approach and recommendations.
[3] See archived version of our Sunrise Movement report. Access date: Nov 24, 2020.
[5] For additional commentary, see our The Sunrise Movement: Deep Dive, last updated November 2021.
[6] “Your gift enables us to continue improving the effectiveness of action within the climate community. Our work is entirely independent and supported by people like you! By clicking "Donate" you will be re-directed to an IDinsight webpage that will accept your payment details. Giving Green is an initiative of IDinsight, which is a registered 501(c)3 in the US. Please note the the Donate page below supports our research efforts. If you would like to support the organizations we recommend, please use the donation links within our recommendations.” Archived version of Giving Green: Give. Access date: January 16, 2021.
Comments