Methods Summary

In this report, we detail our logic for focusing on organisations working to accelerate climate policy and emissions reduction activities in Australia, how we determined organisations to analyse in-depth, and the criteria we used to evaluate those organisations.

This report was last updated in December 2021.

Table of Contents

1. Introduction

2. How we determined which organisations to analyse in-depth

   1. Step 1: Literature review and research theory

   2. Step 2: Expert interviews

   3. Step 3: Identification of key ‘approaches’ to policy change

   4. Step 4: Use the Importance, Tractability, and Neglectedness Framework to determine priority approaches

   5. Step 5: Identification of a longlist of organisations for ‘shallow dive’ analysis

   6. Step 6: Narrow down to a shortlist of high potential organisations

   7. Step 7: ‘Deep dive’ analysis

   8. Step 8: Final recommendations

3. Endnotes

Introduction

There are well over a hundred organisations working to accelerate climate action in Australia. The Climate Action Network Australia alone has 125 member organisations. They range from large international non-government organisations (NGOs), through to small-scale local charities and community groups. Giving Green has tasked itself with identifying which of the organisations working to accelerate climate action in Australia stand to make the greatest impact with a marginal donation.

To make the task feasible, it was necessary first to narrow the scope of our research focus. We settled on policy change because effective public policy has consistently proven to be a key driver of technological, human, business and industry behaviour change. In Australia, there has been a distinct lack of climate policy leadership at a federal level since the election of the Liberal-National Coalition Government in 2013.

In the absence of federal government leadership, many of Australia’s state and territory governments have forged ahead, supporting a dramatic expansion of Australia’s renewable energy industry, assisting other sectors, like transport, to decarbonise, and adopting significantly more ambitious 2030 emissions reductions targets than the federal government.

Nevertheless, Australia remains a major emitter, and the world’s third largest fossil fuel exporter. Improving Australia’s climate policy, both government and corporate, therefore remains a high priority, having potential to deliver globally significant emissions reductions benefits and accelerate international efforts to address the climate crisis.

How we determined which organisations to analyse in-depth

To create recommendations of the highest impact organisations working to improve climate policy in Australia that would make the best use of a marginal donation, we took the steps outlined below. For a more detailed explanation of our research process, see our document Giving Green Australia: 2021 Research Process.

Step 1: Literature review and research theory

We first surveyed the academic literature on advocacy and policy change, and qualitative research methods. We decided to rely heavily on interviews with experts in the field of climate policy, advocacy, and philanthropy (hereafter ‘experts’) to inform our research. We further decided to use Grounded Theory[1] to inform our approach to the expert interview piece of our research.

Step 2: Expert interviews

We conducted hour-long semi-structured interviews with 23 experts around three topics: (1) barriers to improving climate policy in Australia, (2) strategies or methods to overcome those barriers, (3) effective climate organisations that would make the best use of a marginal donation. Interviews were transcribed, coded using the qualitative data analysis software, and analysed to uncover emergent themes.

Step 3: Identification of key ‘approaches’ to policy change

The interviews were transcribed and coded using the qualitative data analysis software, and analysed to uncover emergent themes. We synthesised the expert opinion into five archetypal barriers to improving climate policy in Australia and five key approaches for overcoming those barriers.

The major barriers identified were:

• the economics of fossil fuel extraction;

• state capture by the fossil fuel industry;

• the Liberal-National Coalition Government;

• the communications challenge; and

• the climate movement itself.

The key approaches identified were:

• ‘outsider advocacy’ – applying external pressure to change government policy;

• ‘insider advocacy’ – lobbying and other forms of insider influence designed to change government policy from within;

• ‘influencing elections’ – direct involvement in election-focussed campaigns;

• ‘changing the story’ – identifying and scaling messages and messengers that increase pro-climate literacy, concern and behaviour change; and

• ‘applying and expanding the law’ – bringing court cases aimed at delivering positive climate outcomes.

Step 4: Use the Importance, Tractability, and Neglectedness Framework to determine priority approaches

We surveyed 52 experts, asking them to:

1. order the barrier archetypes in terms of most to least important;

2. rank each key approach to policy change according to the Importance, Tractability, Neglectedness (ITN) Framework; and

3. name their top three climate organisations that could make the best use of a marginal donation.

We also convened two expert focus groups to discuss the survey findings and discover if any positions or ITN rankings changed significantly through facilitated conversation.

This process allowed us to identify ‘state capture by the fossil fuel industry’ and ‘the Liberal-National Coalition Government’ as the greatest barriers to accelerating climate action in Australia, and ‘insider advocacy’, ‘outsider advocacy’, and ‘changing the story’ as the highest priority approaches for delivering policy change at present. It also provided us with data on the organisations the expert community thought would make the best use of a marginal donation.

Step 5: Identification of a longlist of organisations for ‘shallow dive’ analysis

In parallel with Step 4, we used the expert interview data from Step 2 to develop a longlist of fifteen organisations to investigate further through ‘shallow dives’. Each shallow dive drew on desk research and further insights from the expert consultations. In particular, we assessed each organisation on the list using the following questions:

1. What are they and what do they do?

2. What have they accomplished or claimed to have accomplished?

3. What potential do they have for impact?

4. How strong is the organisation and what are their risks?

5. What is their financial need?

Based on that initial assessment, we narrowed the list down to twelve organisations that were asked to complete a short survey focussed on the assessment criteria. Several organisations also participated in hour-long semi-structured interviews. The final twelve ‘shallow dives’ can be viewed here.

Step 6: Narrow down to a shortlist of high potential organisations

We narrowed down our organisation longlist by first removing any organisations that did not have a major focus on the key approaches of ‘insider advocacy’, ‘outsider advocacy’, or ‘changing the story’. We narrowed the list further by carefully considering what organizations could do the most good with extra funding. In general, we believe that smaller organizations can make the most use of the marginal dollar, and therefore we excluded a number of large organizations that we believed were effective but also well-funded. The final ‘deep dive’ analysis list was identified based on the remaining organisations that received the highest number of nominations in our expert survey. Those organisations were: Beyond Zero Emissions, Farmers for Climate Action, and Original Power.

Step 7: ‘Deep dive’ analysis

Our ‘deep dive’ analysis of these final shortlisted organisations was undertaken based on the following elements:

1. Operational context

2. History of the organisation, structure, and budget

3. Activities, tactics and achievements

4. Room for additional funding

5. Theory of change analysis

6. Risks

Our assessment was informed by our ‘shallow dives’, and additional in-depth interviews and consultations with each organisation, expert interviews, and desk-top research.

Operational context

For each deep dive, we started with an assessment of the context in which the organisation is operating. Understanding that context is key to not only determining an organisation’s past effectiveness, but also its potential future impact and theory of change.

History of organisation, structure, and budget

We then undertook a detailed examination of the organisation’s history, looking at how and why it was founded, how its work has evolved over time, its structure, strength, and budget.

Activities, tactics and achievements

Next, we reviewed each organisation’s key activities, tactics, and achievements. We asked questions like: What kinds of work does the organisation do? What tactics and strategies does it use to deliver climate policy change? What has it accomplished to date? What evidence is there to support its claims?

Room for additional funding

One of our key assessment criteria was the impact a marginal donation would have on the organisation. As such, we analysed how additional funds would be used by the organisation, and the relative impact that funding would have on the organisation going forward. We also assessed the organisation's capacity to absorb additional donations.

Theory of change analysis

The centrepiece of our ‘deep dive’ assessments were in-depth analyses of each organisation’s theory of change. We constructed a diagrammatic theory of change for each organisation to describe how they seek to deliver impact. For each theory of change, we used the framework: inputs, activities, outputs, outcome, and impact.

After constructing each theory of change, we noted each of the major assumptions underpinning how the organisation’s activities and outputs lead to the desired outcomes and impact. We then undertook a critical assessment of how likely those assumptions were to hold. Finally, we categorized each assumption examined as most likely holds, may hold, or is unlikely to hold.

Risks

For each organisation we evaluated, our research team discussed what we thought were the greatest risks facing the organisation. We then tested those risks with the organisations to better understand whether they had identified those risks and how they plan to manage them. We also looked at broader risks to the organisation delivering the impact intended.

Step 8: Final recommendations

Based on the ‘deep dive’ assessments, final recommendations were made on each of the three organisations. We concluded that all three organisations are undertaking high impact work to improve climate policy in Australia. We further concluded that the impact of a marginal donation to each organisation would be high. We therefore recommended all three organisations for additional funding.

Endnotes

[1] See for eg. Charmaz, Kathy. 2008. “Constructionism and Grounded Theory.” In Handbook of Constuctionist Research, edited by J Holstein and J Gubrium, 397–412. New York.