Our Monitoring and Evaluation
We measure what changes, not just what we do. Climate Cardinals measures progress toward the three goals in our Theory of Change: Youth Leadership, Community Resilience, and Language Justice.
What follows is what we measure, how we check it, how we protect the people behind the data, what our evidence has already changed, and what we are still building.
The framework
Three instruments carry the work from intention to evidence.
-
Defines the change
Theory of Change
Sets out the change we seek and how our programs contribute to Youth Leadership, Community Resilience, and Language Justice.
-
Makes it measurable
Impact Matrix
Translates those goals into measurable indicators, while program-level balanced scorecards set quarterly targets across financial, customer, internal process, and learning perspectives.
-
Makes it public
Live Impact Dashboard
Brings key indicators together in one place, updated from approved program data, so progress is visible outside the organization as well as inside it.
What we measure
Every program answers the same four questions.
Each program maintains a balanced scorecard on the Kaplan and Norton model, the management framework that fixes four perspectives: financial, customer, internal processes, and learning. Every perspective carries strategic objectives, the measures that test them, a target for each quarter, and the initiatives meant to get us there, and every measure connects back to an outcome in our Theory of Change.
-
Financial
Are we funded to deliver, and funded sustainably?
We track funds raised and distributed against quarterly targets, alongside a diversification index, one minus the largest funder’s share of total funding, so that reliance on any single source is visible rather than assumed.
-
Internal processes
Are we running well enough to grow?
We track participation, reach, retention, program delivery, and partnerships, plus the health of the machinery behind them: manual workload reduced through automation, bottlenecks identified and resolved, and how consistently programs report their activity.
-
Learning
Are we getting better at this?
We track how much of what we learn makes it back into the work: materials revised after feedback, operating procedures and guidebooks documented and rated, peer-learning sessions held, and training completed across the network.
-
Customer
Are the people we serve better off?
Customer is the Kaplan and Norton term, and we keep it rather than soften it. For us it means the young people, chapters, and partners our programs exist to serve, and holding to the word keeps the question honest: it asks whether they are better off, not whether we were busy.
We track outcomes such as changes in knowledge, confidence, behavior, access to information, local climate action, and institutional practice, together with satisfaction and how far participants pass their learning on to others.
Where possible we also measure additionality, whether change would have happened without our involvement, and longevity, whether it continues over time.
How we measure
Three kinds of evidence, because no single kind is enough.
Program data tells us what happened. Surveys tell us what changed. Qualitative evidence tells us how and why, particularly across different linguistic and community contexts.
-
Continuous
Program data
Collected continuously through delivery.
- Participation, activity, and digital engagement records
- Quarterly chapter reports and grassroots partner reports
- Financial and operational data, including funding, opportunities, and resources distributed or mobilized
-
By purpose
Surveys and assessments
Different programs use different instruments, grouped by what each is for.
- Change over time. Pre- and post-program surveys using harmonized questionnaires and compared across matched respondents, measuring knowledge, skills, confidence, and attitudes, plus follow-up surveys that test whether outcomes continue beyond the program period
- Experience. Satisfaction and NPS surveys to monitor participant experience and identify areas for improvement
- Program-specific. Partner impact surveys on how translated resources are used and whether they remain in use; chapter reporting on local activities, leadership, partnerships, preparedness, and community outcomes; fellowship assessments and alumni follow-ups on networks, opportunities accessed, and longer-term pathways
-
Beyond numbers
Qualitative and long-term evidence
Numbers do not capture every form of impact.
- Interviews and open-ended feedback
- Case studies drawn from chapters, partners, and fellows
- Follow-up data gathered after programs conclude
Data quality
We hold our numbers to a rising standard.
Not everything that happens becomes a data point. Consistent inclusion rules are applied before anything reaches a report or the dashboard, and the same rules apply to internal and public figures. We are extending those rules across our full body of reporting as we go.
-
What counts
What counts as a project
We check chapter reports and track only high-quality, completed project reports. Activities still in progress are excluded until they are finished, so our figures reflect work delivered rather than work started.
-
What we exclude
What we deliberately exclude
Workshop and conference attendance is excluded from project reports unless a substantial, generative activity is associated with it. Attendance on its own is not treated as a result.
-
Cleaned weekly
How data reaches the dashboard
Data is cleaned weekly and entered into our Weekly Programmatic Audits. Those audits are the single source for the Live Impact Dashboard, so the figures we publish and the figures we manage against are the same numbers.
Where we are in this transition.
Climate Cardinals has grown quickly from a student-led organization into a professional one, and our methods have matured alongside it. We are centralizing our datasets and strengthening the methodologies behind our headline numbers so that every figure rests on the same basis. Where a method has improved, we restate the figure to match it.
We publish the sample size alongside survey results, as in the Fellowship pilot below, and will do the same for every survey we report, so the strength of the evidence is always visible next to the finding.
How we learn and adapt
Measurement only counts if it changes decisions.
Evidence moves through a fixed cycle each year, beginning with goal-setting each December, so findings reach the people making decisions about program design, funding, staffing, partnerships, and strategy.
-
Every December
Set goals
We set organizational and program goals for the year ahead each December, and the quarterly targets on every balanced scorecard follow from them.
We hold longer-term goals as well, but we plan against a one-year horizon by choice. The organization has grown quickly out of being student-led, and until we hold several years of comparable data, annual goals are the ones we can set with confidence and be held to.
-
Continuous, reviewed quarterly
Measure
Program teams monitor their balanced scorecards throughout the year and investigate significant changes in performance. Each scorecard is then formally reviewed every quarter against the targets set for that quarter.
-
Twice a year
Review
Twice a year Climate Cardinals conducts an organization-wide performance review across all of our data, not only what sits on the scorecards. We look at what is meeting expectations, what is not, why results may be changing, and where additional resources or attention are needed.
-
Following each review
Adapt
Evidence informs decisions about program design, funding, staffing, partnerships, and strategy. We use what we learn to continue what works, improve what can be stronger, and scale or conclude initiatives based on evidence.
Evaluation in practice
What the Fellowship pilot changed.
Our first formalized Fellowship ran for an academic year and was evaluated through matched pre- and post-program surveys, fellow feedback calls, speaker feedback calls, and learning calls with peer organizations. This is what that evidence produced, and what we did about it.
What we found
- +34ppIncrease in fellows’ confidence accessing resources, such as grants and implementation partners, for their initiatives
- +24ppIncrease in fellows’ confidence taking climate action
- 100%Of fellows would recommend the Fellowship to someone else
Based on matched pre- and post-surveys from all 11 fellows who completed the Fellowship, a full matched response.
What we changed
- Time zones and internet access determined who got the full experience from a single live session, so content delivery, speaker access, peer connection, and applied practice are being distributed across separately administered parts of the program
- Community building needed more structure, so peer connection and project responsibilities now begin earlier in the year
- Fellows wanted clearer pathways beyond graduation, so they now join Climate Cardinals teams as full members, working toward a capstone or portfolio output
Why we published it.
We released the full post-mortem so that other organizations in the climate and youth development sectors do not have to build their programs from scratch. Excellence in programming is not the absence of imperfection. It is the discipline of listening, iterating, and designing the next version with greater care.
What we are still learning
Strong evaluation means naming what we do not yet know.
Measuring long-term change across many countries, languages, and communities is genuinely difficult. These are the three areas where our evidence is thinnest, and where we are actively building it.
Long-term outcomes
We are improving how we track
- Fellowship alumni and their longer-term education, career, and leadership pathways
- Whether chapter initiatives and local partnerships continue after their initial activities
- Whether translated resources remain in use six months or longer
Behavior and institutional change
We are developing stronger evidence on
- Whether access to translated information changes participation, preparedness, or community behavior
- Whether outcomes would have occurred without Climate Cardinals’ involvement
- Whether institutions adopt our research, tools, and recommendations in their policies, practices, or funding decisions
Community resilience
We are strengthening how we
- Measure changes in local capacity and preparedness over time
- Understand our contribution to resilience alongside other actors
- Capture locally defined outcomes that standardized global indicators may miss
Explore our evidence
Read the results, the methods, and the lessons.
-
Theory of Change
How our programs contribute to Youth Leadership, Community Resilience, and Language Justice.
Theory of Change -
Reports, memos, and post-mortems
Our research, program findings, and organizational learning, published as white papers, memos, and post-mortems.
Explore Our Resources -
Fellowship pilot evaluation
A deeper look at our Fellowship methodology, results, and lessons, including what did not work.
Fellowship Evaluation