Evidence

“Evidence is not everything that can be displayed; it is what can survive questions about how it was produced. ”

Sustainability organisations are surrounded by data. Satellite layers, supplier declarations, audit findings, household surveys, emission factors, photographs and dashboards can all be called evidence. Yet evidence is not a synonym for information. It is information considered in relation to a proposition. The first question is therefore not how much data exists, but what claim the data is capable of supporting.

The distinction matters because a dataset can be accurate within its own conditions and still be misleading when used for a broader conclusion. In 2015, the United States Environmental Protection Agency found that certain Volkswagen diesel vehicles contained software that detected laboratory testing and activated full emission controls during the test.

The cars could meet the standard in the laboratory while emitting nitrogen oxides at levels up to forty times the standard during normal driving. The test result existed. What failed was the relationship between the test condition and the real-world claim. Good evidence has provenance.

A user should be able to identify who produced it, for what purpose, through which method, during which period and under what incentives. A supplier declaration may be suitable evidence that a document was submitted. It is weaker evidence that the underlying practice occurred, and weaker still that the practice produced an environmental or social outcome.

The source does not become stronger because it enters a digital platform. Relevance comes before prestige. A peer-reviewed global model may be less useful for a farm-level decision than a carefully collected local observation.

A worker interview may reveal retaliation that payroll records cannot. Satellite imagery may identify canopy loss while field evidence explains whether the change was harvest, fire, storm damage or conversion. Different sources answer different questions. Evidence becomes stronger when methods are combined deliberately rather than ranked mechanically.

Quality also includes the conditions under which information was produced. Sampling determines who could appear. Measurement determines what could be seen. Data governance shapes who could correct an error. Incentives influence what respondents, auditors and suppliers may conceal. Independence can reduce conflicts of interest, but it does not repair an unsuitable method.

A third party can apply the wrong criteria consistently. Uncertainty is part of evidence, not an embarrassment to remove before publication.

Modelled estimates, remotely sensed classifications and self-reported practices carry different uncertainty. A credible account explains confidence intervals, error rates, missing observations, alternative explanations and the sensitivity of the conclusion to assumptions. A number presented without its limits can appear more authoritative precisely because the evidence has been simplified beyond recognition.

Contradictory evidence deserves a place in the file. Many sustainability systems preserve material that supports a conclusion and treat disagreement as noise. The stronger approach asks what evidence would make the preferred explanation less likely. If programme monitoring reports adoption while independent interviews report abandonment after incentives ended, the conflict is not a communication problem.

It is evidence about durability. Absence of evidence should also be handled carefully. A grievance database with no recorded cases may indicate good conditions, or it may indicate fear, inaccessibility or mistrust. A satellite model that detects no forest loss may have insufficient resolution for small clearings.

Not observing a condition is evidence of absence only when the method had a reasonable chance of detecting it. Qualitative evidence is sometimes dismissed because it cannot be reduced to a single estimate. Testimony, participatory mapping and case analysis can identify mechanisms, meanings and harms that aggregated indicators hide.

The discipline is not to pretend that one account represents a population, but neither is it to disregard an account because the harm is difficult to count.

The form of evidence should match the question. For practitioners, evidence should be organised as a chain of reasoning. State the proposition, identify the evidence that bears directly on it, explain the method and boundary, record uncertainty and contradictory findings, and show how the conclusion follows. A large annex is not a substitute for that logic.

The purpose of evidence is not to make a claim look technical. It is to make the claim open to informed challenge.

Practical application

Create a claim-evidence matrix before analysis or communication. For every proposed conclusion, record the subject, population, period, boundary, source, method, owner, uncertainty and evidence that could contradict it. Distinguish direct observations from proxies, assumptions and expert judgement. Maintain provenance and version history so another competent person can reproduce the reasoning.

Use triangulation where sources have different weaknesses, and require analysts to document why excluded or conflicting evidence did not change the conclusion.

Why it matters

Evidence determines whether sustainability decisions respond to reality or merely to what is easiest to record. Weak evidence can direct finance, scrutiny and remedy away from the people and ecosystems most affected while giving decision-makers false confidence.

Common misconception

Evidence is often confused with data volume or formal documentation. More records can increase precision, but they do not correct an irrelevant question, biased selection, unsuitable boundary or method designed to produce the preferred answer.

Connections

Data Quality addresses fitness for use. Sampling and Representativeness shape who and what the evidence describes. Uncertainty qualifies the conclusion. Substantiation links evidence to a public claim, while Assurance tests defined information against suitable criteria.

A question worth asking

What evidence would make your organisation revise its preferred sustainability conclusion, and is that evidence currently being sought?

Selected references

OECD. 2020. Mobilising Evidence for Good Governance. United States Environmental Protection Agency. 2015. Notice of Violation and Volkswagen Clean Air Act Enforcement Materials. National Academies of Sciences, Engineering, and Medicine. 2019. Reproducibility and Replicability in Science. Munafò, M. R. et al. 2017. A Manifesto for Reproducible Science. Nature Human Behaviour 1: 0021. Ioannidis, J. P. A. 2005.

Why Most Published Research Findings Are False. PLoS Medicine 2(8): e124.