Basics
What is an evidence gap map?
An evidence gap map is a structured overview of the research on a topic: one dimension, often interventions, runs down the rows, another, often outcomes, runs across the columns, and each cell shows the studies that fit both.
A short definition
The International Initiative for Impact Evaluation (3ie) developed and spread evidence gap maps in international development, building on earlier evidence-mapping methods. The Campbell Collaboration now publishes them too. Campbell's guidance (White and colleagues, 2020) says maps follow the same principles as systematic reviews: the scope is defined up front, the search is comprehensive, studies are screened against explicit inclusion and exclusion criteria, and each included study is coded systematically.
The result is usually a matrix. In the common layout, interventions form the rows and outcomes the columns. A mark in each cell, often a bubble or a shaded square, shows how many studies address that combination. Other features, such as population, region or study design, become filters or a third dimension.
How it differs from a systematic review and a scoping review
The three products share methods but answer different questions.
- A systematic review asks a focused question, such as whether an intervention works for a defined group, and synthesises the findings of the included studies, sometimes in a meta-analysis. The Cochrane Handbook describes the methods in detail.
- A scoping review describes the extent and nature of the research on a broader topic, often in tables and narrative. PRISMA-ScR (Tricco and colleagues, 2018) is its reporting guideline.
- An evidence gap map also covers a broad topic, but organises the studies in a predefined framework and shows them visually. It usually describes the evidence rather than synthesising what it found. Some maps add a confidence rating for each included systematic review, as 3ie's do. That is still not the same as pooling results.
The terms overlap in practice. "Evidence map", "systematic map" (the term in the Collaboration for Environmental Evidence's guidelines) and "evidence and gap map" name related products. What matters for a reader is the method section, not the label.
What a dense cell tells you
A dense cell tells you that many studies address that combination. It does not tell you that the intervention works, that the studies are good, or that they agree. Ten small studies from one research group can fill a cell as much as ten large trials from different countries.
Density also depends on the coding rules. If a study is coded under every outcome it measures, a study with many outcomes appears in many cells. That is correct, but it means a cell count is a count of studies per combination, not a share of the whole.
A map shows where studies are. What they found is a separate question.
What an empty cell means, and what it does not
An empty cell means that no study in this collection was coded to that combination. Before you call it a gap in research, rule out the other explanations:
- the search did not reach the studies (databases, languages, dates or grey literature not covered);
- the eligibility criteria excluded them, for example by study design;
- the studies were coded to a neighbouring category because the definitions overlap;
- the combination makes little sense and nobody would study it.
3ie's maps highlight two kinds of thin spot: cells with few or no studies, and cells with many impact evaluations but no recent high-quality systematic review. The first points to a need for new research, the second to a need for synthesis. Either is a statement about this map's search and criteria, and should be written as one.
Typical uses
- Research funders and commissioners look for areas where little has been studied before they set priorities.
- Review teams check where a systematic review is feasible, because enough studies exist, and where it would be premature.
- Policy and practice teams get a quick view of what is known on a broad topic and where to look for detail.
- Researchers starting a project, including students, scope a field and position their own question.
Limits to keep in mind
- No findings. Unless the map codes the direction or quality of results, it says nothing about effects.
- Only as good as the search. A map inherits every limit of its search strategy.
- Only as good as the coding. If two people would code the same study into different cells, the map partly reflects the codebook. Coding a sample twice and checking agreement shows how much.
- It ages. A map is a snapshot. Its date and search period belong next to the picture.
Making a first map
Examine Evidence's quick maps start from a file of studies with titles and abstracts. The AI suggests dimensions and categories, which you edit. A decision model then classifies each study from its title and abstract, and you can open any cell to see its studies and check the classification. The free quick map takes up to 1,000 studies and up to three dimensions.
A quick map is for screening information: it shows where studies are. It is not a systematic review and does not replace the search, criteria and reporting described above. Screening with several reviewers and agreement checks are part of the Studio workspace, not of the free quick map.
Sources
- White H, Albers B, Gaarder M, et al. Guidance for producing a Campbell evidence and gap map. Campbell Systematic Reviews, 2020;16(4):e1125.
- Snilstveit B, Vojtkova M, Bhavsar A, Gaarder M. Evidence gap maps: a tool for promoting evidence-informed policy and prioritizing future research. World Bank Policy Research Working Paper 6725, 2013.
- Tricco AC, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR). Annals of Internal Medicine, 2018;169:467-473.
- Pullin A, Frampton G, Livoreil B, Petrokofsky G, eds. Guidelines and Standards for Evidence Synthesis in Environmental Management, version 5.1. Collaboration for Environmental Evidence, 2022.
- Cochrane Handbook for Systematic Reviews of Interventions (current version).
Map the studies you already have
Make a map