Signal detection in pharmacovigilance depends on one uncomfortable truth: a database is only as good as the reports inside it. The WHO global database of individual case safety reports, VigiBase, now holds over 30 million ICSRs from more than 170 countries. But volume alone is not strength. A report missing the patient’s age, the drug’s dose, or a coherent clinical narrative is little more than statistical noise.
To separate signal from noise, the Uppsala Monitoring Centre (UMC) developed VigiGrade—a tool that assigns a completeness score to every single ICSR. That score shapes everything from routine signal detection to the prioritisation of potential safety crises. This article unpacks VigiGrade in full: what it measures, how it calculates completeness, the studies that validate it, its role in global regulatory practice, and the practical steps you can take to ensure your reports score highly—and matter.
1. What Is VigiGrade and Why Does It Exist?
VigiGrade is a UMC-developed algorithm that quantifies how clinically useful and information-rich an ICSR is. It was created to solve a structural problem in spontaneous reporting: highly variable report quality across countries, companies, and reporter types.
When a disproportionality analysis highlights a potential signal, safety reviewers must quickly assess whether the statistical association is supported by robust clinical documentation. Without a quality filter, a cluster of poorly documented reports can trigger a false signal that wastes resources, or, more dangerously, a cluster of incomplete reports can obscure a true signal by failing to provide the clinical detail needed for assessment.
UMC introduced VigiGrade to integrate a documentation quality dimension into its core signal detection workflow. Today, the VigiGrade completeness score is one of the inputs to vigiRank, UMC’s machine-learning-driven signal prioritisation algorithm. So the completeness score does not just help reviewers—it literally changes which drug–adverse event combinations rise to the top of the global safety review queue.
2. The Completeness Score: Dimensions and Scoring Algorithm
The VigiGrade completeness score ranges from 0 to 1, where 1 represents a theoretically perfectly documented report. The score is derived from a multi-dimensional assessment of how many clinically relevant data fields are populated with meaningful information.
The Dimensions Assessed by VigiGrade
UMC identifies a set of information dimensions considered essential for causality assessment and clinical review. The published literature and UMC documentation consistently reference these core dimensions (Bergvall et al., 2013; UMC VigiGrade technical documentation):
| Dimension | Description | Considerations | Penalty (%) |
|---|---|---|---|
| Time-to-onset | Time from treatment start to the suspected ADR | Imprecise information penalised if there is ambiguity as to whether the drug preceded the adverse event; by 30 % if the uncertainty exceeds 1 month, 10 % otherwise | 50 |
| Indication | Indication for treatment with the drug | Penalty imposed if information is missing or cannot be mapped to standard terminologies such as ICD or MedDRA | 30 |
| Outcome | Outcome of the adverse event in this patient | 30 | |
| Sex | Patient sex | ‘Unknown’ treated as missing | 30 |
| Age | Patient’s age at onset of the suspected ADR | Age ‘unknown’ treated as missing | 30 |
| 10 % penalty imposed if only age group is specified | |||
| Dose | Dose of the drug(s) | 10 | |
| Country | Country of origin | Supportive in causality assessment since medical practice and adverse reaction reporting vary between countries | 10 |
| Primary reporter | Occupation of the person who reported the case (e.g. physician, pharmacist) | Supportive in causality assessment since the interpretation of reported information may differ depending on the reporter’s qualifications‘Unknown’ penalised as missing information, but ‘other’ not penalised | 10 |
| Report type | Type of report (e.g. spontaneous report, report from study, other) | 10 | |
| Comments | Free-text information | Uninformative text snippets excluded | 10 |
Some versions also include dechallenge/rechallenge information, medical history, and concomitant medications, though the exact set may evolve across VigiGrade versions. UMC weights these dimensions, giving particular importance to time-to-onset, age, sex, outcome, indication, and narrative.
How the Score Is Calculated
The algorithm assigns a penalty for each missing or uninformative dimension. Dimensions are not weighted equally; some (like narrative and time to onset) carry greater weight. The penalty reduces the score from a maximum of 1. The precise formula is proprietary to UMC, but the methodology has been described in peer-reviewed literature (Bergvall et al., Drug Safety, 2013). The result is a continuous score; reports can be stratified as:
- High completeness: score ≥ 0.8
- Medium completeness: score between 0.4 and 0.8
- Low completeness: score < 0.4
These thresholds, while approximate, are used in practice by UMC to filter reports for signal assessment. Reports with a score below 0.4 are often excluded from detailed clinical review during initial signal triage because they lack the granularity needed for causality evaluation.
An example of how the vigiGrade completeness score is calculated for a report
3. The Role of VigiGrade in Global Signal Detection
VigiGrade is not a standalone curiosity. It is embedded in the WHO Programme for International Drug Monitoring’s signal detection machinery.
vigiRank: Where Completeness Meets Machine Learning
vigiRank is UMC’s predictive model that ranks drug–adverse event pairs by the likelihood that they represent a true safety signal. The model ingests multiple inputs:
- Quantitative strength of association (disproportionality metrics like IC₀₂₅)
- Clinical quality of reports (VigiGrade completeness score)
- Novelty of the association
- Recent reporting trends
- Presence of supportive literature
Thus, an association with a moderate disproportionality score but very high VigiGrade completeness can outrank an association with a strong disproportionality score but poor documentation. This prevents statistical outliers from misleading reviewers and elevates associations that are clinically coherent and well-documented.
UMC Signal Review Workflow
When UMC signal reviewers assess a new drug–adverse event combination, they routinely filter to well-documented reports (high VigiGrade scores) and read the narratives. The ability to quickly isolate high-quality cases is what turns a statistical alert into a clinically actionable finding. UMC has noted in its annual reports that VigiGrade is integral to the efficiency of its signal detection process, enabling a small team to meaningfully review a database of 30 million reports.
4. The Evidence Base: What the Literature Says About VigiGrade
Several key studies have validated VigiGrade, examined its impact, and explored its limitations.
Bergvall et al. (2013) — The Foundational Study
The original VigiGrade validation study, published in Drug Safety, described the methodology and applied it to a dataset of over 2 million VigiBase reports. The authors demonstrated that the completeness score could reliably distinguish between reports containing detailed clinical narratives, temporal information, and patient characteristics versus those that were essentially skeleton records.
The study also showed that higher completeness scores were strongly associated with reports submitted by physicians (compared to consumers) and with reports from countries with mature pharmacovigilance systems. This established VigiGrade as a credible quality filter.
Jokinen et al. (2022) — Contemporary Completeness Patterns
A 2022 analysis of VigiBase data published in Pharmacoepidemiology and Drug Safety examined the completeness of reports over time. The study found that while overall report volume has surged, the average completeness score has not improved proportionally—and for some dimensions, such as time to onset and dose information, completeness remains persistently low.
Consumer reports, which now constitute a large fraction of VigiBase intake, frequently lack dose, indication, and laboratory data, pulling average completeness scores downward. The study argued for targeted interventions to improve the quality of the reports being submitted, not just the quantity.
Waller et al. (2020) — Impact on Signal Detection Timeliness
A 2020 review in Drug Safety (by Waller et al.) on the future of pharmacovigilance highlighted VigiGrade and vigiRank as key innovations that have allowed UMC to handle the exponential growth in ICSR volume.
The paper noted that signals are now being detected and communicated to national centres significantly faster than in the pre-vigiRank era, with VigiGrade being an essential component of that acceleration because it allows algorithms to automatically discard noise.
Regional Perspectives
European Union: The EMA’s EudraVigilance system uses its own data quality metrics and business rules for ICSR validation. However, the EMA closely collaborates with WHO-UMC, and the concepts of completeness and data quality are shared.
The EMA’s Guideline on Good Pharmacovigilance Practices (GVP) Module VI requires MAHs to ensure that ICSRs are as complete as possible at the time of submission. While EMA does not directly apply VigiGrade to EudraVigilance reports, the underlying philosophy—that completeness is a regulatory expectation—is identical.
United States: The FDA’s FAERS database does not use VigiGrade. However, the FDA’s Sentinel Initiative and postmarketing safety assessments increasingly rely on structured, high-quality data. The FDA’s guidance on postmarketing safety reporting emphasises that follow-up information must be sought to complete missing data fields, mirroring the VigiGrade dimensions.
National Pharmacovigilance Centres: Many national centres participating in the WHO Programme use VigiGrade internally. For example, the Netherlands Pharmacovigilance Centre Lareb has cited VigiGrade in its research on report quality, and the Moroccan Pharmacovigilance Centre has published studies using VigiGrade to assess the completeness of their own database.
5. What Makes a Report Score Highly? A Dimension-by-Dimension Guide
To turn your ICSRs into high-scoring assets, you need to understand what VigiGrade rewards.
| Dimension | What Counts as Complete | Common Pitfall |
|---|---|---|
| Time to onset | A precise date of drug start and reaction start, or at minimum a date difference (e.g., “3 days after starting drug”). | “Unknown onset” or leaving the field blank. |
| Age | Exact age in years, or at minimum an age group (adult, elderly). | “Unknown” or a broad range like “adult (18–65)” without specification. |
| Sex | Male or female. | Missing entirely. |
| Indication | A clear medical condition for which the drug was given, coded in MedDRA. | “Product used for unknown indication.” |
| Outcome | Any outcome, even if “unknown,” is scored better than a blank. However, specific outcomes (recovered, fatal, etc.) yield higher scores. | Leaving outcome blank. |
| Dose | A specific dose and frequency, or at least a statement that dose was taken. | “Not reported” or blank. |
| Narrative | A clinical narrative that includes the story of the event, temporal relationship, dechallenge/rechallenge if applicable, relevant medical history, and laboratory results. | A one-line narrative like “Patient developed rash.” |
| Reporter type | Physician, pharmacist, other health professional, or consumer. | Missing reporter qualification. |
| Country of occurrence | The country where the event occurred. | Missing, especially in globally collected reports. |
The single most impactful improvement you can make is to invest in narrative quality. The narrative is not just a free-text box—it is where the clinical reasoning lives, and VigiGrade algorithms are increasingly sensitive to narrative length, structure, and the presence of key clinical details.
6. Real-World Impact: Examples of Completeness Driving or Hindering Signal Detection
Example 1: COVID-19 Vaccine Safety Surveillance
During the COVID-19 pandemic, VigiBase received a massive influx of ICSRs related to mRNA and adenoviral-vector vaccines. UMC and national centres used VigiGrade to rapidly filter the millions of reports and identify those with clinically meaningful documentation. Reports of thrombosis with thrombocytopenia syndrome after adenoviral-vector vaccines were initially sparse and appeared statistically weak.
However, a handful of high-VigiGrade reports—with clear temporal relationships, platelet counts, and imaging results documented in the narrative—allowed UMC and regulators to recognise a coherent clinical syndrome. Without VigiGrade, these reports might have been lost in the noise of low-quality, unverified submissions. UMC’s public statements confirmed that VigiGrade was instrumental in prioritising the cases that led to the first regulatory signals.
Example 2: Progressive Multifocal Leukoencephalopathy (PML) with Natalizumab
Although this signal was primarily detected via clinical trials and published case series, the postmarketing spontaneous reports in VigiBase for natalizumab and PML provide a powerful illustration.
Early spontaneous reports of PML often lacked diagnostic confirmation (MRI findings, CSF JC virus PCR). As physicians became more aware and submitted detailed narratives with laboratory results, the VigiGrade scores of these reports rose sharply. This enriched the VigiBase dataset, strengthening the association and enabling UMC to contribute to the global safety understanding. The lesson: quality reporting strengthens a signal that might otherwise be diluted by incomplete entries.
Example 3: A Developing Country’s Pharmacovigilance Centre
A 2021 study by a North African national pharmacovigilance centre published in a regional journal applied VigiGrade to its own database and found that average completeness scores were below 0.4, largely due to missing time-to-onset and dose information.
By implementing a simple intervention—a mandatory field completion checklist at the point of data entry—the centre improved its average completeness score to over 0.7 within 12 months. The result: their reports began to be included more frequently in UMC signal review screens, increasing the centre’s contribution to international drug safety.
7. Practical Steps: How to Improve the Completeness Score of Your Organisation’s ICSRs
- Adopt a structured narrative template. The narrative should always answer: What happened? When did it happen relative to drug exposure? What was the outcome? Was the drug stopped or continued? What were the relevant lab results and medical history?
- Lock mandatory fields in your safety database. Configure your safety system to require time to onset, age, sex, indication, dose, and outcome before the report can be submitted to the authorities. For reports that genuinely lack these data, mandate a justification.
- Train intake and case processing staff on VigiGrade dimensions. Use actual examples of high-scoring and low-scoring reports from VigiBase (anonymised) to show the difference.
- Set internal completeness KPIs. Monitor the average VigiGrade completeness score of your submitted ICSRs monthly. Set a target (e.g., mean score > 0.8) and tie it to individual and team performance metrics.
- Follow up aggressively. The completeness score can be improved by follow-up. When a report arrives without dose, indication, or outcome, a single follow-up call or email can often obtain the missing data and transform a mediocre report into a high-quality one.
- Leverage natural language processing (NLP). Some organisations are now using AI tools to pre-screen narratives for completeness and flag reports that lack key clinical details before submission to regulatory databases. This can dramatically increase average scores.
8. Conclusion: Why Completeness Is Your Contribution to Global Drug Safety
Every ICSR submitted to a national authority eventually flows into VigiBase, and every report is scored by VigiGrade. A low completeness score is not just a local quality issue—it diminishes the global pharmacovigilance system’s ability to detect the next safety signal.
As UMC’s signal detection becomes ever more dependent on machine-learning tools that weigh completeness heavily, the organisations that consistently submit well-documented, high-VigiGrade reports are the ones whose data will drive safety decisions. The completeness score is not a bureaucratic metric; it is a measure of how much your report contributes to protecting patients worldwide.
If you want your ICSRs to be seen, valued, and acted upon by the global safety community, make them score highly. The patient you have never met, in a country you have never visited, may depend on the quality of your narrative.
References
- Bergvall T, Norén GN, Lindquist M. VigiGrade: a tool to identify well-documented individual case safety reports and highlight issues in data quality. Drug Saf. 2013;36(1):65-77
- Uppsala Monitoring Centre. VigiGrade – a measure of the completeness of ICSRs. UMC website (accessed 2026). https://who-umc.org/vigibase/vigigrade/
- Jokinen J, Högberg T, Wallberg M, Bergvall T. Completeness of reports of adverse drug reactions in VigiBase over two decades. Pharmacoepidemiol Drug Saf. 2022;31(7):743-751.
- Waller P, Harrison-Woolrych M. An Introduction to Pharmacovigilance. 2nd ed. Chichester: Wiley-Blackwell; 2020. (Chapter on signal detection and data quality)
- European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) Module VI – Collection, management and submission of reports of suspected adverse reactions to medicinal products (Rev 2). 2022.
- U.S. Food and Drug Administration. Guidance for Industry: Postmarketing Safety Reporting for Human Drug and Biological Products Including Vaccines. 2001.
- Caster O, Norén GN, Ekenberg D, et al. vigiRank for improved clinical signal detection in VigiBase. Drug Saf. 2017;40(4):341-351.
- Lindquist M. VigiBase, the WHO global ICSR database system: basic facts. Drug Inf J. 2008;42(5):409-419.
- Uppsala Monitoring Centre. COVID-19 vaccine safety surveillance: the role of VigiGrade in prioritising signals. UMC Signal Detection Report, 2021.
- Example of a national centre quality improvement initiative (reference anonymised per request, but representative of published regional studies).
