Investigators
Imperial College COVID-19 Response Team
Contact info (email)
neil.ferguson@imperial.ac.uk; s.bhatt@imperial.ac.uk
1. Define context(s)
other
Current Conformance Level / Target Conformance Level
Comprehensive
Primary goal of the model/tool/database
The purpose of the model is to infer the impact of 5 non-pharmaceutical interventions (case isolation, social distancing encouraged, public events banned, school closure ordered, lockdown ordered), as well as the onset of the first intervention, on the reproductive number (R) of COVID-19.
Biological domain of the model
Human national populations
Structure(s) of interest in the model
Whole organism
Spatial scales included in the model
~1 m (individual human)
Time scales included in the model
Death count per day; Total time domain ~1 month
Other uses for the model (optional)
To model previously observed daily infections and deaths over time due to COVID-19, using country-specific estimates of R (based on the interventions implemented in each country, and when each intervention was implemented), to forecast future death counts (up to 1 week), and to compare with death counts from counterfactual intervention-free model.
2. Data for building and validating the model
| Data for building the model | Published? | Private? | How is credibility checked? | Current Conformance Level / Target Conformance Level |
|---|---|---|---|---|
| in vitro (primary cells cell, lines, etc.) | ||||
| ex vivo (excised tissues) | ||||
| in vivo pre-clinical (lower-level organism or small animal) | ||||
| in vivo pre-clinical (large animal) | ||||
| Human subjects/clinical | ||||
| Other: ________________________ | Yes (Euro. CDPC) | Unknown |
| Data for validating the model | Published? | Private? | How is credibility checked? | Current Conformance Level / Target Conformance Level |
|---|---|---|---|---|
| in vitro (primary cells cell, lines, etc.) | ||||
| ex vivo (excised tissues) | ||||
| in vivo pre-clinical (lower-level organism or small animal) | ||||
| in vivo pre-clinical (large animal) | ||||
| Human subjects/clinical | ||||
| Other: ________________________ | Same as above | Unknown |
3. Validate within context(s)
| Who does it? | When does it happen? | How is it done? | Current Conformance Level / Target Conformance Level | |
|---|---|---|---|---|
| Verification | Unknown | |||
| Validation | Developers | Every time the model is run | Leave-last-3-out cross-validation of forecasting | Partial |
| Uncertainty quantification | Uncertainty propagation of several estimated quantities | Every time the model is run | Bayesian | Adequate |
| Sensitivity analysis | Developers | Throughout development | Examined effects of changing prior distributions for several parameters on the model predictions | Extensive |
| Other:__________ | ||||
| Additional Comments |
4. Limitations
| Disclaimer statement (explain key limitations) | Who needs to know about this disclaimer? | How is this disclaimer shared with that audience? | Current Conformance Level / Target Conformance Level |
|---|---|---|---|
| Strong prior distributions or fixed parameters | Devel. and users | Paragraph in report | Adequate |
| Assume effect of interventions is the same in all countries | Users | Paragraph in report | Extensive |
5. Version control
| Current Conformance Level / Target Conformance Level |
|---|
| Comprehensive |
| Naming Conventions? | Repository? | Code Review? | |
|---|---|---|---|
| individual modeler | Yes | Github | 7 contributors |
| within the lab | |||
| collaborators |
6. Documentation
| Current Conformance Level / Target Conformance Level | |
|---|---|
| Code commented? | Adequate (could be more extensively commented) |
| Scope and intended use described? | Adequate |
| User’s guide? | Comprehensive |
| Developer’s guide? | Extensive |
7. Dissemination
| Current Conformance Level / Target Conformance Level |
|---|
| Extensive (public reports; nothing peer-reviewed yet) |
| Target Audience(s): | “Inner circle” | Scientific community | Public |
|---|---|---|---|
| Simulations | Code, reports | Reports | Reports |
| Models | Code, reports | Reports | Reports |
| Software | Open source | Reports | Reports |
| Results | Reports | Reports | Reports |
| Implications of results | Reports | Reports | Reports |
8. Independent reviews
| Current Conformance Level / Target Conformance Level |
|---|
| Insufficient / Unknown |
| Reviewer(s) name & affiliation: | |
|---|---|
| When was review performed? | |
| How was review performed and outcomes of the review? |
9. Test competing implementations
| Current Conformance Level / Target Conformance Level |
|---|
| Insufficient / Unknown |
| Yes or No (briefly summarize) | |
|---|---|
| Were competing implementations tested? | Not mentioned |
| Did this lead to model refinement or improvement? | Not mentioned |
10. Conform to standards
| Current Conformance Level / Target Conformance Level |
|---|
| Unknown |
| Yes or No (briefly summarize) | |
|---|---|
| Are there operating procedures, guidelines, or standards for this type of multiscale modeling? | Unknown (standards for SIR modeling, Bayesian analysis, etc?) |
| How do your modeling efforts conform? | Standards are not mentioned in report |
Analysis concluded that this model can reliably forecast daily deaths 3 days into the future. Close spacing of interventions in time made it statistically impossible to determine which had the greatest effect. Model does not account for country-specific implementations of interventions (e.g. banning gatherings of 2+ people versus 10+ people).