Investigators
Hernan Makse
Contact info (email)
hmakse@ccny.cuny.edu
1. Define context(s)
other
Current Conformance Level / Target Conformance Level
Extensive
Primary goal of the model/tool/database
The primary goal of this project is to develop, calibrate, test, deploy and commercialize an optimized digital contact tracing algorithm based on network optimization theory to find infected contacts and their secondary contacts to deploy directed quarantines, and testing/isolation protocols to halt the COVID-19 epidemic spreading with minimal social disruptions.
For this purpose, the lab of Makse at City College of New York is leading a partnership between Grandata, Inc, a business leader in the IT industry across Latin America (LATAM) and USA, and the Government of the State of Ceara, Brazil. Our path to commercialization starts by algorithm developing in Phase 1 to deploy, in Phase 2, the contact tracing algorithm at the back-end of a COVID dedicated government-backed mobile application (the app) developed by the Government and Health Department of the State of Ceara.
Biological domain of the model
Human network of contacts
Structure(s) of interest in the model
Contact network of people spreading the disease
Spatial scales included in the model
Contact modeling within 8 meters.
Time scales included in the model
Time of exposure within 30 minutes and contact tracing developed within 14 days.
2. Data for building and validating the model
| Data for building the model | Published? | Private? | How is credibility checked? | Current Conformance Level / Target Conformance Level |
|---|---|---|---|---|
| in vitro (primary cells cell, lines, etc.) | ||||
| ex vivo (excised tissues) | ||||
| in vivo pre-clinical (lower-level organism or small animal) | ||||
| in vivo pre-clinical (large animal) | ||||
| Human subjects/clinical | No | GPS trajectories from apps. Personal data is fully annonymized | Identifying first layer of infected people and comparing with the cumulative cases of infected people in Ceara, Brazil. | Adequate |
| Other: ________________________ |
| Data for validating the model | Published? | Private? | How is credibility checked? | Current Conformance Level / Target Conformance Level |
|---|---|---|---|---|
| in vitro (primary cells cell, lines, etc.) | ||||
| ex vivo (excised tissues) | ||||
| in vivo pre-clinical (lower-level organism or small animal) | ||||
| in vivo pre-clinical (large animal) | ||||
| Human subjects/clinical | No | Data is obtained from partners running mobile apps and government-backed app for contact tracing | Tested with documented cases of infections. The model is being developed, calibrated, tested and validated by using billions of data points from real-time individual’s geolocalization collected from hundreds of mobile apps during the ongoing COVID-19 pandemic in all LATAM, initially focusing on the state of Ceara ́, Brazil. | Adequate |
| Other: ________________________ |
3. Validate within context(s)
| Who does it? | When does it happen? | How is it done? | Current Conformance Level / Target Conformance Level | |
|---|---|---|---|---|
| Verification | Developers of the contact tracing algorithm | Before release of the algorithms to the end-user app developers | Performed by static source code verification and dynamics by running specific test cases. Formal verification will be implemented before releasing by creating artificial data on location and contacts between to verify the contact tracing code. | Adequate |
| Validation | Developers of the algorithm and software developers of the mobile app | When the algorithm is deployed in the back-end of the government backed app | The algorithm is validated by putting it through a typical case scenario and testing its performance. We will validate the algorithm with the data provided by the state of Ceara on infected people. Addressable by Push Notification in the App to the end-user. Validation will be performed by fitting the hyperparameters of the model: (1) time interval T, (2) space of contact as defined by radius r of contact circle, (3) threshold p_c in probability of contagion. Validation will be implemented by fitting the hyperparameters (T, r, p_c) in the model to obtain the value of the basic reproduction number R_0. Validation will proceed by determining optimal contact tracing hyperparameters to reproduce the known tree of infections and the R_0 of the pandemic. | Adequate |
| Uncertainty quantification | Developers of the algorithm | During Phase 1 of algorithm development. | Before deploying the contact tracer algorithm in the back-end of the app | Adequate |
| Sensitivity analysis | Developers of the algorithm | Before deploying algorithm in the app | Sensitivity analysis of the hyperparameters of the model: (1) time interval T, (2) space of contact as defined by radious r of contact circle, (3) threshold p_c in probability of contagion. Sensitivity analysis will be implemented by changing the hyperparameters (T, r, p_c) to the value of the basic reproduction number R_0. Study will determine optimal contact tracing hyperparameters to reproduce the known tree of infections and the R_0 of the pandemic. | Adequate |
| Other:__________ | ||||
| Additional Comments |
4. Limitations
| Disclaimer statement (explain key limitations) | Who needs to know about this disclaimer? | How is this disclaimer shared with that audience? | Current Conformance Level / Target Conformance Level |
|---|---|---|---|
| The algorithm attempts to determine the people at high risk of infection of a disease by applying probabilistic models of contact dynamics. There is a risk that an identification of a proximity contact identified by the model does not lead to a contagious event. | The end-user of app who obtain a notification of high probability of infection, even thought the contagion did not occur. | Disclaimer will be shared with the end-user audience by the Term of Services of the app. | Extensive |
| False negative test: infected person with negative test | Patients. False negatives can be captured by subsequent hospitalization if symptoms develop. | Not addressable by the app. | Extensive |
| False positive test: healthy person with positive test | Patients. Lead to unnecessary quarantines | Not addressable by the app. | Extensive |
| False negative contact tracing: infected user without contact tracing detection. This defines our range of uncertainty. | End-users of the app. Population will be warned that a green light from the tracker does not mean that they haven't been in contact with infected people, but that the tracer did not find such contacts. Under full adherence to app false negative can be avoided. Problem is ameliorated by self-reporting or doctor reporting of infected person at the time of testing, which will provide a measurement of false negative rates. | Addressable by Puch Notification in the App to the end-user. | Extensive |
| False positive contact tracing: a healthy person with a tracing warning leading to additional quarantines. | End-user. This case will reinforce local lockdown policy. User will receive the time and location of the contact with a COVID-19 patient, and will be able to confirm or reject the information. | Addressable by Puch Notification in the App to the end-user. | Extensive |
5. Version control
| Current Conformance Level / Target Conformance Level |
|---|
| Extensive |
| Naming Conventions? | Repository? | Code Review? | |
|---|---|---|---|
| individual modeler | Kcore Tracing Algorithm | github.com | yes |
| within the lab | yes | github.com | yes |
| collaborators | yes | github.com | yes |
6. Documentation
| Current Conformance Level / Target Conformance Level | |
|---|---|
| Code commented? | Extensive: inline comments in Python code. |
| Scope and intended use described? | Extensive: described in white paper and publications in the specialized literature. |
| User’s guide? | Partial: partial description in the specialized literature |
| Developer’s guide? | Partial: Documents will be created for internal users of the algorithm. |
7. Dissemination
| Current Conformance Level / Target Conformance Level |
|---|
| Extensive |
| Target Audience(s): | “Inner circle” | Scientific community | Public |
|---|---|---|---|
| Simulations | Reported in specialized literature | ||
| Models | Reported in specialized literature | ||
| Software | github.com | reported in the specialized literature | |
| Results | Specialized literature | Conferences and website | |
| Implications of results | Specialized literature | Conferences and app deployment |
8. Independent reviews
| Current Conformance Level / Target Conformance Level |
|---|
| NA |
| Reviewer(s) name & affiliation: | |
|---|---|
| When was review performed? | |
| How was review performed and outcomes of the review? |
9. Test competing implementations
| Current Conformance Level / Target Conformance Level |
|---|
| Adequate |
| Yes or No (briefly summarize) | |
|---|---|
| Were competing implementations tested? | Not yet |
| Did this lead to model refinement or improvement? | Not yet |
10. Conform to standards
| Current Conformance Level / Target Conformance Level |
|---|
| Extensive |
| Yes or No (briefly summarize) | |
|---|---|
| Are there operating procedures, guidelines, or standards for this type of multiscale modeling? | There are no standard procedures to test this kind of multiscale modeling |
| How do your modeling efforts conform? | No |