The Reference Model for Disease Progression

What is being modeled?
Disease progression, currently COVID-19 or Diabetic Cardiovascular
Description & purpose of resource

The Reference Model is:

The Reference Model can now:

  • Estimate COVID-19 parameters for US states and territories
  • Figure out best fitting infectiousness and mortality profiles for COVID-19
  • Determine CVD models that significantly behave better on several diabetic populations
  • Deduce that CVD probability halves every 5 years due to medicine improving - according to information from the last 3 decades
  • Calculate life tables for diabetics
  • Include human interpretation of potentially ambiguous outcome definitions
  • Interface with ClinicalTrials.Gov
  • Create an interactive map of our cumulative computational knowledge gap

 

 

Spatial scales
group/society
Temporal scales
days
weeks to months
human lifetime
This resource is currently
mature and useful in ongoing research
a demonstration or a framework to be built upon (perhaps with a sample implementation)
likely to require significant study prior to effective reuse
likely to be usable without detailed knowledge of its internals
Has this resource been validated?
Yes
How has the resource been validated?

The Reference Model is actually a validation tool that validates many other models - it is an ensemble model that optimizes multiple models and merges information to fit outcomes, interpretation, and other assumptions.

At the time of initially writing these words (March 2020) it is the most validated diabetes cardiovascular model known worldwide validating its results against 30 / 123 cohorts containing 471 validations - this number increased through the years and will continue increasing periodically when more data is added to ClinicalTrials.Gov. 

 

The Reference Model was since then used to model COVID-19 and on July 29th, a preliminary version of the model was published validating against infection information of US states and territories. The work continued to include an ensemble of models including infectiousness, transmission, response, and mortality. The work was published as an interactive paper and as a webinar . 

 

Link to Resource Credibility Assessment
The Reference Model for Disease Progression
Can this resource be associated with other resources? (e.g.: modular models, linked tools and platforms)
Yes
Which resources?

The model currently contains the following elements that are externally add to it:

- Risk equations: equations that predict the probability of reaching an outcome depending on multiple biomarkers such as age, male, blood pressure, cholesterol levels, A1c , etc...

- Observed Outcomes: This empirical data collected from multiple clinical trials is used as validation information. It includes value such as mortality, stroke, MI and composite endpoints such as CVD. Those numbers are either extracted manually from publications, or extracted via an interface from ClinicalTrials.Gov

The model uses the MIcro Simulation Tool (MIST) as its engine that enables it to run in High Performance Environments and it used the Amazon cloud for simulations in the past. 

The Reference Model uses the Inpyred  python library that enables population generation through Evolutionary Computation 

The COVID-19 model uses an ensemble of models that include those:

Most models were re-implemented in in python and publicly available in:  [github repository]

 

It uses data from the multiple sources including:

  • The Covid Tracking project [link] for initialization adn validation
  • Age and state/territory statistics from US Census [link1],[link2].
  • Number of Interaction per individual according to [link1],[link2].
  •  Apple mobility data [link] for response/behavior models

 

Key publications (e.g. describing or using resource)

There are over two dozen publications of the model, many as posters at the MSM-IMAG meeting starting in 2012.

 

The most recent publication is an interactive paper:

Another recent publication was uploaded to this page and provides an overview. It is an interactive poster that can be accessed through this link.

When loading interactive content, please use a strong computer/smartphone. Some interactive features require use of a mouse and will take above a minute to load on some machines. So please be patient.

Also, please visit the model portal on SimTK: https://simtk.org/projects/therefmodel

Here is a full list of publications in reverse chronological order:

  1. Jacob Barhak, The Reference Model is a Multi-Scale Ensemble Model of COVID-19. Self Published Interactive Manuscript. Publication Date: 27-May-2021 http://doi.org/10.34235/b7eaa32b-1a6b-444f-9848-76f83f5a733c

  2. Jacob Barhak: The Reference Model for COVID-19 - The First Multi-Scale Ensemble Disease Model, MIDAS Webinar - 28 May 2021, https://jacob-barhak.github.io/COVID19_Ensemble_MIDAS_Webinar.html Video: https://youtu.be/-z8N40TdKDk?t=1860

  3. Barhak J , The Reference Model initial Use Case for COVID-19. Cureus. http://dx.doi.org/10.7759/cureus.9455 , Online: https://www.cureus.com/articles/36677-the-reference-model-an-initial-use-case-for-covid-19 . Interactive Results: https://jacob-barhak.netlify.app/thereferencemodel/results_covid19_2020_06_27/combinedplot

  4. Jacob Barhak, The Reference Model Accumulates Knowledge With Human Interpretation. Interagency Modeling and Analysis Group - IMAG wiki - MODELS, TOOLS & DATABASES Uploaded 16 March 2020. Poster: https://jacob-barhak.github.io/Poster_MSM_IMAG_2020.html Code: https://github.com/Jacob-Barhak/Presentations/tree/master/MSM_IMAG2020
  5. Jacob Barhak, The Reference Model for Disease Progression Handles Human Interpretation, MODSIM World 2020, May 5-7, 2019, Norfolk, VA Paper: Presentation
  6. Jacob Barhak, The Reference Model Accumulates Knowledge With Human Interpretation. MSM/IMAG meeting. IMAG Wiki le Modeling (MSM) Consortium Meeting March 16. Interactive Poster Code:
  7. Jacob Barhak, Clinical Data Modeling with Python, AnacondaCon , Austin, Texas, April 3-5, 2019. Video: https://youtu.be/fQIYMf5wKGE , Presentation: https://jacob-barhak.github.io/AnacondaCon_2019.html
  8. Jacob Barhak, The Reference Model is the most validated diabetes cardiovascular model known. MSM/IMAG meeting. IMAG Multiscale Modeling (MSM) Consortium Meeting March 6-7, 2019 @ NIH, Bethesda, MD . Poster: https://jacob-barhak.github.io/InteractivePoster_MSM_IMAG_2019.html Poster Code: https://github.com/Jacob-Barhak/Presentations/tree/master/PosterIMAG2019
  9. J. Barhak, The Reference Model Visualizes Gaps in Computational Understanding of Clinical Trials, 2018 IMAG Futures Meeting March 21-22, 2018 @ NIH, Bethesda, MD. http://sites.google.com/site/jacobbarhak/home/Poster_IMAG_MSM2018_Map_Upload_2018_03_17.pdf
  10. J. Barhak, The Reference Model: A Decade of Healthcare Predictive Analytics with Python, PyTexas 2017, Nov 18-19, 2017, Galvanize, Austin TX. Presentation: http://sites.google.com/site/jacobbarhak/home/PyTexas2017_Upload_2017_11_18.pptx Video: https://youtu.be/Pj_N4izLmsI
  11. J. Barhak, The Reference Model Models ClinicalTrials.Gov. SummerSim 2017 July 9-12, Bellevue, WA. Paper: https://doi.org/10.22360/SummerSim.2017.SCSC.022 http://dl.acm.org/citation.cfm?id=3140087 Presentation: http://sites.google.com/site/jacobbarhak/home/SummerSim2017_Upload_2017_07_09.pptx
  12. J. Barhak, The Reference Model Predicting Life Expectancy for Diabetics, MODSIM world 2017. 26-28 Apr, Virginia Beach Convention Center, Virginia Beach, VA. Paper: http://www.modsimworld.org/papers/2017/The_Reference_Model_Predicts_Life_Expectancy_for_Diabetics.pdf Presentation: http://sites.google.com/site/jacobbarhak/home/MODSIM2017_Submit_2017_04_25.pptx
  13. J. Barhak, The Reference Model Estimates Medical Practice Improvement in Diabetic Populations. SpringSim 2017 April 12 - 15, Alexandria, VA, USA. Paper: https://doi.org/10.22360/SpringSim.2017.MSM.006 http://dl.acm.org/citation.cfm?id=3108762 Presentation: http://sites.google.com/site/jacobbarhak/home/SpringSim2017_Present_2017_04_25.pptx
  14. J. Barhak, The Reference Model Interface with ClinicalTrials.Gov , IMAG Multiscale Modeling (MSM) Consortium Meeting March 22-24, 2017 @ NIH, Bethesda, MD. . http://sites.google.com/site/jacobbarhak/home/PosterImportClinicalTrialsGov_IMAG_MSM2017_Upload_2017_03_18.pdf
  15. J. Barhak, The Reference Model for Disease Progression Combines Disease Models. I/IITSEC 2016 28 Nov – 2 Dec Orlando Florida. Paper: http://www.iitsecdocs.com/volumes/2016 Presentation: http://sites.google.com/site/jacobbarhak/home/IITSEC2016_Upload_2016_11_05.pptx
  16. J. Barhak, The Mount Hood Diabetes Challenge 2016. The Reference Model for Disease Progression - participated in challenges 1 and 2. 17-18 Sep 2016, St.Gallen Switzerland 2016. Program: http://media.wix.com/ugd/4e5824_36cb1fd0aca94f1980d8a2228cf7e6e8.pdf Presentations and results: http://sites.google.com/site/jacobbarhak/home/TheReferenceModel_MH2016_Submit_2016_09_04.zip
  17. J. Barhak, The Reference Model for Disease Progression and Latest Developments in the MIST, PyTexas 2015. College Station, TX, 26-Sep-2015. Presentation: http://sites.google.com/site/jacobbarhak/home/PyTexas2015_Upload_2015_09_26.pptx Video: https://www.youtube.com/watch?v=htGRRjia-QQ
  18. J. Barhak, The Reference Model Uses Modular Population Generation! Object Oriented Population Generation on the Fly with MIST. IMAG Multiscale Modeling (MSM) Consortium Meeting 9-10 September 2015. Poster: http://sites.google.com/site/jacobbarhak/home/PosterModularPopulationGeneration_IMAG_MSM2015_Upload_2015_09_03.pdf
  19. J. Barhak, The Reference Model uses Object Oriented Population Generation. SummerSim 2015 July 26-29, Chicago IL, USA. Paper: http://dl.acm.org/citation.cfm?id=2874946 Presentation: http://sites.google.com/site/jacobbarhak/home/SummerSim2015_Upload_2015_07_26.pptx
  20. J. Barhak, Modeling Clinical Data from Publications, SpringSim 2015. April 12 - 15, Alexandria, VA, USA. Paper: http://dl.acm.org/citation.cfm?id=2873011&CFID=575392711&CFTOKEN=46270544 Presentation: http://sites.google.com/site/jacobbarhak/home/SpringSim2015ModelingDataFromPublications_Present_2015_04_13.pptx
  21. J. Barhak, Generating Populations for Micro Simulation from Publicly Available Data - Populations in the MIST! IMAG Multiscale Modeling (MSM) Consortium Meeting 3-4 September 2014. Poster: http://sites.google.com/site/jacobbarhak/home/PosterPopulationGenerationMIST_IMAG_MSM2014_Upload_2014_08_31.pdf
  22. J. Barhak, The Reference Model for Disease Progression – Data Quality Control. SummerSim 6-10 July 2014, Monterey CA. Paper: http://dl.acm.org/citation.cfm?id=2685666 Presentation: http://sites.google.com/site/jacobbarhak/home/SummerSim2014_Upload_2014_07_06.pptx
  23. J. Barhak, The Mount Hood Diabetes Challenge 2014. The Reference Model for Disease Progression - participated in the challenge. 17-19 June 2014, Stanford. Presentation: http://sites.google.com/site/jacobbarhak/home/MH2014TheReferenceModel_Submit_2014_06_15.pptx
  24. J. Barhak, The Reference Model for Disease Progression uses MIST to find data fitness. PyData Silicon Valley 2014 held at Facebook Headquarters: Abstract: http://pydata.org/sv2014/abstracts/#195_ Presentation: http://sites.google.com/site/jacobbarhak/home/PyData_SV_2014_Upload_2014_05_02.pptx Video: https://www.youtube.com/watch?v=vyvxiljc5vA
  25. J. Barhak, The Reference Model for Disease Progression Sensitivity to Bio-Marker Correlation in Base Population - The Reference Model Runs with MIST Over the Cloud! 2013 IMAG Multiscale Modeling (MSM) Consortium Meeting, October 2-3, 2013, Poster: http://sites.google.com/site/jacobbarhak/home/PosterTheReferenceModel_IMAG_MSM2013_Submit_2013_09_23.pdf
  26. J. Barhak, The Reference Model Scores Fitness of Models and Populations. Poster Presentation. ISPOR 18th Annual International Meeting, May 18-22, 2013, Sheraton New Orleans, New Orleans, Poster Presentation. http://sites.google.com/site/jacobbarhak/home/PosterTheReferenceModel_ISPOR_Submit_2013_05_14.pdf
  27. J. Barhak, H.S. Leff, Modeling a Chronic Disease Model and a Mental Health Model Using the Same Modeling Tools, MODSIM World 2013, April 30 - May 2nd, Hampton Roads Convention Center in Hampton, VA. Paper: http://sites.google.com/site/jacobbarhak/home/MODSIM_World2013_Submitted_04Apr2013.pdf , Presentation: http://sites.google.com/site/jacobbarhak/home/MODSIM_World_Presented_2013_05_2.pptx
  28. J. Barhak, The Reference Model: Improvement in Treatment Through Time in Diabetic Populations, The Fourth International Conference in Computational Surgery and Dual Training. The Joseph B. Martin Conference Center at Harvard Medical School. Boston, MA, USA. December 9-10-11, 2012. Video: http://web.cs.uh.edu/~cosine/?q=node/140 , Presentation: http://www2.cs.uh.edu/~cosine/talks_cosine4/monday/MultidisciplinaryTalks/2_JacobBarhak.pptx Slides Copy: http://sites.google.com/site/ComputationalSurgery_Presneted_2012_12_LateUploadToOwnWebSite_2014_2_27.pptx
  29. J. Barhak, The Reference Model for Chronic Disease Progression. 2012 Multiscale Modeling (MSM) Consortium Meeting, October 22-23, 2012, Poster: http://sites.google.com/site/jacobbarhak/home/PosterTheReferenceModel_IMAGE_MSM_Submit_2012_10_17.pdf
  30. J. Barhak, The Reference Model for Disease Progression. SciPy 2012, Austin Tx, 18-19 July 2012. Paper: http://dx.doi.org/10.25080/Majora-54c7f2c8-007 , https://github.com/Jacob-Barhak/scipy_proceedings/blob/2012/papers/Jacob_Barhak/TheReferenceModelSciPy2012.rst, Poster: http://sites.google.com/site/jacobbarhak/home/PosterTheReferenceModel_SciPy2012_Submit_2012_07_14.pdf
DOI link to publication describing this resource

http://doi.org/10.34235/b7eaa32b-1a6b-444f-9848-76f83f5a733c

http://dx.doi.org/10.7759/cureus.9455

https://doi.org/10.22360/SummerSim.2017.SCSC.022

https://doi.org/10.22360/SpringSim.2017.MSM.006    

http://dx.doi.org/10.25080/Majora-54c7f2c8-007 

Link to resource
Poster Created for this Wiki - The Reference Model Accumulates Knowledge With H…
The Reference Model Portal on SimTK
Jacob Barhak: The Reference Model for COVID-19 - The First Multi-Scale Ensemble…
Jacob Barhak, The Reference Model is a Multi-Scale Ensemble Model of COVID-19. …
Collaborators
Jacob Barhak
PI contact information
jacob.barhak@gmail.com
Keywords
Ensemble model
Diabetes
Cardiovascular Disease
micro-simulation
optimization
Evolutionary Computation
machine learning
Population Modeling
COVID-19
Related Documents
TheReferenceModel_OneSlideSummary_2020_03_16.pptx140.35 KB
TheReferenceModel_OneSlideSummary_2020_03_16.pdf139.96 KB
Table sorting checkbox
Off
Model type
statistical
agent-based

The MIcro Simulation Tool (MIST)

What is being modeled?
Population / Disease Progression
Description & purpose of resource

The MIcro Simulation Tool is free software to support disease modeling. It is a Monte-Carlo micro-simulation compiler that employs High Performance Computing.

MIST has unique population generation capabilities.

MIST runs over the cloud!

The Reference Model for Disease Progression uses MIST as its engine

Spatial scales
whole organism
group/society
Temporal scales
<10-6 s (chemical reactions)
10-6 - 10-3 s
10-3 - 1 s
1 - 103 s
hours
days
weeks to months
human lifetime
multiple generations
This resource is currently
mature and useful in ongoing research
a demonstration or a framework to be built upon (perhaps with a sample implementation)
likely to be usable without detailed knowledge of its internals
Has this resource been validated?
Yes
How has the resource been validated?

MIST distribution includes tests that are made public that test its integrity and validity. This includes critical tests, statistical tests, and reproducibility tests.

The test examples are documented and documentation is released with the software. 

The Reference Model uses MIST for 7 years as its engine and MIST was used to model mental health model and Sepsis.

Can this resource be associated with other resources? (e.g.: modular models, linked tools and platforms)
Yes
Which resources?

MIST uses the Inspyred library to perform Evolutionary Computation.

The Reference Model for Disease Progression uses MIST as the simulation engine

 

Key publications (e.g. describing or using resource)

The Reference Model provides additional publications describing uses of MIST:

https://www.imagwiki.nibib.nih.gov/resources/models-tools-databases/reference-model-disease-progression

 

DOI link to publication describing this resource

https://doi.org/10.22360/SummerSim.2017.SCSC.022

https://doi.org/10.22360/SpringSim.2017.MSM.006    

Link to resource
SimTk MIST portal
MIST on GitHub
Collaborators
Jacob Barhak
PI contact information
jacob.barhak@gmail.com
Keywords
microsimulation
population generation
Evolutionary Computation
high performance computing
Disease Modeling
Table sorting checkbox
Off

COVID-19 Open Research Dataset

What is being modeled?
COVID-19
Description & purpose of resource

A Free, Open Resource for the Global Research Community

covid-image

In response to the COVID-19 pandemic, the Allen Institute for AI has partnered with leading research groups to prepare and distribute the COVID-19 Open Research Dataset (CORD-19), a free resource of over 29,000 scholarly articles, including over 13,000 with full text, about COVID-19 and the coronavirus family of viruses for use by the global research community.

This dataset is intended to mobilize researchers to apply recent advances in natural language processing to generate new insights in support of the fight against this infectious disease. The corpus will be updated weekly as new research is published in peer-reviewed publications and archival services like bioRxiv, medRxiv, and others.

Spatial scales
molecular
cellular
tissue
organ
whole organism
group/society
Temporal scales
<10-6 s (chemical reactions)
10-6 - 10-3 s
10-3 - 1 s
1 - 103 s
hours
days
weeks to months
human lifetime
multiple generations
This resource is currently
of historical interest (e.g., as a reference implementation of a well-known model or computational method)
mature and useful in ongoing research
likely to be usable without detailed knowledge of its internals
Has this resource been validated?
Yes
How has the resource been validated?

This an open research dataset updated regularly by the community in response to COVID-19.

Link to Resource Credibility Assessment
COVID-19 AI Challenges
Can this resource be associated with other resources? (e.g.: modular models, linked tools and platforms)
Yes
Which resources?

 

Additional Resources

 

Key publications (e.g. describing or using resource)

Publisher Resources

DOI link to publication describing this resource

https://pages.semanticscholar.org/coronavirus-research

Link to resource
COVID-19 Open Research Dataset
Collaborators
Allen Institute for AI
PI contact information
https://allenai.org/
Table sorting checkbox
Off
Subscribe to