Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Model #99

Open
sam-writer opened this issue Mar 19, 2020 · 7 comments
Open

Improve Model #99

sam-writer opened this issue Mar 19, 2020 · 7 comments
Labels
documentation Improvements or additions to documentation models Correct/improve the underlying models

Comments

@sam-writer
Copy link
Collaborator

Overview

Currently, we use a deterministic SIR model (see sir and sim_sir in models.py) to predict everything. It does not have many parameters, which I think contributes to the ease of use and adoption of the tool... however, accuracy is also of paramount importance. There have been multiple proposed improvements.

Proposed Improvements

Concerns

  • "my understanding is the SIR model's more of a guesstimate that can be fit retrospectively but isn't that predictive for changing circumstances. It doesn't account for household contact, or hordes of folks driving their dying relatives from one jam-packed hospital to the next, or the larger consequences of jamming 200 octogenarians into a group home manned by underpaid attendants with a shortage of tests and protective gear. But we're fighting the epidemic blind, so it's what we've got." src
  • "There has been a lot of talk about using more complex models, but the hurdles are (1) usability (2) uncertainty/unavailability of the required inputs. I think that the consensus is that better models would be better if they had well-constrianed inputs and didn't make the tool harder for users to adapt to their local contexts. Otherwise better models would be worse." src
  • "What are some additional inputs that are missing? Am very interested in modelling with different values for various activities, e.g. school transmission, intrahousehold transmission, workplace transmission. Obviously this is not straightforward. Moreover the SIR model totally leaves out the fact that different populations are more or less vulnerable, if one is looking at hospital capacity there are pretty sizeable regional population variations. For a first order guess it's useful, but the real world is full of cascading impacts that are very hard to guess." src

Definition of Done

This ticket is complete when we have a plan for improving the model, which takes into account the concerns.

@sam-writer sam-writer added help wanted Extra attention is needed python hospital-request A hospital needs this to make a decision models Correct/improve the underlying models labels Mar 19, 2020
@briglass
Copy link

Regarding incoming infections, this simulator: http://covidsim.eu/ has an "Infections from outside of the population [per day]", which I am assuming to be accomplished via a monotonic daily increase to S, although I am not sure

@hakocon
Copy link

hakocon commented Mar 19, 2020

I really like the idea of social distancing (or I was also thinking population density as a proxy based on ZIP or something like that). Also, would it be valuable to bring in the age distribution of a location? This is specifically dangerous for older folks, so locations with a higher number of elderly might see different distributions of acute/critical care patients than those with lower numbers. Just my thoughts, thanks!

@sam-writer
Copy link
Collaborator Author

@hakocon demographics seem very important, I agree, and that is probably just an API call away.

Population distancing as a proxy for social distancing, however, I feel like that assumption could be very wrong - e.g. San Fransisco, with it's shelter in place order, vs somewhere in rural Alaska which, as of today, has a Representative who is saying that fears over "the beer virus" are "blown out of proportion" and that "it’s not nearly as deadly as the other viruses we have."

I don't think we should infer this... but if we were going to automatically infer social distancing for a location, I would rather do it by an algorithm like (pseudocode):

social_distancing_phrases = ["shelter in place", "social distancing", ...]
hoax_phrases = ["hoax", "overblown", ...]

sd_score = 0
hoax_score = 0

def infer_social_distancing(city_name):
    search_results = get_google_top_100_pages_for_term(city_name + " coronavirus")
    texts = get_text_from_pages(search_results)
    for sdp in social_distancing_phrases:
        if sdp in text:
            sd_score += 1
        if hoax_score in text:
            hoax_score += 1
    score = somehow_convert_these_to_a_score(sd_score, hoax_score)
    return score

@selahlynch
Copy link

selahlynch commented Mar 20, 2020

Are daily counts of COVID related hospital/ICU bed usage at Penn Medicine or another hospital system available? If so we can use this to test some of the models.

@cjbayesian
Copy link
Collaborator

We need to have an improved way to estimate current # infected. I like the approach taken here: https://github.com/jwrichar/COVID19-mortality/blob/master/State-wise%20Unreported%20COVID-19%20Cases%20in%20the%20U.S..ipynb

@quinn-dougherty
Copy link
Collaborator

issue too broad, but has a lot of useful notes and discussion on it so i'm reticent to close it

@sam-writer
Copy link
Collaborator Author

@quinn-dougherty I agree, this is too broad, but the collection of ideas is valuable. Is there some way to make the output of this ticket:

  • an artifact, summarizing these ideas and the ideas on slack? Not sure what that would look like, or if this issue is possibly the best format for an ongoing brainstorming session on model improvement...
  • a ticket, explaining what model improvements are to be done at this time?

@quinn-dougherty quinn-dougherty added documentation Improvements or additions to documentation and removed help wanted Extra attention is needed hospital-request A hospital needs this to make a decision python labels Mar 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation models Correct/improve the underlying models
Projects
None yet
Development

No branches or pull requests

6 participants