Issue 44: Approximate inference vignette

[athowes]

Description

This PR closes #44.

• Algorithm working
• Explain what the different algorithms do
• Compare the inferences from each. Doing this will help with scoping of how to keep post-processing functions inference method agonistic (if indeed we want to do this)
• Perhaps demo something cool you can do with the faster inference methods (I think out of scope, move to new issue)
• Perhaps make suggestions about when you might want to use the faster inference methods (I think out of scope, move to new issue)

Extensions moved to issue: #141

Checklist

• My PR is based on a package issue and I have explicitly linked it.
• I have included the target issue or issues in the PR title in the for Issue(s) issue-numbers: PR title
• I have read the contribution guidelines.
• I have tested my changes locally.
• I have added or updated unit tests where necessary.
• I have updated the documentation if required.
• My code follows the established coding standards.
• I have added a news item linked to this PR.
• I have reviewed CI checks for this PR and addressed them as far as I am able.

[athowes]

Updates based on #70:

• Change basic implementation of approximate methods to run with epidst function -- perhaps will be simpler

[athowes]

@kgostic @seabbs @parksw3 could you give some midway feedback before I finish up this first version of the approximate inference vignette?

My view of what needs to be done before a first merge:

• Better description of how ADVI works
• Better description (and understanding) of how pathfinder works
• Some description and perhaps figure to explain what is going wrong with pathfinder in this case
• Fix use of forcats package (either get rid or add to recommends or whatever correct thing is)
• A figure to show how the actual delay distributions vary by inference method (perhaps). I should also do a proper method for this in the get started vignette (right now it's just take mean of mean and SD and plot)
• Get the time taken for each method and compare them

My view of what can be left to a second try at improving this:

• Fixing pathfinder running
• Demo something cool you can do with the faster inference methods
• Make suggestions about when you might want to use the faster inference methods

[seabbs]

Content so far looks okay. I assume its an initialisation issue for pathfinder. have you given multipathfinder a spin?

[seabbs]

Content so far looks okay. I assume its an initialisation issue for pathfinder. have you given multipathfinder a spin?

[athowes]

Thanks for the suggestion Sam, haven't tried it yet (though the algorithm did have four runs, and perhaps 1-2 of them had this error, so I may already have been using the "multi" version).

To try r.e. fixing pathfinder (open to other suggestions!):

• Investigate if I'm using "pathfinder" or "multi-pathfinder"
• Try different initialisations

[athowes]

Steps needed to merge this are:

• Pass more tests (finding it hard to see what to do here)
• Make new issue with the extensions to this vignette that I might like to see in future

[seabbs]

Do you actually want a review given outstanding tasks and failing CI or advice on something or?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 38.69%. Comparing base (fa1be26) to head (61d4bc5).
Report is 8 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #69       +/-   ##
===========================================
+ Coverage   28.49%   38.69%   +10.20%     
===========================================
  Files          10       10               
  Lines         565      553       -12     
===========================================
+ Hits          161      214       +53     
+ Misses        404      339       -65     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jamesmbaazam]

Hi @athowes thanks for writing this vignette. Really insightful and similar to what I'm writing for EpiNow2 here epiforecasts/EpiNow2#695. I'd be happy for you to review when it's ready and if you're interested.

Two comments:

• Could you consider adding the time units to the runtimes? In the EpiNow2 vignette, some examples run for minutes compared to seconds for others, and without the units, it was easy to misinterpret what they meant.

• As a Bayesian newcomer, I also found the explanations quite technically heavy. Could you consider moving that to an "appendix" in the vignette so that it doesn't distract the reader who only cares about the immediate results?

[kgostic]

One small typo noted but other than that this is looking good to me!

[athowes]

Thanks so much for reading @jamesmbaazam! I've added the units, and included your suggestion about moving some of the technical detail to an appendix in the follow-up issue about this vignette (#141).

[parksw3]

This looks amazing. I'm shocked at how fast other approximate methods run and how well they work... I found one typo.

• "being being" -> "being"

[seabbs]

This is really nice now. Good to see all the love! I agree with the idea of moving some technical terms to an appendix.