GUI vs INCAWrapper validation #18
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reviewer asked for validation that the a GUI run gives the same results as the INCAWrapper. To accomudate this, we manually setup the simple model INST simulated dataset through the INCA GUI and ran estimation and parameter continuation. The model is save as simpale_model_inst_gui.mat and the resulting flux map is saved as simple_model_inst_gui_flux_map.mat.
…cawrapper into compare_gui_vs_incawrapper
This was an attempt to improve the information in the data, but the results did not change a lot.
Still few reactions with large CIs. KPIs frac well determined reactions: 0.36 frac flux rel error < 0.01: 0.813
Simulated all single labeled fructose and glycerol only one tracer pr experiment. KPIs frac well determined reactions: 0.613 frac flux rel error < 0.01: 0.813
Simulated two experiments one using U-13C versions of either fructose or glycerol. Additionally all pool sizes are left at 1 and measured. KPIs frac well determined reactions: 0.613 frac flux rel error < 0.01: 0.813
Appart from adding all AAs to ms measurements, MS measurements of CO2 was added and biomass exchange measurement. KPIs true flux in interval: 0.973 well determined (cutoff=1): 0.868 Rel. error below 0.01: 0.855
This commit is used to port the code to a VM for estimation as this will take longer time.
…valitation This commit is for porting scripts and data to remove machine.
reverted to commit: 2e64ac1 This model had less measurements but still achieved a reasonable fit. This is important for the test against the INCA GUI to limit the amount of data which should be manully imported.
The simulation strikes a balance by finding a reasonable fit with not too many measurements. The measurements haven't been added to the inca_gui file yet
Because rows are inserted in the mdv dataframe the artificial noise is affected. Data in GUI NOT yet updated.
This reverts commit 51839de. Decided to use the simulated data which includes missing measurements for valine260 M3, because of too much work manually inputting new data into INCA GUI. Both INCA and the INCAWrapper handles this missing data correctly.
SSR: 197
Inconsistency in the loaded data for the GUI and INCAwrapper result files. The mistake occured because a the the simulation was modified to simulate the missing M3 measurement in Valine260, however, the new version fo the simulated data wasn't moved into the INCA GUI. SOLUTION: revert back to simulation which has a missing value in Valine260 MDV.
The notebook describes the validation setup, compares the two flux distributions with uncertainty both visually and numerically. Finally, the notebook concludes that the two executions are equivalent.
Old files that was not used anymore was removed. Then the simulation .mat files was moved to simulated data folder. Finally, fluxmap from incawrapper fit was renamed to follow same naming style as the gui fluxmap .mat file.
We during merge we took the full incoming version of c_necator_simulated_data.ipynb
This merge brings this branch up to date with large changes that were done in the develop branch since the creation of the compare_gui_vs_incawrapper branch
… lack running with 1000 restarts
Before the GUI goodness of fit was printed twice.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR brings in two new notebooks in the docs/examples. Both compare estimated flux distributions from analysis run in the GUI and through INCAWrapper. One notebook uses a small toy model, while the other uses a medium-sized (60 reactions) model.