cs234_final

This is Michael Kossyrev's repo for the default final project of CS234: Reinforcement Learning.

What each file does

All scripts use warfarin_binary_supersparse.xslx unless otherwise stated.

• appx.pdf - appendix for the Warfarin research paper - where I extracted the S1f formula (from section S1f...)
• baselines.py - Runs the two baselines and outputs the fraction correct (on the subset of data as discussed in the paper)
• final.py - Runs my LinUCB bandit on the data, outputs information into output.xlsx; First 6 sheets: 0 = 5mg baseline, 1 = S1f baseline, 2 = my bandit. 10 runs total, run x corresponds to the same run across all sheets. 'regret_warfarin_cmp' sheet holds the values of all of the runs for just the patients that yield valid results in S1f. First 10 runs are S1f, next 10 are bandit, last 10 are 5mg. Runs 0,10,20 are all the same run; runs 1,11,21 are all the same run, etc.
• final_nn.py - Runs the neural network. Outputs info to output_NN_arc{i}.xlsx. Right now will output to arc2, as that was the last network I ran. 5 sheets, one for each alpha; 10 runs per sheet.
• output.xslx - As discussed above. This file is HEAVILY modified to actually display the items I need; no data tampered with (obviously), just rearranged. You'll find graphs, the sheet titles are self-explanatory.
• output_NN_{}.xlsx - As described above. A bit of rearranging done, just to get the average/std_dev values.
• pre_process_for_S1f process.py - self-explanatory. Takes warfarin_orig.xlsx, pulls out the relevant columns for S1f, 0s out patients that have "N/A" in any of the columns. Outputs warfarin_S1f.xlsx
• pre_process_sparse - Processes warfarin_orig.xlsx. Outputs warfarin_binary_supersparse.xlsx; contains the 91-length patient vectors
• warfarin_orig.xlsx - 'warfarin' sheet contains unmodified values. Sheets 1 and 2 contain therapeutic doses of warfarin (mg/day). Sheet 3 contains 'warfarin', after pruning things like comorbidities and genotypes, etc. This sheet gets input to pre_process_sparse

How to replicate my results:

1. Delete everything except the scripts and warfarin_orig.xlsx
2. run both pre processing scripts
3. Go into the generated sheets and delete the first column (generated by Pandas, unnecessary). Add a column of 1s to supersparse and S1f at the end
4. Run final.py, and final_nn.py
5. Look at the outputs
6. ???
7. Profit!