05-futurework.Rmd

\addtocounter{chapter}{1}\setcounter{section}{0}\specialchapt{CHAPTER 5. FUTURE WORK}

There are a number of areas demanding further research and exploration to turn this procedure into something that can and should be used in a criminal justice setting. Ultimately, the biggest limitation of the algorithms described is in terms of the reference database. At the time of this writing, the Hamby and Cary studies remain the only publicly available scanned bullets in the 3D surface format. From Hamby, the two sets comprise 70 bullets that were fired from 10 barrels. The Cary study is a single barrel. In other words, there are 11 total gun barrels used to generate the results derived in this document. Naturally, this is a limiting factor in terms of the out-of-sample performance. While the initial results are promising, confidence can not and should not be too high in the procedure before the reference database contains a wide variety of gun barrels, fired under a number of real world scenarios, and the algorithms have been tested using bullets recovered from crime scenes. In particular, the simulation of degraded data may or may not match real world degradation scenarios, and it would be far more useful to assess on a controlled set of true degraded data. The good news is that as bullets are added to the database, the algorithms can be rerun seamlessly to generate updated results.

Another area to explore is the extensive amount of parameters available for tweaking. While we have attempted to use cross-validation to optimize a number of these, including the optimal smoothing factor, there are also other parameters that have been fixed. Ideally, we could tune the parameters for each cross-comparison of lands. This would naturally require significant computational power and time, but would likely lead to more accurate predictions compared with the fixed parameter choices. Even if fixed values are chosen, they can also be optimized such that the fixed values lead to the best accuracy given cross-validation.

While we have documented the presence of operator or scan effects, we didn't make many recommendations in terms of how these should be accounted for. It is simple to verbalize a solution - rigorous procedures for scanning, consistent across labs and operators. However, in practice this is challenging. We could attempt to quantify the scan parameters across operators and adjust for them in some manner, but if the scans themselves are too low quality, this is unlikely to be effective. On the other hand, the model themselves could include an operator or microscope component that is included to handle this automatically. At this point, with two sets of scans, we have too little data to work with, but this will certainly need to be explored as time goes on.

One area that may limit the adoption of automated or semi-automated computer-based algorithms in the forensic community is the desire to still perform manual alignment and matching using expert training. To supplement the examiners, a full-featured 3D viewer that operates as a comparison microscope is needed. We have a very basic version of this, allowing the zooming, panning, and rotating of the bullet lands. But fine-grained alignment of the striations is challenging or impossible in the current framework.

Finally, there should be more exploration into a generative model for land signatures. If an accurate generative model could be created, we could begin to create likelihood ratios that truly assess the odds that a particular bullet was fired from a gun, in a way that could have significant sway in court. Unfortunately, given the wide variety of firing conditions, barrels, ammunition, etc., such a generative model could be very challenging to create. We have chosen the route of avoiding this issue, and instead using an empirical model based on the features generated on land-to-land comparisons. Still, this is an area of research that will likely be a significant pursuit going forward.