A collection of notes on my research workflow in physical oceanography.
- Setting up macOS
- Setting up linux
- Setting up github
- Starting a new scientific project
- Creating simple python modules and packages
- Working with HPCs
- Additional learning resources
- Working collaboratively
- Archiving and publishing (zenodo, DOIs)
- Working at sea
Other bits and bobs:
- Code snippets for getting files
- Useful bash code
- Code snippets for making movies
- Tips for exploring data, ipython widgets?
This repository is a collection of notes on my computational scientific workflow. It is constantly under refinement as I learn new things or as software tools improve (I am always happy to hear about new efficient ways of working!) The notes may not be relevant to everyone, but might be useful to those who do similar things to me, such as:
- write code in python/matlab/R/Julia
- analyse environmental data (both large and small datasets derived from in situ measurements or simulated numerically)
- use high performance computing (HPC) clusters
- work on unix-like machines (macOS/linux)
Two important underlying goals of my workflow are that it be well documented, easily repeatable and easily shared. To achieve this I made extensive use of tools such as:
- version control (e.g. git and github.com)
- environments (e.g. conda)
- the shell (e.g. command line interfaces) and scripting
Recently, scientific journals have started to require that code and data are archived and accessible prior to publication. I have found that working with tools commonly used for software development streamlines the process of meeting open-access requirements considerably.
There are loads of good resources on reproducible open science:
- Stoudt, S., Vásquez, V. N., & Martinez, C. C. (2021). Principles for data analysis workflows. PLOS Computational Biology, 17(3), e1008770. https://doi.org/10.1371/journal.pcbi.1008770
- Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013). Ten Simple Rules for Reproducible Computational Research. PLoS Computational Biology, 9(10), 1–4. https://doi.org/10.1371/journal.pcbi.1003285