Algorithm was adapted from this tutorial: https://towardsdatascience.com/reinforcement-learning-w-keras-openai-actor-critic-models-f084612cfd69
These instructions are for a docker container containing all of the dependancies. If you are using ubuntu you could follow these instructions outside of a container.
You need nvidia-docker installed to utilize the gpu from a docker container.
Start a container
$ nvidia-docker run -it -v /path/to/storage/:/path/in/container/ tensorflow/tensorflow:latest-gpu /bin/bash
Within the container, set up gym and keras
$ apt update
$ pip install gym
$ pip install keras
Clone crystal-ac-ml and pycrysfml. For ease of transition, either put them in the home directory or change all filepaths in hkl.py.
$ apt install git
$ git clone https://github.com/scattering/crystal-ac-ml.git
Follow instructions in pycrysfml/doc/Ubuntu_deps.txt to set up pycrysfml
Replace pycrysfml/hklgen/sxtal_model.py with the sxtal_model.py file in this repository. This updates the model class to accept empty lists for the observed data, so you can create a model without giving it all of the initial data.
Replace the prnio.cfl file in pycrysfml/hklgen/examples/sxtal/prnio.cfl with the prnio_optimized.cfl file. This file contains optimized atomic position values from fitting the data with FullProf. It will allow you to get better fits on the praseodymium nickolate crsytal model when only fitting a single parameter.
Edit the filepath in the top of hkl.py to match your directory structure (i.e., enter the path to the prnio.int file in pycrysfml).
Copy init.py and hkl.py into /usr/local/lib/python2.7/dist-packages/gym/envs/ to register our environment (hkl.py) with openai gym.
Now you are all set up to run actor_critic.py from anywhere in your filesystem.
If you are going to run the code remotely, using slurm rather than directly from inside the container, you need to first save this container as an image.
Exit the container and use 'docker ps -a' to get the container id. Then commit it.
$ docker commit cont-id image-name
Now that the image is commited, it can be run from slurm
cd into your storage folder. Then run:
/storage/you$ srun --mem=0 --nodelist=ncnrgpu1 bash -c '/storage/etc/sudocker run runtime=nvidia -d -v /storage/you/:/path/in/container/ img-name python /path/to/actor_critic.py'
The parts of the srun command are:
--nodelist: the node to run on
bash -c: run a bash command
--runtime=nvida: use nvidia docker, remove this to run with regular docker
To move containers around nodes, save the container to a tar file using
$ docker save container-id