diff --git a/evaluation/README.md b/evaluation/README.md
new file mode 100644
index 0000000..e524be0
--- /dev/null
+++ b/evaluation/README.md
@@ -0,0 +1,21 @@
+# How to Evaluate
+
+## Human Eval
+
+The following steps are required to run the Human Eval step:
+1. Ensure you are using python3.7 as required by [human-eval](https://github.com/openai/human-eval). We recommend conda:
+```
+conda create -n human-eval python=3.7
+```
+2. Install the dependencies in this folder
+```
+pip install -r requirements.txt
+```
+3. Install human-eval by following the instructions on the [human-eval repo](https://github.com/openai/human-eval#usage)
+
+
+With the following requirements performed you can now run the `evaluation.py` script:
+```
+python evaluate.py --model_name_or_path=model_name_or_path --human_eval_path=<path/to/human-eval/data/HumanEval.jsonl.gz> --out_path=./model_results
+```
+So for example if you want to evaluate the EleutherAI GPT Neo 125M
\ No newline at end of file
diff --git a/evaluation/requirements.txt b/evaluation/requirements.txt
new file mode 100644
index 0000000..e4060e8
--- /dev/null
+++ b/evaluation/requirements.txt
@@ -0,0 +1,4 @@
+torch
+fastcore
+transformers
+tqdm
\ No newline at end of file