clean up repo (#188)

* add mixed precision script

* cleanup

* move scripts

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 default_language_version:
     python: python3
-exclude: ^(delphes/tev14_pythia8_*.)|^(images/)|^(clic/dumper_hepsim.py)|^(mlpf/pyg/__init__.py)|^(fcc/clicRec_e4h_input.py)|^(fcc/clic_steer.py)|^(fcc/pythia.py)|^(fcc/PandoraSettings)
+exclude: ^(scripts/delphes/tev14_pythia8_*.)|^(images)|^(scripts/fcc/clicRec_e4h_input.py)|^(scripts/fcc/clic_steer.py)|^(scripts/fcc/pythia.py)|^(scripts/fcc/PandoraSettings/.*)
 
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks

README.md

@@ -31,8 +31,3 @@ MLPF focuses on developing full event reconstruction based on computationally sc
   - paper: https://doi.org/10.1140/epjc/s10052-021-09158-w
     - code: https://doi.org/10.5281/zenodo.4559587
     - dataset: https://doi.org/10.5281/zenodo.4559324
-
-Short instructions with a single test file in [notebooks/delphes-tf-mlpf-quickstart.ipynb](notebooks/delphes-tf-mlpf-quickstart.ipynb).
-
-Long instructions for reproducing the full training from scratch in [README_delphes.md](README_delphes.md).
-The plots can be generated using the notebook [delphes/delphes_model_analysis.ipynb](delphes/delphes_model_analysis.ipynb).

apptainer/python_base.txt

@@ -0,0 +1,60 @@
+POT
+PyYAML
+astropy
+awkward
+awkward0
+black
+bokeh
+boost-histogram
+corner
+cupy-cuda11x
+dask
+distributed
+docopt
+emcee
+energyflow
+fastjet
+fastparquet
+flake8
+girder-client
+hdbscan
+healpy
+hydra-core
+imageio
+imageio-ffmpeg
+ipyparallel
+isort
+jupyter
+line_profiler
+lmfit
+lz4
+matplotlib
+memory_profiler
+mpl_scatter_density
+mplhep
+networkx
+notebook
+numba
+numpy
+pandas
+papermill
+parsl
+particle
+plotly
+pre-commit
+pyarrow
+pydot
+pygraphviz
+pymultinest
+pynbody
+pytest
+scikit-learn
+scipy
+seaborn
+tables
+tensorboard
+tqdm
+uproot
+vector
+xxhash
+zenodo_get

apptainer/python_tf.txt

@@ -0,0 +1,17 @@
+comet-ml
+datasets
+keras-flops
+keras-tuner
+keras_core
+onnx
+onnxruntime
+ray[default]
+ray[tune]
+tensorboard_plugin_profile
+tensorflow-datasets
+tensorflow-estimator
+tensorflow-model-optimization
+tensorflow-text
+tf-models-official
+tf2onnx
+transformers

apptainer/tf-2.13.0.singularity

@@ -0,0 +1,29 @@
+Bootstrap: docker
+
+From: tensorflow/tensorflow:2.13.0-gpu-jupyter
+
+%files
+    python_base.txt /opt/python_base.txt
+    python_tf.txt /opt/python_tf.txt
+
+%post
+    apt update -y --fix-missing
+    apt install -y make cmake parallel gcc g++ gfortran binutils
+    apt install -y libblas3 libblas-dev liblapack3 liblapack-dev libatlas3-base libatlas-base-dev
+    apt install -y libtcmalloc-minimal4
+    apt install -y graphviz graphviz-dev
+    apt install -y git
+    apt install -y wget
+
+    python3 -m pip install --upgrade pip
+    python3 -m pip install -r /opt/python_base.txt
+    python3 -m pip install -r /opt/python_tf.txt
+    python3 -m pip install hls4ml[profiling]
+
+%environment
+    export PIP_DEFAULT_TIMEOUT=500
+    export LC_ALL=""
+    export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
+
+%runscript
+    /bin/bash