Metis is an open scientific framework, materials data organizer, and collaborative online platform for the nanotechnology research. It was designed for the materials research teams at the physical and virtual laboratories. Metis is an AI-ready solution, aiming to bring the recent advances of computer science into a rather conservative area of new materials development and quality control. Metis currently focuses on the X-ray powder diffraction and atomistic simulations. It was started in 2021 in BASF (Ludwigshafen am Rhein, Germany) by Bernd Hinrichsen and Evgeny Blokhin.
This is the third part of the whole Metis infra: GUI ⇄ BFF ⇄ backend.
Metis backend presents minimalistic utility toolset in materials informatics and Flask-based CRUD-server for the nano-scale crystalline data, XRPD measurements, and cloud simulations.
The basic requirements are Python, Numpy, and PostgreSQL. Note that the Numpy depends on the low-level Fortran numeric system libraries, which might be absent in your system.
A scientific cloud scheduler yascheduler is required for a separate cloud orchestration. An optional requirement is AiiDA, which can be linked to the cloud scheduler. The AiiDA is a Python framework for the complex scientific workflows, requiring PostgreSQL database and RabbitMQ message broker.
Thus, metis-backend consists of the 3 independent parts, each using PostgreSQL:
metis-backendPython server per seaiida_coreworkflow engineyaschedulercloud scheduler
Optionally, a frontend server is Nginx (conf/nginx.conf goes to /etc/nginx), and all these guys are controlled by the Supervisor daemon (conf/supervisord.conf goes to /etc/supervisor).
Refer to conf/install.sh for installation of Nginx, PostgreSQL, RabbitMQ, and Supervisor, as well as configuring them. Run conf/install.sh and then modify global options conf/env.ini. Alternatively, feel free to install each component on your own (or all together in a container, see below).
The AiiDA can be installed and configured separately. First, a possibility for ssh-ing into a localhost should be ensured:
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh $USER@localhostThen a yascheduler plugin is configured (conf/yascheduler.conf goes to /etc/yascheduler). After that run yainit to start a yascheduler service. Then, if you have already run conf/install.sh script, it should be enought to do:
pip install aiida_core
bash conf/aiida_setup.shThen setup your virtual env, if needed, and install the Python requirements as simply aspip install -r requirements.txt.
Finally, apply the database schema: /data/pg/bin/psql -U postgres -d metis -f schema/schema.sql.
Run supervisorctl status to see all the services in Supervisor.
One by one, all the parts are managed as follows:
metis-backendserver is started withindex.pyyascheduleris started simply withyaschedulercommand- AiiDA is managed with
verdi, e.g.verdi process listorverdi node show - AiiDA daemon is started separately with
verdi daemon start - PostgreSQL database(s) can be seen with
/data/pg/bin/psql -U postgres -l - RabbitMQ is controlled with
rabbitmqctl status - (Nginx can be added to Supervisor as well depending on your taste)
A script scripts/assign_phases.py should be run regularly to organize users' uploaded data, e.g. in cron job scheduler
*/5 * * * * /path/to/metis-backend/scripts/assign_phases.py
This is an experimental feature intended primarily for development and testing.
It is assumed that you have the metis-backend, metis-bff, and metis-gui
repositories cloned on the same level. Also, you need docker (or podman
and podman-compose) installed.
Now, you can run docker compose up (podman-compose up) in metis-backend
directory. This should start all dependencies and services.
metis-gui should be available at http://localhost:10000/
metis-bff should be available at http://localhost:3000/
- CIF
- POSCAR
- Optimade JSON
- XY patterns (TSV-alike)
- Bruker's RAW (binary)
- Bruker's Topas CLI
- Synchrotron HDF5 (binary)
Copyright 2021-2023 BASF SE
BSD 3-Clause