UPDATE 9/2020 - major change - using solely NetworkAPI of HTM.core - fully AUTOMATIC baking process ! UPDATE 6/2020 - major change - SQLITE3 instead of TCP sockets + Dash plots visualization in web browser !
See video tutorial or presentation video!
Screenshots for visualization of the 2D recognition project
This project aspires to create tool that helps visualize HTM systems in 3D by using opensource framework for 3D rendering https://www.panda3d.org/
It allows to see architecture of the system in 3D space, e.g. connection of several layers and inputs and to see input representation, activity of columns and even individual cells in each simulation step. User can observe vast scalable space by moving as "ghost" and interact with objects. It is supposed as tool for educational purpose or as an inspect tool.
I was inspired by following:
The visualization is application written purely in Python3.
- Data for visualization are generated by so called "baking". This process generates sqlite3 database file, optionally folder with binary dump files.
- User can open these data in HTMpandaVis and explore them
You can also browse SQLite3 dabase file (.db) directly with ordinary browser such as Sqlite Browser.
To bake your simulation, you must use htm.core networkAPI.
Only difference is, that you need to import instead of
from htm.bindings.engine_internal import Network
this
from pandaBaker.pandaNetwork import Network
That is all. PandaVis will generate automatically all data.
If you don't want to change imports and keep using panda, you can modify flag Network.bakePandaData = True/False
See baking readme.
HTMpandaVis can be used also to record custom dataStreams with pandaBaker and then visualize it in web browser. Dash plotly library is used. These plots are interactive!. For creating layout arrangement, axis and plot labels, there are JSON layout configuration files. They are located in HTMpandaVis\dashVis\layouts. See hotgym example for more informations.
Within htm.core, there is databaseRegion in networkAPI that allows to output scalar values into sqliteDatabase. Data can be then plot in the same way as above. See htm.core examples/napi_hello_database.
Python >3.6 is recommended. Also using one of the python environment managers is recommended, like Anaconda
Install htm.core (here building from source, see repo readme if you need other installation instructions)
sudo apt-get install cmake
git clone https://github.com/htm-community/htm.core.git
python setup.py install --user --force
Install prerequisities & clone pandaVis
sudo apt-get install python3-tk
git clone https://github.com/htm-community/HTMpandaVis.git
python -m pip install -r requirements.txt
#this installs pandaBaker package for baking process
python setup.py install
There is NetworkAPI hotgym example in the htm.core repository.
All you have to do to bake it, is changing one line!
- For baking change :
from htm.bindings.engine_internal import Networktofrom pandaBaker.pandaNetwork import Network
when you run the script, database and dump files will be created inside script folder in /bakedDatabase
- Run client - pandaVis tool
cd HTMpandaVis
python run.py
And choose run 3D explorer or "run both" if you want to run dash visualization also.