Python3 CSI decoding and plotting

README.md

@@ -44,7 +44,7 @@ After following the [getting started](#getting-started) guide for your device be
 
 ## Analyzing the CSI
 
-Each UDP packet containing collected CSI has 10.10.10.10 as source address and is destined to 255.255.255.255 on port 5500. The payload starts with four magic bytes 0x11111111, followed by the six byte source mac address as well as the two byte sequence number of the Wi-Fi frame that triggered the collection of the CSI contained in this packet. The next two bytes contain core and spatial stream number where the lowest three bits indicate the core and the next three bits the spatial stream number, e.g. 0x0019 (0b00011001) means core 0 and spatial stream 3. The chanspec used during extraction can be found in the subsequent two bytes. After two bytes identifying the chip version, the actual CSI data follows. Relative to using 20, 40, or 80 MHz wide channels those are 64, 128, or 256 times four bytes long. For the bcm4339 and bcm43455c0 the data contains interleaved int16 real and int16 imaginary parts for each complex CSI value. The bcm4358 and bcm4366c0 return values in a floating point format with one bit sign of the following nine or twelve bits of a real part and the same for an imaginary part, followed by an exponent of five or six bits. We provide matlab scripts under utils/matlab/ for reading and plotting both formats. Make sure to compile a mex file from utils/matlab/unpack_float.c before reading values of the bcm4358 or bcm4366c0 for the first time. Then fill in the configuration section in utils/matlab/csireader.m and run the script. There is an example capture file utils/matlab/example.pcap holding four UDPs of a capture on a bcm4358 for two cores and two spatial streams.
+Each UDP packet containing collected CSI has 10.10.10.10 as source address and is destined to 255.255.255.255 on port 5500. The payload starts with four magic bytes 0x11111111, followed by the six byte source mac address as well as the two byte sequence number of the Wi-Fi frame that triggered the collection of the CSI contained in this packet. The next two bytes contain core and spatial stream number where the lowest three bits indicate the core and the next three bits the spatial stream number, e.g. 0x0019 (0b00011001) means core 0 and spatial stream 3. The chanspec used during extraction can be found in the subsequent two bytes. After two bytes identifying the chip version, the actual CSI data follows. Relative to using 20, 40, or 80 MHz wide channels those are 64, 128, or 256 times four bytes long. For the bcm4339 and bcm43455c0 the data contains interleaved int16 real and int16 imaginary parts for each complex CSI value. The bcm4358 and bcm4366c0 return values in a floating point format with one bit sign of the following nine or twelve bits of a real part and the same for an imaginary part, followed by an exponent of five or six bits. We provide matlab scripts under utils/matlab/ for reading and plotting both formats. Make sure to compile a mex file from utils/matlab/unpack_float.c before reading values of the bcm4358 or bcm4366c0 for the first time. Then fill in the configuration section in utils/matlab/csireader.m and run the script. There is an example capture file utils/matlab/example.pcap holding four UDPs of a capture on a bcm4358 for two cores and two spatial streams. We also provide Python3 scripts to read and plot CSI values from bcm43455c0 and bcm4339 in utils/python.
 
 ## Example
 

utils/python/.gitignore

@@ -0,0 +1,160 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

utils/python/README.md

@@ -0,0 +1,60 @@
+# CSI Explorer
+
+A fast and simple CSI decoder written in Python.
+
+You can:
+- Plot CSI samples from .pcap files generated
+- Read CSI samples to use in your Python programs
+
+## Plotting
+
+1. Install dependencies: `pip install numpy matplotlib`
+2. Open the `config.py` file and select your WiFi chip. Default is bcm43455c0 (Raspberry Pi)
+3. Copy your pcap files to the `pcapfiles` folder. This folder can be changed in the config file.
+4. Run csi-explorer: `python3 csiexplorer.py`
+
+Enter the pcap's filename. Typing the `.pcap` part is optional.
+Type an index to show it's plot.
+Type indexes separated by `-` to play animation (example: `10-20`).
+
+![Screenshot](./docs/screenshot.png)
+
+Type `help` to see all options.
+Null-subcarriers are hidden by default, you can change this in the config file.
+
+## Using csi-explorer in your programs.
+
+You can integrate csi-explorer into your python programs.
+Here is an example for the 'Interleaved' decoder for bcm43455c0 and bcm4339,
+
+```Python
+from decoders import interleaved as decoder
+
+samples = decoder.read_pcap('pcapfiles/example-80.pcap')
+
+samples.csi # Access all CSI samples as a numpy matrix
+
+# Get the 200th sample, but remove Null
+# and Pilot OFDM subcarriers.
+csi = samples.get_csi(
+    index=200,
+    rm_nulls=True,
+    rm_pilots=True
+)
+
+mac = samples.get_mac(index=200)    # Source Mac ID
+sc, fn = samples.get_seq(index=200) # Sequence and fragment number
+css = samples.get_css(index=200)    # Core and Spatial Stream
+
+samples.bandwidth # Bandwidth, automatically inferred from the pcap file
+samples.nsamples  # Number of samples
+
+# Print info about the 200th sample
+samples.print(200)
+
+```
+
+## Authors
+* [@Gi-z](https://github.com/Gi-z) - Glenn Forbes
+* [@zeroby0](https://github.com/zeroby0) - Aravind Voggu
+* [@tweigel-dev](https://github.com/tweigel-dev) - Thomas Weigel

utils/python/config.py

@@ -0,0 +1,83 @@
+# Basic Configuration
+# ===================
+
+# Chip - Uncomment *one* of the lines below
+# ---------------------------------------
+
+# chip = 'bcm4339'    # Nexus 5
+chip = 'bcm43455c0' # Raspberry Pi 3B+ and 4B
+# chip = 'bcm4358'    # Nexus 6P
+# chip = 'bcm4366c0'  # Asus RT-AC86U
+
+
+# Fileroot - Path to the directory with .pcap files
+# -------------------------------------------------
+
+pcap_fileroot = 'pcapfiles'
+
+
+# Miscellaneous
+# -------------
+
+print_samples = True  # Set this to False to stop printing to terminal
+plot_samples = True   # Set this to False to stop plotting
+plot_animation_delay_s = 0.005 # Delay between csi plots.
+
+# Setting this option to True removes Null Subcarriers.
+# Null subcarriers have arbitrary values, and are used to
+# help WiFi co-exist with other wireless technologies.
+# https://www.oreilly.com/library/view/80211ac-a-survival/9781449357702/ch02.html
+remove_null_subcarriers = True
+
+# Pilot subcarriers are used to control the WiFi link,
+# while other subcarriers carry data. I found pilot
+# subcarriers sometimes have inconsistent CSI compared
+# to the rest, and so I remove them. You may not necessarily
+# face such issues.
+remove_pilot_subcarriers = False
+
+
+
+
+# Advanced configuration. You don't need to change this
+# =====================================================
+
+# Decoder
+# -------
+
+if chip in ['bcm4339', 'bcm43455c0']:
+    # The Real and Imaginary values of CSI
+    # are interleaved for these chips
+    decoder = 'interleaved'
+elif chip in ['bcm4358', 'bcm4366c0']:
+    # Right now, there is no support for
+    # CSI encoded as floating point values.
+    # If this is important to you, please raise
+    # an issue.
+    decoder = 'floatingpoint'
+else:
+    decoder = chip
+
+
+help_str = f'''
+CSI Reader
+==========
+
+A simple Python utility to
+explore nexmon_csi CSI samples.
+
+Change the config.py to match your
+WiFi chip and bandwidth. Current chip
+is {chip}.
+
+To explore a sample, type it's
+index from the pcap file. Indexes
+start from 0.
+
+To plot a range of samples as animation,
+type their indexes separated by '-'.
+
+Type 'help' to see this message again.
+
+Type 'exit' to stop this program.
+'''

utils/python/csiexplorer.py

@@ -0,0 +1,76 @@
+import time
+import importlib
+import config
+from plotters.AmpPhaPlotter import Plotter # Amplitude and Phase plotter
+decoder = importlib.import_module(f'decoders.{config.decoder}') # This is also an import
+
+def string_is_int(s):
+    '''
+    Check if a string is an integer
+    '''
+    try:
+        int(s)
+        return True
+    except ValueError:
+        return False
+
+
+if __name__ == "__main__":
+    pcap_filename = input('Pcap file name: ')
+
+    if '.pcap' not in pcap_filename:
+        pcap_filename += '.pcap'
+    pcap_filepath = '/'.join([config.pcap_fileroot, pcap_filename])
+
+    try:
+        samples = decoder.read_pcap(pcap_filepath)
+    except FileNotFoundError:
+        print(f'File {pcap_filepath} not found.')
+        exit(-1)
+
+    if config.plot_samples:
+        plotter = Plotter(samples.bandwidth)
+
+    while True:
+        command = input('> ')
+
+        if 'help' in command:
+            print(config.help_str)
+
+        elif 'exit' in command:
+            break
+
+        elif ('-' in command) and \
+            string_is_int(command.split('-')[0]) and \
+            string_is_int(command.split('-')[1]):
+
+            start = int(command.split('-')[0])
+            end = int(command.split('-')[1])
+
+            for index in range(start, end+1):
+                if config.print_samples:
+                    samples.print(index)
+                if config.plot_samples:
+                    csi = samples.get_csi(
+                        index,
+                        config.remove_null_subcarriers,
+                        config.remove_pilot_subcarriers
+                    )
+                    plotter.update(csi)
+
+                time.sleep(config.plot_animation_delay_s)
+
+        elif string_is_int(command):
+            index = int(command)
+
+            if config.print_samples:
+                samples.print(index)
+            if config.plot_samples:
+                    csi = samples.get_csi(
+                        index,
+                        config.remove_null_subcarriers,
+                        config.remove_pilot_subcarriers
+                    )
+                    plotter.update(csi)
+        else:
+            print('Unknown command. Type help.')