Guitar Transcription with Inhibition

Code for the paper "A Data-Driven Methodology for Considering Feasibility and Pairwise Likelihood in Deep Learning Based Guitar Tablature Transcription Systems". This repository contains scripts which do the following (and more):

• Acquire annotations in JAMS format from GuitarPro data using PyGuitarPro
• Create a matrix of inhibition weights from a collection of tablature (e.g., DadaGP)
• Implement the proposed tablature output layer formulation with inhibition
  • Interchangeable with the TabCNN output layer formulation
• Perform six-fold cross-validation experiments on GuitarSet

The repository heavily utilizes amt-tools, a more general music transcription repository.

Installation

Clone the repository, install the requirements, then install the package:

git clone https://github.com/cwitkowitz/guitar-transcription-with-inhibition
pip install -r guitar-transcription-with-inhibition/requirements.txt
pip install -e guitar-transcription-with-inhibition/

Usage

Convert GuitarPro to JAMS

Update <base_dir> (defined at the bottom of the script) to point to the top-level directory containing GuitarPro files, then run the following:

python guitar_transcription_inhibition/gpro/process_guitarpro.py

The corresponding JAMS annotations will be placed under <base_dir>/jams.

Computing Inhibition Weights

The process for computing inhibition weights for an arbitrary collection of symbolic tablature data involves extending the SymbolicTablature dataset wrapper and employing an InhibitionMatrixTrainer.

Please see acquire_dadagp_matrix.py and acquire_guitarset_matrices.py under guitar_transcription_inhibition/inhibition for examples on how to compute inhibition weights for DadaGP and GuitarSet, respectively.

Using Proposed Output Layer

The proposed tablature output layer can be initialized as follows:

from guitar_transcription_inhibition.models import LogisticTablatureEstimator

tablature_layer = LogisticTablatureEstimator(dim_in=<dim_in>,
                                             profile=<profile>,
                                             matrix_path=<matrix_path>,
                                             lmbda=<lmbda>)

Note that tablature_layer is an instance of torch.nn.Module, and can be used as such. It also acts as a standalone instance of amt_tools.models.TranscriptionModel, and therefore implements the relevant functions TranscriptionModel.pre_proc and TranscriptionModel.post_proc.

Please see the documentation in guitar_transcription_inhibition/models/tablature_layers.py and amt-tools for more information regarding the input arguments and usages within the amt-tools framework.

Six-Fold Cross-Validation on GuitarSet

The scripts experiment.py and evaluation.py under six_fold_cv_scripts are also available as a more complete example of how to train and evaluate the proposed model under the six-fold cross-validation schema using amt-tools.

Generated Files

Execution of six_fold_cv_scripts/experiment.py will generate the following under <root_dir> (defined at the top of the script):

• n/ - folder (beginning at n = 1)¹ containing sacred experiment files:
  • config.json - parameter values used for the experiment
  • cout.txt - contains any text printed to console
  • metrics.json - evaluation results for the experiment
  • run.json - system and experiment information
• models/ - folder containing saved model and optimizer state at each checkpoint, as well as an events file (for each execution) readable by tensorboard
• results/ - folder containing separate evaluation results for each track within the test set
• _sources/ - folder containing copies of scripts at the time(s) execution

Additionally, ground-truth and features will be saved under the path specified by gset_cache, unless save_data=False. Scripts related to GuitarPro -- JAMS conversion, inhibition matrix acquisition, visualization, and evaluation will also generate the respective files.

¹An additional folder (n += 1) containing similar files is created for each execution with the same experiment name <EX_NAME>.

Analysis

During training, losses and various validation metrics can be analyzed in real-time by running:

tensorboard --logdir=<root_dir>/models --port=<port>

Here we assume the current working directory contains <root_dir>, and <port> is an integer corresponding to an available port (port = 6006 if unspecified).

After running the above command, navigate to http://localhost:<port> with an internet browser to view any reported training or validation observations within the tensorboard interface.

Cite

SMC 2022 Paper (Link)

@inproceedings{cwitkowitz2022data,
  title     = {A Data-Driven Methodology for Considering Feasibility and Pairwise Likelihood in Deep Learning Based Guitar Tablature Transcription Systems},
  author    = {Frank Cwitkowitz and Jonathan Driedger and Zhiyao Duan},
  year      = 2022,
  booktitle = {Proceedings of Sound and Music Computing Conference (SMC)}
}