Skip to content

Latest commit

 

History

History
88 lines (71 loc) · 2.66 KB

README.md

File metadata and controls

88 lines (71 loc) · 2.66 KB
Raw

TorchFuzz

TorchFuzz is an open-source for fuzzing of pytorch models.

Base code

The code is based on EvalDNN code.
You can find original code here: https://github.com/yqtianust/EvalDNN

Mutation methods used are as follows.

  • translation
  • scale
  • shear
  • contrast
  • rotation
  • brightness
  • blur
  • GaussianBlur
  • MedianBlur
  • bilateraFilter

You can find more information about mutation method in paper as follows.
https://doi.org/10.1145/3293882.3330579

Usage

Installation

Place torchfuzz folder and requirements.txt to your workspace folder.

pip install -r requirements.txt

Evaluate a model

from torchfuzz.models.pytorch import PyTorchModel

measure_model = PyTorchModel(net, device='cuda' if torch.cuda.is_available() else 'cpu')
measure_model.run_fuzzing(trainloader, isTrain=True, threshold=0.5, isRandom=0)
measure_model.run_fuzzing(testloader, isTrain=False, threshold=0.5, isRandom=0)

Wrap model with PytorchModel().

  • device : string
    Choose which device to run.
    ex. cpu, cuda, cuda:0

use .runfuzzing() to start fuzzing.

  • dataloader : instance of torch.utils.data.dataloader Dataset loader to load the data.
  • isTrain: boolean
    Check wheter dataset is train data or test data
  • isRandom: interger
    0 when want to check all parameters else positive integer
  • threshold: float
    Neuron coverage activate threshold
  • params_list: two-dimensional list or empty
    Empty if want to use base parameters else two-dimensional list of parameters

Base Parameters list

params_list = [
[-3, -2, -1, 1, 2, 3],          # translation
[7, 8, 10, 11, 12,],            # scale
[-6, -5, -3, 3, 5, 6],          # shear
[5, 7, 9, 11, 13],              # contrast
[-50, -40, -30, 30, 40, 50],    # rotation
[-20, -10, 10, 20],             # brightness
[1, 2, 3, 5, 7, 9],             # blur
[1, 3, 5, 7, 9, 11],            # GaussianBlur
[1, 3, 5],                      # MedianBlur
[6, 9]                          # bilateraFilter
]

Output

All datas are stored in ./cache folder.

  • nc_arr.npy: Coverage of train data
  • corpus.pickle: Mutationed data that are classified well
  • crash_increase.pickle: Mutationed data that are classified wrong and incrase coverage compared to train data
  • crash_no_increase.pickle: Mutationed data that are classified wrong and didn't incrase coverage compared to train data

All pickle files structure is as follows.

[list of mutation data, list of label, list of original data, list of mutation parameter]