Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
dedeswim committed Jun 5, 2023
1 parent 2219343 commit 299c90d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ pip install -r requirements.txt

## Evaluate your own attack

To evaluate your own attack against our benchmarks, your attack should implement the `BaseAttack` interface, defined in [src/attacks/base.py](src/attacks/base.py). In particular, if it follows the HSJA scheme of optimizing a perturbation, we suggest to implement the `PerturbationAttack` interface, while if it follows the OPT scheme of optimizing a direction, we suggest to implement the `DirectionAttack` interface. Of course, your attack does not have to follow one of these schemes. When calling the model, you should make sure that you are using the `is_correct_boundary_side` method from `BaseAttack` to make sure that the queries counter is updated correctly. This method returns a tensor of booleans (`True` if the input is classified as *good* and `False` otherwise) and the updated queries counter (which is **not** updated in place to avoid side effects!).
To evaluate your own attack against our benchmarks, your attack should implement the `BaseAttack` interface, defined in [`src/attacks/base.py`](src/attacks/base.py). In particular, if it follows the HSJA scheme of optimizing a perturbation, we suggest to implement the `PerturbationAttack` interface, while if it follows the OPT scheme of optimizing a direction, we suggest to implement the `DirectionAttack` interface. Of course, your attack does not have to follow one of these schemes. When calling the model, you should make sure that you are using the `is_correct_boundary_side` method from `BaseAttack` to make sure that the queries counter is updated correctly. This method returns a tensor of booleans (`True` if the input is classified as *good* and `False` otherwise) and the updated queries counter (which is **not** updated in place to avoid side effects!). We show examples of we adapted how pre-existing attacks to this interface in the [`src/attacks`](src/attacks) directory.

Once you evaluated your own attack, you can submit it to our leaderboard by opening an issue or a pull request.

Expand Down

0 comments on commit 299c90d

Please sign in to comment.