Skip to content

Commit

Permalink
Use uv run to run the Python-based fuzzer
Browse files Browse the repository at this point in the history
  • Loading branch information
AlexWaygood committed Aug 23, 2024
1 parent 1f2cb09 commit 9e4337b
Show file tree
Hide file tree
Showing 9 changed files with 309 additions and 65 deletions.
4 changes: 1 addition & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -294,8 +294,6 @@ jobs:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python requirements
run: uv pip install -r scripts/fuzz-parser/requirements.txt --system
- uses: actions/download-artifact@v4
name: Download Ruff binary to test
id: download-cached-binary
Expand All @@ -307,7 +305,7 @@ jobs:
# Make executable, since artifact download doesn't preserve this
chmod +x ${{ steps.download-cached-binary.outputs.download-path }}/ruff
python scripts/fuzz-parser/fuzz.py 0-500 --test-executable ${{ steps.download-cached-binary.outputs.download-path }}/ruff
uv run scripts/fuzz-parser.py 0-500 --test-executable ${{ steps.download-cached-binary.outputs.download-path }}/ruff
scripts:
name: "test scripts"
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/daily_fuzz.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@ jobs:
python-version: "3.12"
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install Python requirements
run: uv pip install -r scripts/fuzz-parser/requirements.txt --system
- name: "Install Rust toolchain"
run: rustup show
- name: "Install mold"
Expand All @@ -49,7 +47,7 @@ jobs:
# but this is outweighed by the fact that a release build takes *much* longer to compile in CI
run: cargo build --locked
- name: Fuzz
run: python scripts/fuzz-parser/fuzz.py $(shuf -i 0-9999999999999999999 -n 1000) --test-executable target/debug/ruff
run: uv run scripts/fuzz-parser.py $(shuf -i 0-9999999999999999999 -n 1000) --test-executable target/debug/ruff

create-issue-on-failure:
name: Create an issue if the daily fuzz surfaced any bugs
Expand Down
11 changes: 5 additions & 6 deletions crates/ruff_python_parser/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,19 +58,18 @@ cargo test --package ruff_python_parser
The Ruff project includes a Python-based fuzzer that can be used to run the parser on
randomly generated (but syntactically valid) Python source code files.

To run the fuzzer, first install the required dependencies:
To run the fuzzer, first [install `uv`](https://docs.astral.sh/uv/getting-started/installation/):

```sh
uv pip install -r scripts/fuzz-parser/requirements.txt
```
- Unix: `curl -LsSf https://astral.sh/uv/install.sh | sh`
- Windows: `powershell -c "irm https://astral.sh/uv/install.ps1 | iex"`

Then, run the fuzzer with the following command:

```sh
python scripts/fuzz-parser/fuzz.py
uv run scripts/fuzz-parser.py
```

Refer to the [fuzz.py](https://github.com/astral-sh/ruff/blob/main/scripts/fuzz-parser/fuzz.py)
Refer to the [fuzz-parser.py](https://github.com/astral-sh/ruff/blob/main/scripts/fuzz-parser.py)
script for more information or use the `--help` flag to see the available options.

#### CI
Expand Down
23 changes: 23 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -111,3 +111,26 @@ version_files = [
"crates/ruff_wasm/Cargo.toml",
"scripts/benchmarks/pyproject.toml",
]

[tool.uv]
# Dependencies for our various Python scripts in `scripts/`.
# These could possibly use inline script metadata, but then the exact
# version of the dependency installed wouldn't be recorded in `uv.lock`.
dev-dependencies = [
# Required by `generate_known_standard_library.py`
"stdlibs>=2022.10.9",
# Required by `fuzz-parser.py
"pysource-codegen",
"pysource-minimize",
"rich-argparse",
"ruff",
"termcolor",
# Docs dependencies are deliberately omitted,
# as they're unfortunately quite complicated.
# See `docs/requirements.txt` and `docs/requirements-insiders.txt`
# for the requirements of `generate_mkdocs.py`, `_mdformat_utils.py`
# and `check_docs_formatted.py`
]

[tool.uv.workspace]
members = []
13 changes: 6 additions & 7 deletions scripts/fuzz-parser/fuzz.py → scripts/fuzz-parser.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
"""
Run the parser on randomly generated (but syntactically valid) Python source-code files.
To install all dependencies for this script into an environment using `uv`, run:
uv pip install -r scripts/fuzz-parser/requirements.txt
Example invocations of the script:
Example invocations of the script using `uv` from the project root
(which will automatically run the script in an environment
with all dependencies installed):
- Run the fuzzer using seeds 0, 1, 2, 78 and 93 to generate the code:
`python scripts/fuzz-parser/fuzz.py 0-2 78 93`
`uv run scripts/fuzz-parser.py 0-2 78 93`
- Run the fuzzer concurrently using seeds in range 0-10 inclusive,
but only reporting bugs that are new on your branch:
`python scripts/fuzz-parser/fuzz.py 0-10 --new-bugs-only`
`uv run scripts/fuzz-parser.py 0-10 --new-bugs-only`
- Run the fuzzer concurrently on 10,000 different Python source-code files,
using a random selection of seeds, and only print a summary at the end
(the `shuf` command is Unix-specific):
`python scripts/fuzz-parser/fuzz.py $(shuf -i 0-1000000 -n 10000) --quiet
`uv run scripts/fuzz-parser.py $(shuf -i 0-1000000 -n 10000) --quiet
"""

from __future__ import annotations
Expand Down
5 changes: 0 additions & 5 deletions scripts/fuzz-parser/requirements.in

This file was deleted.

36 changes: 0 additions & 36 deletions scripts/fuzz-parser/requirements.txt

This file was deleted.

8 changes: 3 additions & 5 deletions scripts/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[project]
name = "scripts"
version = "0.0.1"
dependencies = ["stdlibs"]
requires-python = ">=3.8"
# This `pyproject.toml` file provides linting configuration for our Python scripts
# It doesn't list dev dependencies requried by those scripts;
# those are listed in the `pyproject.toml` file in the project root.

[tool.black]
line-length = 88
Expand Down
Loading

0 comments on commit 9e4337b

Please sign in to comment.