Test arbitrarily generated opshin validators with hypothesmith

[juliusfrost]

Is your feature request related to a problem? Please describe.

So far we need to create manual unit tests for compilation with opshin. Using hypothesis to generate opshin code would help catch bugs in compilation at a higher rate and give more assurance of what opshin can do.

Describe the solution you'd like
A clear and concise description of what you want to happen.

I found a cool library called hypothesmith that can generate arbitrary python code.
It would be awesome if we could extend this to test validators with a reduced subset of python that's valid for opshin.

I have not taken an in depth look at this library yet, but the next steps would be to do some experiments generating opshin code.

Describe alternatives you've considered
None

Additional context
N/A

Bug bounty: 1000 ADA

[nielstron]

I strongly support this. We will have to take care when combining this with the new constant propagation which essentially executes whatever part of the python program it can - leading to the big disclaimer in the package warning about arbitrary code execution :)

[juliusfrost]

I strongly support this. We will have to take care when combining this with the new constant propagation which essentially executes whatever part of the python program it can - leading to the big disclaimer in the package warning about arbitrary code execution :)

Can we turn off constant propagation just for testing?

[nielstron]

Yes definitely. Though this will reduce the amount of stuff we can test to not break constant folding. Goes into the direction of #168

[juliusfrost]

Perhaps we can limit the scope of operations such that it won't modify the host system state so that we can safely evaluate certain operations for constant folding. As a start, maybe avoiding any IO ops and loops that might create memory or runtime issues.

[nielstron]

Yes ideally this would be the case. But I am not sure how feasible this is :) Anyways we can start thinking about it once we have hypothesmith running