Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spawn new Permuters when improvements are found #158

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

ethteck
Copy link
Contributor

@ethteck ethteck commented Mar 23, 2023

fixes #154

This is very much a WIP. I've used it to match some stuff for PM already, but it can't be merged as-is because I don't know how to properly do multithreading in Python. There are 4 main issues I see:

  1. I'm not sure if the "new permuter queue for each process" approach I have is ideal, and I'm not sure if the things I lifted into the context class are appropriate there. This is more of a design concern.
  2. Quite often, right after a new Permuter is created, the worker proceses get an incoming task for the new permuter before they have added it to their internal list, so they are unable to run the task. I don't know how to write the permuter pass-off mechanism to avoid this scenario. For now I'm just discarding the task.
  3. As the process goes and more sub-Permuters are created, there seems to be a considerable slowdown. I suspect this might be related to 2. but I'm not sure.
  4. I haven't really considered p@h yet, and I don't know how this will fit into it.

To-dos:

  • Get assurance that the design is okay or fix it (1.)
  • Fix bug (2.) with task that we can't complete
  • Fix gradual slowdown (3.)
  • Test / fix p@h compatibility (4.)
  • Add a constructor for Permuter which takes an existing Permuter and new source / a new AST and which doesn't log startup info to console
  • Add args that influence when a new Permuter is created
  • Fix bug where sub-permuters break on gfx macros
  • Fig bug where diff files from sub-permuters contain tons of empty lines

Open to all and any suggestions for how to write this better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Permute improvements
1 participant