parser instead of program gen?

[majidaldo]

Hi Saul. Great work!

I'm trying to digest your sklearn NDArray example. I see that you wrote a program_gen.py (which I just got the idea of ). At this point, why not just use a parser to generate the executable code which I think has better ergonomics.

(you could even use a parser for egglog itself. Sorry if I'm missing something obvious.).

[saulshanabrook]

Are you asking why we generate executable code within egglog, through rewrites and rules, instead of just extracting the egglog expression, and then translating that to Python code outside of the system?

[majidaldo]

Yes

[saulshanabrook]

Thanks for the question. Both options are doable, and a few reasons off the top of my head why I choose to do it that way:

1. Trying to see how much of the whole input -> output pipeline can happen within egglog, so that if we have different analysis we would be able to compose them more easily, through composition of rewrite rules and different types of expressions. So for an end user, if they did something like expr.execute() (like is available in dask) this would happen all as a series of rewrites which could be extended through some kind of global context around which rules are available. This is still aspirational, and the details would have to be ironed out with specific use cases.
2. It lets us rely on the existing matching functionality built into egglog, instead of having to rebuild that in Python.
3. It helps us test and refine the semantics of egglog to support this use case, which, in a general case, can be seen as a way of linearizing a functional code graph into a sequence of imperative instructions. This type of transformation is useful for many other cases (for example, translating into LLVM, say, instead of Python). It also might want to be done in different phases, going back and forth between imperative and graph forms, all within the same e-graph, but for different use cases.

Sorry, that's still a bit vague!