[Core feature] Parallelize Node Executions in FlytePropeller

[hamersaw]

Motivation: Why do you think this is important?

Currently, FlytePropeller uses a modified DFS to traverse the FlyteWorkflow DAG where each node evaluation is synchronous. This means that in certain scenarios DAG traversal can be a bottleneck. For example, extremely large workflows or tasks that operate synchronously.

Goal: What should the final outcome look like, ideally?

FlytePropeller should evaluate nodes in parallel. This could be done adding a layer of futures on top of the current traversal algorithm so that node scheduling remains consistent but the actual evaluation is done in the background.

Describe alternatives you've considered

Leaving it as is may result in poor performance in certain scenarios as described.

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[hamersaw]

Closing in favor of a pluggable scheduler in flytepropeller, which will automatically require parallelized node executions.