Truly sequencial execution is hard (all existing iterators are concurrent)

[emil14]

Previous node doesn't wait for the next one to finish its work, before doing next task. Example: in "99 bottles" problem it's unsafe to use range with for 99..-1 -> for. Neva-like pseudocode:

99..-1 -> for { :data -> firstLine -> secondLine -> :sig }

Imagine this sequence of events:

1. 99 is sent from range it received by firstLine
2. firstLine printed the line and sent 99 to secondLine
3. 98 is send from range and received by firstLine
4. firstLine prints the line before secondLine prints 99 <--- RACE
5. secondLine finally prints the line, but order already messed up

This is completely possible (e.g. scheduler awakes secondLine to late), and even more than that - this is expected (and generally desired) behaviour. This is basically what implicit concurrency is. Previous node starts to work on next task as soon as possible, without waiting for its downstream node to finish whatever he's busy with at the moment.

---

Might be related to #666 and #711, as well as #723 and #739

[emil14]

Always use While

There's one non-concurrent iterator called While, see #723 for details. So one possible solution to this problem might be to use that each time we need to avoid concurrency.

[emil14]

Not just While

I don't like the idea of having 2 versions for iterators - concurrency safe and unsafe, but probably we need to be able to avoid concurrency where it matters, and not just for "while-like" tasks, but also for stream processing.

I'm not 100% sure but I feel like it does matter when component is not pure (especially when it does writing operations), so maybe FP stuff (map/filter/reduce/etc) could be free from this constraint (but then it's up to user to use them properly, not to have write operations in them)

UPDATE: OTOH it would make sense to make all iterators non concurrent because by keeping e.g. map/filter/reduce concurrent we assume that programmer will not do anything that make lead to out of order inside of handler, which is bold

[emil14]

At Runtime

Is it possible to somehow guarantee at runtime level that node won't receive a message until it will send result?

Problems

1. There's no concept of node at runtime at all, it operates in terms of functions and channels. To change that we would have to revisit the whole language. Implementing correct algorithm for runtime was very hard and took several months
2. What does it mean "until it will send result"? A node can have any amount of outports, and they are usually conditional (fire under different circumstances). Do we mean "until it sends at least one message"? What if e.g. sends N messages to port A and then sends 1 message to port B and that's is it's cycle?
3. Last but not least but if that would be implemented, we wouldn't be able to have concurrency in the pipelines at all, while with components like while/for we can limit it, but not give up on it competely

[emil14]

This problem better described at #857