optimizing memory usage

[ThomasWaldmann]

py-esp32-ulp is able to assemble about 3kB of source code (when it is running on a standard ESP32 "WROOM" chip) - more gives a MemoryError (out of memory). Bigger amounts of source code can be assembled when running on a device with more RAM (like on a development machine running the UNIX port of micropython).

PR #36 improved this a bit by calling the garbage collector now and then.

The limit looks rather low, but considering that the ULP has only a total of 4kiB of RAM and that we will likely only be allowed to use ~2kiB of it from micropython and that usually some of it is needed as a buffer (thus: not for code), the limit doesn't look too bad.

(moved from #35)

We can collect ideas here about how to improve memory usage in case we need to.

[ThomasWaldmann]

The place where it currently usually runs out of memory is a list comprehension doing a transformation

• from A) a list of strings (source lines, comments already were removed)
• to B) an equivalent list of tuples (label, opcode, args).

Having both A and B in memory seems to be too much. We could avoid this by restructuring the code to be more line-by-line (doing everything that is needed to process one source line to binary in one go, so it does not have to collect intermediate results in such big lists).

OR, we could try to free the memory needed for elements of A while building B - and use B as input for pass1 and pass2.

Also we (or the caller of our code) could (after A is computed) try to free the memory where the full source code is stored as one big string.

[mattytrentini]

yielding on a line-by-line basis seems like it could be efficient...