Py'n'APL?

[xpqz]

It was suggested we bring Py'n'APL onto the GSoC table, and my initial suggestion was:

Py'n'APL is Dyalog's bi-directional APL-Python bridge. It enables Python to be executed and results passed to
the APL session, and APL to be used inside Python as a super-powered array DSL. Currently, the PynAPL
system speaks on a socket to a running interpreter, meaning that a user must first have a fully functional Dyalog
interpreter installed. Can we implement a version of PynAPL that is a native extension hooking up to a dyalog.dll?
That would likely be faster, and enable a better installation experience.

Having peeked a bit deeper under the hood of this, the suggested idea is a pretty large rewrite. It's interesting that the author chose to bypass Conga for sockets, and instead ⎕NA directly to the standard c libraries for sockets, according to comments "to avoid the overhead of Conga". Max has discovered DWA, with some examples published on https://github.com/Dyalog/NativeLib, but I conjecture that doing this "properly" would mean something much closer to what kX are doing with their Python integration. This would be such a digression from the current Py'n'APL as to be a completely new thing, a native Dyalog module written in Cython and DWA.

Don't get me wrong, this would absolutely be a super-interesting project, but I can't help but feel it's beyond scope for GSoC. If we look at what kX does, it's not just "execute this string as k/q", but seamless (and probably no-copy) in-memory data access. I also suspect that it would place demands on DWA or dyalog-as-a-dll that it currently doesn't support, and we'd need help from the interpreter team to make the most of it.

[MaxCan-Code]

Agreed. Py'n'APL's approach is somewhat orthogonal to what PyKX does. Use of DWA (Direct Workspace Access) can be interesting to the community, but before we proceed we should see if the interpreter team is willing to be mentors and seek mentors from the community.

[MaxCan-Code]

Related works I know of:

• apple
• sml-aplparse
• sml-aplparse fork

Any thoughts/comments/ideas? @vmchale @melsman @BobMcDear

[BobMcDear]

Hi,

This would be a very helpful tool since setting up and running Py'n'APL is not a smooth process at this time. I'm not very familiar with the workings of Py'n'APL or the Dyalog interpreter and therefore can't make any definite statements, but this seems like quite a vast project. The difficulty lies in both Dyalog and (C)Python being interpreted, so compiling them can facilitate the process. Co-dfns and, as already mentioned, Cython immediately come to mind, but these are beyond the "standard" scope of the two languages. Besides that, Python scientific computing packages already have excellent C interoperability, so implementing, e.g., no-copy data access for NumPy arrays shouldn't be too challenging for the Dyalog interpreter.

[jph00]

Can I suggest first at least fixing the current version? It is currently in a broken state, and there are already working solutions in the gh issues which could be quite quickly included (it took me just a few mins to fix in my own local version).

[xpqz]

I think @MaxCan-Code's recent PR addresses those.