Add ANTLR4 grammar and C++ parser

[HackerFoo]

This PR replaces the textX pearser with a C++ implementation using ANTLR to improve parsing speed 16x: 80 seconds down to 5 seconds, parsing the LiteX 35T fasm with this:

import fasm

for line in fasm.parse_fasm_filename('top.fasm'):
    pass

[mithro]

As you are adding C++ code this repo, you should also set up our normal set of C++ compliance checks (I believe there are copies in the prjxray repository?).

[mithro]

This PR replaces the textX pearser with a C++ implementation using ANTLR to improve parsing speed 30x.

Can you give some more concrete numbers?

[litghost]

Should delete textx schema?

[mithro]

What about tests?

[litghost]

Overall looks like a good start. Relatively simple C++ -> Python boundry. I think we are missing some internal state assertions on the Python side, and I've tried to comment on them when I see them. What do you think about having more focused unit tests on the Python and C++ side?

[litghost]

@HackerFoo Please transition the CI from Travis to GH actions in a separate PR. It will significantly aid in CI runtime.

[HackerFoo]

What about tests?

I've been using tests/test_simple.py. I'll need to look into creating a GitHub action to run it.

[litghost]

I've tested the fallback, seems to work as expected.

If the fallback is used during building, how does the user know why the fallback was used? Should there be a warning on pip install?

I've figured out how to get the error, running pip install -v fasm will show the error during install. Please add that information to the fallback warning, so that users can determine why the fallback implementation is being used. Specifically, if the user uninstalls and re-installs the package, e.g.:

pip uninstall fasm
pip install -v fasm
# or
pip install -v git+https://github.com/HackerFoo/fasm.git@antltr_cpp
# or
pip install -v git+https://github.com/Symbiflow/fasm.git

[litghost]

Can you add an argument to fasm/tool.py to select the parse? Similar to what you've done in test_simple.py?

[litghost]

@HackerFoo, review is in.

[HackerFoo]

@litghost I've addressed all of your comments, thanks.

[litghost]

@litghost I've addressed all of your comments, thanks.

Yep, things are looking pretty good. I noticed something wrong when uninstalling the package.

Try this, with ANTLR4 runtime installed:

python3 -mvenv test_env
source test_env/bin/activate
pip install -v git+https://github.com/HackerFoo/fasm.git@antltr_cpp
pip uninstall fasm

I get the following message when running pip uninstall fasm:

pip uninstall fasm
Found existing installation: fasm 0.0.2
Uninstalling fasm-0.0.2:
  Would remove:
    test_pip/bin/fasm
    test_pip/lib/python3.7/site-packages/fasm-0.0.2-py3.7.egg-info
    test_pip/lib/python3.7/site-packages/fasm/*
  Would not remove (might be manually added):
    test_pip/lib/python3.7/site-packages/fasm/parser/libparse_fasm.so
    test_pip/lib/python3.7/site-packages/fasm/parser/tags.py
Proceed (y/n)? 

Note the lines under Would not remove (might be manually added):. Even if I say y, it doesn't remove the .so and tags.py. Do you know why?

[litghost]

Please fix package uninstall not removing all parts of the package.

[HackerFoo]

@litghost It doesn't look like there's a way to fix this presently. Here's the code responsible for deciding what to skip:

https://github.com/pypa/pip/blob/master/src/pip/_internal/req/req_uninstall.py#L167-L215

It won't remove anything without a *.pyc extension.

From https://www.activestate.com/resources/quick-reads/how-to-uninstall-python-packages/

Be aware that there are a few exceptions that cannot be uninstalled with pip, including:

Distutils packages, which do not provide metadata indicating which files were installed.
Script wrappers installed by the setup.py develop command.

GitHub

pypa/pip

The Python package installer. Contribute to pypa/pip development by creating an account on GitHub.

ActiveState

How to Uninstall Python Packages

The Pip Package Manager is the de facto standard for managing Python distributions, and can be used to uninstall Python packages.

[litghost]

tags.py should definitely be able to be removed. Is the issue that the relevant files are missing from the manifest / metadata?

[HackerFoo]

I just tried it. Adding them to MANIFEST.in has no effect. I think MANIFEST.in is only meant for source files.

[HackerFoo]

I think this is just how pip uninstall is meant to work. I can add an issue for it.

[HackerFoo]

@litghost I've created an issue to track this: #37

[litghost]

@litghost I've created an issue to track this: #37

Please fix this. The solution took me 15 minutes of research to find, which is that you need to supply add pyproject.toml in the root with the following content:

[build-system]
requires = ["setuptools", "wheel"]

[HackerFoo]

@litghost I've created an issue to track this: #37

Please fix this. The solution took me 15 minutes of research to find, which is that you need to supply add pyproject.toml in the root with the following content:

[build-system]
requires = ["setuptools", "wheel"]

@litghost Thanks, this works.

Could you please include where you found this solution on #37 for future reference? Thanks.