First steps and planning

[simonegiacomelli]

@hoodmane @rth @ryanking13

These are the first steps where I could focus my energies for now:

• Copy the source code of the generator in this repo (the linked repository contains more than that, so I will not bring in the history of everything there. Just that folder in the current state.
• Organize the project with the src layout.
• Run the tests of the generator on every push through GitHub Actions
• Write some documentation
• Create some script/automation to generate the js.pyi artifact and push it to pypi
• Bring in the existing open issues from the other repo
• Plan something to join manual webidls (sadly I noticed that no one webidl source is enough...) now the best webidl source I know is from w3c/webref/curated

And then these points are for adding comments to the api:

• Investigate how to gather the information about api platform availability (probably from the webidl files)
• Gather/harvest comments from MDN (maybe there is some structured information about the previous point?)

What do you think?

[hoodmane]

Sounds great to me. I'll move the relevant issues over. Thanks so much @simonegiacomelli!

[rth]

Great, thanks for working on this @simonegiacomelli ! You wouldn't be interested in writing a blog post about this at https://blog.pyodide.org/ once it's more or less ready by any chance?

Sorry I missed the pings in the parent issues about naming, didn't have access to email notifications for a few days. A few more name suggestions are in #2 if it's not too late. But of course, feel also free to keep the current name.

[simonegiacomelli]

You wouldn't be interested in writing a blog post about this at https://blog.pyodide.org/ once it's more or less ready by any chance?

Yes, absolutely!

In the coming days/weeks I will work towards producing the first usable js.pyi through pypi. In this regard I just noticed that I put the generator sources in the src folder but probably it is not the best: the generator will not be released on pypi but just the pyi; so I could move those files in a package in the root of the repo (at the same level of tests folder). The src folder will contain only a js.py (empty) and a js.pyi (with the generated stubs).

After the first pypi release I will proceed to add features like: manual webidl, comments, platform support and fixing issues.

Thank you all for the support!
Any comment, suggestion or help is welcome!
Let me know if you have ideas to improve the project :)

[hoodmane]

If a project generates files they shouldn't be placed in src, that's going to confuse people.

I think the best structure is to generate a single js-stubs folder with an __init__.pyi file. See https://peps.python.org/pep-0561/#stub-only-packages

[rth]

a single js-stubs folder with an init.pyi file

It still needs to be importable as import js though, as far as I understand? Maybe what they mean in that PEP is the project name, not the module name. So in this case it could be something like, webtypy/js/__init__.pyi (or adds an extra js-stubs level if we really want to). Agreed that I would expect source files to be in stc/ not generated ones. But in end maybe what matters is what's in the published wheel so this organization would not be very critical for users.

[hoodmane]

It still needs to be importable as import js though, as far as I understand?

Aren't we creating a type stubs only package?

[hoodmane]

I guess it would make sense to also make top level imports from js work at runtime. If you wanted to do that, you could use normal .py files for everything and just raise a RuntimeError if any of the functions is actually called.

[rth]

Well yes but we did discuss the package name in #2 unless you think it should be named webypy-stubs? js-stubs is the package name is not really searchable IMO. Technically it's not a stubs only package since it will have also the generation code. Also that logic works when you create stubs for an existing package, "django" / "django-stubs". In our case, the js package is exposed by Pyodide, but wouldn't say it defined those API but rather proxies them from JS, so it's a bit of a weird case.

About whether it needs to be importable, not sure, I never did a stubs only package. I guess someone would have to try and see if it works.

[hoodmane]

I thought we were discussing the pypi distribution name in #2, so that people will pip install webypy to get the type stubs. Was the idea that webypy would be a typed synonym for js? This seems like it would be pretty confusing to me tbh.

I thought we are creating type stubs for js so that mypy will understand the types we import from js:

from js import setTimeout 

If we just want type stubs, then we need to upload to pypi a distribution with any name we like (maybe webypy) containing a package called js-stubs as specified in PEP 561. If we also want to allow top level imports, we could ship a real package called js with imports that always throw. But I think we'd be better off starting with stubs only.

[rth]

Hmm, I was under the assumption that the package name would be == pip installable name, but indeed it doesn't have to be the case. +1 for what you are proposing. But I think the package name is just specified in the pyproject.toml no need to create a subfolder?

Anyway, it's a long weekend in France, I have only had a quick glance at it, so also feel free to ignore my comments :)

[hoodmane]

If you have a single file package, it's permisable to call it mypackage.py and not put it in a subfolder. PEP 561 says that if you want to make a type stubs only package, you cannot have a single file named mypackage.pyi, it needs to be a folder called mypackage-stubs containing a file called __init__.pyi. I also tried the single file approach and I can confirm that it doesn't work, as the PEP warns.

[simonegiacomelli]

Thanks for linking PEP 561!

After your comments I renamed the folder js to js-stubs.
Now the only content of js-stubs is __init__.pyi.

For the project naming, the pep says about the package naming but I didn't see any indication for the project name; so webtypy should be fine...?

There are still 2 things to fix:

1. the folder above is still in src; it's wrong and it should be removed so the js-stubs stays in the root of the repo. I tried but I was not able to get the right setup.cfg keys to make a workable package
2. To add an additional clarification/safety measure, add an header to the generated file with the disclaimer # this file is generated and should not be modified ... etc.

If you have some pointer for bullet 1, please let me know; I looked at the project mentioned by the pep as a stub-only project (numpy-stubs) but they use setup.py also and I hoped to do that using only setup.cfg. Btw I don't know if it's possible.

Did I forget or misunderstood something?
@hoodmane @rth thanks for the comments!

[hoodmane]

For the project naming, the pep says about the package naming but I didn't see any indication for the project name; so webtypy should be fine...?

The pep indeed places no constraint on the distribution name. If we like webtypy and it is available on pypi we can use it =)

[hoodmane]

the folder above is still in src; it's wrong and it should be removed so the js-stubs stays in the root of the repo. I tried but I was not able to get the right setup.cfg keys to make a workable package

I can help with this, it's quite annoying to get right but I just did it recently. But also I think it would be good to place it under a subfolder like generated_package or something for maximum clarity.

[hoodmane]

they use setup.py also and I hoped to do that using only setup.cfg. Btw I don't know if it's possible.

I think it's possible using a data_files pattern like *.pyi but it's also not the end of the world to use setup.py to compute the data_files.

[simonegiacomelli]

I can help with this, it's quite annoying to get right but I just did it recently. But also I think it would be good to place it under a subfolder like generated_package or something for maximum clarity.

@hoodmane yes please, let's move the js-stubs folder inside a \generated_package. Do you have the conf handy?
Adding setup.py is good for me.