Online API browser

[Hywan]

Hello fellow @hoaproject/hoackers and users!

I would like to have a nice online API browser for our libraries. Ready?

Intro

So far, we have tried http://api.hoa-project.net/, which is the result of Doxygen output.

Is it a solution we could maintain? Do we have better alternatives? What do you expect from an API browser?

Solutions for PHP

• Doxygen,
• Sami,
• ApiGen,
• PHPDox,
• phpDocumentor.

Personally, I don't like any of them.

Solutions in other languages

• In Rust with rustdoc, example, source code,
• In Elm, example,
• In Haskell, hoogle, maybe not relevant for us since it is strongly-base on types,
• In Caml, example, haha, kidding…

Build our own?

I really like the approach of rustdoc, the style, the fact that the code must be extremely documented, and the API is not the first-citizen. Look at Tagua VM, the https://tagua-vm.github.io/parser/ page represents the documentation of the tagua-parser library.

For each class, we have a long description. Then we have modules (namepaces in PHP) with a description.

For each function (for instance parse), we have a description, and an example. The API is even not documented. Well, this discussion is part of the RFC #58.

But the browsing experience is great.

Outro

All API documentation generator I tried in PHP didn't catch me. In other languages, like Rust or Elm, we have very nice ideas. We must get inspired from them.

I am not afraid of building a new tool. I personally thing this is the direction to take.

Thoughts?

[vonglasow]

Have you tried apidoc ?

[1e1]

Quick and dirty answer:
How can it be filled by Praspel and typed functions (new in PHP7)?
Can we use Atoum for extracting some examples?
I will enjoy if the generator need no more comments.

[1e1]

Remember the code is the best documentation!
Notice: in an online API browser we are mainly looking for public methods.

[Hywan]

@vonglasow APIDoc is for Rest API, no?
@1e1 We need to expose all methods, whatever the visibility they have, you don't know who is using your libray and for what. For the rest, I didn't understand what you meant, sorry :-/.

[1e1]

This API browser would not add extra comments for generating the documentation.
There is enough constraints from other tools. It should use these constraints as inputs.

[Pierozi]

As RFC #52 will produce high quality and dynamic content
We should be able to extract Markdown from code and generate HTML verison.

Maybe http://www.mkdocs.org could be useful?

[Hywan]

MKDocs looks nice, but I am afraid this is too much oriented for documentation, and not API documentation, aka code-oriented. What do you think? Do you think it will be difficult to bend MKDocs for an API browser?

[Hywan]

Current implementation https://github.com/Hywan/Kitab.

[harikt]

It looks like you love parsing the docblocks. I have came across lithium which does something similar. Not a standalone one. But it has some plugin when installed your whole framework docs all will be shown.

The plugin is : https://github.com/UnionOfRAD/li3_docs

The documentation written in class is as

https://github.com/UnionOfRAD/lithium/blob/5d9a06b8667e20490ef5ccb3275be8824f77666b/action/Controller.php#L15

once it is rendered you will see something as : http://li3.me/docs/api/lithium/1.1.x/lithium/action/Controller .

I guess this is what you are looking for. I looked at Kitab, but as it has less information how the docs should be written etc I am not very sure also.

[Hywan]

@harikt The format is defined in RFC #58. The output is not exactly what we want. We are looking for something closer to what rustdoc does.

[Hywan]

Yesterday progressions

A namespace page

On the left, we have the navigation, which is “all the sibling data”. You must understand it as “namespaces in the same level”.

On the center, we see one namespace in details, with all entities it contains: Classes, interfaces, traits, and functions.

A class page

Today progression

The class page has been enhanced with more data. Markdown is converted into HTML for the doc comment attached to methods. We have syntax highlighting. Markdown to HTML does not auto-convert entity names (like class names) into links yet. This is expected in a close future.

The class documentation will be placed just before the “Methods” heading.

[Hywan]

Today progression

And now we have a static search engine

It is kind of slow when we have more than 600 files to analyse, but I am working on a cache (the slow part is to build the search index, we can cache it, but I don't know how to orchestrate that).

Anyway, it works great 🙂. I have decided to write it with Elm, because why not.

Here is the Search.elm module.

[Hywan]

Great news! The search data are now compiled ahead-of-time. What does it mean? 3 files are now computed:

1. search-database.json, contains all the search information (never used by the client-side, never loaded, only needed for the compilation step, moved to the frontend just for debugging),
2. search-index.js, contains the serialized and compressed index, it is compiled ahead-of-time by Kitab, the client only unserializes it and here we go,
3. search-metadata.js, contains metadata useful to print results to the user, like URL, real symbol name, and description.

Having the index and the metadata separated really helps to reduce the size of the index (by a factor of 65% with my recent tests).

Why doing this? To reduce the initialisation of the search on the client-side. Now, initialisation of the search drops from 17s to… 369ms 🎉 with an index representing 606 files. Here are more numbers:

Description	Kitab only	Kitab and its dependencies
Number of PHP files to analyse	29	606
Time to compile everything	0.25s	6.57s
Size of search-database.json	5.4Kb	149Kb
Size of search-index.js	4.4Kb	199Kb
Size of search-metadata.js	3.9Kb	117Kb
Time to initialize the search on client-side	?	369ms

I am very happy with these results. Not to say that search-index.js and search-metadata.js are easily cachable with a 3xx HTTP code. They are purely static.

The size of the search module search.elm.js is 92Kb. So for Kitab only, the client loads 100.3Kb, and for Kitab and its dependencies, the client loads 408Kb, once.

We have a complete reactive UI with an advanced search for 600+ files for 400Kb. This is still less than Ember or Angular alone —just the framework, not the app— (source).

[Hywan]

Interface support for the HTML target are better.
Class and interface constants are supported by both the IR and the HTML target (including constant visibility, and default values, for inline or grouped form, resp. const A; const B; or const A, B;), see Hywan/Kitab@ad3a722.

[Grummfy]

Seems really nice. Just a nice feature, for future usage outside Hoa could be a way to +/- compile it to other template style/design

[Hywan]

@Grummfy Sure. This is always the case. Choose another target, and here you go, https://github.com/Hywan/Kitab/blob/ad3a72231c9e997b8be065dfab59cd724fc08e2a/src/Compiler/Compiler.php#L91.

[Hywan]

Progression summary:

1. Attributes are supported (Hywan/Kitab@9f37627),
2. Introduce namespace description, see below please (Hywan/Kitab@9474b34 and Hywan/Kitab@c6b8f3e),
3. Better global/unnamespaced entities support (Hywan/Kitab@b61fce8),
4. Functions are supported (Hywan/Kitab@1c753b1),
5. Links to source have been added, please see below (Hywan/Kitab@7fa57a1 and some others).

Note about namespace description. Any classes, interfaces, traits, or functions can have a block comment attached to it, in a single place. However, for namespaces, we have many locations where to write the documentation. Because this is not obvious and undeterministic, I have decided to look for a README.md file in the directory representing the namespace —with regards to the PSR-4 standard. If such a file exists, then we use it as the namespace description. Thus, I have introduced a new --with-composer option to read the PSR-4 mappings from a composer.json file. Thoughts? My first idea was to add a new special Description.md, but using README.md makes a lot of sense: It describes the content of the current directory, and it is displayed as HTML on Github, and Gitlab, which is very useful too.

About links to source, it is attached to each entities (classes, interfaces, traits, functions, and even methods).

Here is an online example https://hywan.github.io/Kitab/kitab/index.html of https://github.com/Hywan/Kitab. On this documentation homepage, you can see the README.md of the src/ directory. Below that, you have the Classes and Namespaces Sections. I have found a bug just right now, so links to namespaces are broken, but here is the link to the Kitab\Compiler namespace, https://hywan.github.io/Kitab/kitab/compiler/index.html, and here is the documentation for the Kitab\Compiler\Compiler class, https://hywan.github.io/Kitab/kitab/compiler/Compiler.html. You can see the Attributes, and Methods Section. All the [src] links are working. For example, links to the compile method, https://hywan.github.io/Kitab/kitab/compiler/Compiler.source.html#L.91-106. The static search engine is also working.

Please, fill bugs or discuss here :-). Feedbacks are really appreciated!

[Grummfy]

can you add an example where you use php 7 synthaxe with return, scalar type, etc.

Otyherwise nothing special to say it seems really nice

[Hywan]

@Grummfy Like this one https://hywan.github.io/Kitab/kitab/compiler/intermediaterepresentation/Into.html?

[Grummfy]

Well in this case, I would expect that "Type" or "File" would be clickable because thier are class that exist inside the documentation. For that's the only expected things that I see.

As extra information/plugins, I really enjoy to have stuff like phpxref to cross check the usage. But not directly accessible.

[Hywan]

As extra information/plugins, I really enjoy to have stuff like phpxref to cross check the usage. But not directly accessible.

Can you give an example please?

[Grummfy]

http://phpxref.sourceforge.net/phpxref-xaraya/nav.html.gz?index.html.gz

[Hywan]

This a tree explorer, yes, and… ? I don't understand what you are asking :-).

[Hywan]

Bug in namespace links is now fixed, https://hywan.github.io/Kitab/kitab/index.html.

Next steps:

• Being able to configure the logo,
• Add a index.html file at the root of the project to redirect to a particular namespace index (becase https://hywan.github.io/Kitab/ does not link to anything, we have to choose the kitab directory),
• Make more code “clickable”, like class-like types in parameters,
• Add page titles.

After that, we will be good to try a 1.0 beta, and deploy it on Hoa.

[Hywan]

Live: https://hywan.github.io/Kitab/kitab/compiler/Compiler.html.

And here we go, since Hywan/Kitab@1950f62, all qualified names are clickable. I still have an issue with extends that are not linkable, a small conflict with Prism (c-like profile).

[Hywan]

Progression summary

• I am writing the documentation of Kitab itself,
• New IR: Documentation, to represent the whole documentation, or get the description (first line),
• Add entity descriptions on the namespace index (e.g. https://hywan.github.io/Kitab/kitab/compiler/intermediaterepresentation/index.html, the Classes Section lists all classes with the description inlined), since Hywan/Kitab@6811602,
• Support static attribute in the IR, since Hywan/Kitab@b3ef5a1,
• Support variadic parameter,
• Kitab works as a dependency or if installed globally, since Hywan/Kitab@0c05267 (thanks @jubianchi!),
• The HTML target is responsive, since Hywan/Kitab@ae04cce.

I find interesting how to document things without saying: “Return bla bla” or “Get bla bla”, because the name of the function or method must be self-explanatory! So taking the example of inNamespace in https://hywan.github.io/Kitab/kitab/compiler/intermediaterepresentation/Entity.html, I used the following description:

An entity is in a namespace if its fully-qualified name contains at least one namespace separator (aka \).

Actually, this is the only explanation we expect at this step. We know the method is called inNamespace so saying something like: “Check if the name is in a namespace” is totally stupid.

This is interesting to rethink how we write documentation.

[Pierozi]

Really nice, I like the Namespace page related to Readme, a good one 👍

[Hywan]

It renders on Github too! On Github: https://github.com/Hywan/Kitab/tree/master/src/Compiler, on Kitab: https://hywan.github.io/Kitab/kitab/compiler/index.html.

[Hywan]

And now, Kitab can test your examples with ./bin/kitab test. More at https://github.com/Hywan/Kitab/pull/3.

[Hywan]

What's next?

• Kitab can be used as a dependency.
• Documenting everything inside Kitab.
• Try to use Kitab with a non-PSR-4 project.
• Move Kitab into the Hoa organisation.
• Release the 1.0 version.

[brzuchal]

None of the links to generated documentation doesn't work. Could you provide working one?

[Hywan]

Yes, we are migrating the repository into the Hoa organisation since this morning. Give me some hours to fix everything :-).

[Hywan]

@brzuchal https://hywan.github.io/Kitab/kitab/index.html :-)

[Hywan]

The 0.10 version has been tagged https://github.com/hoaproject/Kitab/releases. The version is stable. We use it at work.

It is implemented inside hoa/test. We will start to deploy the documentation on the hoa-project.net site soon.