Add json schema

[autoantwort]

To validate cps files. And personally I find it easier to read than the current documentation 😅

Additions: The example has a description field, but it is missing in the schema

I used draft 7 because that is the latest fully supported version by vscode (microsoft/vscode#155379)

[mwoehlke]

Looking at this again, while I like the idea, the duplication of attribute descriptions is... concerning. Actually, this seems like it would generally be a non-trivial amount of work to maintain. Do you think there would be a way to generate the schema from the specification reST? Or vice versa? (reST → schema is probably preferred, though.)

[autoantwort]

Or vice versa?

I have found https://github.com/stoplightio/json-schema-viewer which can generate very nice browse-able documentation for json schema. I now use this component to generate the html for the documentation:

Untitled.mp4

[mwoehlke]

I have found https://github.com/stoplightio/json-schema-viewer which can generate very nice browse-able documentation for json schema. I now use this component to generate the html for the documentation.

Interesting! However:

• I am concerned that this reduces the expressiveness of the documentation. As the schema seems to not allow any form of markup, we lose the ability to e.g. include links in the schema documentation.
• I am concerned that this does not integrate well with the rest of the documentation.
• I am concerned that this may reduce the editability of the schema.
• I don't see an obvious way to designate "tiers" of schema with this system. For example, right now we have "core" and "supplemental" schemas. I expect that we will eventually have proposed extensions as well.

Accordingly, I would much prefer either generating the schema from the reST, or some other format which could generate both reST equivalent to what we have now as well as a schema. Tacking on the "interactive" browser as an alternate means of viewing just the schema documentation could be added on top of that.

[autoantwort]

I am concerned that this reduces the expressiveness of the documentation. As the schema seems to not allow any form of markup, we lose the ability to e.g. include links in the schema documentation.

This is wrong. The schema does support markdown. Imo this change greatly improves comprehensibility.

I am concerned that this does not integrate well with the rest of the documentation.

Can you elaborate?

I am concerned that this may reduce the editability of the schema.

Why should it?

I don't see an obvious way to designate "tiers" of schema with this system. For example, right now we have "core" and "supplemental" schemas. I expect that we will eventually have proposed extensions as well.

How does the renderer influences this?

Accordingly, I would much prefer either generating the schema from the reST, or some other format which could generate both reST equivalent to what we have now as well as a schema.

I am not going to implement that, but i have found https://pypi.org/project/jsonschema2rst/

What is the benefit of creating a new format and generators to generate json schemas and rst/documentation from the new format over directly generating the rst/documentation from the schema?

[mwoehlke]

The schema does support markdown.

Hmm, perhaps I am just not seeing where you actually used any?

I am concerned that this does not integrate well with the rest of the documentation.

Can you elaborate?

Right now, we have one set of documentation, all generated by Sphinx and all capable of cross-referencing. In its current state, this essentially splits things into two completely-independent documents (losing or at least obscuring the "core" / "supplemental" split in the process) and breaks all the non-schema links to the schema. There are ways, in theory, to make external references visible to Sphinx, but this PR isn't doing so.

I also don't know how if there's any way for the schema to link to non-schema documentation short of writing out links in entirety. At best, I suspect that any such links are going to be fragile, as I don't know how to verify links in the schema.

I would much prefer either generating the schema from the reST, or some other format which could generate both reST equivalent to what we have now as well as a schema.

I am not going to implement that

Okay, I'll look into it. Extracting the schema from reST doesn't seem hard (with minor changes to how the reST is written to facilitate things); it looks like the worst part is dealing with type specifiers.

What is the benefit of creating a new format and generators to generate json schemas and rst/documentation from the new format over directly generating the rst/documentation from the schema?

I don't need to worry about combining hand-written and generated reST input files. I can trivially split the schema across multiple documents (which we're doing at present). As previously noted, I can trivially link between the schema and other documentation in either direction.

Really, though, the deal-breaker for me is editability. Compare:

"attribute-name": {
  "description": "This is a description. It has to be one line and is subject to string escaping.\n\nI have to use escape sequences to denote paragraphs. Few if any editors will support syntax highlighting of markup. The file is sensitive to correct use of syntactic characters. Context (i.e. what objects use this attribute) is not readily apparent, and changing such can be tricky.",
  "type": "string"
}

.. attribute:: attribute-name
  :type: string
  :context: package
  :required:

  This is a description.
  Lines are short and easily readable.
  Many editors support syntax highlighting.

  Paragraphs are easy.
  Context is obvious and changing it is easy.

One of those is easy and pleasant to read (in raw source form) and edit. The other... isn't.

I also noticed at least a few "conversion errors" in the schema descriptions. Finding these is... not easy, since the .json looks nothing like its source material and is hard to read.

[mwoehlke]

If you're curious / want to help, you can see my WIP at https://github.com/cps-org/cps/tree/generate-json-schema. Right now it just builds a model of the schema. Turning the model into a schema.json still needs to be implemented. Most of that should be boilerplate, but translating the attribute type descriptions, e.g. map(requirement) may be non-trivial.

[mwoehlke]

Okay, a first-draft generated schema can be found here (see also #63). I'd appreciate any feedback on whether this seems "correct" or if there are any problems. (Aside from the root description being sub-optimal; right now it just uses the description of the root object type, which is a package.)

[mwoehlke]

See #63. @autoantwort, per comments there, the json-schema-viewer stuff is still interesting, but needs to work for local builds also. Please feel free to open an issue or PR for that. (If you want to work on this yourself, schema.rst should have links to the raw schema and also the JSV version; I'd prefer a new "JSON Schema" section added to the end. I can help write the content that should go under that.)