Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autogenerated documentation for bundle config #2033

Merged
merged 52 commits into from
Jan 29, 2025

Conversation

ilyakuz-db
Copy link
Contributor

@ilyakuz-db ilyakuz-db commented Dec 18, 2024

Changes

Documentation autogeneration tool. This tool uses same annotations_*.yml files as in json-schema

Result will go there and there

Tests

Manually

@eng-dev-ecosystem-bot
Copy link
Collaborator

Test Details: go/deco-tests/12401106836

Copy link

github-actions bot commented Jan 3, 2025

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 2033
  • Commit SHA: c6703c13637c0d85e6754e4c802f38052304a5d6

Checks will be approved automatically on success.

Makefile Outdated
schema:
go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

docs: vendor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs: vendor

We don't add this implicitly anymore to other targets. If you're on CI, you can always specify both with "make vendor docs".

BTW, do you intended to run it on CI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, do you intended to run it on CI?

Currently the only use case of using that tool is to checkout CLI repo and run locally

We don't add this implicitly anymore to other targets

By implicitly you mean using make deps or in general?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean the make entry should be:

docs:
  ...

no dependency on vendor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate little bit more on the reasoning behind "We don't add this implicitly anymore to other targets" so I can understand how to better fix it?

Is it just that we don't want to rely on make deps syntax and this is still allowed?

docs:
  make vendor
  go run ....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed and update the readme

Makefile Outdated
schema:
go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

docs: vendor
go run ./bundle/internal/docs ./bundle/internal/schema ./bundle/internal/docs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this part if /bundle/internal/ ?

internal is meant for code that is not meant to be used outside of the package, but this is clearly intended to be used outside the package, that's why it's in Makefile.

How about placing this at top level? /docsgen/ ?

I see you're following existing structure, so it's a more of a question for @pietern

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you're following existing structure,

Yes I reused same approach as with json schema generation which is used in .codegen.json along with other internal scripts

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is bundle specific, so bundle/docsgen would be more appropriate.

It doesn't need to be internal.

@ilyakuz-db ilyakuz-db requested a review from pietern January 16, 2025 11:28
@ilyakuz-db ilyakuz-db requested a review from denik January 20, 2025 16:17
@ilyakuz-db
Copy link
Contributor Author

@pietern @denik could you take another look? Want to merge it so docs team could start using this tool

)

// Parsed file with annotations, expected format:
// github.com/databricks/cli/bundle/config.Bundle:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is copied, right?

It would be much easier to review if PRs was split into

  • refactoring: copy / renames
  • new functionality

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this was moved as is from schema package into the new package

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be much easier to review if PRs was split into

refactoring: copy / renames
new functionality

Agree but it feels too painful to split now. For next PRs I'll try to follow more convenient approach

@ilyakuz-db ilyakuz-db changed the title feat: Config reference documentation autogeneration Config reference documentation autogeneration Jan 21, 2025

// Examples of the value for properties in the schema.
// https://json-schema.org/understanding-json-schema/reference/annotations
Examples []any `json:"examples,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if the schema check we perform also validates that entries here match the containing schema?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it only tests yamls against the schema but not the schema consistency itsef

I.e. this is allowed
image

In general spec says that example values should validate against schema but it isn't used and not required
https://json-schema.org/understanding-json-schema/reference/annotations see draft 6

Worth to mention that this field currently is not used in schema generation

Makefile Outdated
schema:
go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

docs: vendor
go run ./bundle/internal/docs ./bundle/internal/schema ./bundle/internal/docs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is bundle specific, so bundle/docsgen would be more appropriate.

It doesn't need to be internal.

@ilyakuz-db ilyakuz-db changed the title Config reference documentation autogeneration Autogenerated documentation for config reference Jan 29, 2025
@ilyakuz-db ilyakuz-db changed the title Autogenerated documentation for config reference Autogenerated documentation for bundle config Jan 29, 2025
@ilyakuz-db ilyakuz-db added this pull request to the merge queue Jan 29, 2025
Merged via the queue into main with commit 708c4fb Jan 29, 2025
9 checks passed
@ilyakuz-db ilyakuz-db deleted the feat/config-reference-doc-autogen branch January 29, 2025 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants