v2 Template Format

[dm-p]

Deneb's template format has been a Vega or Vega-Lite specification, with everything specific to Deneb in the usermeta section. This works reasonably well but requires work to manage mappings between template format and Power BI's internal properties.

With the introduction of JSONC into Deneb, and with PBIR now a thing, and while we have a fantastic suggestion in #462 and an idea for MS to implement it, we should probably assume that in short to medium term (even with GA of PBIR, whenever that is), that we will be using the same methods of persistence.

I think it's time to propose a new template format that is more "top-down" in its design, to provide better integration with Power BI's property/persistence system and provide a good balance of portability when "packaged" to those who wish to keep a definition within a report in source control.

We would ideally want to take the learnings from what has been implemented since 0.1 through 1.7 and squeezed into the "v1" template format to give us something that should help us scale going forward. As this is a "breaking change" (although we would support loading and migration of v1 templates as part of this to ensure backward compatibility), I would regard it as a catalyst for moving Deneb's version to 2.0 also.

High-Level Requirements

We would also want to collect ideas from the current backlog for this design and some things I haven't formally written about other than in my head. At a high level, I would see the objectives/deliverables as:

1. Templates preserve text as entered in each editor, preserving the developer's original coding styles and conventions. A scenario like the one outlined in URL instead of JSON in specification/config #269 would likely not permit a template to be used, so this would be out of scope.

2. 1:1 mapping of template component properties to Power BI properties where possible.

3. In addition to the dataset, include a framework for "dependencies" not part of the core Values field. These would need to be fulfilled by making sure they are assigned before a visual can be created from a template. This could include:

   • Drill down fields - Drill Down #196

   • Allowing a 'series' field, which provides dedicated orthogonality for a specific column that we can use to apply series-based selectors (i.e., allowing tooltips, drill through and cross-filtering by a single value without having to 'collect' all component rows).

     ^ This has been prototyped, and while it works, it creates some challenges with consolidation, as columns get mapped at the category/row level and measures get mapped under the series. It is not intuitive for the end-user to see how data gets mapped internally by Power BI. Its uses are marginal compared to a high-demand feature like field parameters (which would fall under this problem, depending on whether a developer is using columns, measures, or a combination), making it harder for the visual and developer to reconcile.

   • Any measures supplied via conditional formatting, e.g.:

     • Exposing a property to allow a map layer to be supplied via conditional formatting and internalized as a dedicated dataset (Choropleth using geospatial data stored in PBI data model #194)
     • Other conditional formatting or external properties that can set signal names (highly unlikely given the lack of flexibility in custom visual capabilities)

4. Persistence of template metadata to allow incremental development/export for content creators without creating brand new metadata - Template generator removes "usermeta" property from Vega specification #285

5. Declaration and exposure of external widgets for users that do not affect the visual footprint:

   • Feature request: binding format options #275
   • Custom container for input bindings #346

6. Dataset sorting information - Support Sorting API #360

7. Static "assets" (with some sensible size limits, as these will be persisted and exportable) such as:

   • Shape files - Choropleth using geospatial data stored in PBI data model #194
   • Images
   • Pattern defs - Functionality to Add & Maintain Custom SVG Fill Pattern Defs #115

8. Better organization of existing exportable properties and accommodation of new ones, such as how we might handle bindings of params to interactivity behavior for Vega-Lite to satisfy a bunch of outstanding asks:

   • Attempt to Resolve Selectors for Interval/Brush Selection #176
   • Cross-filtering not working with brush selection.  #223
   • Zoom in/out (i.e selection) is glitchy when cross-filtering is turned on #225
   • Add support for lasso/rectangular select #327
   • Cross-filtering does not work with option nearest = true in select parameter #405

   This would likely need to be in line with Vega-Lite's current limitations for selections, if we can achieve them.

9. A top-level schema for Deneb templates, rather than ensuring the template is valid JSON, contains a usermeta property, and then this usermeta object validates against the current Deneb template schema.

Others TBC