Ability to inspect the configuration mapping model

[dmlloyd]

As a framework developer, I'd like to be able to be involved in the early process of configuration mapping, specifically gathering information about the configuration model before mapping occurs. In this way, I can examine the structure, and use additional annotations for my own purposes. For example, I could define a command-line argument configuration source, using annotations to configure the mapping of command-line options to configuration property names. Or, I could provide annotations which specify additional information for documentation (e.g. a help screen), or correlate configuration options with UI elements. I could detect properties that are @Deprecated or @Experimental to supplement my documentation and/or runtime behavior.

I imagine the overall lifecycle of configuration establishment looking like this:

flowchart TD
    start((( )))
    defmodel("Define the mapping model(s)")
    defparse("Combine model(s) into parsing model")
    mkdefs("Establish default value set")
    create(Create configuration)
    load(Load configuration)
    parse(Parse configuration into mapping model)
    end_((( )))
    start --> defmodel
    defmodel --> mkdefs
    defmodel --> defparse
    defparse --> parse
    start --> create
    mkdefs --> load
    create --> load
    load --> parse
    parse --> end_

Loading

To support my use case, I could use information from the mapping model to create a new configuration source. This would be done after the model is defined, but before the configuration is loaded. With a coherent model API, we could use the same techniques to handle the existing annotation set as well as providing general extensibility. It's really a question of establishing a usable model abstraction.

I imagine the model API could look something like this:

// a familiar example
@ConfigMapping(prefix = "hello-world")
public interface HelloWorldConfig {
    @MyAnnotation("extra info!")
    @WithDefault("hello")
    String helloMessage();

    @WithDefault("5")
    int numberOfTimes();

    Optional<Path> outputPath();
}

MappingModel<HelloWorldConfig> myModel = MappingModel.forInterface(HelloWorldConfig.class);
MappingProperty helloMessageProp = myModel.getProperty("helloMessage");
if (helloMessageProp.hasDefaultValue()) System.out.printf("Default is %s%n", helloMessageProp.defaultValue());
if (helloMessageProp.hasAnnotation(MyAnnotation.class)) System.out.printf("Has annotation with %s%n", helloMessageProp.annotation(MyAnnotation.class).value());
if (helloMessageProp.isGroup()) {
    MappingModel childModel = helloMessageProp.groupModel();
    // do something with the child model
}
System.out.printf("The property %s maps to the configuration path %s%n", helloMessageProp.name(), helloMessageProp.key());
// ...Plus more methods to query the type, converter information, optionality, validation rules, etc.

// Also: these would be reasonable methods to have, and would give a nicer input into the mapping process, given a couple other bits of work that would need to be coordinated.
HelloWorldConfig config = myModel.applyTo(someConfigView); // see #981
HelloWorldConfig config2 = myModel.builder().build(); // see #1001
Class<? extends HelloWorldConfig> genClass = myModel.implementationClass(); // see #1001
// ...

This API could be the entry point into creating a builder for a given mapping model as described in #1001. It could be used as a simple API to create a mapping for an arbitrary configuration view as described by #981.

[radcortez]

I was already planning to build such API.

In Quarkus, we can generate the metamodel at build time to generate the implementation bytecode, so we don't have to load it dynamically. In runtime, we still need the metamodel to apply the mapping rules, and because we don't have an API that can leverage what we could collect in build time, we have to read and parse the metamodel again in runtime.

We need this to avoid that extra work that happens at runtime. I plan to work on this piece once I'm done with some optimizations on the Quarkus side.

[dmlloyd]

I was already planning to build such API.

In Quarkus, we can generate the metamodel at build time to generate the implementation bytecode, so we don't have to load it dynamically. In runtime, we still need the metamodel to apply the mapping rules, and because we don't have an API that can leverage what we could collect in build time, we have to read and parse the metamodel again in runtime.

We need this to avoid that extra work that happens at runtime. I plan to work on this piece once I'm done with some optimizations on the Quarkus side.

Allowing the bytecode generation to happen ahead of time is a good idea. The runtime could check to see if a pregenerated implementation class exists (maybe use a naming convention like $$Impl) before dynamically generating one, resulting in faster startup.

For native image it matters less - the generation could happen in build-time class init for example - but pregenerating in this case is not harmful either.

[dmlloyd]

My proposal above shows the idea of having a model be able to be applied directly to a configuration. However, this is not actually a good idea. Quarkus for example combines many configuration models into one configuration parsing action, and there are extra rules (such as detecting extra config keys under certain namespaces, or applying a model under some prefix).

What I would propose instead is a separate API to compose mapping models into a single configuration parser, with additional rules, something like this (as always please ignore the names I pulled out of thin air):

ConfigParserBuilder b = ...;
b.withModel(/*prefix*/"foo.bar", myFooBarModel);
b.withModel(/*no prefix*/mainModel);
b.rejectExtraKeys(/*prefix*/"foo.bar");
ConfigParser p = b.build();
ModeledConfig mc = p.accept(config);
FooBar fooBar = mc.getMappedConfig("foo.bar", FooBar.class);
// ..etc..

By having a separate, well-encapsulated API for each step of the process, we can compose the pieces in different ways based on the desired application (end-user standalone program, user config within a container like Quarkus or WildFly, configuration of larger systems themselves e.g. Quarkus, WildFly, qbicc, etc.)

• Mapping model for a given config interface with introspection capabilities
• Factory for realizing config interface implementation (with an API to pregenerate classes per Roberto's earlier comment)
• Factory for independent instantiation of config interfaces (API for using mapped configurations in a standalone way #1001)
• Combining mappings into parsers
• Applying parsers to configurations
• Removal of MP Config as the primary API, which enables:
• Hierarchical configuration organization for simplified parsing and maybe lower memory footprint

These pieces would work together to greatly simplify the implementation while also making the API much more powerful than it is now.

[radcortez]

Allowing the bytecode generation to happen ahead of time is a good idea. The runtime could check to see if a pregenerated implementation class exists (maybe use a naming convention like $$Impl) before dynamically generating one, resulting in faster startup.

Actually, we already generate the implementations during build-time, but we still need to generate the mapping metadata, because it is required for the mapping functions (and we need to move it to build time, since you notice the difference when moving from the old config to the new config).

As a first step, I think we can provide a way to create the model programmatically (it would still be inspected during build time and the result feeds to the runtime code). After that, we can probably start to iterate on it and improve it.