ParametricExpression within a TemplateExpressionSpec?

[Andrea-gm]

Hello Miles,

I am trying to find an equation to describe the Bremsstrahlung X-rays collected within an energy-dispersive X-ray spectroscopy spectrum. I am using the following TemplateExpressionSpec:

templated_expr = TemplateExpressionSpec(
    function_symbols=["k", "f", "g", "h"],
    combine="((; k, f, g, h), (x1, x2, x3)) -> k() * g(x2) * h((x3-x1)/x1, x3)* f(x1,x2,x3)")

I was wondering if there was the possibility of making k parametric, such as the parameters within ParametricExpressionSpec?
k is simply a scaling factor, and should be allowed to be different in every spectrum. The idea of a parametric expression would do the job, but I am not sure at the moment this can be integrated with a TemplateExpressionSpec? Any suggestion on how to do this?

[MilesCranmer]

This feature is not yet available, though I would also like it! If you are very eager to have this feature (or anybody else reading this comment), I can help walk you through what needs to be done in the backend. I don't think it will be too bad (and Julia is pretty similar to Python in syntax), but it requires some care to make sure everything is compatible.

If you want something quick and dirty, and don't have too many categories, I would recommend just passing in an additional feature i that indexes the category. Then you can have the "parameters" be functions like p1(i), p2(i), and so on. (Or just pass i to the other functions directly).

e.g., if I just write down some random parameters [0.8, 0.2, -4.2, 0.9], I could learn an expression to represent them with p1(i) = (0.82028 / i) + ((0.45242 / (i + -3.0969)) + 0.19716) which is 1e-6 level accurate. So for the general case you would just assume each p1, p2, p3, is going to learn some mapping like that - and you would just remove those expressions at the end and replace them with the numerical values for each category. Maybe you could even write a custom complexity that downweights the complexity in the parameter mappings or something.

But this is not going to be as clean or as accurate as a full implementation of a "TemplateParametricExpressionSpec".

[MilesCranmer]

And I guess "category" in this case is just the enumeration of the specific spectra you are looking at. Like i=1 for spectra 1, i=2 for spectra 2, and so on.

[Andrea-gm]

Thanks for the prompt suggestion! I am not familiar with julia unfortunately, so I think I will try your workaround first and see how that works out

[gm89uk]

Would like to echo support for parametric within template, particularly to support the flexibility of structure within parametric expression. That essentially gives you the power of a custom loss function. I have been doing my own workarounds for a problem with a variable that has 12 categories within template expression.

[MilesCranmer]

@gm89uk @Andrea-gm do you have any ideas for what would be the most intuitive API for something like this? How would you use the parameters inside the template expression?

[MilesCranmer]

The main cost function here is that I want the interface to be as flexible as possible. It should cover all potential use-cases users may have, while also being intuitive and robust.

With this in mind, I wonder if it would be easier if I just allow the user to request a vector of parameters of some length, and then index that however they want? Like some param vector, and they index it using param[classVar1] or param[classVar2 + maximum(classVar1)] to get unique parameters.

I guess it puts a bit more effort on the user in that they would have to request the number of parameters they'd like. But I feel like it's a bit more flexible and generic?

[Andrea-gm]

Hi Miles,

Thanks so much for considering adding this feature. Personally, I was thinking of something like:

expression_spec = TemplateExpressionSpec(
    function_symbols=["f", "g"],
    combine="((; f, g), (x1, x2, x3, p1, p2)) -> p1*sin(f(x1, x2)) + p2*g(x3)^2",
    parametric_parameters = ["p1", "p2"]
)

The category would be passed as usual in model.fit(category = int/list), either as a single value or as a list of length equal to the number of parametric parameters. I the former case, a single category would be used for all parametric parameters.

I think this interface would be quite straightforward for the user, but I am not sure if it's doable?

[MilesCranmer]

A single int should already be doable, because you can use 0-argument functions in a template expression. e.g., ((; f, g, p), (x1, x2)) -> f(x1) + g(x2) - p() * x1 * x2

[Andrea-gm]

But in that case isn't the int common across all data points? My point was to define a different int for different categories of data points (i.e. category = spectrum in my case)

[MilesCranmer]

Right, I thought you meant that by saying model.fit(…, category=int)

[gm89uk]

I guess the main difference / difficulty between the approaches is a means to permit parameter optimisation within and also outside the functions of templateExpressions.

[MilesCranmer]

I think it makes sense to let the user pass the parameters explicitly to expression as additional arguments. The advantage is that a user could also do things like require the same parameters for multiple expressions. Or have the same expressions but different parameters.

[MilesCranmer]

@gm89uk @Andrea-gm what about this API?

structure = TemplateStructure{(:f, :g)}(
    ((; f, g), (x1, x2, x3, i1, i2), params) -> let
        p1 = params[i1]
        p2 = params[i2 + 15]
        f(x1, x2) + g(x3, p2) + p1 * x1 * x3
    end,
    num_params=20
)

Now, you would need to manually input the max number of categories in i1 in advance (so you can offset them) - here, the example is 15. And we have specified there are 20 parameters in total.

This params would basically be a special type of Vector with a special method when you index it with a ValidVector it creates another ValidVector.

The other part I like about this API is it is super super flexible. params is just a vector. You could manually just index one of the parameters, like params[1]. This is actually a better option than the current approach of doing f() - because f() is actually treated as an arbitrary symbolic expression (and is evolved like every other symbolic expression – even if it doesn't look like that under the hood – which is kind of inefficient).

So you basically get access to this vector of parameters and can use them however you want. They'll be optimized like any other constant in the expression.

---

I should also mention that whatever API we decide on is probably there permanently, which is why I'm trying to think very carefully about this.