Efficiency of macros

[MarekSuchanek]

Hello,
We are using Ginger to generate documents based on questionnaires where are follow-up questions leading us to recursion. For that, we use macros in the template, but the "deeper" is the questionnaire, it gets significantly slower. I found out that macros are causing this problem, the "more" is the output inside a macro, the worse it gets. I made a simple example (of course this could be written totally without any macro, but we need recursion in the template):

{%- macro lipsum() %}
  <!-- some content, e.g., 5 paragraphs of lorem ipsum -->
{% endmacro -%}
{%- macro nLipsum(xs) -%}
  {%- for x in xs -%}
    {{ lipsum() }}
  {%- endfor -%}
{%- endmacro -%}
{%- macro twiceNLipsum(xs) -%}
  {{ nLipsum(xs) }}
  {{ nLipsum(xs) }}
{%- endmacro -%}
{{  twiceNLipsum(xs) }}

{%- macro lipsum() %}
  <!-- some content, e.g., 5 paragraphs of lorem ipsum -->
{% endmacro -%}
{%- for x in xs -%}
    {{ lipsum() }}
{%- endfor -%}
{%- for x in xs -%}
    {{ lipsum() }}
{%- endfor -%}

The first one takes me approx. 4.5-5 seconds but the second one less than 0.4 seconds, both with 200 calls of lipsum macro in total (i.e. xs of length 100). In our real case, it was around 25 seconds; by removing macros, I managed to get to something like 3 seconds, but it is still quite a lot 😞

Can we avoid this effect somehow?

Thank you in advance 😸

[tdammers]

I think the reason for this is because macros capture their output in a value of type h (usually Html or Text), and whenever the macro emits anything, it gets appended to that value, causing a Shlemiel-the-painter problem. The solution to this would be to either allow the macro to output directly (but this is problematic when filters are involved, or the macro output is somehow processed further), or to accumulate macro output in a Builder instead of a raw Text. The latter is what I'm planning to do (see also #53), but it's a bit tricky because the Run monad transformer is generic over h, so we will need something like a fundep typeclass or type family, e.g.:

class Buildable h b | h -> b where
    toBuilder :: h -> b
    fromBuilder :: b -> h

...and then provide instance Buildable Text Text.Builder and instance Buildable Html HtmlBuilder (where HtmlBuilder is a newtype wrapper over Text.Builder just like Html is a newtype wrapper over Text).

It's also possible that we need a couple strictness annotations to get rid of unnecessary thunks; I will look into that as well.