Not using bytestring-builder internally?

[saurabhnanda]

I'm planning to use ginger as the underlying library for a static site generator. However, I'm puzzled why the only two types types implementing ContextEncodable are Text and Html (which itself is a newtype over Text).

Doesn't ginger internally use lazy-text, lazy-bytestring, or a bytestring-builder? Wouldn't appending to a strict-text repeatedly be very expensive?

[saurabhnanda]

Sorry, submitted too soon. Editing the description now.

[tdammers]

The way this works internally is that the caller supplies a contextWrite function to the template execution context, which takes a (typically small) bit of HTML or plaintext output and streams it to wherever the caller needs it to go. In the ginger CLI program (found in the cli project subdirectory), this contextWrite function does the moral equivalent of Data.Text.putStr, so there is no need for a Builder - no concatenation happens in memory, things just get written directly into the system-provided output buffer. For a static site generator, the same can be done, except you stream into a file - contextWrite would then do the moral equivalent of Data.Text.hPutStr outputFile. In a dynamic web application, you would probably have contextWrite stream output directly to the HTTP response - again, no need for a Builder, because if you do it right, you use the underlying HTTP framework's response buffering mechanism.

There is one issue though, and that is macros. The semantics of macros are such that whatever the macro code emits when run can be captured into a ginger variable, and of course in order to do that, it must be buffered in memory, and the mechanism that does that currently involves concatenating Texts. Changing this to use Builders instead is fundamentally the right thing to do, the only problem is that the mechanism must remain generic enough to support user-supplied types for the h parameter - that is, if the caller wants to use something other than Text or Html for h, it should still work.

A solution for this would be to introduce a type family or fundep typeclass, something along the lines of:

class Buildable t where
    type Builder t :: *
    fromBuilder :: Builder t -> t
    toBuilder :: t -> Builder t

And then capture macro output not into h, but into a Builder h, and add the required constraints everywhere.

On a side note: using lazy texts wouldn't actually fix anything, because the Tower Of Hanoi problem still persists, we would just postpone the inevitable and build up a bunch of thunks - the only situation where we'd win anything would be when the output of a macro is thrown away, or truncated.

If you feel like taking a stab at this, by all means go ahead; if not, I might be persuaded to do it myself, though I'm currently in between holiday trips, so I won't be able to work on it for at least another 2 weeks.

HTH.