Bug: Incorrect formatting when using \left\{ and \right. causing unexpected tabbing issues

[Charle4]

It appears that tex-fmt has an issue when handling the combination of \left\{ and \right.. When these are used together in a math environment, the formatter fails to correctly recognize the environment closure. This results in unexpected additional tabs in subsequent lines of the document.

This issue is particularly problematic in documents with multiple occurrences of this pattern, as the incorrect tabbing propagates, making the document increasingly misaligned and harder to read.

Using the following minimal working example (MWE), the issue can be reproduced:

\[
    f(x) =
    \left\{
        \begin{aligned}
            & g(x), & & \text{if } x \geq 0, \\
            & h(x), & & \text{otherwise.}
        \end{aligned}
    \right.
\]
xyz abc xyz ...

After formatting this code with tex-fmt, the last two lines inside the aligned environment are incorrectly indented. Furthermore, all subsequent content in the document is shifted to the right by an additional tab.

\[
    f(x) =
    \left\{
        \begin{aligned}
            & g(x), & & \text{if } x \geq 0, \\
            & h(x), & & \text{otherwise.}
        \end{aligned}
        \right.
    \]
    xyz abc xyz ...

Changing \right. to \right\} resolves the tabbing issue, confirming that the problem lies in the formatter's failure to recognize the closure of \left when paired with \right.. The formatter should properly interpret \left\{ and \right. as a valid delimiter pair, which is a legitimate LaTeX syntax, ensuring that the subsequent lines are not misaligned.

This bug is particularly disruptive in documents with multiple instances of \left\{ ... \right., as the issue compounds with each occurrence, requiring extensive manual intervention to fix. Thank you for investigating this issue and improving the usability of tex-fmt!

[cdesaintguilhem]

Hi @Charle4, I agree with you that tex-fmt should accept all valid delimiter pairs and I'll see if I have some time to look into this unintended behaviour. However this might be tricky because I believe it's the { character that is causing the indentation increase, and it would be the } character that would be required to decrease it again.

I just wanted to point out that you seem to be trying to produce output that could be achieved with the cases environment. The following latex might help remove the mis-matched \left\{ and \right..

\[
    f(x) =
    \begin{cases}
        g(x), & \text{if } x \geq 0, \\
        h(x), & \text{otherwise.}
    \end{cases}.
\]

[Charle4]

Hi @cdesaintguilhem,

Thank you for taking the time to respond and for considering looking into this unintended behavior. I really appreciate your efforts in maintaining tex-fmt.

You are absolutely correct that the cases environment would be a more natural way to format the example I provided. However, the purpose of my example was not to recommend the use of \left\{ and \right. in such simple cases, but rather to demonstrate an edge case where tex-fmt fails to properly handle valid LaTeX delimiter pairs. In more complex mathematical scenarios, the use of \left\{ and \right. (or other similar delimiter pairs) is quite common, especially when combined with nested or invisible delimiters. The issue here is not about promoting a specific style but showcasing a limitation in tex-fmt's current parsing logic, which could cause significant formatting issues in such cases. I believe addressing this could improve the robustness of tex-fmt when dealing with valid LaTeX syntax, even in edge cases.

Thank you again for considering this, and please let me know if I can provide further examples or clarify anything further.

[WGUNDERWOOD]

Thanks for this discussion; this is an interesting case. I see the argument for expecting \right. to match its preceding \left\{, for example, but is this definitely what we want? I can also envision scenarios in which the current behaviour is actually preferable. For example, consider

\left\{
\right.
\left.
\right\}

As far as I know, this is the typical use case for \left. and \right.. Here it might be clearer to have the \left\{ match to \right\} rather than to the internal "delimiters".

[YDX-2147483647]

Here it might be clearer to have the \left\{ match to \right\} rather than to the internal "delimiters".

Well, maybe it makes no difference to match { } or { . + . }? I think the following is still acceptable.

\left\{
    …
\right.
\left.
    …
\right\}

Besides, LaTeX Workshop in VS Code just matches { . + . }.
James-Yu/LaTeX-Workshop#2503

[Binero]

Can confirm this issue. Is there some temporary workaround? I'd like to keep tex-fmt elsewhere. Right now it just sets all my indents to 0.

[cdesaintguilhem]

@Binero You can add % tex-fmt: skip at the end of the lines containing the delimiters and then tex-fmt should ignore them.

Or you can use % tex-fmt: off to turn off tex-fmt for some lines, and then % tex-fmt: on to turn it back on afterwards.