Convert ≔ to \coloneqq from Word to LaTeX

[ZhuangQu]

≔ is a unicode-math character (U+2254) in A.docx. Run

pandoc A.docx -o B.tex

and then B.tex is

\[≔\]

which causes an error. The correct output is

\[\coloneqq\]

where \coloneqq is the LaTeX command corresponding to ≔.

[jgm]

This should be fine if you use xelatex or lualatex when you compile the LaTeX code; have you tried that?

[ZhuangQu]

You're right. Only pdflatex throws an error. However, pandoc translates all unicode-math characters into corresponding LaTeX commands. From what I've observed, ≔ is the only exception, breaking the consistency.

[jgm]

Actually texmath (which we use for conversion) does support this, but only when unicode-math is enabled in the environment:

https://github.com/jgm/texmath/blob/master/src/Text/TeXMath/Unicode/ToTeX.hs#L698

Pandoc assumes only amsmath and amssymb. This is what you want when generating LaTeX that can be processed with pdflatex, since the default templates only load unicode-math for lualatex and xelatex.

Hence I think pandoc is doing the right thing here.

[ZhuangQu]

You mean: pdflatex does not support unicode-math package, and \coloneqq belongs to unicode-math, so pandoc does not support \coloneqq. Is my understanding correct?

However, \coloneqq is in mathtools package, and pdflatex support mathtools. Actually, mathtools is proposed to fixes various deficiencies of amsmath you mentioned, see https://ctan.org/pkg/mathtools

[jgm]

Yes, correct. I did not know about mathtools.
Currently we have

\usepackage{amsmath,amssymb}

in the default LaTeX template. Would you recommend adding mathtools to this list? Is it compatible with unicode-math (which we load when pdflatex is not being used) and mathspec (which we load instead of unicode-math when the mathspec variable is set and xelatex is being used)?

[ZhuangQu]

Yes, I recommend adding mathtools to the list.

I did some experiments and found no incompatibilities between mathtools and unicode-math/mathspec. By searching on stackexchange, I saw a few incompatible reports on special cases. However, there are also incompatible reports between unicode-math and amsmath. unicode-math also recognizes that "Care should be taken when transitioning from amsmath".

Compatibility issues among LaTeX packages is a perennial problem, not something we can avoid. All pandoc can do is to translate commands as many as possible.

[jpcirrus]

I have been using mathtools with unicode-math in a custom template for almost 10 years without issue. This is with both xelatex and lualatex engines. mathtools.sty contains the following:

\RequirePackage{amsmath}[2016/11/05]
\@ifpackageloaded{unicode-math}{
  \PackageWarning{mathtools}{For best results, please load mathtools
    *before* \MessageBreak unicode-math}
}{}

So line 20 of default.latex can be changed to \usepackage{mathtools,amssymb}. unicode-math is already loaded later in fonts.latex, so no change required there.

Also, mathtools loads calc unconditionally, so the 2 conditional loads of the calc package in common.latex can be removed.

[jgm]

To clarify, if we load mathtools, it replaces amsmath and we don't need to load it too?

[jpcirrus]

Correct.

[jgm]

OK, so two action items here:

• Update the default.latex template as described above by @jpcirrus
• Tell pandoc to call texmath with the unicode-math environment, so it will use \coloneqq and other symbols not available otherwise

[ZhuangQu]

For the same unicode-math character, you want to translate it into different commands based on different LaTeX environments. Is my understanding correct? This is too complicated, not only hard to implement, but also hard to maintain.

I recommend that you specify a "standard" environment where all unicode-math characters have a unique translation. This is much easier to implement and maintain. What is the "standard" environment like? My suggestion is to align with GitHub markdown. Markdown's formulas are expected to be consistent with LaTeX's formulas. GitHub does not allow users to specify which LaTeX package to use, so this environment is unique (standard).

In GitHub markdown, the four commands you mentioned are shown as:

As you can see, only \mathtt{9} and \coloneqq are rendered correctly, so they are standard translations.

[jgm]

texmath is designed to allow this kind of flexibility. We could revisit that design. However, it is important that in converting from tex math all of these symbols can be recognized.

Pandoc has never taken advantage of this flexibility; it just always tells texmath that is has amsmath and amssymb.

I assume that GH is using MathJax for math rendering?

[ZhuangQu]

I checked the github documentation and confirmed that MathJax is used for rendering. I have an idea: add MathJax mode to texmath, and pandoc told texmath to use MathJax. MathJax supports these command

[jgm]

I think it could be quite problematic to assume that all of the symbols in this list are defined in rendering LaTeX. That would be fine for HTML + MathJaX, but it would mean that users who are producing PDFs via LaTeX would need to include a large number of packages to ensure that everything works.
e.g. \curl requires the physics package and \centernot requires the centernot package and longRightleftharpoons requires mwchem and \upzeta requires upgreek. We don't want to put all of these in our default template.

[ZhuangQu]

I agree with you. It's up to you to decide which packages to support.

[ZhuangQu]

I find that when using unicode-math, there is no need to translate unicode-math characters into latex commands. For example, the following code is OK:

\documentclass{article}
\usepackage{unicode-math}
\begin{document}
$α≔∅$
\end{document}

without translating α to \alpha, ≔ to \coloneq, ∅ to \emptyset.

So texmath only needs to have two modes:

1. unicode-math package is used, and unicode-math characters remain unchanged.
2. unicode-math package is not used, and unicode-math characters are translated into latex commands. As @jpcirrus said, \usepackage{mathtools,amssymb} and use them as the standard.

[jgm]

• texmath already has a unicode-math environment.
• Adding awareness of the mathtools environment to texmath would be desirable, but would require some work as noted above. I added an issue Add mathtools support for tex reader/writer texmath#249 that outlines what would need to be done. It's a pretty significant job. Without that step, you still won't get \coloneqq even if we change the latex template to include mathtools.
• Note also that pandoc can't tell texmath to use the unicode-math environment, because pandoc seeks to produce a latex file that can be processed either by pdflatex or by xelatex/lualatex. \ifpdftex and the like are used in the template to achieve this, but it means that we can't tailor the symbols used to whether unicode-math is used.

[ZhuangQu]

Great!

[jgm]

As a short-term workaround, you could simply use a Lua filter to replace U+2254 with \coloneqq in Math elements.