Why does hyphen-minus get converted to minus in text when outputing to ms?

[tkurtbond]

Why does pandoc convert hyphen-minus to minus signs in text when outputing to ms?

For instance, why does running

# Trying out Hyphen-Minus in pandoc -w ms

This is a hyphen-minus, -, which pandoc -w ms turns into a Unicode minus sign.

Use the command ``ls -a`` to see all the dotfiles in a directory.

through

pandoc -r markdown -w ms --output=hyphen-minus.ms  hyphen-minus.md

become

.SH 1
Trying out Hyphen\-Minus in pandoc \-w ms
.pdfhref O 1 "Trying out Hyphen-Minus in pandoc -w ms"
.pdfhref M "trying-out-hyphen-minus-in-pandoc--w-ms"
.LP
This is a hyphen\-minus, \-, which pandoc \-w ms turns into a Unicode
minus sign.
.PP
Use the command \f[CR]ls \-a\f[R] to see all the dotfiles in a
directory.

with all those blasted \-s, which are the troff form for entering a minus sign?

Automagically changing hypen-minus to minus signs is never a good thing to do. It changes the example ls -a command to something that won't work because instead of a hyphen-minus it has a stupid minus sign, which is not the character that introduces an option in a command line program, so the command doesn't work right.

The discussion of thesmart extension in the documents doesn't say anything aobut translating hypen-minus to minus, but I tried -r markdown-smart, which didn't seem to do anything.

$ pandoc --version
pandoc 3.6.2
Features: +server +lua
Scripting engine: Lua 5.4
User data directory: /home/tkb/.pandoc

(Didn't I have this discussion on the old pandoc-discuss mailing list once?)

[jgm]

Why don't you search the old archive? Probably this is a bug that needs to be fixed, but it would be helpful to be reminded of what was said then.
https://inbox.vuxu.org/pandoc-discuss

[jgm]

Relevant commit:

commit 5132f1ef330d3eb2a0bf87037035beaeaf19d3f3
Author: John MacFarlane <[email protected]>
Date:   Thu Nov 2 19:36:25 2023 -0700

    T.P.RoffChar: escape `-` as `\-`.
    
    The groff_man (7)` man page indicates that `-` characters will be
    treated as typographic hyphens and are not appropriate for cases
    where the output should be copy-pasteable as an ASCII
    hyphen-minus character.  (E.g. in command line options.)
    
    However, until a recent update groff man did not actually do this;
    it treated `-` and `\-` the same.  With the new update (1.23.0)
    the two are distinguished (see https://lwn.net/Articles/947941/
    for background), so now it is important that pandoc escape `-`.
    
    This reverts ee60ba5252360d2efbf9cf30197236a21a15a924.
    That change was motivated by a problem with backslash-escaping
    `-` in a filename for .PSPIC. That's simply a separate issue;
    we shouldn't do the normal escapes in such a context. It has
    been addressed in the previous commit.

[jgm]

The groff_char man page says:

  The hyphen‐minus is a particularly unfortunate case of
   overloading.  Its awkward name in ISO 8859 and later standards
   reflects the many distinguishable purposes to which it had
   already been put by the 1980s, including a hyphen, a minus sign,
   and (alone or in repetition) dashes of varying widths.  For best
   results in roff systems, use the “-” character in input outside
   an escape sequence only to mean a hyphen, as in the phrase “long‐
   term”.  For a minus sign in running text or a Unix file name or
   command‐line option dash, use \- (or \[-] in groff if you find it
   helps the clarity of the source document).  (Another minus sign,
   for use in mathematical expressions, is available as \(mi.)  AT&T
   troff supported em‐dashes as \(em, as does groff.

[jgm]

This is the article that motivated my change, IIRC: https://lwn.net/Articles/947941/

[jgm]

It is possible that there is now a relevant difference between ms and man...

[jgm]

I found the old discussion. You noted:

The practice of using \- in man pages for the option indicator started because \- looks better typeset, and in nroff it was interpreted as an ASCII minus anyway, since that's all the old terminals could display.  But if you look at groff's an-old.tmac maco file you'll find near the bottom the following section:
.\" For UTF-8, map some characters conservatively for the sake
.\" of easy cut and paste.
.
.if '\*[.T]'utf8' \{\
.  rchar \- - ' `
.
.  char \- \N'45'
.  char  - \N'45'
.  char  ' \N'39'
.  char  ` \N'96'
.\}

This section removes the \- character (along with some others) and explicitly redefines it as code point 45, the HYPHEN-MINUS.  They had to do this for groff's -Tutf8 output mode, because so many man pages use \- for the option character and people looking at man pages in UTF-8 were getting the wrong character when they cut and pasted.

So it seems there is a relevant difference here between man and ms. The change to groff_man noted above may require us to use \- in man pages, but we may need to treat ms differently.

[jgm]

We use the same code for escaping for both formats, so this is going to require some fiddling, but please feel free to submit an issue referring to this discussion.