Curly, dotted and dashed underlines

[spacekitteh]

So, I'm working on a patch which will add more underline styles; the problem is that the SGR codes for them is as follows:

<ESC>[4:0m  # no underline
<ESC>[4:1m  # straight underline
<ESC>[4:2m  # double underline
<ESC>[4:3m  # curly underline
<ESC>[4:4m  # dotted underline
<ESC>[4:5m  # dashed underline
<ESC>[4m    # straight underline (for backwards compat)
<ESC>[24m   # no underline (for backwards compat)

(https://sw.kovidgoyal.net/kitty/underlines/)

Note that they are separated by colons, not by semicolons; if I were to include these into the Underlining type, then in sgrToCode, these would have to be output as, say, [4, 3], which, in an CSI sequence, means "underline and italics".

There are a number of ways to deal with this:

1. In sgrToCode, just output "4" for the new styles, and deal with it in setSGRCode, and add a note in the docs for sgrToCode about it
2. Create a separate setUnderlineStyle function which will do it, and use a different datatype than Underlining
3. Bump to the next major version, create a new type CSICode (or similar), and change the type of sgrToCode to return a list of CSICodes. This is probably the most principled approach, but certainly the most invasive.
4. Don't submit the patch.

What do you suggest?

[mpilgrem]

@spacekitteh, thanks for looking into this. I think the starting point is to understand the extent to which these features have actually been adopted by (or are on the blocks for) the most popular terminals on Windows (including the native Windows Terminal), macOS (including the native Terminal app) and on Linux distributions. As the ansi-terminal package aims to be cross-platform, it does not rush to implement features that are, in a sense, niche or not part of the 'ANSI' standards (as that term is loosely used in the Haddocks for System.Console.Ansi). There is not a hard and fast rule in that regard, but it is helpful context. Here the Kitty URL you provide above seems to suggest to me the codes are not 'standard' in any sense, but unique to that software? The package does not seek to implement codes that are unique to a single terminal.

Where a niche feature is implemented, the package strives for high quality Haddock documentation to give users a heads up on the limitation of using the feature cross-plaform.

[spacekitteh]

I know that VTE, minTTY, and iTerm2 support them as well, and I'm pretty sure I'm forgetting a couple of others. I think Windows Terminal supports them too? Apparently xterm.js and ST do too.

However, this still leaves open the question about the CSICode type: The spec says that subparameters can be separated by colons, so the current situation where CSI codes are always separated by semicolons doesn't allow for that. So for spec correctness, I think that that is still something that should be done anyway.

[mpilgrem]

This - microsoft/terminal#7228 - seems to be a request to add the functionality to Windows Terminal; the issue provides a review of supporting terminals.

[spacekitteh]

Yeah out of those it looks like only Windows Terminal doesn't support it yet.

[mpilgrem]

In respect of CSI, : (03/10) is not a valid separator of numerical parameters but it is valid within a parameter - see ECMA-48 (5th ed) 5.4.2(b) and (c) - for example, acting like a 'decimal point'. A parameter is meant to represent a number in decimal notation (5.4.1(a)), so I suppose a : can feature only once in a valid parameter. The package's csi utility function assumes that the parameters are integers. If you had a more general version of csi, one that allowed parameters to be decimal numbers with a finite number of digits after the decimal point, I am wondering what type you would use - remembering that this package seeks to keep its dependencies on other packages to a minimum. That seems to me to be a non-trivial decision.

[mpilgrem]

microsoft/terminal#7223 is double underlying only. Double underlying is already supported by this package, as it is SGR 21. See 8.3.117 of ECMA-48.

[spacekitteh]

If you had a more general version of csi, one that allowed parameters to be decimal numbers with a finite number of digits after the decimal point, I am wondering what type you would use - remembering that this package seeks to keep its dependencies on other packages to a minimum.

I think something like this:

data CSICode = Simple Int | Complex String

[mpilgrem]

As 'fancy' underlining does seems to be supported by a number of terminals, and seems to be in active consideration for Windows Terminal too, then implementing it here is fine.

There is a tension over whether the API of this package should reflect (when there is a conflict): (a) the structure of 'ANSI' codes; or (b) the structure of what end users want to achieve. This 'fancy' underlying functionality is clearly outside of SGR in terms of codes, but it does something 'like' what parts of SGR do. Personally, I fall in the former camp and I am attracted to your option No. 2 in your initial post - perhaps the type is FancyUnderlining to contrast it with SGR's pedestrian (but more widely supported) underlying choices?

[spacekitteh]

Is it actually outside of SGR though? Is SGR specifically defined to accept no non-integer parameters?

[spacekitteh]

Something like this would be more accurate:

data CSIParameter = SimpleParameter Int
                  | ComplexParameter [Int] [Int] -- Before and after colon separator
data CSICodes = CSICodes [CSIParameter] Char -- Parameters + Control function

[spacekitteh]

Of course, you could just define an enum with 0-9 to restrict the types even further.

[mpilgrem]

Is it actually outside of SGR though?

I see what you mean. I was thinking that SGR was limited to the Ps listed at ECMA-48 8.3.117, but this could be viewed as an extension to SGR (SGR '4 and a bit').

[mpilgrem]

In which case, I think a breaking change to extend the constructors of System.Console.ANSI.Types.Underlining is tolerable. I suspect very few people out there are currently pattern matching on Underlining.

[spacekitteh]

Yeah, also see the note: NOTE The usable combinations of parameter values are determined by the implementation.

That could be interpreted as "what parameter values mean" as well as "what parameters are accepted", in addition to "which parameters work with which others"

[spacekitteh]

You could also rename the existing constructors, and introduce some parameter synonyms, and mark them as complete, defaulting curly/dotted/dashed to just mean single underline, too.

[mpilgrem]

One can get carried away with types. Another way to go might be new functions sgrToCode' :: SGR -> String and csi' :: [String] -> String -> String. They are, after all, only helper functions that happen to be exposed as 'utilities' in case anyone found them useful.

[spacekitteh]

That might be the simplest way forward.

[spacekitteh]

One other thing is underline colours; the simplest way forward is to add "UnderlineColor" to the ConsoleLayer type; this fits in nicely due to it being code 58 and working the same as 38/48.

[mpilgrem]

@spacekitteh, I've returned to this because the implementation of the feature request for Windows Terminal has recently been merged and so this looks to be available cross-platform soon. I've raised two pull requests (#161 - subparameters, and #162 - changes to API) which are draft while I test that they do what I intend them to do.

You will see that I picked Underlining for the new ConsoleLayer.

I also settled on (Int, [Maybe Int]) for a parameter value followed by a parameter substring (as T.416 (03/93) calls it) - although I did provide some type synonyms.

The package emphasises backwards compatibility and stability, so I provided csi' and sgrToCode' rather than change the types of the existing utility functions csi and sgrToCode :: Either ParamWithSubs [Parameter].

[mpilgrem]

Now merged.