Intelligent cursor restoration after auto-formatting

[danlamanna]

I've never used tree-sitter so let me know if this is not possible or doesn't make much sense.

Restoring buffer position after applying styling fixes can be difficult. My frequent use case is with the black formatter using https://github.com/lassik/emacs-format-all-the-code on save.

Black makes a guarantee that the AST is unmodified after formatting, so it seems to me that combining tree-sitter with this the user could be placed on the same tree node as they were before the format. This would make it so that autoformatting would leave your cursor in the same logical place as it was before, even if the number of lines of code was radically changed.

[ubolonton]

This is a nice idea! I think it can be done as followed:

• Before formatting, store the "path" leading from the root node to the current node.
• After formatting, use that path to search for the corresponding node in the new syntax tree.

[ubolonton]

There's a macro in 0.5.0 for this: tree-sitter-save-excursion. It keeps the cursor on the same node, and on the same vertical position.

The following works for me in Python (provided that tree-sitter-mode is turned on in the buffer), with Black formatter:

(require 'tree-sitter-extras)

(define-advice format-all-buffer (:around (f &rest args) tree-sitter-save-excursion)
  (if tree-sitter-mode
      (tree-sitter-save-excursion
        (apply f args))
    (apply f args)))

I think it's better to combine this with the diff-based approach mentioned in the issue you linked, though.

[leungbk]

@raxod502, who wrote apheleia.el, is knowledgeable about this. If we use Needleman-Wunsch as with apheleia, it's not clear to me how we'd combine this with a tree-sitter-based approach.

[raxod502]

Yes, there are basically two good options for handling this issue:

• The smart approach, which is to have an understanding of the AST and the cursor's position within a specific node
• The dumb approach, which is to use a string alignment or regexp-based algorithm

If you've got the resources available to do the first approach, it'll probably work better. The reason I think the second approach is good is because the first approach requires a lot of work for each language / formatter, whereas the second approach works for all languages and formatters.

The disadvantages of the second approach are:

• it needs some heuristics to optimize for large diff regions, because the dynamic programming algorithm scales quadratically (Impose maximum length for dynamic programming algorithm radian-software/apheleia#4)
• it needs some heuristics to nudge the cursor to the most appropriate position sometimes, because the dynamic programming algorithm can select one of several different string alignments which score equally well but which do not seem equally appropriate to a user (Point is sometimes moved small distances radian-software/apheleia#2)

[shackra]

I wonder if you see emacs-tree-sitter as a reliable and potential source of information that takes care of the requirements of your first approach, if not, why not

I ask because my mind is dense like a cannonball.

[raxod502]

I mean, you can do smart parsing, but you're going to have to implement it independently for every single language you support, and it probably will have heavy dependencies, and you have to run all your code formatters through the same framework. So I don't view it as a good value proposition versus treating the formatting operation as a black box and reconciling the differences after the fact---at least in the present ecosystem.

[ubolonton]

Unless you mean something else by smart parsing, the first approach doesn't require any additional effort for each language. It works as long as there's a tree-sitter grammar for the language.

It doesn't work when:

• There's no grammar.
• The AST doesn't change, but the concrete syntax tree changes. For example, the Black formatter may wrap some expressions in parentheses.

The first approach (parsing) is more precise, and easier to implement (a couple of lines above), but the second approach (dealing with text only) is more robust.

A hybrid approach can be done pretty easily. Either:

• Try the first approach. If it fails to locate the node, use the second approach.
• Use the second approach. Additionally try the first approach to fine-tune it.

[raxod502]

the first approach doesn't require any additional effort for each language. It works as long as there's a tree-sitter grammar for the language

Well... yeah! Someone has to write a grammar for every language. That's effort, right? :)

But yes, I agree with your points. Using tree-sitter sounds like a fine solution for code formatting, and there's no reason it couldn't be combined with the replace-buffer-contents approach for unsupported languages.

[ubolonton]

Yes, but the effort is orthogonal to writing a cursor-restoration logic, thus not being spent by such a package. That's the important distinction.

[raxod502]

It's since been pointed out to me that Emacs 26 and later ship a function which takes care of all this stuff for you: radian-software/apheleia#38