Run fzf

[andychu]

https://github.com/junegunn/fzf/blob/master/shell/completion.bash

From #320

[andychu]

If anyone uses fzf, I could help on this!!

https://github.com/oilshell/oil/wiki/How-To-Test-OSH

https://github.com/oilshell/oil/wiki/Shell-Programs-That-Run-Under-OSH

[mrshu]

@andychu I actually do run fzf (from time to time at least) and since I really wanted to give OSH a try, I've decided to see if I can get it to work.

Sadly, when I tried to source FZF's bash injection script, I ended up with the following stack trace:

osh$ source /home/user/.fzf.bash 2> /tmp/trace.txt                                                                                         
Traceback (most recent call last):                   
  File "/home/user/oil/bin/oil.py", line 859, in <module>      
    main(sys.argv)                                             
  File "/home/user/oil/bin/oil.py", line 827, in main                                                                       
    sys.exit(AppBundleMain(argv))     
  File "/home/user/oil/bin/oil.py", line 797, in AppBundleMain 
    status = ShellMain('osh', argv0, main_argv, login_shell)   
  File "/home/user/oil/bin/oil.py", line 602, in ShellMain     
    return main_loop.Interactive(opts, ex, c_parser, display, errfmt)
  File "/home/user/oil/core/main_loop.py", line 90, in Interactive
    is_control_flow, is_fatal = ex.ExecuteAndCatch(node)
  File "/home/user/oil/osh/cmd_exec.py", line 1188, in ExecuteAndCatch
    status = self._Execute(node, fork_external=fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 1128, in _Execute
    status, check_errexit = self._Dispatch(node, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 671, in _Dispatch
    status = self.RunSimpleCommand(arg_vec, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 538, in RunSimpleCommand
    return self._RunBuiltin(builtin_id, arg_vec, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 324, in _RunBuiltin
    status = self._RunBuiltinAndRaise(builtin_id, arg_vec, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 290, in _RunBuiltinAndRaise
    status = self._Source(arg_vec)
  File "/home/user/oil/osh/cmd_exec.py", line 241, in _Source
    status = self._EvalHelper(c_parser, source.SourcedFile(path, call_spid))
  File "/home/user/oil/osh/cmd_exec.py", line 177, in _EvalHelper
    return main_loop.Batch(self, c_parser, self.arena)
  File "/home/user/oil/core/main_loop.py", line 162, in Batch
    is_control_flow, is_fatal = ex.ExecuteAndCatch(node)
  File "/home/user/oil/osh/cmd_exec.py", line 1188, in ExecuteAndCatch
    status = self._Execute(node, fork_external=fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 1136, in _Execute
    status, check_errexit = self._Dispatch(node, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 671, in _Dispatch
    status = self.RunSimpleCommand(arg_vec, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 538, in RunSimpleCommand
    return self._RunBuiltin(builtin_id, arg_vec, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 324, in _RunBuiltin
    status = self._RunBuiltinAndRaise(builtin_id, arg_vec, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 290, in _RunBuiltinAndRaise
    status = self._Source(arg_vec)
  File "/home/user/oil/osh/cmd_exec.py", line 241, in _Source
    status = self._EvalHelper(c_parser, source.SourcedFile(path, call_spid))
  File "/home/user/oil/osh/cmd_exec.py", line 177, in _EvalHelper
    return main_loop.Batch(self, c_parser, self.arena)
  File "/home/user/oil/core/main_loop.py", line 162, in Batch
    is_control_flow, is_fatal = ex.ExecuteAndCatch(node)
  File "/home/user/oil/osh/cmd_exec.py", line 1188, in ExecuteAndCatch
    status = self._Execute(node, fork_external=fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 1136, in _Execute
    status, check_errexit = self._Dispatch(node, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 1026, in _Dispatch
    status = self._ExecuteList(arm.action)
  File "/home/user/oil/osh/cmd_exec.py", line 1157, in _ExecuteList
    status = self._Execute(child)  # last status wins
  File "/home/user/oil/osh/cmd_exec.py", line 1136, in _Execute
    status, check_errexit = self._Dispatch(node, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 1021, in _Dispatch
    status = self._ExecuteList(arm.cond)
  File "/home/user/oil/osh/cmd_exec.py", line 1157, in _ExecuteList
    status = self._Execute(child)  # last status wins
  File "/home/user/oil/osh/cmd_exec.py", line 1136, in _Execute
    status, check_errexit = self._Dispatch(node, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 693, in _Dispatch
    status = self._Execute(node.child)
  File "/home/user/oil/osh/cmd_exec.py", line 1136, in _Execute
    status, check_errexit = self._Dispatch(node, fork_external)
  File "/home/user/oil/osh/cmd_exec.py", line 726, in _Dispatch
    result = self.bool_ev.Eval(node.expr)
  File "/home/user/oil/osh/expr_eval.py", line 566, in Eval
    b = self.Eval(node.child)
  File "/home/user/oil/osh/expr_eval.py", line 647, in Eval
    raise NotImplementedError(op_id)
NotImplementedError: <Id_t BoolUnary_o 269>

I am not sure what BoolUnary_o is but if you point me into the right direction, I'd be inclined to try to add its implementation to OSH, so that we can get a bit closer to FZF running in it.

Thanks!

[mrshu]

I gave it some more time and this is what I found:

The line this breaks on is

if [[ ! -o vi ]]; then

which basically checks whether the vi option is set. Does OSH currently have "shell options"? I am not sure if this is part of POSIX, so I am not even certain if this is something OSH would like to port, but I am afraid it most probably will be necessary, as many scripts like this one depend on it.

https://www.gnu.org/software/bash/manual/bash.html#Bash-Conditional-Expressions

[mrshu]

And one last thing before I finish here: once I've patched OSH to always evaluate the expression above as False, I was able to successfully source the .fzf.bash file which basically enables its use in bash, although with a couple of errors:

osh$ source /home/xg/.fzf.bash 
    bind '"\er": redraw-current-line'
    ^~~~
/home/user/.fzf/shell/key-bindings.bash:70: Can't execute 'bind': No such file or directory
    bind '"\e^": history-expand-line'
    ^~~~
/home/user/.fzf/shell/key-bindings.bash:71: Can't execute 'bind': No such file or directory
    if [ $BASH_VERSINFO -gt 3 ]; then
                        ^~~
/home/user/.fzf/shell/key-bindings.bash:74: (test) Expected unary operator, got '-gt' (2 args)
      bind '"\C-t": " \C-u \C-a\C-k`__fzf_select__`\e\C-e\C-y\C-a\C-y\ey\C-h\C-e\er \C-h"'
      ^~~~
/home/user/.fzf/shell/key-bindings.bash:79: Can't execute 'bind': No such file or directory
    bind '"\C-r": " \C-e\C-u`__fzf_history__`\e\C-e\e^\er"'
    ^~~~
/home/user/.fzf/shell/key-bindings.bash:83: Can't execute 'bind': No such file or directory
    bind '"\ec": " \C-e\C-u`__fzf_cd__`\e\C-e\er\C-m"'
    ^~~~
/home/user/.fzf/shell/key-bindings.bash:86: Can't execute 'bind': No such file or directory

In general the bind command does not seem to be implemented and $BASH_VERSINFO does not seem to be populated, but otherwise it seems to be on a good track.

[andychu]

@mrshu Perfect, thanks for trying this. I can implement -o easily.

bind is a whole other story... I will have to understand how it works.

Oil won't have $BASH_VERSINFO, so that would require a patch on their side (or a fork). That shouldn't be too big a deal.

[andychu]

Oh since you said you were interested in implementing it -- if you look at osh/state.py, OSH has many shell options. Also osh/builtin.py contains the set and shopt builtin, both of which set options.

[mrshu]

@andychu Thanks for the updates and pointers!

I was actually able to get FZF to run after applying both #349 and #348, although it worked in a somewhat strange way.

When you invoke FZF in bash (by pressing Ctrl/Cmd+R) you basically get a fzf-ied window with your bash history. In OSH, after pressing Ctrl/Cmd+R, the command line changed to

osh$ `__fzf_history__`

and I only managed to get to the fzf-ied window with OSH history by pressing Enter. It therefore seem that bind interprets these backticks and runs the appropriate function(s).

Would you happen to have some view as to how to proceed here? If my understanding is correct, something like interpreting those backticks would have to happen somewhere close to where readline is being wrapped around (i.e. native/line_input.c) but I am really not certain as to whether that is the best way of doing so.

[andychu]

@mrshu Great thanks for making such quick progress on this!

I will try to reproduce this later with your patches.

But one thing that might help is the --debug-file flag to OSH.

https://github.com/oilshell/oil/wiki/Debugging-Completion-Scripts

Basically I do mkfifo _tmp/debug, and then use a tmux window to have one side with osh --debug-file, and the other side does cat _tmp/debug.

So you can see what happens when you hit various keys. The output can be very messy and hard to read, but I'm open to improving it!

This is how I debugged a lot of bash-completion.

---

I don't know exactly why an extra Enter would be required. But I do know that native/line_input.c from Python uses GNU readline in a way that's not exactly like how bash uses it. And GNU readline has tons of global flags and a complicated interface.

At times I have had to read bash's code to figure out what options they're using. It could be some silly global flag.

I forked line_input.c from Python's readline.c precisely so we could make incompatible changes that are more suited for shell.

We should chat on oilshell.zulipchat.com if you have more questions with this! I'm excited to be able to run fzf, as I would like more programs on this wiki page:

https://github.com/oilshell/oil/wiki/Shell-Programs-That-Run-Under-OSH

[andychu]

TODO: try this!

[mrshu]

@andychu it's been a while I guess 🙂

Do you think it's still worth trying out?

[andychu]

Sorry for the delayed response :-/

I definitely think running fzf is valuable. I would like to put more programs here:

https://github.com/oilshell/oil/wiki/Shell-Programs-That-Run-Under-OSH

Though looking back on the history of this bug, I suspect the issue is the lack of the bind builtin. I honestly have no idea how that works, but I would appreciate help on it!

For some context about the project, see this comment:

https://news.ycombinator.com/item?id=28547185 (should appear on the blog at some point)

Basically I'm punting the interactive shell (which includes bind) to "other places", and concentrating on the Oil language. At first Oil will be like a shell that you can use for automation -- for stuff like continuous builds, Github Actions, Docker files, etc.

But I would very much like it if someone pushes on the interactive shell in the meantime! Chat with me on Zulip about it :) I think the bind builtin is a bit tricky because it can't be tested as easily with "spec tests" ??? But maybe I'm wrong

[mcepl]

osh-0.21.0$ source /usr/share/fzf/shell/completion.bash
      if [[ "$l" =~ ^(.*\ -F)\ *([^ ]*).*\ ([^ ]*)$ ]]; then
                                    ^
/usr/share/fzf/shell/completion.bash:60: Expected ]]
osh-0.21.0$ rpm -qf /usr/share/fzf/shell/completion.bash
fzf-0.52.0-1.1.x86_64
osh-0.21.0$

[mcepl]

Yes, with 0.22.0 source /usr/share/fzf/shell/completion.bash works now, and ls ~/somewhere/**<TAB> opens fzf menu. Alt+C and Ctrl-T doesn’t work, but I guess I am supposed to configure it myself (and I don’t have to do anything with bash)?