Asynchronous Reliable Extensible Sleek RPC Server for Guile. First of all it’s a library providing a lot of tools for evaluating and exploring the code, from simple reflection API to interruptible asyncronous evaluation. In addition to that it provides an async RPC server implementation based on nREPL protocol and can be used for programmable interactions with a running guile processes, for implementing REPLs, IDEs, test runners or other tools. It also can be potentially used as a backend or library for LSP server.
WARNING: It’s in early stage of development, the API is a subject to change.
guile-ares-rs
was previously known as guile-nrepl
(because it started
as just nREPL implementation in Guile Scheme). It was renamed to
avoid any confusion with network/socket REPL, guile-ares-rs
is not a
usual REPL, it’s actually not a REPL at all, it’s an RPC Server,
however it’s possible to implement REPL-like experience with it (and
respective client).
It’s intended for use with Arei IDE, but you can use it with other generic nREPL clients.
The are multiple ways to utilize Ares RS, but most straightforward is to spawn a standalone server and connect to it from a client (Emacs Arei for example):
guix shell guile-next guile-ares-rs -- \
guile -c '((@ (ares server) run-nrepl-server))'
Don’t forget to add your project source code and other dependencies to load path.
It depends, on fibers, and on guile-next (custom textual ports), tests are executed with gider.
Send patches to rde-devel mailing list in format:
[PATCH guile-ares-rs 0/3] SUBJECT-HERE
.
Send feedback to rde-discuss.
Arei and Ares 1.0 milestone.
- How to work with fibers in nrepl? Maybe C-2 C-c C-e to entered to the clonned session with fibers spawned.
- Bootstrap from guile network repl? (Send guile-ares-rs implementation to remote network repl server to make it nrepl).
- Multiple guile languages support.
How to bypass continuation barrier in evaluation thread, when using previously stored continuation.- Translation level for shrothanded nrepl base operations?
- Do we need to support meta-commands? (Probably not, it just functions, which can be exported to user or repl module scope).
One of the reasons this project started is unfortunate missbehave of current REPL soultions in different situation and here is a list of some of them:
- Text based REPL over socket is not interruptible.
- call/cc works when pasted into prompt of the repl, but fails when evaling from buffer with C-x C-e. (when saving continuation to symbol with set!)
- It’s not possible to see stdout until evaluation completed.
- Gider tests hangs repl on errors.
- CAPF function is injected, even if there is no connection to REPL and throws errors preventing other CAPFs to work.
- There is no way to provide stdin input to running expression.
- Go to definition doesn’t work for with-input-from-port.
- After using
,i
in REPL, go to definition hangs up. - Go to definition opens non-existing file.
- If expression being evaluated from buffer requires input from stdin it will be frozen.
- Rebinds M-. instead of using xref.
- https://spritely.institute/files/docs/guile-goblins/0.11.0/TEMPORARY-Live-hacking-_0060_0060fix_0027_0027.html
- https://www.reddit.com/r/scheme/comments/qyqxcd/geiser_sucks_is_there_a_way_to_make_it_suck_less/
- Comment by Olical, Conjure author on guile-ares-rs integration
- https://github.com/nrepl/nrepl
- default clojure implementation.
- https://nrepl.org/nrepl/1.0/building_servers.html
- some tips.
- https://gitlab.com/technomancy/ogion/-/blob/master/main.rkt
- very simple Racket nREPL.
- https://gitlab.com/technomancy/jeejah
- lua nREPL server.
- https://github.com/babashka/babashka.nrepl
- probably most advanced 3rd party nREPL server implementation, doesn’t support interrupts.
- https://docs.cider.mx/cider-nrepl/nrepl-api/ops.html#info
- extensions of nREPL for CIDER.
- http://www.flow-storm.org/
- https://github.com/clojure-emacs/sayid
- https://github.com/philoskim/debux
- https://github.com/jpmonettas/flow-storm-debugger
- https://docs.cider.mx/cider/debugging/debugger.html
- https://www.gnu.org/software/guile/manual/html_node/Interactive-Debugging.html
- https://www.gnu.org/software/guile/manual/html_node/Debugging.html
- https://github.com/sanel/monroe/
- https://github.com/Sasanidas/Rail
- Potentially Conjure, when nREPL for guile will be implemented in it.
- https://github.com/flatwhatson/grump/blob/master/grump-lib/bencode.scm
- https://github.com/caolan/chicken-bencode
- http://www.bittorrent.org/beps/bep_0003.html#bencoding
- https://wiki.theory.org/BitTorrentSpecification#Bencoding
Q: Can I use guile-ares-rs with an existing network REPL?
A: Theoretically it’s possible to upgrade existing REPL to guile-ares-rs, see bootstrap and infect modules (they can be not implemented yet).
Thanks for inspiration, help, support and motivation to Jos´e Antonio Ortega Ruiz, Nikita Domnitskii, Maxime Devos, Andy Wingo, Bozhidar Batsov, Andrey Listopadov, Dmitrii Bogdanov, David Thompson, Dmitry Polyakov.
@startuml
/'
' scale 350 width
'/
state ForkState <<fork>>
[*] --> ForkState
ForkState --> EvaluationThread
ForkState --> EvaluationThreadManager
state EvaluationThread {
[*] --> Idle
Idle --> Running : Scheduled
Running --> ReturningValue1 : Finished
Running --> ReturningValue2 : Interrupted
Idle --> ReturningValue3 : Interrupted
/'
' ReturningValue -[dotted]-> EvaluationThreadManager : Returned Value
'/
}
state EvaluationThreadManager {
state "Waiting" as TWait
TWait: Waiting for Command or Thread Value
[*] --> TWait
TWait --> RunEval : Eval Command Received
TWait --> Interrupt : Interrupt Command Received
state Interrupt {
Try -[dotted]-> EvaluationThread : Interrupt
}
RunEval -[dotted]-> EvaluationThread : Schedule
}
/'
' state Configuring {
' [*] --> NewValueSelection
' NewValueSelection --> NewValuePreview : EvNewValue
' NewValuePreview --> NewValueSelection : EvNewValueRejected
' NewValuePreview --> NewValueSelection : EvNewValueSaved
'
' state NewValuePreview {
' State1 -> State2
' }
'
' }
'/
@enduml
@startuml
!theme reddress-lightblue
/' needed for &, but unfortunatelly breaks diagram '/
/'
' !pragma teoz true
'/
title Ares-rs Architecture
actor "nREPL Client" as Client
participant "Ares Server" as Server
participant "Socket" as Socket
participant "Event Loop Fiber" as EventLoop
participant "Evaluation Thread" as EvaluationThread
group New Connection
Client -> Server: Establish Connection
Server -> Socket **: Create Socket
Server -> EventLoop **: Setup Event Loop
EventLoop -> Client: Connection Established
end
group Eval Processing
Socket ->> EventLoop: Evaluation Request (eval op)
EventLoop ->> EvaluationThread: Schedule Evaluation
EvalutaionThread ->> Socket: Reply 1
EvalutaionThread ->> Socket: Reply 2
EvalutaionThread ->> Socket: Evaluation Value
end
/'
' participant "Evaluation Supervisor" as Supervisor
' participant "Evaluation Manager" as Manager
' actor "External Actor 2" as Actor2
'/
/'
' group New Session
' Actor -> Supervisor **: Create
' activate Supervisor
' group Basic Evaluation with stdout and stderr
' Actor ->> Supervisor: nREPL eval operation\nCommand: process-nrepl-message
'
' Supervisor -> Supervisor: Enqueue.\nCommand: evaluate
' Supervisor ->> Manager **: Run Evaluation
' activate Manager
' Manager ->> Actor: Send Standard Output
' Manager ->> Actor: Send Error Output
' Actor2 ->> Supervisor: nREPL eval operation\nCommand: process-nrepl-message
' Supervisor -> Supervisor: Enqueue Evaluation
' Manager ->> Actor: Send Standard Output
' Manager ->> Actor: Send Evaluation Value
' return Done\nCommand: finished
' destroy Manager
' end
' group Queued Evaluation with interrupt
'
' Supervisor -> Supervisor: Command: evaluate
'
' Supervisor ->> Manager **: Run Evaluation
' activate Manager
' Manager ->> Actor2: Send Standard Output
' Actor ->> Supervisor: nREPL interrupt operation\nCommand: process-nrepl-message
' Supervisor ->> Actor: interrupt-id-mismatch
'
' Actor2 ->> Supervisor: nREPL interrupt operation\nCommand: process-nrepl-message
' Supervisor ->> Manager: Signal Interrupt
'
' Manager ->> Actor2: Succesful Interruption
' return Done\nCommand: finished
' destroy Manager
' end
' deactivate Supervisor
' end
'/
@enduml
@startuml
!theme reddress-lightblue
/' needed for &, but unfortunatelly breaks diagram '/
/'
' !pragma teoz true
'/
title Evaluation Supervisor
actor "External Actor" as Actor
participant "Evaluation Supervisor" as Supervisor
participant "Evaluation Manager" as Manager
actor "External Actor 2" as Actor2
group New Session
Actor -> Supervisor **: Create
activate Supervisor
group Basic Evaluation with stdout and stderr
Actor ->> Supervisor: nREPL eval operation\nCommand: process-nrepl-message
Supervisor -> Supervisor: Enqueue.\nCommand: evaluate
Supervisor ->> Manager **: Run Evaluation
activate Manager
Manager ->> Actor: Send Standard Output
Manager ->> Actor: Send Error Output
Actor2 ->> Supervisor: nREPL eval operation\nCommand: process-nrepl-message
Supervisor -> Supervisor: Enqueue Evaluation
Manager ->> Actor: Send Standard Output
Manager ->> Actor: Send Evaluation Value
return Done\nCommand: finished
destroy Manager
end
group Queued Evaluation with interrupt
Supervisor -> Supervisor: Command: evaluate
Supervisor ->> Manager **: Run Evaluation
activate Manager
Manager ->> Actor2: Send Standard Output
Actor ->> Supervisor: nREPL interrupt operation\nCommand: process-nrepl-message
Supervisor ->> Actor: interrupt-id-mismatch
Actor2 ->> Supervisor: nREPL interrupt operation\nCommand: process-nrepl-message
Supervisor ->> Manager: Signal Interrupt
Manager ->> Actor2: Succesful Interruption
return Done\nCommand: finished
destroy Manager
end
deactivate Supervisor
end
@enduml
@startuml
/'
' !theme cerulean-outline
' !theme plain
' !theme reddress-lightblue
' !theme sketchy-outline
' !theme vibrant
'/
!theme reddress-lightblue
title Evaluation Workflow
actor User
participant "NREPL Server" as Server
participant "Operation Dispatcher" as Dispatcher
participant "Evaluation Watcher" as Watcher
participant "Evaluation Thread" as Thread
/'
' group Connection
' User -> Server: Establish Connection
' Server -> Server: Create Socket
' Server -> User: Acknowledge
' end
'
'/
group New Session
User ->> Server: Clone Request
Server ->> Dispatcher: clone-op
Dispatcher ->> Dispatcher: Create New Session
Dispatcher -->> Server: New Session Object
Server -->> User: New Session Response
group Evaluation
User ->> Server: Eval Request
Server ->> Dispatcher: eval-op
note right Dispatcher: session context
Dispatcher ->> Watcher **: eval-op
Watcher -> Watcher: Setup Channels and Conditions
Watcher -> Printer **: Attach to Out Ports
activate Printer
Watcher -> Thread ** : Run Evaluation Thread
Thread -> Thread ++: Setup dynamic-wind
Thread -> Thread ++: Start evaluation
loop Output
Thread ->> Printer: Flush Output
Printer ->> Dispatcher: Eval Object
end
opt evaluation interrupted
User ->> Server: Interrupt Request
Server ->> Dispatcher: interrupt-op
Dispatcher ->> Watcher: Interrupt Request
Watcher ->> Thread: cancel-thread
return interrupted
end
Thread ->> Watcher: Signal Finished
Thread ->> Printer: Signal Finished
return done
opt evaluation finished successfully
Watcher -> Thread: join-thread
Thread --> Watcher: evaluation value
end
Printer -> Printer !!: Close Port
deactivate Printer
deactivate Thread
end
end
@enduml