(print (fp 2 3)) )
(let [fpp (partial foo/fp 2)] (print (fpp 3)) )
(t ((p 1) 2) [1 2]))
(let [f (fn [x y] [x y]) p (partial partial f)] (t ((p 1) 2) [1 2]))
(let [mypartial partial pp (mypartial partial f) p (pp 1)] (t (p 2) [1 2]))
(let [mypartial partial pp (mypartial mypartial f) p (pp 1)] (t (p 2) [1 2]))
(let [mypartial partial pp (partial mypartial f) p (pp 1)] (t (p 2) [1 2]))
(do (defn bax [] bax) (t (= (bax) bax) true) )
(defn get-bax2 [] (fn [] bax2))
(do (def bax2 (get-bax2)) (t (= bax2 (get-bax2)) false) (t (= bax2 bax2) true) )
(do (def bax3 (let [x 1 y (fn [] bax3)] y)) (t (= bax3 (bax3)) true))
(let [bax (fn bax4 [] bax4)] (t (= bax (bax)) true))
(do (def q (let [r q] r)) (t (bound? (var q)) false))
(defn get-bax4 [] (fn bax4 [] bax4))
(do (def bax (get-bax4)) (t (= bax (get-bax4)) false))
(defn bix [n] (when (> n 0) (cons n (bix (- n 1)))))
(t (= (bix 5) (list 5 4 3 2 1)) true)
(let [foobar (fn bix2 [n] (when (> n 0) (cons n (bix2 (- n 1)))))] (t (= (foobar 5) (list 5 4 3 2 1)) true))
(defn bix [n] (when (> n 0) (cons n (bix (- n 1)))))
(def bix2 (fn bix3 [n] (when (> n 0) (cons n (bix3 (- n 1))))))
(def bix4 (fn [n] (when (> n 0) (cons n (bix4 (- n 1))))))
(defn bix5 [n] (let [b bix5] (when (> n 0) (cons n (b (- n 1))))))
(defn main [& args] (t (= (bix 5) (list 5 4 3 2 1)) true) (t (= (bix2 5) (list 5 4 3 2 1)) true) (t (= (bix4 5) (list 5 4 3 2 1)) true) (t (= (bix5 5) (list 5 4 3 2 1)) true) )
in a lval_wasm_lambda any partials should all be retained, meaning we can release all of the partials when releasing the lval_wasm_lambda
so get interpreter to end with all slots free again!
we need to free its closure and partials!!
In eval.c, eval_lambda_call: Namespace* old_ns = state->current_ns; // TODO: bit iffy, look into more if (lval_fn->ns) state->current_ns = lval_fn->ns; Lval* ret = do_list(bindings_env, lambda->body, RETURN_ON_ERROR);
state->current_ns = old_ns;
Maybe make sure things are included in the lval_fn’s closure
when the fn is defined the fn itself is not in the env yet. So (defn foo [] (foo)) will not work since foo is not yet defined when we add the foo fn to the env.
So after we define a fn we need to add it to the closure. This will fix it for the interpreter.
Also, clojure allows it.
So LOCAL and PARAM (don’t set to local, since we just retrieve a local for these ones) vs closure (set to local because its a computed value, and we don’t want to insert it every time we refer to it)
Partial: And what about (partial partial f 2) or where either partial is an lval_ref or result of fn call!!!
retained, since at compile time we don’t know if it’s been a real fn call, so we’ll pretend it was, and the result of the whole partial call can then be released when required, but we still want to compile all args, well the fn calls at least.
https://github.com/clojure/clojure/blob/38bafca9e76cd6625d8dce5fb6d16b87845c8b9d/src/clj/clojure/core.clj#L660 Use core lib fn to reduce args to a single list Then call wasm implementation of apply* sys fn that receives a list and puts them on wajure operand stack and calls fn
But we can optimize this depending on:
- apply is called as apply (so no (let [a apply] (a + 1 2))), so apply is not a lval_wasm_lambda
- the function to apply is not a lval_wasm_lambda, I think
- the last arg (a list) is a list at compile time and not the result of a compilation
Scenario:
- If we know we’re calling apply at compile time and we know the last arg is a
list we can make a lval_list with apply’s second arg (the fn) as its first element and all the args as the rest and call apply(wasm, lval_list) to compile the apply call, it doesn’t matter whether the fn to apply is a wasm_ref, sys fn, imported fn etc since appl(wasm, lval_list) will take care of that
- we fill the stack with the compiled args, if we know the last
arg is a list we compile the list members and add them to the stack assume the last arg is a list (put a check in if it’s a lval_compiler)
- When datafying the apply fn we don’t know anything at compile time and we need to rely on a apply fn such as in clojure.core. But we need multiarity fns for that first.
map is special case of reduce
http://clojure-doc.org/articles/language/macros.html
We can sometimes know what lval_refs actually are, we don’t always have to retrieve by local index and then get the data out of them.
For instance the ‘self’ ref can probably be called by fn_table_index if not by name directly
Also when assigning literals to a lval_ref we might as well substitute the literal iso retrieving the wval by local index.
Same when fns are assigned directly to a lval_ref/local and we want to call them we can call by name I think.
Also no need to assign a lval_ref/local/wval to another local when it already is retrieved by local index.
Do assign closure to a local because the are retrieved with a load and a local_get is faster.
replace list implementations of maps, sets and vectors with permanent data structures other than list
-> vector and map hamt.
(defmacro foo [x] (if (= x 1) ‘(fn [] 1) ‘(fn [] 2))) (defn main [x y] (let [f (foo 0) f2 (foo 0)] (print (f)) (print (f2)))) This will create a wasm fn for (fn [] 1) twice. See add_wasm_function: // TODO: compare lval_fn with other fns added so far. If a match don’t create // the wasm fn again but retrieve its fn-table_index and set that on lval_fun // This can happen when macros return a fn for instance.
Don’t record compiler data on them like data_offset, Context and offset. It’s just asking for trouble. But alternative is using a hashtable for lookup of lvals to get their compiler data. We’re already efficiently wiping compiler data from root lvals (as found in env of namespace). When we use macros in fns we can encounter literals that have their compiler data set already. This is ok unless compiler data is set when compiling another namespace, the data_offset is not applicable then. So we check if the lval is from this namespace or not. See logic and comments in datafy.c/datafy_lval So the trouble seems managable so far....
So we need to store the name as we do param_count, rest_arg_index etc now as well in the wval
benchmark whether internal module calls are faster than calling imported fns or calling imported table fns
As per warning in Binaryen docs. When reusing optimisations might screw things up.
Currently only ints work
hamt
Currently vars of a namespace are/will be hardcoded into the fns that then refer to them statically. Alternatively we could store them in a namespace env and refer to them dynamically. This way we could have a ‘image’ that we can modify in a repl. We could then redefine values quite easily (with an interpreter built into the runtime). However interpreter fun objects are different from compiled fun objects. So they would have to be bridged. Either by building in a compiler, but the wasm would have to be reloaded then, or by relaying any call to an interpreted fn to the interpreter’s repl. Interpreter and runtime can easily share env though.
But our algorhitm to compute a hash needs 64bits operations, so we need to rewrite it or find another c algorhitm
in repl you can (re)compile namespaces. Also, it can watch directory and if any clj source file gets modified, recompile. Because it’s a live env we can expand macros at compile time if needed, not sure how yet. But we do need a live env for that be possible when macros use referred values and fns from required namespaces when expanded.
You should also be able to switch namespace.
(defmacro when2 [cond body] `(if ~cond ~body)) (print (macroexpand ‘(when 1 2))) (print (macroexpand ‘(when2 1 2)))
-> (when 1 2) -> (if 1 2)
implement: loop/recur, doseq, keywords, map, reduce, and, or, xor, doseq, multimethods, destructering, meta data
replace list implementations of maps, sets and vectors with permanent data structures other than list
-> vector and map hamt.
Also, clojure allows it.
Don’t cut off compiling, try to continue, produce list of errors.
&form - the actual form (as data) that is being invoked
(defmacro foo [a] (print &form)) (foo (+ 1 1)) prints out (foo (+ 1 1))
&env - a map of local bindings at the point of macro expansion. The env map is from symbols to objects holding compiler information about that binding. (do (let [bar 123] (defmacro foo [a] (print &env)) (foo (+ 1 1)))) prints: {bar #object[clojure.lang.Compiler$LocalBinding 0x7fad7051 clojure.lang.Compiler$LocalBinding@7fad7051]}nil
and or when when-not when-let when-first if-not if-let cond condp cond-> cond->>
for doseq dotimes while
ns declare defmethod defmulti defn- defonce
.. doto -> ->>
assert comment doc
- direct to NOT_A_FN rt error by default
- fns should receive ptr to lval, not just to closure as first arg
WARNING (src/refcount.c:70): Warning: trying to release data that’s not managed by ref_count (data_p: 70264, slot->data_p: 112).
we only need to call this once, then hang it off the lambda and reuse it when we call by name
Similar to clojure.core. Probably needs namespaces implemented first
into a fn, such as {},[],:kw etc Not needed with our fn_call_relay_array. We can just add the partials.
It returns a nil lval if list is empty because it’s used for the rest arg,but make sure not to use it when we expect an empty list!!!!
4 bytes aligned. Probably better for loading data!!!
obsolete if dirty compile everything, and stdlib first. we can then move:
assign_fn_table_index_to_native_fns(wasm); register_native_relay_arrays();
to init_wajure
It’s used for multiple purposes:
- fn_table_index
- param and local index (lval_ref)
- DONE export/record the fns table indices somehow, so we can use them when compiling other modules
- add fn_call_relay_arrays to wajure.core data for and export the pointers to use in make_lval and in datafy
1-4 {}, #{}, [], :kw
- Not a fn runtime error.
- write fns to dispatch to when first arg of list is a kw, map, set, vector.
- And set the fn_call_relay_array properly on on all lvals!!!!
runtime stacksize, runtime data_end, wajure data_end, heap_base get_memory() nodejs: initial_page_count, max_page_count makefile: initial-memory and stack size
(block $args_4 (if (local.get $7) (memory.copy (local.get $5) (call $get_wval_partials (local.get $6) ) (local.tee $9 (i32.mul (local.get $7) (i32.const 4) ) ) ) (nop) ) (local.set $10 (i32.add (local.get $5) (local.get $9))))
(block $args_4 (local.set $10 //only if there are args to the fn (if (result i32) (local.get $7) (block (result i32) (memory.copy (local.get $5) (call $get_wval_partials (local.get $6)) (local.tee $9 (i32.mul (local.get $7) (i32.const 4)))) (i32.add // only if there are args to the fn (local.get $5) (local.get $9))
) (local.get $5) //only if ther args, otherwise nop ) )
when looking up function index to relay to.
Don’t call native partial fn in compile_partial_call can be more optimized
(let [p partial] (p f 1)) so when the partial fn is a LVAL_REF, we need to be able to datafy the partial fn
So find a way to call native partial fn!!!!, when we just have a pointer to an lval and that’s supposed to be the native partial fn:
So we need to have a native fn that does the right thing. And it should receive all of its args in an arg block!!! Because why bother putting it all in a list like we do for sys fns
(defn f [x y z] [x y z]) (def fp (partial f 1 2)) We’ll get two identical fns, f and fp
trying to release that’s not managed by
(defn f [x y z] [x y z]) (def f2 f) (defn f [x] 1)
(defn main [x y] (print (f 1 2 3)))
This is because f gets replaced by the second f, but and the second f is already processed and added to wasm because it came first in the env (so when compiling f2, which still refers to the old f we get the error, because it’ll get added as f), and that’s because we do lenv_put, and not lenv_prepend, which would solve this problem. Well, that is, if we check for the function in wasm in add_wasm_function and remove it and replace it with the update one when compiling.
You could be a bit smarter about it at compile time.
- When a symbol resolves to a sys fn you can check arg count
- When a symbol resolves to a root lambda fn (as found in compiler env) you can check arg count
- When a symbol resolves to lval_ref we can know whether the lval_ref is a ref to a lambda, and which one eg: (let [f (fn [x] x)] (f 123)) but also in: (let [f (fn [x y] ..) g (partial f 1)] (g 2))
Clojure has max of 20 args, not sure what happens in (foo a1..a20, & rest-arg)
See if we can put args on stack from low to high iso of high to low as we have now. If so we can do apply easier as well.
malloc(sizeof(str)) iso malloc(_strlen(str)) !!!!
So propagate errors properly till we the last return from compile_main
Ideally you’d want to have to compile only the source files that have changed at all since last compilation. However dependencies come into play here. In our case, because we reduce any non function values to a single lval at compile time, and because we use global imports to refer to external (from another namespace) in functions at runtime we only have to deal with external refs in non fn values as in: (def a foo/b).
When a namespace refers to a var in another ns from a non fn, that namespace will be recompiled when its required ns gets recompiled. To avoid this don’t refer do this, better is to refer to it in a fn. Or wrap the value in a fn: Instead of (def a foo/bar) write (defn a[] foo/bar).
If this is not desirable and too many namespaces are recompiled in development it’s an option to add the feature of wrapping all (def …), in a parameterless fn put a flag on the symbol and use a fn call to retrieve the value (by using a global) iso datafying the value. But this would/might slow down the program, and increase compilation time.
but also the deps!!
throw result of condition through fn that returns 0 if condition is false or nil, otherwise 1
This is a lval_namespace. lval_namespace->head points to a Namespace struct. We need to add a mempool type NAMESPACE and a destroy method for it, so we can release namespace->namespace and namespace->as/refer
So break string into two, and inter strings only once, and do two prints
This shouldn’t be too hard. We just need to pass a closure to the f lambda
- so we can more easily see if we just compiled a fn call. iso relying on is_fn_call flag
- we might be able to do optimisations, such as mutually cancelling retain and release calls
So wasmified sys fns are called eg sys_print
And lambdas (such as foobar) found in compile env should be renamed and numbered, eg: l1_foobar, no I don’t think that’s needed: we use the latest lval defined for a symbol in the compiler env.
Anonymous lambdas found in fns become foobar#1, foobar#2 etc.
`(let [a 1] a)
#include <stdarg.h> //va_start, va_list #include <stdio.h> //printf, puts #include <stdlib.h> //malloc, calloc, realloc
and link them to compiled wajure code runtime includes:
- builtin fns
- memory management
M-x gdb gdb -i=mi cd ..; file out/lispy run -c wajure/main.clj
// Every lval is either the result of a fn/lambda call, special form or a // retrieving of interred values or previously calculated dynamic values. This // flag keeps track of what we just put on the wasm stack is the result of // retrieving of a value, or the result of wasn fn call or special form (in a // wasm block). We need to keep track of this because we want release all // calculated values after they’ve been passed to another fn, eg in (f (+ 1 2) // some-var 123) we want to release the result of (+ 1 2) after f returns, but // not some-var and 123. // // Similarly at the end of a do/let block or fn body we want release all // values that were the result of a fn call eg: in (do 123 some-var (+ 1 2) 1) // we want to release (+ 1 2) and retain 1. In (do 123 (+ 1 2)) we want to // retain (+ 1 2). In (do 123 (+ 1 2) some-var) we want to release (+ 1 2) and // retain some-var. // // In (let [x 1 y (+ 1 x) some-var (+ 1 x)] x some-var) we want to retain // some-var, but also release also all bindings that are result of fn calls // (so y and some-var)
In the CResult of a lval_compile we have info on whether we just compiled a fn call or not (result.is_fn_call)
Before we call a fn we put all args on the stack, then adjust the stackpointer to point to the first free mem again. After returning we set the sp back again. When calling fn we know how many args are passed so we can hardcode the sp adjustment. When in the fn we have to subtract offset from the sp to get at the args.
Alternatively we could adjust the sp in the fn itself but we’d have to rely on the wasm arg count arg that any fn gets passed in. We’d add that arg count to the sp before adding args to the stack frame and then calling a fn. On return we’d subtract it again. When getting at the lispy params on the stack we’d have to first add the arg count, then subtract the expected arg count, again relying on the passed in arg count in second wasm param.
First solution uses hardcoded values, second doesn’t.
Stack looks like this btw:
arg2 arg1 arg0 | x x rest_arg arg1 arg0 | etc.
where sp points at the |’s and we extract the args from the stack frame just before the sp.
This is so that we can easily add partial args on top (as found in a lval_wasm_lambda)
make clean bear make
rc -J
https://github.com/Andersbakken/rtags/wiki/Usage
make clean make run
make clean make compile
PLATFORM=wasm make clean PLATFORM=wasm make runtime
bin/clj-repl
When using in-ns also evoke (clojure.core/use ‘clojure.core)
Path to clj dir is set in deps.edn in project root
In Emacs connect with inf-clojure (connect to localhost:5555) https://github.com/clojure-emacs/inf-clojure ;; (add-hook ‘clojure-mode-hook #’inf-clojure-minor-mode)
(setq inf-clojure-custom-startup ‘(“localhost” . 5555)) (setq inf-clojure-custom-repl-type ‘clojure)
Add a builtin function cons that takes a value and a Q-Expression and appends it to the front. Add a builtin function len that returns the number of elements in a Q-Expression. Add a builtin function init that returns all of a Q-Expression except the final element.
Create builtin logical operators or ||, and && and not ! and add them to the language. Define a recursive Lisp function that returns the nth item of that list. Define a recursive Lisp function that returns 1 if an element is a member of a list, otherwise 0. Define a Lisp function that returns the last element of a list. Define in Lisp logical operator functions such as or, and and not.
Adapt the builtin function join to work on strings. Adapt the builtin function head to work on strings. Adapt the builtin function tail to work on strings. Create a builtin function show that can print the contents of strings as it is (unescaped). Add functions to wrap all of C’s file handling functions such as fopen and fgets.
load wval_ptr + fn_table_index indirect call br_table args_count (indirect call)
load wval_ptr + call_table_index limit args_count add call_table args_count load fn_table_index from call_table indirect call