diff --git a/NEWS.rst b/NEWS.rst index 2604f8ebc..ac9068e1f 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -3,6 +3,18 @@ Unreleased ============================= +Removals +------------------------------ +* `(defn/a …)` is now `(defn :async …)`. +* `(fn/a …)` is now `(fn :async …)`. +* `(with/a […] …)` is now `(with [:async …] …)`. + + * As with `for`, `:async` must precede each name to be bound + asynchronously, because you can mix synchronous and asynchronous + types. + +* `(yield-from …)` is now `(yield :from …)`. + New Features ------------------------------ * You can now set `repl-ps1` and `repl-ps2` in your `HYSTARTUP` to customize diff --git a/docs/api.rst b/docs/api.rst index 29fa9c5bd..33239611a 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -91,13 +91,7 @@ base names, such that ``hy.core.macros.foo`` can be called as just ``foo``. allowed), and the newly created function object is returned. Decorators and type parameters aren't allowed, either. However, the function body is understood identically to that of :hy:func:`defn`, without any of the - restrictions of Python's :py:keyword:`lambda`. See :hy:func:`fn/a` for the - asynchronous equivalent. - -.. hy:macro:: (fn/a [name #* args]) - - As :hy:func:`fn`, but the created function object will be a :ref:`coroutine - `. + restrictions of Python's :py:keyword:`lambda`. ``:async`` is also allowed. .. hy:macro:: (defn [name #* args]) @@ -117,16 +111,16 @@ base names, such that ``hy.core.macros.foo`` can be called as just ``foo``. 5))``. There is one exception: due to Python limitations, no implicit return is added if the function is an asynchronous generator (i.e., defined with :hy:func:`defn/a` or :hy:func:`fn/a` and containing at least one - :hy:func:`yield` or :hy:func:`yield-from`). - - ``defn`` accepts a few more optional arguments: a bracketed list of - :term:`decorators `, a list of type parameters (see below), - and an annotation (see :hy:func:`annotate`) for the return value. These are - placed before the function name (in that order, if several are present):: + :hy:func:`yield`). - (defn [decorator1 decorator2] :tp [T1 T2] #^ annotation name [params] …) + ``defn`` accepts a few more optional arguments: a literal keyword ``:async`` + (to create a :ref:`coroutine ` like Python's ``async def``), a + bracketed list of :term:`decorators `, a list of type + parameters (see below), and an annotation (see :hy:func:`annotate`) for the + return value. These are placed before the function name (in that order, if + several are present):: - To define asynchronous functions, see :hy:func:`defn/a` and :hy:func:`fn/a`. + (defn :async [decorator1 decorator2] :tp [T1 T2] #^ annotation name [params] …) ``defn`` lambda lists support all the same features as Python parameter lists and hence are complex in their full generality. The simplest case is a @@ -174,11 +168,6 @@ base names, such that ``hy.core.macros.foo`` can be called as just ``foo``. an unpacked symbol (such as ``#* T`` or ``#** T``). As in Python, unpacking and annotation can't be used with the same parameter. -.. hy:macro:: (defn/a [name lambda-list #* body]) - - As :hy:func:`defn`, but defines a :ref:`coroutine ` like - Python's ``async def``. - .. hy:macro:: (defmacro [name lambda-list #* body]) ``defmacro`` is used to define macros. The general format is @@ -1206,10 +1195,10 @@ base names, such that ``hy.core.macros.foo`` can be called as just ``foo``. .. hy:macro:: (with [managers #* body]) - ``with`` compiles to a :py:keyword:`with` statement, which wraps some code - with one or more :ref:`context managers `. The first - argument is a bracketed list of context managers, and the remaining - arguments are body forms. + ``with`` compiles to a :py:keyword:`with` or an :py:keyword:`async with` + statement, which wraps some code with one or more :ref:`context managers + `. The first argument is a bracketed list of context + managers, and the remaining arguments are body forms. The manager list can't be empty. If it has only one item, that item is evaluated to obtain the context manager to use. If it has two, the first @@ -1236,22 +1225,23 @@ base names, such that ``hy.core.macros.foo`` can be called as just ``foo``. Python's ``with e1 as _: …``), ``with`` will leave it anonymous (as Python's ``with e1: …``). + Finally, any variable-manager pair may be preceded with the keyword + ``:async`` to use an asynchronous context manager:: + + (with [:async v1 e1] …) + ``with`` returns the value of its last form, unless it suppresses an exception (because the context manager's ``__exit__`` method returned true), - in which case it returns ``None``. So, the previous example could also be + in which case it returns ``None``. So, the first example could also be written :: (print (with [o (open "file.txt" "rt")] (.read o))) -.. hy:macro:: (with/a [managers #* body]) - - As :hy:func:`with`, but compiles to an :py:keyword:`async with` statement. - -.. hy:macro:: (yield [value]) +.. hy:macro:: (yield [arg1 arg2]) ``yield`` compiles to a :ref:`yield expression `, which - returns a value as a generator. As in Python, one argument, the value to - yield, is accepted, and it defaults to ``None``. :: + returns a value as a generator. For a plain yield, provide one argument, + the value to yield, or omit it to yield ``None``. :: (defn naysayer [] (while True @@ -1259,18 +1249,13 @@ base names, such that ``hy.core.macros.foo`` can be called as just ``foo``. (hy.repr (list (zip "abc" (naysayer)))) ; => [#("a" "nope") #("b" "nope") #("c" "nope")] - For ``yield from``, see :hy:func:`yield-from`. - -.. hy:macro:: (yield-from [object]) - - ``yield-from`` compiles to a :ref:`yield-from expression `, - which returns a value from a subgenerator. The syntax is the same as that of - :hy:func:`yield`. :: + For a yield-from expression, provide two arguments, where the first is the + literal keyword ``:from`` and the second is the subgenerator. :: (defn myrange [] (setv r (range 10)) (while True - (yield-from r))) + (yield :from r))) (hy.repr (list (zip "abc" (myrange)))) ; => [#("a" 0) #("b" 1) #("c" 2)] diff --git a/hy/core/result_macros.py b/hy/core/result_macros.py index 87f0bead3..c4676f796 100644 --- a/hy/core/result_macros.py +++ b/hy/core/result_macros.py @@ -1068,21 +1068,15 @@ def compile_break_or_continue_expression(compiler, expr, root): # ------------------------------------------------ -@pattern_macro( - ["with", "with/a"], - [ - brackets(oneplus(FORM + FORM)) - | brackets(FORM >> (lambda x: [(Symbol("_"), x)])), - many(FORM), - ], -) +@pattern_macro("with", [ + brackets(oneplus(maybe(keepsym(":async")) + FORM + FORM)) | + brackets((maybe(keepsym(":async")) + FORM) >> (lambda x: [(x[0], Symbol("_"), x[1])])), + many(FORM)]) def compile_with_expression(compiler, expr, root, args, body): - body = compiler._compile_branch(body) - # Store the result of the body in a tempvar + # We'll store the result of the body in a tempvar temp_var = compiler.get_anon_var() name = asty.Name(expr, id=mangle(temp_var), ctx=ast.Store()) - body += asty.Assign(expr, targets=[name], value=body.force_expr) # Initialize the tempvar to None in case the `with` exits # early with an exception. initial_assign = asty.Assign( @@ -1091,7 +1085,18 @@ def compile_with_expression(compiler, expr, root, args, body): ret = Result(stmts=[initial_assign]) items = [] - for variable, ctx in args[0]: + was_async = None + cbody = None + for i, (is_async, variable, ctx) in enumerate(args[0]): + is_async = bool(is_async) + if was_async is None: + was_async = is_async + elif is_async != was_async: + # We're compiling a `with` that mixes synchronous and + # asynchronous context managers. Python doesn't support + # this directly, so start a new `with` inside the body. + cbody = compile_with_expression(compiler, expr, root, [args[0][i:]], body) + break ctx = compiler.compile(ctx) ret += ctx variable = ( @@ -1103,8 +1108,12 @@ def compile_with_expression(compiler, expr, root, args, body): asty.withitem(expr, context_expr=ctx.force_expr, optional_vars=variable) ) - node = asty.With if root == "with" else asty.AsyncWith - ret += node(expr, body=body.stmts, items=items) + if not cbody: + cbody = compiler._compile_branch(body) + cbody += asty.Assign(expr, targets=[name], value=cbody.force_expr) + + node = asty.AsyncWith if was_async else asty.With + ret += node(expr, body=cbody.stmts, items=items) # And make our expression context our temp variable expr_name = asty.Name(expr, id=mangle(temp_var), ctx=ast.Load()) @@ -1454,10 +1463,12 @@ def compile_try_expression(compiler, expr, root, body, catchers, orelse, finalbo ) -@pattern_macro(["fn", "fn/a"], - [maybe(type_params), maybe_annotated(lambda_list), many(FORM)]) -def compile_function_lambda(compiler, expr, root, tp, params, body): - is_async = root == "fn/a" +@pattern_macro("fn", [ + maybe(keepsym(":async")), + maybe(type_params), + maybe_annotated(lambda_list), + many(FORM)]) +def compile_function_lambda(compiler, expr, root, is_async, tp, params, body): params, returns = params posonly, args, rest, kwonly, kwargs = params has_annotations = returns is not None or any( @@ -1483,12 +1494,14 @@ def compile_function_lambda(compiler, expr, root, tp, params, body): return ret + Result(expr=ret.temp_variables[0]) -@pattern_macro( - ["defn", "defn/a"], - [maybe(brackets(many(FORM))), maybe(type_params), maybe_annotated(SYM), lambda_list, many(FORM)], -) -def compile_function_def(compiler, expr, root, decorators, tp, name, params, body): - is_async = root == "defn/a" +@pattern_macro("defn", [ + maybe(keepsym(":async")), + maybe(brackets(many(FORM))), + maybe(type_params), + maybe_annotated(SYM), + lambda_list, + many(FORM)]) +def compile_function_def(compiler, expr, root, is_async, decorators, tp, name, params, body): name, returns = name node = asty.AsyncFunctionDef if is_async else asty.FunctionDef decorators, ret, _ = compiler._compile_collect(decorators[0] if decorators else []) @@ -1672,23 +1685,27 @@ def compile_return(compiler, expr, root, arg): return ret + asty.Return(expr, value=ret.force_expr) -@pattern_macro("yield", [maybe(FORM)]) -def compile_yield_expression(compiler, expr, root, arg): +@pattern_macro("yield", [times(0, 2, FORM)]) +def compile_yield_expression(compiler, expr, root, args): if is_inside_function_scope(compiler.scope): nearest_python_scope(compiler.scope).has_yield = True + yield_from = False + if len(args) == 2: + from_kw, x = args + if from_kw != Keyword("from"): + raise compiler._syntax_error(from_kw, "two-argument `yield` requires `:from`") + yield_from = True + args = [x] ret = Result() - if arg is not None: - ret += compiler.compile(arg) - return ret + asty.Yield(expr, value=ret.force_expr) + if args: + ret += compiler.compile(args[0]) + return ret + (asty.YieldFrom if yield_from else asty.Yield)(expr, value=ret.force_expr) -@pattern_macro(["yield-from", "await"], [FORM]) +@pattern_macro("await", [FORM]) def compile_yield_from_or_await_expression(compiler, expr, root, arg): - if root == "yield-from" and is_inside_function_scope(compiler.scope): - nearest_python_scope(compiler.scope).has_yield = True ret = Result() + compiler.compile(arg) - node = asty.YieldFrom if root == "yield-from" else asty.Await - return ret + node(expr, value=ret.force_expr) + return ret + asty.Await(expr, value=ret.force_expr) # ------------------------------------------------ diff --git a/tests/compilers/test_ast.py b/tests/compilers/test_ast.py index 001c2e55f..4ca70a307 100644 --- a/tests/compilers/test_ast.py +++ b/tests/compilers/test_ast.py @@ -550,11 +550,12 @@ def test_compiler_macro_tag_try(): def test_ast_good_yield_from(): - can_compile("(yield-from [1 2])") + can_compile("(yield :from [1 2])") + can_compile("(yield :from)") def test_ast_bad_yield_from(): - cant_compile("(yield-from)") + cant_compile("(yield :ploopy [1 2])") def test_eval_generator_with_return(): diff --git a/tests/native_tests/comprehensions.hy b/tests/native_tests/comprehensions.hy index bc354a4cb..3ee4cb8cd 100644 --- a/tests/native_tests/comprehensions.hy +++ b/tests/native_tests/comprehensions.hy @@ -387,12 +387,12 @@ (defn [async-test] test-for-async [] - (defn/a numbers [] + (defn :async numbers [] (for [i [1 2]] (yield i))) (asyncio.run - ((fn/a [] + ((fn :async [] (setv x 0) (for [:async a (numbers)] (setv x (+ x a))) @@ -400,12 +400,12 @@ (defn [async-test] test-for-async-else [] - (defn/a numbers [] + (defn :async numbers [] (for [i [1 2]] (yield i))) (asyncio.run - ((fn/a [] + ((fn :async [] (setv x 0) (for [:async a (numbers)] (setv x (+ x a)) diff --git a/tests/native_tests/decorators.hy b/tests/native_tests/decorators.hy index 3fedea468..0603cec8d 100644 --- a/tests/native_tests/decorators.hy +++ b/tests/native_tests/decorators.hy @@ -53,10 +53,10 @@ (assert (= l ["dec" "arg" "foo" "foo fn" "bar body" 1]))) -(defn [async-test] test-decorated-defn/a [] - (defn decorator [func] (fn/a [] (/ (await (func)) 2))) +(defn [async-test] test-decorated-defn-a [] + (defn decorator [func] (fn :async [] (/ (await (func)) 2))) - (defn/a [decorator] coro-test [] + (defn :async [decorator] coro-test [] (await (asyncio.sleep 0)) 42) (assert (= (asyncio.run (coro-test)) 21))) diff --git a/tests/native_tests/functions.hy b/tests/native_tests/functions.hy index fd1d31b61..6c5fd3951 100644 --- a/tests/native_tests/functions.hy +++ b/tests/native_tests/functions.hy @@ -25,8 +25,8 @@ (assert (= (fn-test) None))) -(defn [async-test] test-fn/a [] - (assert (= (asyncio.run ((fn/a [] (await (asyncio.sleep 0)) [1 2 3]))) +(defn [async-test] test-fn-async [] + (assert (= (asyncio.run ((fn :async [] (await (asyncio.sleep 0)) [1 2 3]))) [1 2 3]))) @@ -182,8 +182,8 @@ (setv x [#* spam] y 1))) -(defn [async-test] test-defn/a [] - (defn/a coro-test [] +(defn [async-test] test-defn-async [] + (defn :async coro-test [] (await (asyncio.sleep 0)) [1 2 3]) (assert (= (asyncio.run (coro-test)) [1 2 3]))) @@ -191,15 +191,15 @@ (defn [async-test] test-no-async-gen-return [] ; https://github.com/hylang/hy/issues/2523 - (defn/a runner [gen] + (defn :async runner [gen] (setv vals []) (for [:async val (gen)] (.append vals val)) vals) - (defn/a naysayer [] + (defn :async naysayer [] (yield "nope")) (assert (= (asyncio.run (runner naysayer)) ["nope"])) - (assert (= (asyncio.run (runner (fn/a [] (yield "dope!")) ["dope!"]))))) + (assert (= (asyncio.run (runner (fn :async [] (yield "dope!")) ["dope!"]))))) (defn test-root-set-correctly [] @@ -271,30 +271,6 @@ (assert (= accum [2]))) -(defn test-yield-from [] - (defn yield-from-test [] - (for [i (range 3)] - (yield i)) - (yield-from [1 2 3])) - (assert (= (list (yield-from-test)) [0 1 2 1 2 3]))) - - -(defn test-yield-from-exception-handling [] - (defn yield-from-subgenerator-test [] - (yield 1) - (yield 2) - (yield 3) - (/ 1 0)) - (defn yield-from-test [] - (for [i (range 3)] - (yield i)) - (try - (yield-from (yield-from-subgenerator-test)) - (except [e ZeroDivisionError] - (yield 4)))) - (assert (= (list (yield-from-test)) [0 1 2 1 2 3 4]))) - - (defn test-yield [] (defn gen [] (for [x [1 2 3 4]] (yield x))) (setv ret 0) @@ -355,3 +331,34 @@ (yield "a") (yield "end")) (assert (= (list (multi-yield)) [0 1 2 "a" "end"]))) + + +(defn test-yield-from [] + (defn yield-from-test [] + (for [i (range 3)] + (yield i)) + (yield :from [1 2 3])) + (assert (= (list (yield-from-test)) [0 1 2 1 2 3]))) + + +(defn test-yield-from-exception-handling [] + (defn yield-from-subgenerator-test [] + (yield 1) + (yield 2) + (yield 3) + (/ 1 0)) + (defn yield-from-test [] + (for [i (range 3)] + (yield i)) + (try + (yield :from (yield-from-subgenerator-test)) + (except [e ZeroDivisionError] + (yield 4)))) + (assert (= (list (yield-from-test)) [0 1 2 1 2 3 4]))) + + +(defn test-yield-from-notreally [] + (defn f [] + (yield :from) + (yield :from)) + (assert (= (list (f)) [:from :from]))) diff --git a/tests/native_tests/with.hy b/tests/native_tests/with.hy index fb2b364fb..48f481715 100644 --- a/tests/native_tests/with.hy +++ b/tests/native_tests/with.hy @@ -25,65 +25,44 @@ (defn test-single-with [] (with [t (WithTest 1)] - (assert (= t 1)))) - -(defn test-twice-with [] - (with [t1 (WithTest 1) - t2 (WithTest 2)] - (assert (= t1 1)) - (assert (= t2 2)))) - -(defn test-thrice-with [] - (with [t1 (WithTest 1) - t2 (WithTest 2) - t3 (WithTest 3)] - (assert (= t1 1)) - (assert (= t2 2)) - (assert (= t3 3)))) + (setv out t)) + (assert (= out 1))) (defn test-quince-with [] - (with [t1 (WithTest 1) - t2 (WithTest 2) - t3 (WithTest 3) - _ (WithTest 4)] - (assert (= t1 1)) - (assert (= t2 2)) - (assert (= t3 3)))) - -(defn [async-test] test-single-with/a [] - (asyncio.run - ((fn/a [] - (with/a [t (AsyncWithTest 1)] - (assert (= t 1))))))) + (with [t1 (WithTest 1) t2 (WithTest 2) t3 (WithTest 3) _ (WithTest 4)] + (setv out [t1 t2 t3])) + (assert (= out [1 2 3]))) -(defn [async-test] test-two-with/a [] +(defn [async-test] test-single-with-async [] + (setv out []) (asyncio.run - ((fn/a [] - (with/a [t1 (AsyncWithTest 1) - t2 (AsyncWithTest 2)] - (assert (= t1 1)) - (assert (= t2 2))))))) + ((fn :async [] + (with [:async t (AsyncWithTest 1)] + (.append out t))))) + (assert (= out [1]))) -(defn [async-test] test-thrice-with/a [] +(defn [async-test] test-quince-with-async [] + (setv out []) (asyncio.run - ((fn/a [] - (with/a [t1 (AsyncWithTest 1) - t2 (AsyncWithTest 2) - t3 (AsyncWithTest 3)] - (assert (= t1 1)) - (assert (= t2 2)) - (assert (= t3 3))))))) - -(defn [async-test] test-quince-with/a [] + ((fn :async [] + (with [ + :async t1 (AsyncWithTest 1) + :async t2 (AsyncWithTest 2) + :async t3 (AsyncWithTest 3) + :async _ (AsyncWithTest 4)] + (.extend out [t1 t2 t3]))))) + (assert (= out [1 2 3]))) + +(defn [async-test] test-with-mixed-async [] + (setv out []) (asyncio.run - ((fn/a [] - (with/a [t1 (AsyncWithTest 1) - t2 (AsyncWithTest 2) - t3 (AsyncWithTest 3) - _ (AsyncWithTest 4)] - (assert (= t1 1)) - (assert (= t2 2)) - (assert (= t3 3))))))) + ((fn :async [] + (with [:async t1 (AsyncWithTest 1) + t2 (WithTest 2) + :async t3 (AsyncWithTest 3) + _ (WithTest 4)] + (.extend out [t1 t2 t3]))))) + (assert (= out [1 2 3]))) (defn test-unnamed-context-with [] "`_` get compiled to unnamed context" diff --git a/tests/resources/pydemo.hy b/tests/resources/pydemo.hy index 80c4c0901..8c7ef7ae9 100644 --- a/tests/resources/pydemo.hy +++ b/tests/resources/pydemo.hy @@ -173,12 +173,12 @@ Call me Ishmael. Some years ago—never mind how long precisely—having little pys_accum.append(i)") (setv py-accum (py "''.join(map(str, pys_accum))")) -(defn/a coro [] +(defn :async coro [] (import asyncio tests.resources [AsyncWithTest async-loop]) (await (asyncio.sleep 0)) (setv values ["a"]) - (with/a [t (AsyncWithTest "b")] + (with [:async t (AsyncWithTest "b")] (.append values t)) (for [:async item (async-loop ["c" "d"])] (.append values item))