diff --git a/docs/conf.py b/docs/conf.py index 5425ecdf3..bd5d0410d 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -24,7 +24,7 @@ release = hy_version # The full version identifier, including alpha, beta, and RC tags -hyrule_version = 'v0.5.0' +hyrule_version = 'doc-testing' source_suffix = '.rst' master_doc = 'index' diff --git a/docs/interop.rst b/docs/interop.rst index 028311206..d878640dd 100644 --- a/docs/interop.rst +++ b/docs/interop.rst @@ -4,6 +4,9 @@ Python interoperability ======================= +This chapter describes how to interact with Python code from Hy code and vice +versa. + .. contents:: Contents :local: @@ -16,11 +19,28 @@ Hy and Python. For example, Python's ``str.format_map`` can be written ``hyx_valid_Xquestion_markX`` in Python. You can call :hy:func:`hy.mangle` and :hy:func:`hy.unmangle` from either language. +Keyword mincing +--------------- + +Another kind of mangling may be necessary in Python to refer to variables with +the same name as reserved words. For example, while ``(setv break 13)`` is +legal Hy, ``import hy, my_hy_module; print(my_hy_module.break)`` is +syntactically invalid Python. String literals work, as in +``getattr(my_hy_module, "break")``, but to use what is syntactically a Python +identifier, you'll have to take advantage of Python's Unicode normalization +(via NFKC) and write something like ``my_hy_module.๐›reak``. Here are all the +MATHEMATICAL BOLD SMALL letters (U+1D41A through U+1D433) for convenient +copying: + +.. code-block:: text + + ๐š๐›๐œ๐๐ž๐Ÿ๐ ๐ก๐ข๐ฃ๐ค๐ฅ๐ฆ๐ง๐จ๐ฉ๐ช๐ซ๐ฌ๐ญ๐ฎ๐ฏ๐ฐ๐ฑ๐ฒ๐ณ + Libraries that expect Python ============================ There are various means by which Hy may interact poorly with a Python library because the library doesn't account for the possibility of Hy. For example, -when you run :ref:`hy-cli`, ``sys.executable`` will be set to +when you run the program :ref:`hy-cli`, ``sys.executable`` will be set to this program rather than the original Python binary. This is helpful more often than not, but will lead to trouble if e.g. the library tries to call :py:data:`sys.executable` with the ``-c`` option. In this case, you can try @@ -44,9 +64,15 @@ declaring your package's dependence on Hy there, likely in the below for some related issues to keep in mind. If you want to compile your Hy code into Python bytecode at installation-time -(in case e.g. the code is being installed to a directory where the bytecode be -able to be automatically written later, due to permissions issues), see Hy's -own ``setup.py`` for an example. +(in case e.g. the code is being installed to a directory where the bytecode +can't be automatically written later, due to permissions issues), see Hy's own +``setup.py`` for an example. + +For PyPI packages, use the trove classifier ``Programming Language :: Hy`` for +libraries meant to be useful for Hy specifically (e.g., a library that provides +Hy macros) or other projects that are about Hy somehow (e.g., an instructive +example Hy program). Don't use it for a package that just happens to be written +in Hy. Using Python from Hy ==================== @@ -67,10 +93,10 @@ Using Hy from Python To use a Hy module from Python, you can just :py:keyword:`import` it, provided that ``hy`` has already been imported first, whether in the current module or in some earlier module executed by the current Python process. The ``hy`` -import is necessary to create the hooks that allow importing Hy modules. Note -that you can always have a wrapper Python file (such as a package's -``__init__.py``) do the ``import hy`` for the user; this is a smart thing to do -for a published package. +import is necessary to create the hooks that allow importing Hy modules. you +can have a wrapper Python file (such as a package's ``__init__.py``) do the +``import hy`` for the user; this is a smart thing to do for a published +package. No way to import macros or reader macros into a Python module is implemented, since there's no way to call them in Python anyway. diff --git a/docs/macros.rst b/docs/macros.rst index 232d0d251..4a861ad69 100644 --- a/docs/macros.rst +++ b/docs/macros.rst @@ -48,7 +48,7 @@ A regular macro can be defined with :hy:func:`defmacro` using a syntax similar t (print (seventeen)) -To see that ``seventeen`` is expanded at compile-time, run ``hy2py`` on this script and notice that it ends with ``print(17)`` rather than ``print(seventeen())``. If you insert a ``print`` call inside the macro definition, you'll also see that the print happens when the file is compiled, but not when it's rerun (provided an up-to-date bytecode file exists). +To see that ``seventeen`` is expanded at compile-time, run ``hy2py`` on this script and notice that it ends with ``print(17)`` rather than ``print(seventeen())``. If you insert a ``print`` call inside the macro definition, you'll also see that the print happens when the file is compiled, but not when it's rerun (so long as an up-to-date bytecode file exists). A more useful macro returns code. You can construct a model the long way, like this:: @@ -72,7 +72,7 @@ Arguments are always passed in as models. You can use quasiquotation (see :hy:fu (set-to-2 foobar) (print foobar) -Macros don't understand keyword arguments like functions do. Rather, the :ref:`keyword objects ` themselves are passed in literally. This gives you flexibility in how to handle them. Thus, ``#** kwargs`` and ``*`` aren't allowed in the parameter list of a macro, although ``#* args`` and ``/`` are. +Macros don't understand keyword arguments like functions do. Rather, the :ref:`keyword objects ` themselves are passed in literally. This gives you flexibility in how to handle them. Thus, ``#** kwargs`` and ``*`` aren't allowed in the parameter list of a macro, although ``#* args`` and ``/`` are. See :hy:func:`hyrule.defmacro-kwargs` if you want to handle keyword arguments like a function. On the inside, macros are functions, and obey the usual Python semantics for functions. For example, :hy:func:`setv` inside a macro will define or modify a variable local to the current macro call, and :hy:func:`return` ends macro execution and uses its argument as the expansion. @@ -134,7 +134,7 @@ The form ``(import math)`` here appears in the wrong context, in the macro call (defmacro hypotenuse [a b] `(hy.I.math.sqrt (+ (** ~a 2) (** ~b 2)))) - (hypotenuse 3 4) + (print (hypotenuse 3 4)) A related but distinct issue is when you want to use a function (or other ordinary Python object) in a macro's code, but it isn't available soon enough:: @@ -195,6 +195,8 @@ Note that because reader macros are evaluated at parse-time, and top-level forms (print #up "hello?")) ; LexException: reader macro '#up' is not defined +Of the potential problems discussed above that apply to regular macros, such as surprise shadowing, most also apply to reader macros. + .. _macro-namespaces: Macro namespaces and operations on macros diff --git a/docs/semantics.rst b/docs/semantics.rst index 90df63d87..7041e75b4 100644 --- a/docs/semantics.rst +++ b/docs/semantics.rst @@ -39,6 +39,8 @@ variables are, and they aren't explicitly cleaned up, so theoretically, they can waste memory and lead to :py:meth:`object.__del__` being called later than you expect. When in doubt, check the ``hy2py`` output. +.. _order-of-eval: + Order of evaluation ------------------- diff --git a/docs/syntax.rst b/docs/syntax.rst index 506063e3c..d51a925e7 100644 --- a/docs/syntax.rst +++ b/docs/syntax.rst @@ -48,7 +48,7 @@ or :py:class:`int`, or you can evaluate a model as Hy code with Models can be created with the constructors, with the :hy:func:`quote` or :hy:func:`quasiquote` macros, or with :hy:func:`hy.as-model`. Explicit creation -is often not necessary, because the compiler will autopromote (via +is often not necessary, because the compiler will automatically promote (via :hy:func:`hy.as-model`) any object it's trying to evaluate. Note that when you want plain old data structures and don't intend to produce @@ -118,12 +118,12 @@ Discard prefix Like Clojure, Hy supports the Extensible Data Notation discard prefix ``#_``, which is kind of like a structure-aware comment. When the reader encounters -``#_``, it reads and then discards the following form. Thus ``#_`` is similar -to ``;`` except that reader macros still get executed, and normal parsing -resumes after the next form ends rather than at the start of the next line: -``[dilly #_ and krunk]`` is equivalent to ``[dilly krunk]``, whereas ``[dilly ; -and krunk]`` is equivalent to just ``[dilly``. Comments indicated by ``;`` can -be nested within forms discarded by ``#_``, but ``#_`` has no special meaning +``#_``, it reads and then discards the following form. Thus ``#_`` is like +``;`` except that reader macros still get executed, and normal parsing resumes +after the next form ends rather than at the start of the next line: ``[dilly #_ +and krunk]`` is equivalent to ``[dilly krunk]``, whereas ``[dilly ; and +krunk]`` is equivalent to just ``[dilly``. Comments indicated by ``;`` can be +nested within forms discarded by ``#_``, but ``#_`` has no special meaning within a comment indicated by ``;``. Identifiers @@ -131,11 +131,13 @@ Identifiers Identifiers are a broad class of syntax in Hy, comprising not only variable names, but any nonempty sequence of characters that aren't ASCII whitespace nor -one of the following: ``()[]{};"'`~``. The reader will attempt to read each -identifier as a :ref:`numeric literal `, then attempt to read -it as a :ref:`keyword ` if that fails, then attempt to read it as a -:ref:`dotted identifier ` if that fails, then fall back on -reading it as a :ref:`symbol ` if that fails. +one of the following: ``()[]{};"'`~``. The reader will attempt to read an +identifier as each of the following types, in the given order: + +1. a :ref:`numeric literal ` +2. a :ref:`keyword ` +3. a :ref:`dotted identifier ` +4. a :ref:`symbol ` .. _numeric-literals: @@ -197,9 +199,9 @@ equivalent to ``{"a" 1 "b" 2}``, which is different from ``{:a 1 :b 2}`` (see :ref:`dict-literals`). The empty keyword ``:`` is syntactically legal, but you can't compile a -function call with an empty keyword argument due to Python limitations. Thus -``(foo : 3)`` must be rewritten to use runtime unpacking, as in -``(foo #** {"" 3})``. +function call with an empty keyword argument because of Python limitations. +Thus ``(foo : 3)`` must be rewritten to use runtime unpacking, as in ``(foo #** +{"" 3})``. .. autoclass:: hy.models.Keyword :members: __bool__, __call__ @@ -247,7 +249,7 @@ Lisps). Some example symbols are ``hello``, ``+++``, ``3fiddy``, ``$40``, Dots are only allowed in a symbol if every character in the symbol is a dot. Thus, ``a..b`` and ``a.`` are neither dotted identifiers nor symbols; they're -simply illegal syntax. +syntax errors. As a special case, the symbol ``...`` compiles to the :data:`Ellipsis` object, as in Python. @@ -389,10 +391,10 @@ Expressions Expressions (:class:`Expression `) are denoted by parentheses: ``( โ€ฆ )``. The compiler evaluates expressions by checking the -first element. +first element, called the head. -- If it's a symbol, and the symbol is the name of a currently defined macro, - the macro is called. +- If the head is a symbol, and the symbol is the name of a currently defined + macro, the macro is called. - Exception: if the symbol is also the name of a function in :hy:mod:`hy.pyops`, and one of the arguments is an @@ -402,18 +404,18 @@ first element. ``(hy.pyops.+ #* summands)``, because Python provides no way to sum a list of unknown length with a real addition expression. -- If it is itself an expression of the form ``(. None โ€ฆ)`` (typically produced - with a :ref:`dotted identifier ` like ``.add``), it's used - to construct a method call with the element after ``None`` as the object: - thus, ``(.add my-set 5)`` is equivalent to ``((. my-set add) 5)``, which - becomes ``my_set.add(5)`` in Python. +- If the head is itself an expression of the form ``(. None โ€ฆ)`` (typically + produced with a :ref:`dotted identifier ` like ``.add``), + it's used to construct a method call with the element after ``None`` as the + object: thus, ``(.add my-set 5)`` is equivalent to ``((. my-set add) 5)``, + which becomes ``my_set.add(5)`` in Python. .. _hy.R: - Exception: expressions like ``((. hy R module-name macro-name) โ€ฆ)``, or equivalently ``(hy.R.module-name.macro-name โ€ฆ)``, get special treatment. They :hy:func:`require` the module ``module-name`` and call its macro ``macro-name``, so ``(hy.R.foo.bar 1)`` is equivalent to ``(require foo) (foo.bar 1)``, but without bringing ``foo`` or ``foo.bar`` into scope. Thus ``hy.R`` is convenient syntactic sugar for macros you'll only call once in a file, or for macros that you want to appear in the expansion of other macros without having to call :hy:func:`require` in the expansion. As with :hy:class:`hy.I`, dots in the module name must be replaced with slashes. - Otherwise, the expression is compiled into a Python-level call, with the - first element being the calling object. (So, you can call a function that has + head being the calling object. (So, you can call a function that has the same name as a macro with an expression like ``((do setv) โ€ฆ)``.) The remaining forms are understood as arguments. Use :hy:func:`unpack-iterable` or :hy:func:`unpack-mapping` to break up data structures into individual @@ -429,10 +431,12 @@ meaning. Trying to compile it is an error. For the empty tuple, use ``#()``. List, tuple, and set literals ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Literal :class:`list`\s (:class:`List `), :class:`tuple`\s -(:class:`Tuple `), and :class:`set`\s (:class:`Set -`) are denoted respectively by ``[ โ€ฆ ]``, ``#( โ€ฆ )``, and ``#{ โ€ฆ -}``. +- Literal :class:`list`\s (:class:`List `) are denoted by ``[ โ€ฆ + ]``. +- Literal :class:`tuple`\s (:class:`Tuple `) are denoted by + ``#( โ€ฆ )``. +- Literal :class:`set`\s (:class:`Set `) are denoted by ``#{ โ€ฆ + }``. .. autoclass:: hy.models.List .. autoclass:: hy.models.Tuple @@ -494,9 +498,9 @@ Additional sugar ---------------- Syntactic sugar is available to construct two-item :ref:`expressions -` with certain macros. When the sugary characters are encountered -by the reader, a new expression is created with the corresponding macro as the -first element and the next parsed form as the second. No parentheses are +` with certain heads. When the sugary characters are encountered +by the reader, a new expression is created with the corresponding macro name as +the first element and the next parsed form as the second. No parentheses are required. Thus, since ``'`` is short for ``quote``, ``'FORM`` is read as ``(quote FORM)``. Whitespace is allowed, as in ``' FORM``. This is all resolved at the reader level, so the model that gets produced is the same whether you diff --git a/docs/tutorial.rst b/docs/tutorial.rst index 8f936f64d..9b5f5780d 100644 --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -44,11 +44,12 @@ Here's a more complex example:: (- (* (+ 1 3 88) 2) 8) -This code returns ``176``. Why? We can see the infix equivalent with the +This code returns ``176``. Why? You can see the infix equivalent with the command ``echo "(- (* (+ 1 3 88) 2) 8)" | hy2py``, which returns the Python -code corresponding to the given Hy code, or by passing the ``--spy`` option to -Hy when starting the REPL, which shows the Python equivalent of each input line -before the result. The infix equivalent in this case is: +code corresponding to the given Hy code. Or you can pass the ``--spy`` option to +Hy when starting the interactive read-eval-print loop (REPL), which shows the +Python equivalent of each input line before the result. The infix equivalent in +this case is: .. code-block:: python @@ -119,7 +120,7 @@ But if you start Hy like this:: $ hy --repl-output-fn=repr -the REPL will use Python's native ``repr`` function instead, so you'll see values in Python syntax:: +the REPL will use Python's native :py:func:`repr` function instead, so you'll see values in Python syntax:: => [1 2 3] [1, 2, 3] @@ -228,12 +229,12 @@ Set a function parameter by name with a ``:keyword``:: (test 1 2 :d "y") ; => [1, 2, None, 'y', ()] -Note that unlike Python, Hy doesn't always evaluate function arguments (or -the items in a literal list, or the items in a literal dictionary, etc.) in -the order they appear in the code. But you can always force a particular -evaluation order with :hy:func:`do`, or with other macros that provide an -implicit :hy:func:`do`, like :hy:func:`when ` or -:hy:func:`fn`. +Note that unlike Python, Hy doesn't always evaluate function arguments (or the +items in a literal list, or the items in a literal dictionary, etc.) :ref:`in +the order they appear in the code `. But you can always force a +particular evaluation order with :hy:func:`do`, or with other macros that +provide an implicit :hy:func:`do`, like :hy:func:`when ` +or :hy:func:`fn`. Define classes with :hy:func:`defclass`:: @@ -255,7 +256,7 @@ various means:: Note that syntax like ``fb.x`` and ``fb.get-x`` only works when the object being invoked (``fb``, in this case) is a simple variable name. To get an attribute or call a method of an arbitrary form ``FORM``, you must use the -syntax ``(. FORM x)`` or ``(.get-x FORM)``. +syntax ``(. FORM x)`` or ``(.get-x FORM)``, or call :py:func:`getattr`. Access an external module, whether written in Python or Hy, with :hy:func:`import`:: @@ -263,6 +264,10 @@ Access an external module, whether written in Python or Hy, with (import math) (print (math.sqrt 2)) ; => 1.4142135623730951 +Or use the one-shot import syntax :hy:class:`hy.I`:: + + (print (hy.I.math.sqrt 2)) + Python can import a Hy module like any other module so long as Hy itself has been imported first, which, of course, must have already happened if you're running a Hy program. @@ -313,7 +318,7 @@ simply:: Our macro ``m`` has an especially simple return value, an integer (:py:class:`int`), which at compile-time is converted to an integer model (:class:`hy.models.Integer`). In general, macros can return -arbitrary Hy forms to be executed as code. There are several helper macros that +arbitrary Hy models to be executed as code. There are several helper macros that make it easy to construct forms programmatically, such as :hy:func:`quote` (``'``), :hy:func:`quasiquote` (`````), :hy:func:`unquote` (``~``), :hy:func:`unquote-splice` (``~@``), and :hy:func:`defmacro! @@ -328,9 +333,8 @@ that is, during the translation from Hy to Python. Instead, use :hy:func:`requir which imports the module and makes macros available at compile-time. ``require`` uses the same syntax as ``import``. :: - => (require tutorial.macros) - => (tutorial.macros.rev (1 2 3 +)) - 6 + (require some-module.macros) + (some-module.macros.rev (1 2 3 +)) ; => 6 Hy also supports reader macros, which are similar to ordinary macros, but operate on raw source text rather than pre-parsed Hy forms. They can choose how @@ -339,17 +343,14 @@ any code. Thus, reader macros can add entirely new syntax to Hy. For example, you could add a literal notation for Python's :class:`decimal.Decimal` class like so:: - => (import decimal [Decimal] fractions [Fraction]) - => (defreader d - ... (.slurp-space &reader) - ... `(Decimal ~(.read-ident &reader))) - => (print (repr #d .1)) - Decimal('0.1') - => (print (Fraction #d .1)) - 1/10 - => ;; Contrast with the normal floating-point .1: - => (print (Fraction .1)) - 3602879701896397/36028797018963968 + (defreader d + (.slurp-space &reader) + `(hy.I.decimal.Decimal ~(.read-ident &reader))) + (print (repr #d .1)) ; => Decimal('0.1') + (import fractions [Fraction]) + (print (Fraction #d .1)) ; => 1/10 + ;; Contrast with the normal floating-point .1: + (print (Fraction .1)) ; => 3602879701896397/36028797018963968 ``require`` can pull in a reader macro defined in a different module with syntax like ``(require mymodule :readers [d])``. @@ -361,34 +362,32 @@ Recommended libraries It provides a variety of functions and macros that are useful for writing Hy programs. :: - => (import hyrule [inc]) - => (list (map inc [1 2 3])) - [2 3 4] - => (require hyrule [case]) - => (setv x 2) - => (case x 1 "a" 2 "b" 3 "c") - "b" + (import hyrule [inc]) + (list (map inc [1 2 3])) ; => [2 3 4] + (require hyrule [case]) + (setv x 2) + (case x 1 "a" 2 "b" 3 "c") ; => "b" `toolz `_ and its Cython variant `cytoolz `_ provide lots of utilities for functional programming and working with iterables. :: - => (import toolz [partition]) - => (list (partition 2 [1 2 3 4 5 6])) - [#(1 2) #(3 4) #(5 6)] + (import toolz [partition]) + (list (partition 2 [1 2 3 4 5 6])) + ; => [#(1 2) #(3 4) #(5 6)] `metadict `_ allows you to refer to the elements of a dictionary as attributes. This is handy when frequently referring to elements with constant strings as keys, since plain indexing is a bit verbose in Hy. :: - => (import metadict [MetaDict]) - => (setv d (MetaDict)) - => (setv d.foo 1) ; i.e., (setv (get d "foo") 1) - => d.foo ; i.e., (get d "foo") - 1 - => (list (.keys d)) - ["foo"] + (import metadict [MetaDict]) + (setv d (MetaDict)) + (setv d.foo 1) ; i.e., (setv (get d "foo") 1) + d.foo ; i.e., (get d "foo") + ; => 1 + (list (.keys d)) + ; => ["foo"] Next steps ========== diff --git a/docs/whyhy.rst b/docs/whyhy.rst index 2c617bd04..0bd404966 100644 --- a/docs/whyhy.rst +++ b/docs/whyhy.rst @@ -10,7 +10,7 @@ language) is a multi-paradigm general-purpose programming language in the `Lisp family `_. It's implemented as a kind of alternative syntax for Python. Compared to -Python, Hy offers a variety of extra features, generalizations, and +Python, Hy offers a variety of new features, generalizations, and syntactic simplifications, as would be expected of a Lisp. Compared to other Lisps, Hy provides direct access to Python's built-ins and third-party Python libraries, while allowing you to freely mix @@ -93,11 +93,11 @@ as ordinary first-class functions of the same name, allowing them to be passed to higher-order functions: ``(sum xs)`` could be written ``(reduce + xs)``, after importing the function ``+`` from the module :hy:mod:`hy.pyops`. -The Hy compiler works by reading Hy source code into Hy :ref:`model objects ` and -compiling the Hy model objects into Python abstract syntax tree (:py:mod:`ast`) +The Hy compiler works by reading Hy source code into Hy :ref:`model objects +` and compiling them into Python abstract syntax tree (:py:mod:`ast`) objects. Python AST objects can then be compiled and run by Python itself, byte-compiled for faster execution later, or rendered into Python source code. -You can even :ref:`mix Python and Hy code in the same project, or even the same +You can :ref:`mix Python and Hy code in the same project, or even the same file,` which can be a good way to get your feet wet in Hy. @@ -107,7 +107,7 @@ Hy versus other Lisps At run-time, Hy is essentially Python code. Thus, while Hy's design owes a lot to `Clojure `_, it is more tightly coupled to Python than Clojure is to Java; a better analogy is `CoffeeScript's -`_ relationship to JavaScript. Python's built-in +`_ relationship with JavaScript. Python's built-in :ref:`functions ` and :ref:`data structures ` are directly available:: diff --git a/hy/_compat.py b/hy/_compat.py index 4b83686fa..25e8872be 100644 --- a/hy/_compat.py +++ b/hy/_compat.py @@ -37,6 +37,8 @@ def rewriting_unparse(ast_obj): and keyword.iskeyword(v) and v not in ("True", "False", "None") ): + # We refer to this transformation as "keyword mincing" + # in documentation. setattr(node, field, chr(ord(v[0]) - ord("a") + ord("๐š")) + v[1:]) return true_unparse(ast_obj)