From fc8e7eb8d4667f2ace7a0747bbc33034da5f6249 Mon Sep 17 00:00:00 2001 From: MilesCranmer Date: Sat, 14 Dec 2024 14:27:21 -0800 Subject: [PATCH 1/5] deps: bump backend with diff fix --- pysr/juliapkg.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pysr/juliapkg.json b/pysr/juliapkg.json index 1673a713..56f09199 100644 --- a/pysr/juliapkg.json +++ b/pysr/juliapkg.json @@ -3,7 +3,7 @@ "packages": { "SymbolicRegression": { "uuid": "8254be44-1295-4e6a-a16d-46603ac705cb", - "version": "=1.4.0" + "version": "=1.5.0" }, "Serialization": { "uuid": "9e88b42a-f829-5b0c-bbe9-9e923198166b", From 31e4e912ac6090c415d569fbc77dab89da90119d Mon Sep 17 00:00:00 2001 From: MilesCranmer Date: Sat, 14 Dec 2024 14:42:47 -0800 Subject: [PATCH 2/5] docs: improve operator page --- docs/operators.md | 80 ++++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 50 deletions(-) diff --git a/docs/operators.md b/docs/operators.md index db538d8b..b67c7b1b 100644 --- a/docs/operators.md +++ b/docs/operators.md @@ -7,56 +7,32 @@ takes one or two scalars as input, and returns on scalar as output, is likely to be a valid operator[^1]. A selection of these and other valid operators are stated below. -**Binary** - -- `+` -- `-` -- `*` -- `/` -- `^` -- `max` -- `min` -- `mod` -- `cond` - - Equal to `(x, y) -> x > 0 ? y : 0` -- `greater` - - Equal to `(x, y) -> x > y ? 1 : 0` -- `logical_or` - - Equal to `(x, y) -> (x > 0 || y > 0) ? 1 : 0` -- `logical_and` - - Equal to `(x, y) -> (x > 0 && y > 0) ? 1 : 0` - -**Unary** - -- `neg` -- `square` -- `cube` -- `exp` -- `abs` -- `log` -- `log10` -- `log2` -- `log1p` -- `sqrt` -- `sin` -- `cos` -- `tan` -- `sinh` -- `cosh` -- `tanh` -- `atan` -- `asinh` -- `acosh` -- `atanh_clip` - - Equal to `atanh(mod(x + 1, 2) - 1)` -- `erf` -- `erfc` -- `gamma` -- `relu` -- `round` -- `floor` -- `ceil` -- `sign` +Also, note that it's a good idea to not use too many operators, since +it can exponentially increase the search space. + +**Binary Operators** + +| Arithmetic | Comparison | Logic | +|--------------|------------|----------| +| `+` | `max` | `logical_or`[^2] | +| `-` | `min` | `logical_and`[^3]| +| `*` | `greater`[^4] | | +| `/` | `cond`[^5] | | +| `^` | `mod` | | + +**Unary Operators** + +| Basic | Exp/Log | Trig | Hyperbolic | Special | Rounding | +|------------|------------|-----------|------------|-----------|------------| +| `neg` | `exp` | `sin` | `sinh` | `erf` | `round` | +| `square` | `log` | `cos` | `cosh` | `erfc` | `floor` | +| `cube` | `log10` | `tan` | `tanh` | `gamma` | `ceil` | +| `cbrt` | `log2` | `asin` | `asinh` | `relu` | | +| `sqrt` | `log1p` | `acos` | `acosh` | `sinc` | | +| `abs` | | `atan` | `atanh` | | | +| `sign` | | | | | | +| `inv` | | | | | | + ## Custom @@ -96,3 +72,7 @@ any invalid values over the training dataset. [^1]: However, you will need to define a sympy equivalent in `extra_sympy_mapping` if you want to use a function not in the above list. +[^2]: `logical_or` is equivalent to `(x, y) -> (x > 0 || y > 0) ? 1 : 0` +[^3]: `logical_and` is equivalent to `(x, y) -> (x > 0 && y > 0) ? 1 : 0` +[^4]: `greater` is equivalent to `(x, y) -> x > y ? 1 : 0` +[^5]: `cond` is equivalent to `(x, y) -> x > 0 ? y : 0` From a95c63313ccbb2d07bbf1ab1e72eda38e66924c0 Mon Sep 17 00:00:00 2001 From: MilesCranmer Date: Sat, 14 Dec 2024 14:43:09 -0800 Subject: [PATCH 3/5] chore: bump version with new behavior for `tanh` --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index bffdb372..b2445108 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "pysr" -version = "1.2.0" +version = "1.3.0" authors = [ {name = "Miles Cranmer", email = "miles.cranmer@gmail.com"}, ] From 64168d52b1128b797af614249f209b7b10db97f4 Mon Sep 17 00:00:00 2001 From: MilesCranmer Date: Sat, 14 Dec 2024 15:23:30 -0800 Subject: [PATCH 4/5] docs: add example of differential operator --- docs/examples.md | 65 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 64 insertions(+), 1 deletion(-) diff --git a/docs/examples.md b/docs/examples.md index 84d4d1bb..3523cf82 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -644,7 +644,70 @@ You can then view the logs with: tensorboard --logdir logs/ ``` -## 13. Additional features +## 13. Using differential operators + +As part of the `TemplateExpressionSpec` described above, +you can also use differential operators within the template. +The operator for this is `D` which takes an expression as the first argument, +and the argument _index_ we are differentiating as the second argument. +This lets you compute integrals via evolution. + +For example, let's say we wish to find the integral of $\sqrt(1 - x^2)$. +We can compute the derivative of a function $f(x)$, and compare that +to numerical samples of $\sqrt(1 - x^2)$. Then, by extension, +$f(x)$ represents the indefinite integral of $\sqrt(1 - x^2)$ with some constant offset. + +```python +import numpy as np +from pysr import PySRRegressor, TemplateExpressionSpec + +x = np.random.rand(1000) +y = np.sqrt(1 - x**2) + +expression_spec = TemplateExpressionSpec( + ["f"], + """ + function diff_f_x((; f), (x,)) + df = D(f, 1) # Symbolic derivative of f with respect to its first arg + return df(x) + end + """ +) + +model = PySRRegressor( + niterations=1000, + binary_operators=["+", "-", "*", "/"], + unary_operators=["sqrt", "atan"], + expression_spec=expression_spec, + maxsize=30, + batching=True, + batch_size=32, + parsimony=1e-3 +) +model.fit(x[:, np.newaxis], y) +``` + +If everything works, you should + +Here, we write out a full function in Julia. +But we can also do an anonymous function, like `((; f), (x,)) -> D(f, 1)(x)`. We can also avoid the fancy unpacking syntax and write: +`(nt, xs) -> D(nt.f, 1)(xs[1])` which is completely equivalent. Note that in Julia, +the following two syntaxes are equivalent: + +```julia +nt = (; f=1, g=2) # Create a "named tuple" +(; f, g) = nt +``` + +and + +```julia +f = nt.f +g = nt.g +``` + + +## 14. Additional features For the many other features available in PySR, please read the [Options section](options.md). From d880542c1bbf669919497219fa5ea056aaf4a2e1 Mon Sep 17 00:00:00 2001 From: MilesCranmer Date: Sat, 14 Dec 2024 15:36:34 -0800 Subject: [PATCH 5/5] docs: better integral example --- docs/examples.md | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/docs/examples.md b/docs/examples.md index 3523cf82..6d841b0e 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -652,17 +652,18 @@ The operator for this is `D` which takes an expression as the first argument, and the argument _index_ we are differentiating as the second argument. This lets you compute integrals via evolution. -For example, let's say we wish to find the integral of $\sqrt(1 - x^2)$. +For example, let's say we wish to find the integral of $\frac{1}{x^2 \sqrt{x^2 - 1}}$ +in the range $x > 1$. We can compute the derivative of a function $f(x)$, and compare that -to numerical samples of $\sqrt(1 - x^2)$. Then, by extension, -$f(x)$ represents the indefinite integral of $\sqrt(1 - x^2)$ with some constant offset. +to numerical samples of $\frac{1}{x^2\sqrt{x^2-1}}$. Then, by extension, +$f(x)$ represents the indefinite integral of it with some constant offset! ```python import numpy as np from pysr import PySRRegressor, TemplateExpressionSpec -x = np.random.rand(1000) -y = np.sqrt(1 - x**2) +x = np.random.uniform(1, 10, (1000,)) # Integrand sampling points +y = 1 / (x**2 * np.sqrt(x**2 - 1)) # Evaluation of the integrand expression_spec = TemplateExpressionSpec( ["f"], @@ -675,19 +676,15 @@ expression_spec = TemplateExpressionSpec( ) model = PySRRegressor( - niterations=1000, binary_operators=["+", "-", "*", "/"], - unary_operators=["sqrt", "atan"], + unary_operators=["sqrt"], expression_spec=expression_spec, - maxsize=30, - batching=True, - batch_size=32, - parsimony=1e-3 + maxsize=20, ) model.fit(x[:, np.newaxis], y) ``` -If everything works, you should +If everything works, you should find something that simplifies to $\frac{\sqrt{x^2 - 1}}{x}$. Here, we write out a full function in Julia. But we can also do an anonymous function, like `((; f), (x,)) -> D(f, 1)(x)`. We can also avoid the fancy unpacking syntax and write: