Skip to content

Commit

Permalink
fix grammar in docs
Browse files Browse the repository at this point in the history
  • Loading branch information
orsinium committed Jan 21, 2023
1 parent 80d87f6 commit 5eab9bd
Show file tree
Hide file tree
Showing 12 changed files with 74 additions and 71 deletions.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,24 @@ Features:
+ Exactly-once delivery.
+ Smart and configurable retries.
+ Many integrations.
+ Compatible. You can use walnats to emit events for non-walnats services or consume event emitted by non-walnats services. The tools is flexible enough to adapt for any format of the messages you use.
+ Compatible. You can use walnats to emit events for non-walnats services or consume events emitted by non-walnats services. The tool is flexible enough to adapt to any format of the messages you use.

## Compared to other tools

Compared to other big Python frameworks for background jobs (like [celery](https://docs.celeryq.dev/en/stable/), [dramatiq](https://dramatiq.io/index.html), [rq](https://python-rq.org/), [huey](https://huey.readthedocs.io/en/latest/), and so on), the main difference from implementation perspective is that walnats is younger and so had opportunity to be designed around modern technologies right from the beginning. Namely, mypy-powered type safety, async/await powered concurrency, and nats-powered persistency and distribution.
Compared to other big Python frameworks for background jobs (like [celery](https://docs.celeryq.dev/en/stable/), [dramatiq](https://dramatiq.io/index.html), [rq](https://python-rq.org/), [huey](https://huey.readthedocs.io/en/latest/), and so on), the main difference from an implementation perspective is that walnats is younger and so had an opportunity to be designed around modern technologies right from the beginning. Namely, mypy-powered type safety, async/await-powered concurrency, and nats-powered persistency and distribution.

And when compared to **all** other Python frameworks for background jobs (including new async/await-based ones like [arq](https://arq-docs.helpmanual.io/), [pytask-io](https://github.com/joegasewicz/pytask-io), and [aiotasks](https://github.com/cr0hn/aiotasks)), the main difference is that Walnats is event-driven. While in all these frameworks the job scheduling is conceptually a function call over the network, in walnats publishers instead emit events to which any subscribers can subscribe at any point. This approach is called "[tell, don't ask](https://wiki.c2.com/?TellDontAsk)".

For example, when your webshop sends a parcel to a client, instead of directly calling `send_email` and `send_sms` actors like you'd do with Celery, with Walnats publisher will emit a single `parcel-sent` event, and that event will be delivered by Walnats to all interested actors. It gives you a few nice benefits:

1. Publisher makews only one network request.
1. Publisher makes only one network request.
1. When you add a new actor, you don't need to modify the publisher. That's especially cool for microservice architecture when the publisher and the actor can be different services owned by different teams.
1. It's easier to reason about. When you develop a microservice, you only need to know what events there are you can subscribe to and emit your own events without thinking too much how all other sefvices in the system work with these events.
1. It's easier to observe. Walnats directly translates events into Nats subject and actors into Nats JetStream consumers. So, any Nats observability tool will give you great insights on what's going on in your system.
1. It's easier to reason about. When you develop a microservice, you only need to know what events there are you can subscribe to and emit your own events without thinking too much about how all other services in the system work with these events.
1. It's easier to observe. Walnats directly translates events into Nats subjects and actors into Nats JetStream consumers. So, any Nats observability tool will give you great insights into what's going on in your system.

If you have a big distributed system, Walnats is for you. If all you want is to send emails in background from your Django monolith or a little hobby project, you might find another framework a better fit.
If you have a big distributed system, Walnats is for you. If all you want is to send emails in the background from your Django monolith or a little hobby project, you might find another framework a better fit.

Lastly, compared to you just taking [nats.py](https://github.com/nats-io/nats.py) and writing your service from scratch, Walnats does a better job at handling failures, load spikes, and corner-cases. Walnats is "[designed for failure](https://www.v-wiki.net/design-for-failure/)". Distributed systems are hard, and you shouldn't embark on this journey alone.
Lastly, compared to you just taking [nats.py](https://github.com/nats-io/nats.py) and writing your service from scratch, Walnats does a better job at handling failures, load spikes, and corner cases. Walnats is "[designed for failure](https://www.v-wiki.net/design-for-failure/)". Distributed systems are hard, and you shouldn't embark on this journey alone.

## Installation

Expand Down Expand Up @@ -71,7 +71,7 @@ async def run() -> None:
asyncio.run(run())
```

Create subscriber (a service that listens to events):
Create a subscriber (a service that listens to events):

```python
import asyncio
Expand Down
4 changes: 2 additions & 2 deletions docs/actors.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ async def send_email(user: User) -> None:
...
```

That's it. The handler knows nothing about Nats, Walnats, or the publisher. That means, it's dead simple to test, refactor, or reuse with another framework. This pattern is known as [Functional core, imperative shell](https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell). The handler is the functional core, and walnats is the imperative shell that you don't need to test (at least not in the unit tests).
That's it. The handler knows nothing about Nats, Walnats, or the publisher. That means it's dead simple to test, refactor, or reuse with another framework. This pattern is known as [Functional core, imperative shell](https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell). The handler is the functional core, and walnats is the imperative shell that you don't need to test (at least not in the unit tests).

## Declare actors

Expand All @@ -23,7 +23,7 @@ WELCOME_USER = walnats.Actor(
)
```

That's all you need to get started. Below are the API docs covering different configuration options you can specify for an actor. They allow you to adjust actor's behavior in case if there are too many messages or a handler failure. We'll also talk about handling high load and failures in the [Subscriber](sub) section. So, you can skip the rest of the page for now if you're only getting started.
That's all you need to get started. Below are the API docs covering different configuration options you can specify for an actor. They allow you to adjust the actor's behavior in case there are too many messages or a handler failure. We'll also talk about handling high loads and failures in the [Subscriber](sub) section. So, you can skip the rest of the page for now if you're only getting started.

## API

Expand Down
25 changes: 14 additions & 11 deletions docs/alloy.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ layout: default
---
# Verification

Formal verification usually means that you describe the same algoithm twice (declaratively and imperatively), and then computer checks that both implementations are equivalent. And when it comes to distributed systems, you need not only to describe your system but also how it changes over time. There are currently 2 usable languages that can effectively describe changing systems: [TLA+](https://en.wikipedia.org/wiki/TLA%2B) and [Alloy 6](https://en.wikipedia.org/wiki/Alloy_(specification_language)). On this page, I use Alloy 6 because it's simpler, looks more like a programming language, great for describing relations, and has a nice visualizer.
Formal verification usually means that you describe the same algorithm twice (declaratively and imperatively), and then the computer checks that both implementations are equivalent. And when it comes to distributed systems, you need not only to describe your system but also how it changes over time. There are currently 2 usable languages that can effectively describe changing systems: [TLA+](https://en.wikipedia.org/wiki/TLA%2B) and [Alloy 6](https://en.wikipedia.org/wiki/Alloy_(specification_language)). On this page, I use Alloy 6 because it's simpler, looks more like a programming language, is great for describing relations, and has a nice visualizer.

This page provides a declarative model for a walnats-based system. You can use this model to generate possible failure scenarios or built on top of it verification of your business-specific logic.
This page provides a declarative model for a walnats-based system. You can use this model to generate possible failure scenarios or build on top of it verification of your business-specific logic.

## Learn Alloy

The good news is that there are tons of articles and publications about Alloy. Disproportinally more than this tool is actually used by sane people. The bad news is that Alloy 6 (released in 2021) was a huge release introducing a great support for working with time. Before the release, working with time required a lot of dirty hacks and workarounds, and it all was far from pretty. And most of models and tutorials using Alloy relied on time. Hence if you see a tutorial for Alloy using time and not updated after 2021, you can safely dicard it, it's useless now.
The good news is that there are tons of articles and publications about Alloy. Disproportionally more than this tool is actually used by sane people. The bad news is that Alloy 6 (released in 2021) was a huge release introducing great support for working with time. Before the release, working with the time required a lot of dirty hacks and workarounds, and it all was far from pretty. And most of the models and tutorials using Alloy relied on time. Hence if you see a tutorial for Alloy using time and not updated after 2021, you can safely discard it, it's useless now.

There are some resources to learn Alloy that are still good:

Expand All @@ -26,7 +26,7 @@ Alloy [supports running Markdown files](https://alloy.readthedocs.io/en/latest/t
1. [Install Alloy Analyzer](https://github.com/AlloyTools/org.alloytools.alloy/releases/)
1. Run Alloy Analyzer: `java -jar org.alloytools.alloy.dist.jar`.
1. Press "Open".
1. Navigate to the walnats repository you clonned, and inside go to `docs/alloy.md` and open it.
1. Navigate to the walnats repository you cloned, and inside go to `docs/alloy.md` and open it.
1. Press "Execute" to generate a sample of the model and "Show" to open the graph.

Go to these tutorials to learn using the GUI:
Expand All @@ -37,19 +37,19 @@ Go to these tutorials to learn using the GUI:

## Message

The first "signature" (something like `class` in Python) we'll have is `Message`. It represents specific messages sent for a single event type. I make a model for only one event type to keep it simple. Whenever possible, you should use [inductive](https://en.wikipedia.org/wiki/Inductive_reasoning) approach. Prove your assumptions for one event, prove that every event in the system fits the model, and you have proven the whole system. That's why one event type is sufficient.
The first "signature" (something like `class` in Python) we'll have is `Message`. It represents specific messages sent for a single event type. I make a model for only one event type to keep it simple. Whenever possible, you should use the [inductive](https://en.wikipedia.org/wiki/Inductive_reasoning) approach. Prove your assumptions for one event, prove that every event in the system fits the model, and you have proven the whole system. That's why one event type is sufficient.

```alloy
some sig Message {}
```

The `some` quantifier means that there is always at least one message because models without messages are boring. If nothing happens, what's the point?

if there is a Message, it doesn't mean it has been emitted yet. All that means is that it will be emitted on the interval we consider. In other words, each message atom (instance) at the start is a planned message. I find it helpful for verification. We know in advance what messages we expect, and so it's easier to check if all expected messages actually happened. Lastly, it will help you if you plan to model producer failures.
if there is a Message, it doesn't mean it has been emitted yet. All that means is that it will be emitted on the interval we consider. In other words, each message atom (instance) at the start is a planned message. I find it helpful for verification. We know in advance what messages we expect, so it's easier to check if all expected messages actually happened. Lastly, it will help you if you plan to model producer failures.

## Producer

Producer is a service that emits messages for the event. We can have multiple producers.
A producer is a service that emits messages for the event. We can have multiple producers.

```alloy
sig Producer {
Expand All @@ -71,7 +71,7 @@ abstract sig Actor {
sig Actor1, Actor2 extends Actor {}
```

`Actor1` and `Actor2` are different actor types, such as "send-email" or "generate-report". Each atom (instance) here is a separate instance of that actor. For example, the atom `Actor1$2` will be 2nd instance of Actor1.
`Actor1` and `Actor2` are different actor types, such as "send-email" or "generate-report". Each atom (instance) here is a separate instance of that actor. For example, the atom `Actor1$2` will be the 2nd instance of Actor1.

The `abstract` means that there are no atoms of `Actor` type, it must be either `Actor1` or `Actor2`.

Expand Down Expand Up @@ -127,9 +127,9 @@ Liveness properties are the ones that **eventually** will be true, that "good th

```alloy
pred liveness {
// Every message is eventaully emitted.
// Every message is eventually emitted.
Message = Producer.emitted
// Every emitted message is eventaully processed.
// Every emitted message is eventually processed.
Producer.emitted = Actor1.handled
Producer.emitted = Actor2.handled
}
Expand All @@ -155,4 +155,7 @@ We start with `init` and let the system evolve such that `safety` always holds t

## Stuttering

Since we require
Since we require `liveness` properties to be eventually true, on each step the system will progress towards that goal. There are a few things to keep in mind, though:

+ One step of the spec can take any amount of time in the real world, from nanoseconds to days. The system may die and be dead for days until an engineer comes and fixes a critical bug. All we say is that *eventually* messages will be processed.
+ Individual actors or even all of them still can [stutter](https://www.learntla.com/core/temporal-logic.html?highlight=stutter#anything-can-crash) for any duration of steps. Again, all we say is that they will *eventually* move on.
2 changes: 1 addition & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# API

Are you looking for an API documentation for a specific class or module? If can find most of the answers in docstrings without leaving the IDE. But in case you need a web reference, here is an index of pages covering specific parts of the API.
Are you looking for API documentation for a specific class or module? If can find most of the answers in docstrings without leaving the IDE. But in case you need a web reference, here is an index of pages covering specific parts of the API.

## Public classes

Expand Down
6 changes: 3 additions & 3 deletions docs/clock.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Periodic tasks

You often will have tasks that you need to execute periodically (for instance, every day or every 5 minutes) or by schedule (for instance, 5 days after today). For example, create a database backup every midnight or actualize cache of orders every 5 minutes. Wouldn't it be great to have an event for that? Then you can get for the periodic tasks all the benefits of event-driven actors: scalability, distribution, retries, observibility, and more.
You often will have tasks that you need to execute periodically (for instance, every day or every 5 minutes) or by schedule (for instance, 5 days after today). For example, create a database backup every midnight or actualize a cache of orders every 5 minutes. Wouldn't it be great to have an event for that? Then you can get for the periodic tasks all the benefits of event-driven actors: scalability, distribution, retries, observability, and more.

Meet `walnats.Clock`. It's a worker that emits periodic events. To implement a periodic task, run the clock and subscribe to the event in any other place.

Expand All @@ -11,6 +11,6 @@ Meet `walnats.Clock`. It's a worker that emits periodic events. To implement a p

## Tips

1. It might be a good idea to run a separate clock on each nats cluster, so that in case of network failure between clusters, events will still be coming in each of them. When connection is restored, Nats will make sure there are no duplicates.
1. It might be a good idea to run a separate clock on each nats cluster so that in case of network failure between clusters, events will still be coming in each of them. When the connection is restored, Nats will make sure there are no duplicates.
1. The interval between events may be affected by CPU-bound tasks running in the same process. To avoid it, make sure to use {py:attr}`walnats.ExecuteIn.PROCESS` for all CPU-heavy actors running in the same instance as the clock.
1. If there is just one worker that needs a very specific schedule (for instance, it must be run daily at 12:03 and any other time won't do), prefer using {py:class}`walnats.decorators.filter_time`. If there are multiple workers that need the same schedule (like be run once a day, doesn't matter when), prefer making for them a separate Clock with a fitting duration.
1. If there is just one worker that needs a very specific schedule (for instance, it must be run daily at 12:03 and any other time won't do), prefer using {py:class}`walnats.decorators.filter_time`. If multiple workers need the same schedule (like being run once a day, doesn't matter when), prefer providing for them a separate Clock with a fitting duration.
Loading

0 comments on commit 5eab9bd

Please sign in to comment.