Skip to content

Commit

Permalink
simplified readme
Browse files Browse the repository at this point in the history 
Signed-off-by: Callan Gray <[email protected]>
  • Loading branch information
@calgray
calgray committed Jan 1, 2025
1 parent 4f8f241 commit c24ca5c
Showing 1 changed file with 65 additions and 126 deletions.
191 changes: 65 additions & 126 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,162 +1,94 @@
# athreading

[![Test and build](https://github.com/calgray/athreading/actions/workflows/ci.yml/badge.svg)](https://github.com/calgray/athreading/actions/workflows/ci.yml)
[![PyPI version shields.io](https://img.shields.io/pypi/v/athreading.svg)](https://pypi.python.org/pypi/athreading)
[![PyPI pyversions](https://img.shields.io/pypi/pyversions/athreading.svg)](https://pypi.python.org/pypi/athreading)

`athreading` is an asynchronous threading library for running and synchronizing I/O threads with asyncio.
`athreading` is a Python library that allows you to run synchronous I/O functions asynchronously using `asyncio`. It provides decorators to adapt synchronous functions and generators, enabling them to operate without blocking the event loop.

Note: Due to Python GIL, this library not not provide multi-threaded CPU parallelism unless using Python 3.13ft.
## Features

## Usage

Synchronous I/O functions and generators using sleep/wait operations can be run asynchronously by offloading to worker threads and avoid blocking the async I/O loop.
- **`@athreading.call`**: Converts a synchronous function into an asynchronous function.
- **`@athreading.iterate`**: Converts a synchronous iterator into an asynchronous iterator.
- **`@athreading.generate`**: Converts a synchronous generator into an asynchronous generator.

### Callable[..., ResultT] → Callable[..., Coroutine[None, None, ResultT]]
*Note*: Due to Python's Global Interpreter Lock (GIL), this library does not provide multi-threaded CPU parallelism unless using Python 3.9 with `nogil` or Python 3.13 with free threading enabled.

Use `athread.call` to wrap a function/`Callable` to an async function/function returning a `Coroutine`:
## Installation

#### Synchronous<!--1-->
`athreading` can be installed using pip:

```python
def print_sqrt(x):
time.sleep(0.5)
result = math.sqrt(x)
print(datetime.datetime.now(), result)
return result

res = (print_sqrt(2), print_sqrt(3), print_sqrt(4))
print(res)
```bash
pip install athreading
```

output:
## Usage

`athreading` enables running synchronous functions and iterators asynchronously using `asyncio`.

```log
2023-12-05 14:45:57.716696 1.4142135623730951
2023-12-05 14:45:58.217192 1.7320508075688772
2023-12-05 14:45:58.717934 2.0
(1.4142135623730951 1.7320508075688772 2.0)
```
### 1. Converting a Synchronous Function

#### Asynchronous<!--1-->
The `@athreading.call` decorator to convert a synchronous function into an asynchronous function.

```python
async def amain():
aprint_sqrt = athreading.call(print_sqrt)
res = await asyncio.gather(
aprint_sqrt(2),
aprint_sqrt(3),
aprint_sqrt(4)
import athreading
import time
import math
import asyncio

@athreading.call
def compute_sqrt(x):
time.sleep(0.5) # Simulate a blocking I/O operation
return math.sqrt(x)

async def main():
results = await asyncio.gather(
compute_sqrt(2),
compute_sqrt(3),
compute_sqrt(4)
)
print(res)
print(results)

asyncio.run(amain())
asyncio.run(main())
```

output:

```log
2023-12-05 14:45:59.219461 1.4142135623730951
2023-12-05 14:45:59.220492 1.7320508075688772
2023-12-05 14:45:59.221174 2.0
(1.4142135623730951 1.7320508075688772 2.0)
```

### Iterator → AsyncIterator

Use `athreading.iterate` to convert an `Iterator` interface to an `AsyncIterator` for iterating on the I/O thread.

#### Synchronous<!--2-->

```python
def worker(n):
for i in range(n):
time.sleep(0.5)
yield datetime.datetime.now()

def print_stream(id, stream):
for value in stream:
print("stream:", id, "time:", value)

for sid in range(4):
print_stream(sid, worker(3))
```
In this example, `compute_sqrt` is a synchronous function that sleeps for 0.5 seconds to simulate a blocking I/O operation. By decorating it with `@athreading.call`, it can be awaited within an asynchronous context, allowing multiple calls to run concurrently without blocking the event loop.

output:

```log
stream: 0 time: 2023-12-05 09:37:38.758954
stream: 0 time: 2023-12-05 09:37:39.259106
stream: 0 time: 2023-12-05 09:37:39.759610
stream: 1 time: 2023-12-05 09:37:40.260039
stream: 1 time: 2023-12-05 09:37:40.760152
stream: 1 time: 2023-12-05 09:37:41.260274
stream: 2 time: 2023-12-05 09:37:41.760548
stream: 2 time: 2023-12-05 09:37:42.262526
stream: 2 time: 2023-12-05 09:37:42.762736
stream: 3 time: 2023-12-05 09:37:43.262930
stream: 3 time: 2023-12-05 09:37:43.763080
stream: 3 time: 2023-12-05 09:37:44.263225
```
## 2. Converting a Synchronous Iterator

#### Asynchronous<!--2-->
The `@athreading.iterate` decorator transforms a synchronous iterator into an asynchronous iterator.

```python
async def aprint_stream(id, stream):
async with stream:
async for value in stream:
print("stream:", id, "time:", value)


async def arun():
executor = ThreadPoolExecutor(max_workers=4)
await asyncio.gather(
*[
aprint_stream(sid, athreading.iterate(worker, executor=executor)(3))
for sid in range(4)
]
)

asyncio.run(arun())
import athreading
import time
import datetime
import asyncio

@athreading.iterate
def time_generator(n):
for _ in range(n):
time.sleep(0.5) # Simulate a blocking I/O operation
yield datetime.datetime.now()

async def main():
async for current_time in time_generator(3):
print(current_time)

asyncio.run(main())
```

output:

```log
stream: 0 time: 2023-12-05 09:37:15.834115
stream: 1 time: 2023-12-05 09:37:15.835140
stream: 2 time: 2023-12-05 09:37:15.835749
stream: 3 time: 2023-12-05 09:37:15.836387
stream: 0 time: 2023-12-05 09:37:16.834371
stream: 1 time: 2023-12-05 09:37:16.835346
stream: 2 time: 2023-12-05 09:37:16.835938
stream: 3 time: 2023-12-05 09:37:16.836573
stream: 0 time: 2023-12-05 09:37:17.834634
stream: 1 time: 2023-12-05 09:37:17.835552
stream: 2 time: 2023-12-05 09:37:17.836113
stream: 3 time: 2023-12-05 09:37:17.836755
```

### Generator → AsyncGenerator

Use `athreading.generate` to convert a `Generator` interface to an `AsyncGenerator` for iterating on the I/O thread.

#### Synchronous<!--3-->

TODO

#### Asynchronous<!--3-->

TODO
Here, `time_generator` is a synchronous generator function that yields the current time after sleeping for 0.5 seconds. By decorating it with `@athreading.iterate`, it becomes an asynchronous generator that can be iterated over without blocking the event loop.

## Maintenance

This is a minimal Python 3.11 application that uses [poetry](https://python-poetry.org) for packaging and dependency management. It also provides [pre-commit](https://pre-commit.com/) hooks (for [isort](https://pycqa.github.io/isort/), [Black](https://black.readthedocs.io/en/stable/), [Flake8](https://flake8.pycqa.org/en/latest/) and [mypy](https://mypy.readthedocs.io/en/stable/)) and automated tests using [pytest](https://pytest.org/) and [GitHub Actions](https://github.com/features/actions). Pre-commit hooks are automatically kept updated with a dedicated GitHub Action, this can be removed and replace with [pre-commit.ci](https://pre-commit.ci) if using an public repo. It was developed by the [Imperial College Research Computing Service](https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/).
This is a minimal Python library that uses [poetry](https://python-poetry.org) for packaging and dependency management. It also provides [pre-commit](https://pre-commit.com/) hooks (for [isort](https://pycqa.github.io/isort/), [Black](https://black.readthedocs.io/en/stable/), [Flake8](https://flake8.pycqa.org/en/latest/) and [mypy](https://mypy.readthedocs.io/en/stable/)) and automated tests using [pytest](https://pytest.org/) and [GitHub Actions](https://github.com/features/actions). Pre-commit hooks are automatically kept updated with a dedicated GitHub Action, this can be removed and replace with [pre-commit.ci](https://pre-commit.ci) if using an public repo. It was developed by the [Imperial College Research Computing Service](https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/).

### Development
### Testing

To modify, test and request changes to this repository:

1. [Download and install Poetry](https://python-poetry.org/docs/#installation) following the instructions for your OS.
2. Clone `[email protected]:calgray/athreading.git` and make it your working directory
1. [Download and install Poetry](https://python-poetry.org/docs/#installation) following the instructions for the target OS.
2. Clone `[email protected]:calgray/athreading.git` and make it the working directory.
3. Set up the virtual environment:

```bash
Expand Down Expand Up @@ -184,4 +116,11 @@ To modify, test and request changes to this repository:
### Publishing

The GitHub workflow includes an action to publish on release.

To run this action, uncomment the commented portion of `publish.yml`, and modify the steps for the desired behaviour (ie. publishing a Docker image, publishing to PyPI, deploying documentation etc.)

## License

This project is licensed under the BSD-3-Clause License.

For more information and examples, please visit the [athreading GitHub repository](https://github.com/calgray/athreading).

0 comments on commit c24ca5c

Please sign in to comment.