Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add from_dict method to construct configuration from dictionary #15

Merged
merged 19 commits into from
Aug 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
701ef66
feat: add separate callback module
NucciTheBoss Aug 26, 2024
41c043e
chore(deps): update tox.ini to use new ruff command syntax
NucciTheBoss Aug 26, 2024
4a67164
chore(deps): update pyproject.toml to use new ruff syntax
NucciTheBoss Aug 26, 2024
8e58a9d
feat: simplify base editor implementation
NucciTheBoss Aug 26, 2024
1189f85
refactor: have config editors use explicitly declared methods rather …
NucciTheBoss Aug 26, 2024
2863e1e
feat!: add `from_*` methods for flexible creation of data models
NucciTheBoss Aug 26, 2024
43992ca
feat: add `BaseError` and `ModelError` for more granular exceptions
NucciTheBoss Aug 26, 2024
a28785f
tests: update unit tests for slurmconfig editor
NucciTheBoss Aug 26, 2024
4afd843
chore(deps): remove renovate
NucciTheBoss Aug 26, 2024
6df3071
chore(lint): use explicit re-exports in module __init__.py
NucciTheBoss Aug 26, 2024
72797ff
docs: slurm.conf -> slurmdbd.conf in function docstrings
NucciTheBoss Aug 26, 2024
8b6f6b5
refactor: use debug logging level instead of warning for ignored comm…
NucciTheBoss Aug 26, 2024
1eb6d94
docs: update README to match other Charmed HPC projects
NucciTheBoss Aug 26, 2024
82d9ed4
fix: use f-string to generate error message if config file is not found
NucciTheBoss Aug 26, 2024
b19f9de
feat: version bump and pypi metadata update
NucciTheBoss Aug 26, 2024
87e6fb2
refactor: make `clean` less Go-ish
NucciTheBoss Aug 26, 2024
703303f
refactor: use more clear naming for configuration option sets
NucciTheBoss Aug 26, 2024
67a1bdc
refactor: add type annotation to `generate_descriptors(...)` func
NucciTheBoss Aug 26, 2024
7a09a69
fix: use correct type annotations for SlurmConfig constructor
NucciTheBoss Aug 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 60 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,30 @@
<div align="center">

# slurmutils

Utilities and APIs for interfacing with the Slurm workload manager.

[![Matrix](https://img.shields.io/matrix/ubuntu-hpc%3Amatrix.org?logo=matrix&label=ubuntu-hpc)](https://matrix.to/#/#ubuntu-hpc:matrix.org)
![PyPI - Version](https://img.shields.io/pypi/v/slurmutils)
![PyPI - Downloads](https://img.shields.io/pypi/dm/slurmutils)
![GitHub License](https://img.shields.io/github/license/charmed-hpc/slurmutils)
[![Matrix](https://img.shields.io/matrix/ubuntu-hpc%3Amatrix.org?logo=matrix&label=ubuntu-hpc)](https://matrix.to/#/#hpc:ubuntu.com)

</div>

## Features
Utilities and APIs for interfacing with the Slurm workload manager.

`slurmutils` is a collection of various utilities and APIs to make it easier
slurmutils is a collection of various utilities that make it easier
for you and your friends to interface with the Slurm workload manager, especially if you
are orchestrating deployments of new and current Slurm clusters. Gone are the days of
seething over incomplete Jinja2 templates. Current utilities and APIs shipped in the
`slurmutils` package include:
seething over incomplete Jinja2 templates. Current utilities shipped in the
slurmutils package include:

#### `from slurmutils.editors import ...`

* `slurmconfig`: An editor _slurm.conf_ and _Include_ files.
* `slurmdbdconfig`: An editor for _slurmdbd.conf_ files.
* `slurmconfig`: An editor for _slurm.conf_ configuration files.
* `slurmdbdconfig`: An editor for _slurmdbd.conf_ configuration files.

## Installation
For more information on how to use or contribute to slurmutils,
check out the [Getting Started](#-getting-started) and [Development](#-development)
sections below 👇

## ✨ Getting Started

### Installation

#### Option 1: Install from PyPI

Expand All @@ -32,7 +35,7 @@ $ python3 -m pip install slurmutils
#### Option 2: Install from source

We use the [Poetry](https://python-poetry.org) packaging and dependency manager to
manage this project. It must be installed on your system if installing `slurmutils`
manage this project. It must be installed on your system if installing slurmutils
from source.

```shell
Expand All @@ -41,17 +44,17 @@ $ cd slurmutils
$ poetry install
```

## Usage
### Usage

### Editors
#### Editors

#### `slurmconfig`
##### `slurmconfig`

This module provides an API for editing both _slurm.conf_ and _Include_ files,
and can create new configuration files if they do not exist. Here's some common Slurm
lifecycle management operators you can perform using this editor:

##### Edit a pre-existing _slurm.conf_ configuration file
###### Edit a pre-existing _slurm.conf_ configuration file

```python
from slurmutils.editors import slurmconfig
Expand All @@ -63,7 +66,7 @@ with slurmconfig.edit("/etc/slurm/slurm.conf") as config:
config.proctrack_type = "proctrack/linuxproc"
```

##### Add a new node to the _slurm.conf_ file
###### Add a new node to the _slurm.conf_ file

```python
from slurmutils.editors import slurmconfig
Expand All @@ -77,16 +80,16 @@ with slurmconfig.edit("/etc/slurm/slurm.conf") as config:
RealMemory=1000,
TmpDisk=10000,
)
config.nodes[node.node_name] = node
config.nodes.update(node.dict())
```

#### `slurmdbdconfig`
##### `slurmdbdconfig`

This module provides and API for editing _slurmdbd.conf_ files, and can create new
_slurmdbd.conf_ files if they do not exist. Here's some operations you can perform
on the _slurmdbd.conf_ file using this editor:

##### Edit a pre-existing _slurmdbd.conf_ configuration file
###### Edit a pre-existing _slurmdbd.conf_ configuration file

```python
from slurmutils.editors import slurmdbdconfig
Expand All @@ -99,19 +102,42 @@ with slurmdbdconfig.edit("/etc/slurm/slurmdbd.conf") as config:
del config.auth_alt_parameters
```

## Project & Community
## 🤔 What's next?

If you want to learn more about all the things you can do with slurmutils,
here are some further resources for you to explore:

* [Open an issue](https://github.com/charmed-hpc/slurmutils/issues/new?title=ISSUE+TITLE&body=*Please+describe+your+issue*)
* [Ask a question on Github](https://github.com/orgs/charmed-hpc/discussions/categories/q-a)

## 🛠️ Development

This project uses [tox](https://tox.wiki) as its command runner, which provides
some useful commands that will help you while hacking on slurmutils:

```shell
tox run -e fmt # Apply formatting standards to code.
tox run -e lint # Check code against coding style standards.
tox run -e unit # Run unit tests.
```

If you're interested in contributing your work to slurmutils,
take a look at our [contributing guidelines](./CONTRIBUTING.md) for further details.

## 🤝 Project and community

slurmutils is a project of the [Ubuntu High-Performance Computing community](https://ubuntu.com/community/governance/teams/hpc).
Interested in contributing bug fixes, new editors, documentation, or feedback? Want to join the Ubuntu HPC community? You’ve come to the right place 🤩

The `slurmutils` package is a project of the
[Ubuntu HPC](https://discourse.ubuntu.com/t/high-performance-computing-team/35988) community.
It is an open-source project that is welcome to community involvement, contributions, suggestions, fixes,
and constructive feedback. Interested in being involved with the development of `slurmutils`?
Check out these links below:
Here’s some links to help you get started with joining the community:

* [Join our online chat](https://matrix.to/#/#ubuntu-hpc:matrix.org)
* [Code of Conduct](https://ubuntu.com/community/code-of-conduct)
* [Ubuntu Code of Conduct](https://ubuntu.com/community/ethos/code-of-conduct)
* [Contributing guidelines](./CONTRIBUTING.md)
* [Join the conversation on Matrix](https://matrix.to/#/#hpc:ubuntu.com)
* [Get the latest news on Discourse](https://discourse.ubuntu.com/c/hpc/151)
* [Ask and answer questions on GitHub](https://github.com/orgs/charmed-hpc/discussions/categories/q-a)

## License
## 📋 License

The `slurmutils` package is free software, distributed under the GNU Lesser General Public License, v3.0.
See the [LICENSE](./LICENSE) file for more information.
slurmutils is free software, distributed under the GNU Lesser General Public License, v3.0.
See the [LGPL-3.0 LICENSE](./LICENSE) file for further details.
23 changes: 13 additions & 10 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,14 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "slurmutils"
version = "0.4.0"
version = "0.5.0"
description = "Utilities and APIs for interfacing with the Slurm workload manager."
repository = "https://github.com/canonical/slurmutils"
authors = ["Jason C. Nucciarone <[email protected]>"]
maintainers = ["Jason C. Nucciarone <[email protected]>"]
repository = "https://github.com/charmed-hpc/slurmutils"
authors = ["Jason C. Nucciarone <[email protected]>"]
maintainers = [
"Jason C. Nucciarone <[email protected]>",
"Ubuntu High-Performance Computing <[email protected]>"
]
license = "LGPL-3.0-only"
readme = "README.md"
keywords = ["HPC", "administration", "orchestration", "utility"]
Expand All @@ -41,7 +44,7 @@ classifiers=[
python = ">=3.8"

[tool.poetry.urls]
"Bug Tracker" = "https://github.com/canonical/slurmutils/issues"
"Bug Tracker" = "https://github.com/charmed-hpc/slurmutils/issues"

# Testing tools configuration
[tool.coverage.run]
Expand All @@ -66,6 +69,9 @@ target-version = ["py38"]
# Linting tools configuration
[tool.ruff]
line-length = 99
extend-exclude = ["__pycache__", "*.egg_info"]

[tool.ruff.lint]
select = ["E", "W", "F", "C", "N", "D", "I001"]
extend-ignore = [
"D203",
Expand All @@ -80,9 +86,6 @@ extend-ignore = [
"D409",
"D413",
]
ignore = ["E501", "D105", "D107"]
extend-exclude = ["__pycache__", "*.egg_info", "__init__.py"]
ignore = ["E501", "D107"]
per-file-ignores = {"tests/*" = ["D100","D101","D102","D103","D104"]}

[tool.ruff.mccabe]
max-complexity = 15
mccabe = { "max-complexity" = 15 }
6 changes: 0 additions & 6 deletions renovate.json

This file was deleted.

4 changes: 2 additions & 2 deletions slurmutils/editors/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@

"""Editors for Slurm workload manager configuration files."""

from . import slurmconfig
from . import slurmdbdconfig
from . import slurmconfig as slurmconfig
from . import slurmdbdconfig as slurmdbdconfig
Loading