Skip to content

Commit

Permalink
docs: Revise contribution and documentation overview (#1265)
Browse files Browse the repository at this point in the history
  • Loading branch information
@sasa-tomic
sasa-tomic authored Feb 4, 2025
1 parent ccceed7 commit c211ebd
Show file tree
Hide file tree
Showing 4 changed files with 484 additions and 144 deletions.
207 changes: 108 additions & 99 deletions docs/contributing.md
View file
Original file line number Diff line number Diff line change
@@ -1,167 +1,176 @@
# Pre-requisites
# Contributing to DRE

## Rye installation
Thank you for your interest in contributing to the Decentralized Reliability Engineering (DRE) project. This guide will help you set up your development environment and understand our contribution process.

Rye is a comprehensive project and package management solution for Python.
Rye provides a unified experience to install and manages Python installations,
pyproject.toml based projects, dependencies and virtualenvs seamlessly.
## Table of Contents

Run the following from the repo root:
1. [Development Environment Setup](#development-environment-setup)
2. [Project Structure](#project-structure)
3. [Development Workflow](#development-workflow)
4. [Running Tests](#running-tests)
5. [Submitting Changes](#submitting-changes)

```bash
curl -sSf https://rye.astral.sh/get | bash
```
## Development Environment Setup

### 1. Python Environment (Rye)

Follow the instructions on screen. Once the install is done,
reopen your shell or run `source "$HOME/.rye/env"`.
[Rye](https://rye.astral.sh/) is our preferred Python environment manager. It provides a unified experience for managing Python installations, dependencies, and virtual environments.

You can make sure all dependencies are installed by running
#### Installation

```bash
rye sync
curl -sSf https://rye.astral.sh/get | bash
source "$HOME/.rye/env" # Add to your shell's RC file
```

And you can enter the `venv` manually if needed by running `. .venv/bin/activate`.
This is typically not needed.

It's sufficient to prefix any command with the following:
#### Project Setup

```bash
rye run <command>
rye sync # Install all dependencies
```

to run the `<command>` with all expected dependencies.

### IDE setup

Point your IDE to the Python interpreter inside `.venv/bin`.
#### Common Rye Commands

### Troubleshooting rye

If you face problems in `rye sync`, such as `unknown version cpython@...`, you can try to

* List all available toolchains
```
rye toolchain list --include-downloadable
```bash
rye run <command> # Run a command with project dependencies
rye show # Show current environment info
rye toolchain list --include-downloadable # List available Python versions
```

* Upgrade rye itself
```
rye self update
```
### 2. IDE Configuration

* Ensure rye python is in path
```
which python3
```
Configure your IDE to use the Python interpreter from `.venv/bin/python`. This ensures consistent development settings across the team.

* Show the actively used rye environment in the project
```
rye show
```
#### Troubleshooting Rye

If you encounter issues:
1. Update Rye: `rye self update`
2. Verify Python path: `which python3`
3. Check environment: `rye show`
4. List toolchains: `rye toolchain list --include-downloadable`

### 3. Install pre-commit
### 3. Pre-commit Hooks

Install and enable pre-commit. It's highly recommended in order to prevent pushing code to github that will surely cause failures.
We use pre-commit hooks to ensure code quality and consistency.

```
# cd ~/src/dre
```bash
rye run pre-commit install
```

More detailed instructions at https://pre-commit.com/#installation .
For more information, visit the [pre-commit documentation](https://pre-commit.com/#installation).

### 4.a Install cargo (optional)
### 4. Rust Development Setup (Optional)

If you build with cargo, and not with bazel, you need an installation of `rustup` and `cargo`. You can follow the instructions from https://www.rust-lang.org/tools/install
This is typically as simple as running
If you plan to work on Rust components:

```sh
#### Install Rust Toolchain
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
#### On Linux
```sh
command -v apt && sudo apt install -y clang mold protobuf-compiler || true

#### System Dependencies

For Linux:
```bash
sudo apt install -y clang mold protobuf-compiler
```
#### On Mac OS
No need to install Clang for Mac OS user since it comes with Xcode.
```sh
brew install mold protobuff

For macOS:
```bash
brew install mold protobuf
```
Make sure you add `$HOME/.cargo/bin` to your PATH, as written in the page above.
> In the Rust development environment, all tools are installed to the ~/.cargo/bin directory, and this is where you will find the Rust toolchain, including rustc, cargo, and rustup.

### Check the Rust / Cargo installation
Add Cargo to your PATH:
```bash
export PATH="$HOME/.cargo/bin:$PATH" # Add to your shell's RC file
```

To check if your Rust tooling is set up correctly, you can go to the repo root and then
```sh
#### Verify Rust Setup
```bash
cd rs
cargo check
```

This should succeed.
### 5. Node.js and Yarn

### 4.b Install bazel
Required for frontend development:

To install bazel, do not use the version provided by your OS package manager. Please make sure you use [bazelisk](https://bazel.build/install/bazelisk).

## 5. Install nvm, node, yarn

### 1. Install nvm

https://github.com/nvm-sh/nvm#installing-and-updating

### 2. Install node
1. Install NVM:
```bash
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
```

```sh
2. Install Node.js:
```bash
nvm install 14
nvm use 14
```

### 3. Install yarn

```sh
3. Install Yarn:
```bash
npm install --global yarn
```

### "No disk space left" when building with Bazel on Linux?
## Project Structure

```
sudo sysctl -w fs.inotify.max_user_watches=1048576
```
The DRE repository is organized into several key components:

Bazel eats up a lot of inotify user watches.
- `/dashboard` - Internal DRE dashboard (frontend and backend)
- `/rs` - Rust implementations
- `/pylib` - Python libraries
- `/docs` - Project documentation
- `/k8s` - Kubernetes configurations
- `/scripts` - Utility scripts

# IC Network Internal Dashboard
## Development Workflow

## Pre-requisites
1. Create a new branch for your feature/fix
2. Make your changes
3. Ensure all tests pass
4. Submit a pull request

### 1. Install cargo-watch
## Running Tests

```sh
cargo install cargo-watch
### Backend Tests
```bash
rye run pytest
```

### 2. Install yarn dependencies

### Frontend Tests
```bash
cd dashboard
yarn test
```

## IC Network Internal Dashboard

### Setup
```bash
cd dashboard
yarn install
```

## Running

To start the release dashboard locally, run the following from dashboard folder
### Development
```bash
yarn dev # Starts development server
```

```sh
yarn dev
### Using DRE CLI with Local Dashboard
```bash
dre --dev subnet replace --id <subnet-id> -o1
```

To use the `dre` CLI tool with the local dashboard instance run it with `--dev` flag.
## Common Issues

E.g.
### Linux: "No disk space left" with Bazel

```sh
dre --dev subnet replace --id <subnet-id> -o1
If you encounter inotify issues:
```bash
sudo sysctl -w fs.inotify.max_user_watches=1048576
```

## Need Help?

- Check existing [GitHub Issues](https://github.com/dfinity/dre/issues)
- Join our developer community
- Review our [documentation](https://dfinity.github.io/dre/)
67 changes: 65 additions & 2 deletions docs/index.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,66 @@
# Repo description
# Decentralized Reliability Engineering (DRE)

This repo holds the tools and documentation for IC Decentralized Reliability Engineering (DRE). We manage the IC Mainnet, topology, observability and monitoring.
Welcome to the documentation for DFINITY's Decentralized Reliability Engineering (DRE) tools and processes. This documentation covers everything you need to know about managing, monitoring, and maintaining the Internet Computer network.

## What is DRE?

DRE (Decentralized Reliability Engineering) is a comprehensive suite of tools and practices designed to manage the Internet Computer network in a decentralized manner. Our tools help maintain network reliability, manage updates, and ensure the smooth operation of the Internet Computer Protocol (ICP).

## Key Components

### DRE CLI Tool
A command-line interface tool that provides essential functionality for:
- Managing subnet configurations
- Handling node operations
- Executing network updates
- Monitoring system health

### Internal DRE Dashboard
A web-based interface offering:
- Real-time network monitoring
- Deployment management
- System metrics visualization
- Operational status tracking

### Service Discovery
Maintains an up-to-date registry of:
- IC network targets
- Log aggregation endpoints
- Metrics collection points

### Log Management
Comprehensive logging solutions for:
- Host nodes
- Guest nodes
- Boundary nodes
- Canister operations

## Getting Started

1. [Installation Guide](getting-started.md) - Set up the DRE tools and environment
2. [Contributing Guide](contributing.md) - Learn how to contribute to the project
3. [Documentation Guide](how-to-update-docs.md) - Help improve our documentation

## Core Features

- **Release Management**: Coordinate and execute network updates
- **Subnet Management**: Configure and maintain network subnets
- **Monitoring**: Track network health and performance
- **Decentralization**: Support the Internet Computer's decentralized architecture

## Quick Links

- [Create a Neuron](create-a-neuron.md)
- [Submit NNS Proposals](nns-proposals.md)
- [Run Qualification Tests](qualification/running-qualification.md)
- [Manage Releases](make-release.md)

## Resources

- [GitHub Repository](https://github.com/dfinity/dre)
- [Latest Releases](https://github.com/dfinity/dre/releases)
- [Issue Tracker](https://github.com/dfinity/dre/issues)

## License

This project is licensed under the [Apache 2.0 License](https://github.com/dfinity/dre/blob/main/LICENSE).
Loading

0 comments on commit c211ebd

Please sign in to comment.