powerapi-ng · roda82 · Oct 4, 2024 · Oct 4, 2024 · Oct 7, 2024 · Oct 8, 2024
diff --git a/README.md b/README.md
@@ -2,3 +2,23 @@
 PowerAPI website use [MKDocs](https://www.mkdocs.org/). 
 
 You can push your changes on `master` branch to deploy them on the website. 
+
+## Local development  
+
+When redacting changes for this documentation, you may want to visualize the 
+output the way it will be presented in the final website once compiled.  
+
+One way to do so is to:  
+
+1. Install the necessary packages : mkdocs, mkdocs-material, mkdocs-material-extensions  
+This can be done with: `pip install mkdocs mkdocs-material mkdocs-material-extensions`  
+
+2. Have a local copy of this repository (taking care of *checkout-ing* the right ref)  
+
+3. From the CLI, have this particular repository as PWD  
+
+4. Use mkdocs in order to serve locally (default on http://localhost:8000)  
+This can be done with: `mkdocs serve -o`  
+
+5. Check the URL it serves to, targets will be rebuild on modifications in the 
+current directory or in any nested elements (use -W to add paths to be considered for hot reloading)  
diff --git a/docs/assets/images/reference/sensors/PowerAPI_HWPCSensorOverview.drawio b/docs/assets/images/reference/sensors/PowerAPI_HWPCSensorOverview.drawio
diff --git a/docs/assets/images/reference/sensors/PowerAPI_HWPCSensorOverview.drawio.svg b/docs/assets/images/reference/sensors/PowerAPI_HWPCSensorOverview.drawio.svg
diff --git a/docs/getting_started.md b/docs/getting_started.md
@@ -1,57 +1,92 @@
 # Getting started
 
-If you want to monitor the energy consumption of your process we have some
-ready-to-use tools
+In this tutorial, we will guide you through the first steps to get started with PowerAPI.
+The objective is to get a quick view of the capabilities of PowerAPI, by monitoring a process and getting a quick glimpse at the energy consumption.
+A few things are required before we start: 
 
-???+ info "Source and Destination"
-    In order to use any Formula, you need to run a Source and a Destination. The former is used by a Sensor to store metrics. The later allows the Formula to make available the estimations. For starting, you can use [MongoDB](https://hub.docker.com/_/mongo) as Source and [InfluxDB:2.X](https://hub.docker.com/_/influxdb) as Destination by installing them as Docker containers.
-    For more details about Sources and Destinations please check this [section](reference/database/sources_destinations.md).
+- A compatible processor (you can see the compatible CPU architecture [here](./reference/sensors/hwpc-sensor.md#)), and you can look on the following pages to find your CPU architecture:  
+    * For [Intel Processor](https://en.wikipedia.org/wiki/List_of_Intel_processors)  
+    * For [Intel Xeon Processor](https://en.wikipedia.org/wiki/List_of_Intel_Xeon_processors)  
+    * For [AMD Processor](https://en.wikipedia.org/wiki/Table_of_AMD_processors)  
+- A python installation ready
+- Docker & Docker-Compose ready (refer to [this official documentation](https://docs.docker.com/engine/install/) and the [post-install steps](https://docs.docker.com/engine/install/linux-postinstall/) if needed !)
+- Root access
+- Optional : [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) to proceed by cloning the repository
 
+## Which components to get a complete stack  
 
-<!---
-## **RAPL Formula**
+If you wish to get started as soon as possible, the archive below will allow you to deploy the following elements:  
 
-!!! note ""
-    for monitoring the energy consumption of your device
+1. A MongoDB instance to store the [Sensor](./reference/sensors/hwpc-sensor.md)
+Reports
 
-RAPL Formula is made for tracking the energy consumption of your machine.
-To install RAPL Formula on a baremetal server or a PC run [the following
-script](script/rapl_install.sh) in a Terminal.
+3. An [HWPC-Sensor](./reference/sensors/hwpc-sensor.md) that outputs its 
+[HWPC Reports](./reference/reports/reports.md#hwpc-reports) in a MongoDB Database, 
+within the HWPC Report Collection
 
-The script explains what it will do and then pauses before it does it.
+4. A [SmartWatts](./reference/formulas/smartwatts.md) that streams the 
+[HWPC Reports](./reference/reports/reports.md#hwpc-reports) from the MongoDB 
+Database Collection, processes it and outputs its 
+[Power Reports](./reference/reports/reports.md#power-reports) as CSV files for a 
+quick glimpse 
 
-Please notice that you need a **Linux distribution** in order to use the HWPC Sensor installed by the script as
-well as a **comptible Intel** (Sandy Bridge and newer) or **AMD Processor** (Zen). **Power/ARM/RISCV are not supported** architectures. HWPC Sensor will **not work on a Virtual Machine**. However, you can install the Formula by hand in a Virtual Machine if need it.
--->
+## Preparation
 
-## **SmartWatts Formula**
+You can either download the archive by cloning the repository or using wget.
 
-!!! note ""
-    for monitoring the power consumption of your process
+=== "git"
+    ```sh 
+    git clone https://github.com/powerapi-ng/powerapi-ng.github.io.git
+    cd powerapi-ng.github.io/docs/script/getting-started
+    ```
 
-Smartwatts is made for tracking the power consumption of processes on a
-machine.
-To install Smartwatts on a baremetal server or a PC run [the following
-script](script/smartwatts_install.sh) in a Terminal. Please notice that you will need [pip](https://pip.pypa.io/en/stable/installation/) or [docker](https://docs.docker.com/engine/install/) in order to use the Formula.
+=== "wget"
+    ```
+    wget -c https://raw.githubusercontent.com/powerapi-ng/powerapi-ng.github.io/refs/heads/master/docs/script/getting_started.tar.gz -O - | tar -xz
+    cd getting_started
+    ```
 
-The script explains what it will do and then pauses before it does it.
+At this stage, you will have all the essential files to begin. Let's go through each element in detail.
+### Archive content
 
-Please notice that you need a **Linux distribution** in order to use the HWPC Sensor installed by the script as
-well as a **comptible Intel** (Sandy Bridge and newer) or **AMD Processor** (Zen). You also need [docker](https://docs.docker.com/engine/install/).  **Power/ARM/RISCV are not supported** architectures. HWPC Sensor will **not work on a Virtual Machine**. However, you can install the Formula by hand in a Virtual Machine if need it.
+```sh
+getting_started/
+ |--csv/
+ |--formula/
+ |----smartwatts-mongodb-csv.json
+ |--sensor/
+ |----hwpc-mongodb.json
+ |--start.sh
+ |--start.py
+ |--stop.sh
+ |--pretty_print.py
+ |--docker-compose.yaml
+ |--.env
+```
 
+#### HWPC-Sensor and SmartWatts Configuration
 
+As described in the [HWPC-Sensor Documentation](./reference/sensors/hwpc-sensor.md#global-parameters) and in the [SmartWatts Documentation](./reference/formulas/smartwatts.md#global-parameters), 
+several parameters can be set, both globally and for specific Groups monitored for the sensor or the formula.
 
-#### CGroups
-If you need to monitor a process or a group of process via SmartWatts by using HWPC Sensor **version 1.2 or older**, you can follow this [tutorial](reference/cgroup/cgroup.md). Please notice that **cgroup V1** is required **only** for HWPC Sensor **version 1.2 or older**. If you need to enable this `cgroup` version please follow this [tutorial](reference/cgroup/cgroup_v1_activation.md).    
+The provided docker-compose.yaml file uses configuration files and the **.env** to set those parameters.
+You can find examples of both those configuration files in the archive under the **formula** and **sensor** directories.
 
-<!---
-## **Jouleit**
 
-!!! note ""
-    for mesuring the energy consumption of a program
+## Turn the key 
 
-Jouleit is made for tracking the energy consumption of a program.
-Jouleit need `gawk` to run.
-You can get the script from the [github repository](https://github.com/powerapi-ng/jouleit)
-Start jouleit by using `./jouleit.sh cmd`.
--->
+Once all set, you shall be able to initiate the stack with:  
+
+```sh
+python3 start.py
+```
+
+After the 2 minutes of monitoring, you will be able to see the result inside the **csv** directory.
+If you have trouble understanding the output, you can read the [Power Report documentation](./reference/reports/reports.md#power-reports).
+
+!!! info "Quick results overview"
+    Only in the context of this testing archive, after the monitoring, you can use the following command to get a pretty print of the result directly inside the terminal.  
+
+    ```sh
+    python3 pretty_print.py
+    ```
diff --git a/docs/reference/database/sources_destinations.md b/docs/reference/database/sources_destinations.md
@@ -1,19 +1,23 @@
-# Sources and Destinations
+# Storage Options
 
-A PowerAPI Formula uses Sources and Destinations in order to retrieve metrics and store estimations.
+Different storage options are available to serve different purpose both for [Sensors](../overview.md#Sensor) and [Formulas](../overview.md#Formula).  
 
-For each Source/Destination the parameters to specify are different. For each one of them,
-its parameters are specified in following sections.
+Storage is needed to save reports produced by each components.  
+- Sensors store their usage reports  
+- Formulas retrieve usage reports and store energy consumption reports  
+- Visualization tools or individuals need to access reports for analysis  
 
 ## Summary
-| Name     | Source   | Destination  | CLI `input`/`ouput` parameter value                                      | JSON `type` tag parameter value                             |
-| ------------- | -----  | ------------- | -------------                                      | ------------------------------------    |
-| MongoDB | Yes  | Yes | mongodb                                      | mongodb    |
-| InfluxDB2 | No  | Yes | influxdb2                                      | influxdb2    |
-| CSV | Yes  | Yes | csv                                      | csv    |
-| Socket | Yes  | No | socket                                      | socket    |
-| File Database | Yes  | Yes | filedb                                      | filedb    |
-| Prometheus | No  | Yes | prometheus                                      | prometheus    |
+
+The following table defines the existing storage options for Sensors usage reports :  
+
+| Name      | CLI `ouput` parameter value  | JSON `type` tag parameter value|
+| ------------ | --------------------------------------| -------------------------------------------|
+| MongoDB | mongodb | mongodb |
+| CSV |  csv | csv |
+| Socket | socket | socket    |
+| File Database | filedb | filedb |
+
 
 ## MongoDB
 
@@ -30,7 +34,7 @@ The list of accepted parameters are:
 |`db` (`database` for `HWPCSensor`)           | string | `d` (`D` for `HWPCSensor`)            | N/A | Yes                                                       | The name of your database               |
 |`collection`   | string | `c` (`C` for `HWPCSensor`)          | N/A | Yes                                                       | The name of the collection inside `db`  |
 |`name`         | string | `n`           | `"puller_mongodb"` (Source), `pusher_mongodb` (Destination)| No | The related puller/pusher name. This parameter is not used by `HWPCSensor`                 |
-|`model`        | string | `m`           | `"HWPCReport"` (Source), `PowerReport` (Destination) | No         | The Report type stored by the database  |
+|`model`        | string | `m`           | `"HWPC Report"` (Source), `Power Report` (Destination) | No         | The Report type stored by the database  |
 
 ### JSON File Excerpt
 
@@ -64,7 +68,7 @@ The list of accepted parameters are:
 |`org`          | string | `g`           | N/A           | Yes | The name of the organization associated to the bucket               |
 |`tags`         | string | `t`           | N/A           | No | List of metadata keys of the report separated by `,` that will be kept. `sensor` and `target` are always kept as report metadata                           |
 |`name`         | string | `n`           | `"pusher_influxdb2"` | No                                    | The related pusher name                 |
-|`model`        | string | `m`           | `"PowerReport"`  | No | The Report type stored by the database  |
+|`model`        | string | `m`           | `"Power Report"`  | No | The Report type stored by the database  |
 
 
 InfluxDB2 can only be used as a Destination.
@@ -75,7 +79,7 @@ Below you find an example of configuration excerpt for this kind of Destination.
 
 ```json
 {
-  "model": "PowerReport",
+  "model": "Power Report",
   "type": "influxdb2",
   "uri": "http://127.0.0.1",
   "port": 8086,
@@ -100,7 +104,7 @@ The list of accepted parameters are:
 |`files`(Source)| string  | `f`           | Empty list           | No | The list of input CSV files with the format file1,file2,file3...              |
 |`directory` (Destination and `HWPCSensor`)| string         |`d` (`U` for `HWPCSensor`)          | Current directory           | No |The directory where output CSV files will be written          |
 |`name`         | string | `n`           | `"puller_csv"` (Source), `"pusher_csv"` (Destination)| No | The related puller/pusher name. This parameter is not used by `HWPCSensor`                 |
-|`model`        | string | `m`           | `"HWPCReport"` (Source), `"PowerReport"` (Destination)   | No | The Report type stored in CSV files. This parameter is not used by `HWPCSensor`     |
+|`model`        | string | `m`           | `"HWPC Report"` (Source), `"Power Report"` (Destination)   | No | The Report type stored in CSV files. This parameter is not used by `HWPCSensor`     |
 
 ### JSON File Excerpt
 
@@ -128,7 +132,7 @@ The list of accepted parameters are:
 |`port`         | int    | `P`           | N/A | Yes                                               | The port of communication               |
 |`uri`/ `host`         | int    | `U`           | N/A | Yes                                               | The IP address of the machine running the socket               |
 |`name`         | string | `n`           | `"puller_socket"`| No | The related puller name  |
-|`model`        | string | `m`           | `"HWPCReport"` | No | The Report type managed by the socket |
+|`model`        | string | `m`           | `"HWPC Report"` | No | The Report type managed by the socket |
 
 
 ### JSON File Excerpt
@@ -158,7 +162,7 @@ The list of accepted parameters are:
 | ------------- | -----  | ------------- | -------------| ----------                                        | ------------------------------------    |
 |`filename`     | int    | `f`           | N/A                                                | Yes | The name of the file                    |
 |`name`         | string | `n`           | `"pusher_filedb"` | No | The related pusher name |
-|`model`        | string | `m`           | `"HWPCReport"` (Source) `"PowerReport"` (Destination)| No  | The Report type stored in the file      |
+|`model`        | string | `m`           | `"HWPC Report"` (Source) `"Power Report"` (Destination)| No  | The Report type stored in the file      |
 
 ### JSON File Excerpt
 
@@ -188,7 +192,7 @@ The list of accepted parameters are:
 |`metric-name`  | string | `M`           | N/A | Yes                                              | The exposed metric name                    |
 |`metric-description`  | string | `d`    | `"energy consumption"` | No                             | The exposed metric description                    |
 |`name`         | string | `n`           | `"pusher_prom"` | No | The related pusher name                 |
-|`model`        | string | `m`           | `"PowerReport"` | No | The Report type exposed by Prometheus       |
+|`model`        | string | `m`           | `"Power Report"` | No | The Report type exposed by Prometheus       |
 
 
 Prometheus can only be used as a Destination that monitors reports but they will be not stored by this service.

diff --git a/docs/reference/formulas/configuration_files.md b/docs/reference/formulas/configuration_files.md
@@ -11,14 +11,11 @@ The table below shows basic parameters.
 | `verbose`                            | `bool` (flag)               | `v`           |`NOTSET`                              | Verbose or quiet mode                |
 | `stream`                             | `bool` (flag)  | `s`           |`False`                               | Real time or post-mortem mode         |
 | `sensor-report-sampling-interval`    | `int`         | N/A           | `1000`                                 | The time in milliseconds between two reports (`stream` = `True`) |
-| `input`     | `string` (Source)      | N/A | N/A     | Source used as input. More information about Sources and their related parameters can be found [here](../database/sources_destinations.md)|
-| `output`     | `string` (Destination)| N/A | N/A            | Destination used as output. More information about Destinations and their related parameters can be found [here](../database/sources_destinations.md)|
+| `input`     | `string`      | N/A | N/A     | SmartWatts input, shall match an existing Sensor output and contain HPWCReports. See [here](./smartwatts.md#smartwatts-inputs) |
+| `output`     | `string` | N/A | N/A            | SmartWatts output to store Power Report. See [here](./smartwatts.md#smartwatts-outputs) |
 | `pre-processor`     | `string` | N/A | N/A            | Pre-Processor to modify reports generated by a sensor. More information about Processors and their related parameters can be found [here](../processors/processors.md) |
 | `post-processor`     | `string` | N/A | N/A            | Post-Processor to modify reports generated by a formula. More information about Processors and their related parameters can be found [here](../processors/processors.md) |
 
-???+ tip "Sources and Destinations' values"
-    - Sources: `mongodb`, `csv`, `socket`, `filedb`
-    - Destinations: `mongodb`, `influxedb`, `influxedb2`, `csv`, `socket`, `filedb`, `prom`
 
 ???+ tip "Processors' values"
     - Pre-Processors: `k8s`, `libvirt`
@@ -52,5 +49,5 @@ PowerAPI Formulas use `json` files. These files follow the next template:
   $formula_parameters
 }
 ```
-???+ info "Sources and Destinations' `json` tags"
+???+ info "input and output' `json` tags"
     More information related to `json` tags for each Source/Destination can be found [here](../database/sources_destinations.md)