Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify pipeline #39

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
54 changes: 24 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,47 @@
![pytest](https://github.com/N3PDF/pycompressor/workflows/pytest/badge.svg)
[![documentation](https://github.com/N3PDF/pycompressor/workflows/docs/badge.svg)](https://n3pdf.github.io/pycompressor/)

### pycompressor
## pycompressor

Fast and efficient python implementation of PDF set **compressor** (https://arxiv.org/abs/1504.06469).
Fast and efficient python implementation of PDF **compression** (https://arxiv.org/abs/1504.06469).

#### New features

Additional new features have been added to the following python package. The two main features are:
- **Covariance Matrix Adaptation-Evlotion strategy (CMA-ES):** in addition to the Genetic
Algorithm (GA), there is now the possibility to choose as a minimizer the CMA. The choice
of minimizer can be defined in the `runcard.yml` file.
- **Generative Adversarial Strategy (GANs):** this is a standalone python [package](https://github.com/N3PDF/ganpdfs/tree/master)
that can enhance the statistics of the prior PDF replicas before compression by generating
synthetic replicas. For more details, refer to the [documentation](https://n3pdf.github.io/ganpdfs/)
(still has to be done). In a similar way, in order to trigger the enhancement, one just has to set
the value of `enhance` in the runcard to be `True`. Setting this value to `False` will just run the
standard compression. The GANs also requires extra-parameters (as shown in the example
[runcard.yml](https://github.com/N3PDF/pycompressor/blob/master/runcard.yml)) that defines
the structure of the networks.

#### Installation
### How to install

To install `pyCompressor`, just type:
```bash
python setup.py install
```
or if you are a developer:
```bash
python setup.py develop
python setup.py install # or python setup.py develop (if you want development mode)
```

#### How to use
### How to use

#### Standard compression

The input parameters that define the compression is contained in a YAML file. To run
the `pycompressor` code, just type the following:
The input parameters that define the compression is contained in a YAML file. To run the standard compression,
Radonirinaunimi marked this conversation as resolved.
Show resolved Hide resolved
use the reference [runcard](https://github.com/N3PDF/pycompressor/blob/master/runcards/runcard.yml) as it is by
just replacing the entry of the `pdf` key with the name of the PDF set, then run the following:
Radonirinaunimi marked this conversation as resolved.
Show resolved Hide resolved
```bash
pycomp runcards/runcard.yml [--threads NUMB_THREADS]
```
A detailed instruction on how to set the different parameters in the runcard can be found here.

#### Generating compressed PDF set & post-analysis
#### Using GAN and/or Compressing from an enhanced set

Although it is advised to run the [ganpdfs](https://github.com/N3PDF/ganpdfs) independently, it is possible
to generate enhanced PDF replicas within the `pycompressor`. To do so, just set the entry `enhance` in the
runcard to `True` and specify the total number of replicas (prior+synthetics).

Finally, in order to perform a compression with an enhanced set, set the entry `existing_enhanced` to `True`.

A detailed instruction on how to set the different parameters in the runcard can be found
[here](https://n3pdf.github.io/pycompressor/howto/howto.html).

### Generating compressed PDF set & post-analysis

The code will create a folder named after the prior PDF sets. To generate the
compressed PDF grid, run the following command:
```bash
get-grid -i <PDF_NAME>/compressed_<PDF_NAME>_<NB_COMPRESSED>_output.dat
```
Note that if the compression is done from an enhanced set, the output folder will be append by `_enhanced`.
Note that if the compression is done from an enhanced set, the output folder will be appended by `_enhanced`.

Finally, in order to generate ERF plots, enter in the `erfs_output` directory and run the following:
```bash
Expand All @@ -56,7 +50,7 @@ validate --random erf_randomized.dat --reduced erf_reduced.dat
This script can also plot the ERF validation from the old compressor code by adding the flag
`--format ccomp`.

#### Warning
### Warning

This package cannot be installed with python 3.9 yet due to the numba dependency. This will be resolved
soon according to [#6579](https://github.com/numba/numba/pull/6579).