Skip to content

Commit

Permalink
Switch from python module to install to module to run
Browse files Browse the repository at this point in the history
  • Loading branch information
frodrigo committed Feb 22, 2023
1 parent ad7ce15 commit 5b8fc36
Show file tree
Hide file tree
Showing 7 changed files with 54 additions and 45 deletions.
21 changes: 20 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,15 @@ Il faut pour alimenter la base OSM locale dans laquelle puise BANO :

Autres outils : parallel.

## Installation

Mettre en place un environnement virtuel python :
```
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
```

## Configuration

Première étape avant de lancer les chargements de données : il faut adapter le fichier `config` à votre environnement, en déclarant différents chemins. Ce fichier est utilisé en début de plusieurs scripts pour connaître le chemin de différents répertoires.
Expand Down Expand Up @@ -55,6 +64,16 @@ load_osm_france_db.sh http://download.openstreetmap.fr/extracts/europe/france/fr

## Utilisation

Pour connaître les commandes du module bano : `bano --help`.
### Commande `bano`

Activer l'environnement virtuel python (si pas déjà activé) :
```
source venv/bin/activate
```

Pour connaître les commandes du module bano :
```
python -m bano --help
```

Quasiment toutes les options sont utilisées dans le script `cron_bano`.
Empty file removed bano/__init__.py
Empty file.
8 changes: 6 additions & 2 deletions bano/bin.py → bano/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ def main():

subparser = subparsers.add_parser(
"process_commune_lieux-dits",
help="Traite unitairement les lieux-dits d'une commune",
help="Traite unitairement les lieux-dits d'une commune",
description="Traite unitairement les lieux-dits d'une commune"
)
group = subparser.add_mutually_exclusive_group(required=True)
Expand All @@ -54,7 +54,7 @@ def main():

subparser = subparsers.add_parser(
"ban2fantoir",
help="Ajoute dans FANTOIR des libellés BAN",
help="Ajoute dans FANTOIR des libellés BAN",
description="Ajoute dans FANTOIR des libellés BAN"
)
subparser.add_argument(
Expand Down Expand Up @@ -205,3 +205,7 @@ def main():
import ipdb

ipdb.set_trace()


if __name__ == "__main__":
main()
45 changes: 23 additions & 22 deletions cron_bano.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,18 @@ source config

./arborescence.sh

pip install -e .

bano="python -m bano"

# Sources
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano download_bal CADASTRE {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano download_bal BAL {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano download_ban {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano download_cadastre lieux_dits {1}
# bano update_code_cadastre
bano download_commune_summary
bano update_bis_table
bano remove_chars_in_ban
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano download_bal CADASTRE {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano download_bal BAL {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano download_ban {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano download_cadastre lieux_dits {1}
# $bano update_code_cadastre
$bano download_commune_summary
$bano update_bis_table
$bano remove_chars_in_ban
./copy_table_from_osm_to_cadastre.sh rep_b_as_bis


Expand All @@ -31,17 +32,17 @@ bano remove_chars_in_ban
# le reste de la semaine, ciblage des communes ayant eu des modifs dans OSM dans les dernières 24h
if [ `date +%u` -eq 7 ]
then
bano update_insee_lists
cat insee_locales.csv | parallel -j 4 export LANG=$LANG\; bano process_commune BAL --code_insee {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano process_commune BAN --depts {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano process_commune OSM --depts {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano process_commune_lieux-dits --depts {1}
$bano update_insee_lists
cat insee_locales.csv | parallel -j 4 export LANG=$LANG\; $bano process_commune BAL --code_insee {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano process_commune BAN --depts {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano process_commune OSM --depts {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano process_commune_lieux-dits --depts {1}
else
bano update_insee_lists
cat insee_locales.csv | parallel -j 4 export LANG=$LANG\; bano process_commune BAL --code_insee {1}
cat insee_osm.csv | parallel -j 4 export LANG=$LANG\; bano process_commune BAN --code_insee {1}
cat insee_osm.csv | parallel -j 4 export LANG=$LANG\; bano process_commune OSM --code_insee {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; bano process_commune_lieux-dits --depts {1}
$bano update_insee_lists
cat insee_locales.csv | parallel -j 4 export LANG=$LANG\; $bano process_commune BAL --code_insee {1}
cat insee_osm.csv | parallel -j 4 export LANG=$LANG\; $bano process_commune BAN --code_insee {1}
cat insee_osm.csv | parallel -j 4 export LANG=$LANG\; $bano process_commune OSM --code_insee {1}
cat deplist.txt | parallel -j 4 export LANG=$LANG\; $bano process_commune_lieux-dits --depts {1}
fi

# Mise à jour quotidienne dans la base cadastre des couches des polygones postaux d'OSM et des statuts admin de communes en vue des exports
Expand All @@ -58,11 +59,11 @@ $pgsql_OSM -f sql/create_table_polygones_postaux.sql
$pgsql_CADASTRE -f sql/post_copie_ban_odbl.sql

# exports
cat deplist.txt | parallel -j 4 bano export {1}
cat deplist.txt | parallel -j 4 $bano export {1}

# copie+zip dans le dossier web
cat deplist.txt | parallel -j 4 bano publish {1}
bano publish_aggregate
cat deplist.txt | parallel -j 4 $bano publish {1}
$bano publish_aggregate

# ménage PostgreSQL
$pgsql_CADASTRE -c "VACUUM cumul_adresses;"
Expand Down
5 changes: 5 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
beautifulsoup4==4.6.3
psycopg2-binary==2.9.3
requests==2.21.0
lxml
ipdb==0.13.9
18 changes: 0 additions & 18 deletions setup.cfg

This file was deleted.

2 changes: 0 additions & 2 deletions setup.py

This file was deleted.

0 comments on commit 5b8fc36

Please sign in to comment.