Testing Datamodel and plugin import

[3nids]

Prerequisites

Postgresql+postgis has to be installed either locally or on your server. Access configuration is not described here but can be found on the postgresql documentation.

QGIS has to be installed in a minimal 3.34.8 LTR Version, preferably with the osgeo4w installer.

Python modules psychopg2, sqlalchemy and geoalchemy2 have to be installed for INTERLIS IMPORT and export. pum and pirogue will be required for upgrades and database initialisations in the future.

Setup

0. Create the roles on the database cluster :
https://github.com/teksi/wastewater/blob/main/datamodel/12_0_roles.sql

1. Install TWW data model using a dump
Download from the latest Release https://github.com/teksi/wastewater/releases/tag/2024.0.0

Or, if you have access, take the last build from the main branch from https://github.com/teksi/wastewater/actions/workflows/datamodel-create-dumps.yml

Use the 'Restore' function on your postgres database installation after creating a new database first and select tww_dev_structure_and_demo_data.backup (there is not demo data in there currently)
On Windows:

2. Install plugin from zip
teksi_wastewater.0.0.0.zip

You need a Java Runtime Environment to run ili2pg

If you do not want to compromise your existing QGIS installation create a new User profile first in TEKSI and install the plugin version there.

3. Download the code (only if you need separately)
https://github.com/teksi/wastewater/archive/refs/tags/2024.0.0.zip

4. Open the project in QGIS
Use the downloaded projet from the Release or
Download current version from https://github.com/teksi/wastewater/blob/main/project/teksi_wastewater.qgs

Click the button "Download raw file" and save locally

Or directly https://raw.githubusercontent.com/teksi/wastewater/main/project/teksi_wastewater.qgs

Create a pg configuration entry with the name: pg_tww to match the project files definition

5. Import from INTERLIS
Open the project file

Go to the plugin settings and change to Admin Mode - else INTERLIS Import / Export buttons will not be displayed.
You can also Add Developers Mode and save the logs to have more data available for debugging.
Save settings.

Save project file - it is suggested to name it with your QGIS version to distinguish from the provided version - and restart QGIS.

Now the INTERLIS Import / Export buttons should be visible.

Using files in this order (files can be found in the code source)

1. plugin/teksi_wastewater/tests/data/test-dataset-organisations.xtf (use right click - File save as .. to get xtf directly. Needs to be imported first)
2. download the vsa organisation dataset (needs to bei imported second)
3. download test-dataset-DSS.xtf (use right click - File save as .. to get xtf directly. Data - can only succesfully be imported when all organisations referenced are in teksi)

6. Refresh network views
Should now be included in the workflow with the latest plugin version.

7. Run Update Main Cover query manually
Run the update main cover query manually so symbols on layer vw_tww_wastewater_structure display correctly and objects can be selected - see #75
Should now be included in the workflow with the latest plugin version!

Known issues

• INTERLIS import / export issues have been fixed
• Some 3D geometry errors may happen as they are not fully supported yet. (INTERLIS import is now 2.5D)

LAST EDIT ponceta 19.07.2024

[sjib]

When testing please provide the following imformation - you can copy the template below in the new issue:

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Exact steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots / data**
If applicable, add screenshots or data to help explain your problem.

**Desktop (please complete the following information):**
 - TEKSI plugin version
 - TEKSI datamodel version
 - TEKSI project file version
 - QGIS Version - see the About menu
 - Postgres - you can check with `SELECT version();`
 - Postgis - you can check with `SELECT PostGIS_full_version();`
 - OS
 

**Additional context**
Add any other context about the problem here.

---

[cymed]

Also when using the last build from https://github.com/teksi/wastewater/actions/workflows/datamodel-create-dumps.yml, check you take the main branch

[urskaufmann]

length profil works!!!

[urskaufmann]

I can not reproduce no - I get error as before with qgep QGEP/QGEP#816
but I swear: there was a length profile on my screen 3 days ago!

[sjib]

@domi4484 If data is imported also with new plugin version interlis export vsa-dss runs thru without errors, but configuration for vsa-dss is not complete as there are most vsadss 2020 classes missing - see #98.

[sjib]

@domi4484 labels export is still failing - see #101

[urskaufmann]

Project file has a title (TEKSI Wastewater and General Wastewater Management Plan (GEP))
Because of this title, it's not visible in the QGIS-window-titlebar, which project file is open. We prefer to have an empty title in the projectfile.

[urskaufmann]

Creating a child-records if the parent-record is not safed gives a "red" error with a not very understandable errormessage (CRITICAL Layer vw_access_aid : PostGIS-Fehler beim Attributhinzufügen: ERROR: insert or update on table "structure_part" violates foreign key constraint …).

Is there a possibility to give a better message or even better to tell the user to save first.

Interesting: when inserting a new child-record maintenance event, it works also with this similar red error (I know, it's not a real child because there is a re_table between).

[urskaufmann]

Solution: buffered transaction groups, not automatic transaction groups in the project settings

[ponceta]

I'll have a check that it is set as default on the demo project.

[sjib]

Issue opend for this #104

[urskaufmann]

Action Digitize Drainage Channel: there is a blue hint in the map who tells me what to do.
Action Digitize: There is no blue hint. This hint is missing

[sjib]

• Add hint: Digitize on the map @ponceta

[3nids]

This has to be done in the action in the project. I would suggest to create a dedicated ticket for this.

[sjib]

Issue opened: #103

[domi4484]

@sjib @cymed @urskaufmann I just uploaded the newest plugin in the top post. This should fix:

• Missing classes INTERLIS VSA-DSS 2020 Import / Export #98
• Hide logs button when no logs are available. Part of Export SIA 405 Abwasser: Error #82
• Interlis Import only works when the user owns all triggers #87
• Layer vw_network_node and vw_network_segment not available #79 and Add (option) to run query tww_update_maincover after import #75

Still open (at least on interlis front):

• INTERLIS Export VSA-KEK failing #102
• Labels export INTERLIS Export with labels failing #101 and Export SIA 405 Abwasser: Error #82
• Possibility to export from command line missing #78

[3nids]

datamodel dump updated to 8e0f8f5

[ponceta]

Datamodel dump and plugin updated to df6ee39 05.03.2024

[sjib]

New release candidate published: https://github.com/teksi/wastewater/releases/tag/2024.0.rc2

[sjib]

Documentation adapted.

[TriformMoritz]

Hello,
This installation and testing process is so complicated that it takes hours to do (it took me some hours and I didi it for the 2015 version allready).
Testing and installation is, as a consequence not done by many people. If we want people to test, this has to be much easier. I agree the installation of postgresgl (with postis and pgadmin) and Qgis. And the plugin (via Qgis dépôt).
The rest shouldn't be installed, or shipped with the plugin in Qgis

Problems and suggestions:

• The need to install Java. -> Or it should not be used, or it should come with Qgis.
• The need to install the python packages -> Or it should not be used, or it should come with Qgis.
• User/ login rules -> Should be via administrator tool of the plugin.
• New database upset -> Should be via administrator tool of the plugin.
• New QGS file (with translation) and entry into the pg_service file-> Should be via administrator tool of the plugin.

So the steps should only be:

1. Install postgres with postgis and set it up (pga_hba.conf file).
2. Install Qgis.
3. Install the plugin.
4. Use the plugin to set up the rest.

So once postgres is setup, the rest is easy and can be done by anyone inexperienced!

I personnaly had a problem installing postgresql 12 and 15 alongside on a windows 10 computer used as a server.

• I have two versions of pgadmin installed.
• The old version would le me acces the postgres 15 server (no 'user info'
• The new version wold not work because the port was used (maybe by the old pgadmin version).

Workaround was using pgadmin (newest version) from my local computer.

Hope this comment is in the right spot. Please tell me if I have to move it.

Moritz

[sjib]

New pre-release added https://github.com/teksi/wastewater/releases/tag/2024.0.1

[sjib]

@ponceta @cymed @urskaufmann
Is the additional installation of the linking relationship manager done automatically also from the zip install?

I have it installed by zip and then see it:

How to deal with the 'Document Management System'?

[sjib]

[cymed]

All I did was add the names in the metadata. I lack the experience to know in detail where the problem lies. Maybe @domi4484 can help?

[domi4484]

Strange the plugin name is spelled right but for some reason the plugin manager is not finding it. Maybe it doesn't strip the space after ,. I'll try to find out: #400