Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generalize extensions #23

Merged
merged 10 commits into from
Apr 10, 2024
Merged

Generalize extensions #23

merged 10 commits into from
Apr 10, 2024

Conversation

antoinebhs
Copy link
Contributor

@antoinebhs antoinebhs commented Mar 21, 2024

Please check if the PR fulfills these requirements

  • The commit message follows our guidelines

What kind of change does this PR introduce?

It adds the ability to handle extensions discovered by service loader in the network store in a separate table "extension" instead of storing the extension with the identifiable. This introduces the possibility of not having the implementation of the extension in the network-store.
3 existing extensions have been migrated to this new framework: OperatingStatus, GeneratorStartup, ActivePowerControl.
The migration from the old framework to the new one is handled with SQL script in Liquibase.

This PR relies on powsybl/powsybl-network-store#383

Notes

  • Migration
    → Dump of dev-opf data => 7866159 extensions (OperatingStatus, GeneratorStartup or ActivePowerControl), migrated in 1 minute.
    → The data migration is only done for postgres but the schema migration for all dbms.
    → The migration does not drop the columns activePowerControl, operatingStatus and startup. This will be done in a future migration to be able to fallback on the previous implementation without data loss.
  • Use extension name in the column instead of adding "extensionName" in the stored json
    → I haven't found a way to deserialize the json without adding the "extensionName" field when storing it (e.g. {"extensionName":"activePowerControl","participate":true,"droop":0.0,"participationFactor":"NaN"}). The extensionName value is already stored in the "name" column so we could use it (?).
    → I added a runtime dependencies to powsybl-network-store-iidm-impl to be able to discover ExtensionLoaders in this dependency at runtime.
  • Performance
    → On my computer, on a THT/HT study with 12000 extensions and ~6000 generators, we measure the time spent in NetworkStoreRepository methods only:
main generalize-extension overhead
create all identifiables 8.5 s 9 s 5%
get identifiable (all generators) 0.2 s 0.25 s 25%
update identifiable (tabular modification - 6000 generators) 1 s 1.2 s 20%
delete identifiable (1 generator) 0.002 s 0.004 s 100%

Signed-off-by: BOUHOURS Antoine <[email protected]>
Signed-off-by: BOUHOURS Antoine <[email protected]>
@antoinebhs antoinebhs changed the title [WIP] Generalize extensions Generalize extensions Mar 28, 2024
Signed-off-by: BOUHOURS Antoine <[email protected]>
@antoinebhs antoinebhs force-pushed the generalize-extension branch 3 times, most recently from 06afaf9 to 7abae19 Compare April 3, 2024 11:10
Signed-off-by: BOUHOURS Antoine <[email protected]>
@antoinebhs antoinebhs force-pushed the generalize-extension branch from 7abae19 to e544543 Compare April 3, 2024 12:39
@antoinebhs antoinebhs requested a review from FranckLecuyer April 8, 2024 07:02
Signed-off-by: BOUHOURS Antoine <[email protected]>
" = ? and " + VARIANT_NUM_COLUMN + " = ?";
}

public static String buildExtensionsQuery(String columnNameForWhereClause) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildGetExtensionsQuery ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, thanks!
Fixed in 99469ab.

columnNameForWhereClause + " = ?";
}

public static String buildExtensionsWithInClauseQuery(String columnNameForInClause, int numberOfValues) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildGetExtensionsWithInClauseQuery ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, thanks!
Fixed in 99469ab.

@FranckLecuyer FranckLecuyer self-requested a review April 9, 2024 07:13
Copy link

@antoinebhs antoinebhs merged commit 1aaf81f into main Apr 10, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants