Add support for OData V4 specifications

[cwiechmann]

User story
It should be possible to use the APIM-CLI to import OData V4 Metadata-Specifications into the API-Manager without the need of an additional converter.
For that, the APIM-CLI should internally convert the OData-Metadata-Specification into an OpenAPI 3 spec and import it as any other API.

Please vote only once on the issue (OData V3 or OData V4 (this issue)) that has a higher priority for you. Since OData V3 and V4 are massively different, support for both versions cannot be added at the same time. Maybe V3 will not be supported at all, because it is not widely used.

Please vote on this issue 👍 if you have need for this feature. It would also help, if you can share a OData Metadata-Description which can be used for Unit-Testing the converter.

[davidgeiger]

I really appreciate this. The SAP API Business Hub contains a lot of OData API specs. This one, for instance, is v4 and the edmx file can be downloaded in Overview > API resources > API Specification.

You're probably already aware of the OASIS reference implementation for mapping OData to Open API using JavaScript or XSLT. This seems to be pretty reliable. https://github.com/oasis-tcs/odata-openapi#further-description-of-this-repository

[davidgeiger]

Here's another relevant use case: the Microsoft Graph API

https://graph.microsoft.com/v1.0/$metadata

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 90 days. It will be closed in the next 30 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[davidgeiger]

Any news on this? OData v4 is becoming more prevalent with new SAP releases. Since the CLI already supports v2 and the OASIS converter reference implementation also supports v4, is this a lot of effort to implement?

[axepow]

I would like to second the request as well. In our company we currently have an OData v4 use case we can't convert through the CLI tool and are struggling now on how to move on.

[rathnapandi]

@davidgeiger,

The current code detects odata4 specification and throws an error. We need to write a mapping from odata4 to opeanapi using Apache Olingo.

I will prioritize it for the next release.

[cwiechmann]

To let you know @rathnapandi there is an existing branch that already contains support for OData V4. Maybe it helps as an inspiration.

[davidgeiger]

We need to write a mapping from odata4 to opeanapi using Apache Olingo.

Can't you use one of the OASIS reference implementations for mapping OData to Open API? https://github.com/oasis-tcs/odata-openapi#further-description-of-this-repository

[rathnapandi]

Thanks @cwiechmann.

@davidgeiger I am open with other options too. The github page says it is a POC. Do let me know if you have tested the xslt and works?

[davidgeiger]

Yes, we have worked with both converters (mostly the Node.js version though) and the results are quite good. It does lack support for some things such as adding a root method / and /$metadata, which some SAP-specific applications rely on. This may also be an issue with the underlying specification, though, and can be added after conversion.

[davidgeiger]

Additional remark on this: The aim of at least the OASIS converters appears to be a "nice" human-readable representation of an OData API in Swagger UI. This is obviously very different from using a converted spec to validate API traffic in a gateway. See, for instance, this comment.

That being said, so far we've only run into issues with the lack of the previously mentioned root and /$metadata methods. The linked discussion on handling enums does raise additional concerns, though.

[rathnapandi]

Thanks for sharing, the issue oasis-tcs/odata-openapi#38 handled in odata 4 conversion.

[rathnapandi]

fixed as part v1.13.1 release

[davidgeiger]

Thanks @rathnapandi. Could you share which conversion approach is now used (OASIS conversion, Apache Olingo...)? And does the CLI add endpoints for / and /$metadata?

[rathnapandi]

We are using Apache Olingno as I do see lot of TODO list on xsl implementation. Also, it requires xsl customization to work with API Manager.

The endpoint should use $metadata e.g - https://github.com/Axway-API-Management-Plus/apim-cli/blob/f7d8be10dd5ba1bd71979a9d918f031585035c70/modules/apis/assembly/samples/basic/odata-v4-trippin-api.json

Filtering example - https://github.com/Axway-API-Management-Plus/apim-cli/blob/develop/modules/apis/assembly/samples/basic/odata-v4-trippin-api-filtered.json

We can give edmx file as input with backendBasepath parameter.

[axepow]

I just downloaded v.1.13.1 and tried it out. This is the result:

C:\UserData\x\Tools\apim-cli-1.13.1\scripts>apim api import -username my.user -s dev -a .\import\dev\EXAMPLE_TEST_ODATA\odata_ex.xml -c .\import\dev\EXAMPLE_TEST_ODATA\api-config.json
----------------------------------------------------------------------------------------
API-Manager CLI: 1.5.0

To report issues or get help, please visit:
https://github.com/Axway-API-Management-Plus/apim-cli
----------------------------------------------------------------------------------------
Module: API - I M P O R T (1.5.0)
----------------------------------------------------------------------------------------
0      INFO     APIManagerAdapter| Successfully connected to API-Manager (7.7.20220228) on: https://my_hostname:8075
437    INFO  IImportConfigAdapter| Reading API-Definition (Swagger/WSDL) from file: '.\import\dev\EXAMPLE_TEST_ODATA\odata_ex.xml' (relative path)
523    ERROR         APIImportApp| Detected OData V4 specification, which is not yet supported by the APIM-CLI.
                                 | If you have a need for OData V4 support please upvote the following issue:
                                 | https://github.com/Axway-API-Management-Plus/apim-cli/issues/234

C:\UserData\x\Tools\apim-cli-1.13.1\scripts>

I'm not sure what I could be doing wrong. I think I got the right version (1.13.1), it's properly unzipped and I'm executing the correct executable. Please let me know for any hint. (I was always puzzled why the initial screen keeps on telling me about v1.5.0, however, I just think this is a text relict and has nothing to do with the actual functionality.)

[rathnapandi]

The log says it using version 1.5.0. Can you check the lib folder for apim cli version 1.13.1 jar files?

API-Manager CLI: 1.5.0

[axepow]

Is this what you're looking for?

C:\UserData\x\Tools\apim-cli-1.13.1\lib>dir
 Datenträger in Laufwerk C: ist Windows
 Volumeseriennummer: 5251-5C3D

 Verzeichnis von C:\UserData\x\Tools\apim-cli-1.13.1\lib

Mi, 16.11.2022  18:21    <DIR>          .
Mi, 16.11.2022  18:21    <DIR>          ..
Mi, 09.11.2022  13:41           358.135 aalto-xml-1.3.1.jar
Di, 15.11.2022  10:46           426.235 apimcli-apim-adapter-1.13.1.jar
Di, 15.11.2022  10:46           161.789 apimcli-apis-1.13.1.jar
Di, 15.11.2022  10:47            55.208 apimcli-apps-1.13.1.jar
Di, 15.11.2022  10:47            36.618 apimcli-organizations-1.13.1.jar
Di, 15.11.2022  10:47            48.198 apimcli-settings-1.13.1.jar
Di, 15.11.2022  10:47            22.495 apimcli-spectoconfig-1.13.1.jar
Di, 15.11.2022  10:46             6.417 apimcli-tool-1.13.1.jar
Di, 15.11.2022  10:47            39.898 apimcli-users-1.13.1.jar
Fr, 03.06.2022  08:04            15.001 ascii-table-1.2.0.jar
Mo, 20.12.2021  10:56            58.284 commons-cli-1.5.0.jar
Do, 22.04.2021  08:47           335.042 commons-codec-1.11.jar
Mo, 25.04.2022  12:14            51.322 commons-csv-1.9.0.jar
Fr, 26.11.2021  11:37           327.135 commons-io-2.11.0.jar
Do, 22.04.2021  08:47           587.402 commons-lang3-3.12.0.jar
Do, 22.04.2021  08:47            61.829 commons-logging-1.2.jar
Di, 08.11.2022  10:04           238.400 commons-text-1.10.0.jar
Mo, 25.04.2022  12:14         1.795.405 ehcache-3.9.9.jar
Mo, 25.04.2022  12:14           212.164 gson-2.4.jar
Do, 22.04.2021  08:47           780.321 httpclient-4.5.13.jar
Do, 22.04.2021  08:47           328.593 httpcore-4.4.13.jar
Do, 22.04.2021  08:47            41.790 httpmime-4.5.13.jar
Do, 02.09.2021  08:30            29.807 istack-commons-runtime-3.0.12.jar
Do, 27.10.2022  16:48            75.717 jackson-annotations-2.13.4.jar
Fr, 03.06.2022  08:03           374.895 jackson-core-2.13.3.jar
Do, 27.10.2022  16:48         1.531.560 jackson-databind-2.13.4.1.jar
Fr, 03.06.2022  08:03           122.296 jackson-dataformat-xml-2.13.3.jar
Fr, 03.06.2022  08:03            52.020 jackson-dataformat-yaml-2.13.3.jar
Mo, 02.05.2022  10:07           121.204 jackson-datatype-jsr310-2.13.2.jar
Do, 22.04.2021  08:47            68.453 jakarta.activation-1.2.2.jar
Mo, 03.02.2020  14:05            91.930 jakarta.validation-api-2.0.2.jar
Do, 22.04.2021  08:47           115.638 jakarta.xml.bind-api-2.3.3.jar
Mo, 25.04.2022  12:14           110.928 javax.ws.rs-api-2.0-m10.jar
Fr, 03.06.2022  08:03         1.019.386 jaxb-runtime-2.3.6.jar
Fr, 03.06.2022  08:03           302.511 log4j-api-2.17.2.jar
Fr, 03.06.2022  08:03         1.811.089 log4j-core-2.17.2.jar
Fr, 03.06.2022  08:03            24.248 log4j-slf4j-impl-2.17.2.jar
Di, 15.11.2022  10:39               986 log4j2.xml
Mi, 09.11.2022  13:41           102.922 odata-client-api-4.9.0.jar
Mi, 09.11.2022  13:41           503.534 odata-client-core-4.9.0.jar
Mi, 09.11.2022  13:41           201.116 odata-commons-api-4.9.0.jar
Mi, 09.11.2022  13:41           208.914 odata-commons-core-4.9.0.jar
Fr, 03.06.2022  08:03           203.301 olingo-odata2-api-2.0.12.jar
Fr, 03.06.2022  08:03           610.460 olingo-odata2-core-2.0.12.jar
Do, 04.08.2022  11:38            41.125 slf4j-api-1.7.36.jar
Mo, 25.04.2022  12:14           331.605 snakeyaml-1.30.jar
Do, 22.04.2021  08:47           195.909 stax2-api-4.2.1.jar
Mo, 02.05.2022  10:07            42.096 swagger-annotations-2.2.0.jar
Mo, 02.05.2022  10:07           200.595 swagger-core-2.2.0.jar
Mo, 25.04.2022  12:14           133.825 swagger-models-2.2.0.jar
Di, 21.06.2022  21:51             3.242 swagger-parser-2.0.33.jar
Di, 21.06.2022  21:51             7.132 swagger-parser-core-2.0.33.jar
Di, 21.06.2022  21:51           151.938 swagger-parser-v3-2.0.33.jar
Fr, 03.06.2022  08:03            72.008 txw2-2.3.6.jar
Mo, 25.04.2022  12:14         1.584.634 woodstox-core-6.2.7.jar
              55 file(s),     16.434.705 Bytes
               2 folder(s), 114.697.011.200 Bytes free

C:\UserData\x\Tools\apim-cli-1.13.1\lib>

[davidgeiger]

Perhaps you need to update any env variables such as AXWAY_APIM_CLI_HOME?

https://github.com/Axway-API-Management-Plus/apim-cli/wiki/1.-How-to-get-started#setup-axway_apim_cli_home

[axepow]

That could be it, at least it's ringing a bell. I need to get admin rights to check this and will report back here, thank you.

[axepow]

So yes, it was that AXWAY_APIM_CLI_HOME variable. Once this was set properly, I was able to import an Odata V4 based service and didn't get the error anymore. I can't judge the quality of the import yet, this will take a bit time. It seems to look more different than expected compared with the output from other (open source) tools, so that's a bit surprising to me.

[rathnapandi]

Check the video for how OData2 works on API gateway - https://www.youtube.com/watch?v=K4q7HR3wBqQ&t=455s

OData4 follows same approach. The openapi generated from OASIS TC GitHub repositories and APIM cli won't be same as we tailored the open api specification to work with Axway APIM.