Enhancement of Data Plane Signaling (DPS) Documentation - Need for a well-defined Specification

[f-zimmer]

Hello,

I am about to be working on implementing my own Data Plane following the DPS specifications. However, I've noticed that the documentation for Data Plane Signaling (DPS) is quite sparse and lacks the necessary detail for full adoption. This lack of detailed documentation is making it difficult to understand and implement a compliant Data Plane which can effectively communicate with the EDC Control Plane - or even a custom Control Plane which also needs to adhere to the DPS, thus requiring a specification as well.

The current DPS documentation issues include:

• Insufficient detail on DPS protocol, especially in regards to the message layout and interaction state machines.
• Lack of examples or use cases that illustrate practical implementations.

A well-defined and comprehensive DPS specification is crucial for anyone looking to implement their own Data Plane. It ensures compatibility and effective communication between the Control Plane and Data Plane, which is integral to the system's overall functionality.

I propose that we discuss potential ways to enhance the DPS documentation, including:

• Expanding the existing documentation to include more detailed explanations.
• Including a well-defined protocol specification.
• Adding clearer implementation guidelines.
• Including practical examples and use cases.

I believe that these improvements will greatly benefit the community by making it easier for developers to implement their own Data Planes in accordance with the DPS specification.

Looking forward to your thoughts and any further suggestions.

Thank you!

[jimmarino]

The document in question is an architecture document, not a specification. We do have plans to write a specification under the following guidelines:

1. The specification will be proposed to the Eclipse Dataspace Working Group and will not be done as part of the EDC because it will be implementation-independent.
2. The specification will not contain any implementation advice, examples, or suggestions because it is intended to specify interoperable behavior rather than detail how that behavior should be achieved.

Given these plans and the fact that the current document defines a specific EDC architecture, I don't think it should be changed.

We have other higher-priority items at the moment, so the standardization work will not begin immediately. Feel free to point out items that are unclear about the DPS state machine here so that we can catalog them and pick up the discussion when this work commences.

[f-zimmer]

Thank you for initiating this discussion and for the detailed proposal to enhance the DPS documentation. I appreciate the clarification from @jimmarino about the nature of the current document and the plans to develop a formal specification that will be implementation-independent.

While I agree that a non-implementation-dependent specification is beneficial for broader adoption and interoperability, I do share the concern about the current lack of detail that would enable developers to effectively use the DPS without needing to delve deep into source code. It's crucial for developers to have a clear starting point and a basic understanding of the DPS behavior without extensive code analysis.

Here are a few suggestions to bridge the gap until the formal specification is developed:

1. Enhance the documentation to include references to relevant classes and packages involved in the interactions, such as ApiControllers, Managers, or Message Classes. This would provide a clearer entry point for developers looking to understand or implement the DPS.
2. Outline the internal state machine of the Data Plane, specifically detailing the behavior in response to receiving messages. Ideally, this should be illustrated through a diagram to provide a visual representation of the process, making it easier to comprehend and implement.

These steps could serve as interim solutions that support developers and maintain the momentum of DPS adoption and implementation.

Recognizing the limited capacity of the contributors and the tight schedule constraints, I am definitely keen to assist by contributing to the discussed points. Having invested considerable time exploring the source code and creating beneficial diagrams for other adopters, I am prepared to enhance the documentation and support the community. For more specifics on my contributions, please see this link.

Looking forward to further discussions and your thoughts.

Best regards.

[jimmarino]

Writing something up as part of the end-user documentation would be helpful and much apprectiated. Please keep in mind the following guidelines:

1. The goal of documentation should be to help people who want to write their own data plane and have it interoperate with the EDC, not understand the internals of EDC or the DPF.
2. The DPS API is currently a work in progress, and it is not maintained as a GA API. It is subject to the EDC Deprecation Rules. In other words, data plane implementations should expect to handle changes and evolution of the message protocol.
3. Please do not reference any implementation code, including classes and packages. This is not relevant for a data plane to interoperate with the control plane. In general, many committers (including myself) strongly dislike documentation that directly references code as it becomes quickly outdated.
4. Only the abstract DPS state machine (which is very simple) and message types need to be documented. Message types should be described using JSON Schema, not Java types. Take a look at the Dataspace Protocol specifications for details.

I'll be happy to review anything you come up with.

[f-zimmer]

Hello,

I tried to come up with a first draft of such a specification.

Would love your feedback on this one.

Best regards.