Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stream API methods #60

Merged
merged 12 commits into from
Feb 12, 2025
Merged

Add stream API methods #60

merged 12 commits into from
Feb 12, 2025

Conversation

vaidikcode
Copy link
Contributor

Signed-off-by: vaidikcode [email protected]

Description

  • Added streaming methods for blob operations:

    • pushBlobStream() for uploads
    • getBlobStream() for downloads
    • DigestUtils.sha256(InputStream) for streaming hash calculation
  • Updated artifact operations to use streams:

Closes #25 .

Testing done

No, but I will add tests if needed.

### Submitter checklist
- [ ] If an issue exists, it is well described and linked in the description
- [ ] The description of this pull request is detailed and explain why this pull request is needed
- [ ] The changeset is on a specific branch. Using `feature/` for new feature, or improvements ; Using `fix/` for bug fixes ; Using `docs/` for any documentation changes.
- [ ] If required, the documentation has been updated
- [ ] There is automated tests to cover the code change / addition or an explanation why there is no tests in the description.

Signed-off-by: vaidikcode <[email protected]>
@vaidikcode vaidikcode requested a review from jonesbusy as a code owner January 29, 2025 13:13
@jonesbusy
Copy link
Collaborator

Thanks

Test are failing

Error:    RegistryTest.testShouldPushAndPullMinimalArtifact:187 » Oras Response code: 400
Error:    RegistryTest.testShouldPushCompressedDirectory:226 » Oras Failed to calculate digest

Did you run test locally

Make sure that all new code is tested

@vaidikcode
Copy link
Contributor Author

I have not run the tests locally yet, but after fixing the problems, I will push those changes.

@jonesbusy
Copy link
Collaborator

I have not run the tests locally yet, but after fixing the problems, I will push those changes.

This is the minimum before opening a PR. Thanks for your understanding

I value high quality PR (Don't matter the size) with test implemented and that are valuable for future consumer.

While I appreciate contribution, my feeling is that we open PR as fast as possible hoping this gets merged (I suspect it's because for GsoC proposal or LFX mentorship). But it doesn't work like that.

I will only accept PR that

  • Are well documented
  • Have test (unit, integration, testcontainer, wiremock etc...)
  • Don't introduce regression
  • Explose clear APIs and don't break encapsulation
  • Introduce real value (for example did we test pushing a very large artifact on an OCI registry? Like 5Gi or 10Gi artifact?)

Note I was using also https://github.com/jonesbusy/oras-java-cli to perform some interactive test. Feel free to use it if you want to test SDK performance againts a local registry

@vaidikcode
Copy link
Contributor Author

vaidikcode commented Jan 29, 2025

Thank you for your detailed feedback and for setting clear expectations for contributions. I completely understand the importance of submitting high-quality PRs that are well-documented, properly tested
Before finalizing the PR, I wanted to check:

1- Would you prefer me to add specific types of tests or cover additional scenarios?
2- Are there any other changes or refinements you'd like to see in the code before I proceed?

@vaidikcode
Copy link
Contributor Author

I think adding new unit tests, particularly for these new methods, would be beneficial. What do you think? Should I work on that?

@jonesbusy
Copy link
Collaborator

Yes all new code must be tested if possible. Thanks

@vaidikcode
Copy link
Contributor Author

@jonesbusy
The issue was that the testShouldPushAndPullMinimalArtifact test failed, and after debugging, I identified that the problem was in the pushArtifact method, which I hadn’t directly or indirectly modified. The mediaType for individual files wasn’t explicitly set, causing it to default to "application/vnd.oci.empty.v1+json" instead of the expected "text/plain". To fix this, I added explicit mediaType detection for individual files using Files.probeContentType(path) and set it during layer creation, ensuring the correct mediaType is used. Now the test passes. Is this change acceptable, or does it seem unusual? Also, should I add any additional tests to cover more specific behaviors or make further changes?
Thanks

@jonesbusy
Copy link
Collaborator

Does it comply with https://github.com/opencontainers/distribution-spec/blob/main/spec.md ? If yes I guess it should be acceptable

@jonesbusy
Copy link
Collaborator

Thanks this look good. Just minor things.

Can you fix formatting: https://github.com/oras-project/oras-java/actions/runs/13144635831/job/36699506717 via mvn spotless:apply

Signed-off-by: vaidikcode <[email protected]>
@jonesbusy
Copy link
Collaborator

 Warning:  Javadoc Warnings
Warning:  /home/runner/work/oras-java/oras-java/src/main/java/land/oras/Registry.java:665: warning: no @return
Warning:  public Layer pushBlobStream(ContainerRef containerRef, InputStream input, long size) {
Warning:  ^
Warning: [WARNING] 1 warning

Please ensure to build mvn clean install and fix javadoc

@jonesbusy jonesbusy changed the title add stream methods Add stream API methods Feb 12, 2025
@jonesbusy jonesbusy merged commit dee91aa into oras-project:main Feb 12, 2025
2 checks passed
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
Co-authored-by: Valentin Delaye <[email protected]>
Signed-off-by: AayushSaini101 <[email protected]>
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
Co-authored-by: Valentin Delaye <[email protected]>
Signed-off-by: AayushSaini101 <[email protected]>
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
Co-authored-by: Valentin Delaye <[email protected]>
Signed-off-by: AayushSaini101 <[email protected]>
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
Co-authored-by: Valentin Delaye <[email protected]>
Signed-off-by: AayushSaini101 <[email protected]>
AayushSaini101 pushed a commit to AayushSaini101/oras-java that referenced this pull request Feb 12, 2025
Co-authored-by: Valentin Delaye <[email protected]>
Signed-off-by: AayushSaini101 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure memory consuption when pushing/pulling large artifact
2 participants