Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add PulsarAvroMessageDecoder #14397

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

chunxiaozheng
Copy link

@chunxiaozheng chunxiaozheng commented Nov 6, 2024

With this PR, decoder can directly obtain the schema through the pulsar admin API, instead of setting the avro schema in the table config.
In addition, the following configurations need to be added in the table config

"stream.pulsar.decoder.prop.admin.servers": "http://localhost:8081",
"stream.pulsar.decoder.class.name": "org.apache.pinot.plugin.inputformat.avro.PulsarAvroMessageDecoder"

@codecov-commenter
Copy link

codecov-commenter commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 0% with 60 lines in your changes missing coverage. Please review.

Project coverage is 55.54%. Comparing base (59551e4) to head (6beab2f).
Report is 1376 commits behind head on master.

Files with missing lines Patch % Lines
...gin/inputformat/avro/PulsarAvroMessageDecoder.java 0.00% 56 Missing ⚠️
...gin/inputformat/avro/SimpleAvroMessageDecoder.java 0.00% 4 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (59551e4) and HEAD (6beab2f). Click for more details.

HEAD has 44 uploads less than BASE
Flag BASE (59551e4) HEAD (6beab2f)
integration 7 0
integration2 3 0
temurin 12 3
java-21 7 2
skip-bytebuffers-true 3 1
skip-bytebuffers-false 7 2
unittests 5 3
java-11 5 1
unittests2 3 0
integration1 2 0
custom-integration1 2 0
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14397      +/-   ##
============================================
- Coverage     61.75%   55.54%   -6.21%     
- Complexity      207      791     +584     
============================================
  Files          2436     2102     -334     
  Lines        133233   110725   -22508     
  Branches      20636    17597    -3039     
============================================
- Hits          82274    61499   -20775     
+ Misses        44911    44334     -577     
+ Partials       6048     4892    -1156     
Flag Coverage Δ
custom-integration1 ?
integration ?
integration1 ?
integration2 ?
java-11 55.46% <0.00%> (-6.25%) ⬇️
java-21 55.40% <0.00%> (-6.23%) ⬇️
skip-bytebuffers-false 55.53% <0.00%> (-6.22%) ⬇️
skip-bytebuffers-true 55.35% <0.00%> (+27.62%) ⬆️
temurin 55.54% <0.00%> (-6.21%) ⬇️
unittests 55.54% <0.00%> (-6.21%) ⬇️
unittests1 55.54% <0.00%> (+8.65%) ⬆️
unittests2 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@Jackie-Jiang Jackie-Jiang added documentation Configuration Config changes (addition/deletion/change in behavior) labels Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Configuration Config changes (addition/deletion/change in behavior) documentation ingestion plugins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants