This repository has been archived by the owner on Apr 9, 2022. It is now read-only.
Version 1.4.0 #47
Merged
Version 1.4.0 #47
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, there is only one partition selection mode, which is the round robin one. There's a need to introduce a new selection strategy depending on the keys of messages to determine the partition the messages should go to. It would allow to compact the same messages and reduce heavily the number of messages. This new partition selection strategy guarantees that messages with the same key will be sent to the same partition, given the same number of available partitions. It is not using a consistent hashing though, which means that if new nodes are introduced or old ones are removed from the cluster, new messages with the same key as old messages will not be sent to the same partition. JIRA: WBSC-3909 Change-Id: Ie75b40123fd90550aecafd9c33d2b82952470469
For client of the driver (i.e. the Kafka service) it is useful to be able to create messages for testing purpose, and to be able to manually set their partition/topic etc. Change-Id: Ife4b6e29e0ea4e732f58e9500f085ca06a390312
We currently have an issue with kibana where for some messages the labels are not correctly parsed, and they are considered part of the message. This messes up with getting logs easily. I am trying to change this message to see if it fixes the issue. Change-Id: I45a24df49a1d564a4c8a8dd0453d1fd72a5cc237
Change-Id: I7a054467983c93ee307c8b6410e15c4fd03bd964
- when SerializeOnProduce is on the SerializeKeyValue is now done correctly for records - when SerializeOnProduce is off but the object is not sized, serialization is still done on produce Change-Id: I9aa42448f9914c0b3eab95c806772981d965b1e4
Change-Id: Ia5e0975778586c7c0b46583a0fb24b81295e9fa8
…...} ReadInt32 implies there is an Int32 in the stream but it actually reads a varint and stores it in an Int32 Change-Id: I375566a8173b1a74ea4b9906a14a9ccf29a53d21
This change is to replace this pattern var recordBatch = new RecordBatch(); recordBatch.Deserialize(stream, deserializers, endOfAllBatches); with var recordBatch = RecordBatch.Deserialize(stream, deserializers, endOfAllBatches); The big diff is because I moved static method at the top of the class Change-Id: I381a0145bc147495fa4145fd17e99d8bf14968b5
Change-Id: I84fd77041218e4dd9fb1acadc6dd406103f982b5
Change-Id: I43d6650ee70bbff8661b6406ab5f4086ca0381f2
From the official documentation: "As an optimization the server is allowed to return a partial message at the end of the message set. Clients should handle this case.". We then discard the message, the read offset not being updated for this message, meaning we will get it in full in the next FetchRequest. Change-Id: I0dae2e93641c7a48544d5a3b02287bd10319ab58
Currently, when trying to start consuming a topic, if the first fetch of the available partitions fails (because the topic does not exist for example), the driver will immediately retry. This can translate to hundreds of requests per second, which is useless. As this loop in HandleStart is already blocking indefinitely the ProcessMessage() loop of the driver, introducing a retry delay does not change the behavior. Change-Id: Ice89ab8e4cb1a41ba25fb2bec49bd1be66001a8b
- Add a log when a broker is removed from the nodes list because of a topology change - Remove unknown node (when dead) from the routing table so it is not reused, which was not the case before Change-Id: I32cb7fe19f9935b0abe088950a07f1590b7a2951
So that logs like "...reaching TTL for [topic: toto / partition: -3]" are replaced with "...reaching TTL for [topic: toto / partition: any]" Change-Id: I95f2a8400abd66ebc82aaf0336eeb4f29f39888e
Change-Id: Ibf22b2e88b79ffae73f2cfcd08914438e96ebe7d
ychuzevi
approved these changes
Feb 26, 2020
|
@ychuzevi Nuget package for version 1.4 is available to use? or latest change will be reflected in present version 1.3 only ? |
|
Good question. It should have been published automatically thanks to our appveyor plugin. Not sure why the 1.4.0 nuget is not yet available. |
|
@ychuzevi |
|
Hello @ParthShirolawala |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.