-
Notifications
You must be signed in to change notification settings - Fork 321
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
17 changed files
with
37 additions
and
53 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -28,7 +28,6 @@ | |
* Create Operation | ||
* <p> | ||
* This operation creates topic in a broker or enlarge the partition number of topic. | ||
* </p> | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.2 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -31,7 +31,6 @@ | |
* Jafka will returns all offsets earlier than given time with max number | ||
* limit. The fist offset of result is the biggest and the the last is the | ||
* smallest. | ||
* </p> | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
|
@@ -74,7 +73,6 @@ public class OffsetRequest implements Request { | |
* <li>time>0: the log file offset which lastmodified time earlier | ||
* than the time</li> | ||
* </ul> | ||
* </p> | ||
*/ | ||
public long time; | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -39,8 +39,6 @@ | |
* message: bytes | ||
* </pre> | ||
* | ||
* </p> | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
*/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -31,7 +31,6 @@ | |
* <li>groupid: the consumer group name</li> | ||
* <li>zk.connect: the zookeeper connection string</li> | ||
* </ul> | ||
* </p> | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
|
@@ -79,7 +78,6 @@ public class ConsumerConfig extends ZKConfig { | |
* <li>groupid: the consumer group name</li> | ||
* <li>zk.connect: the zookeeper connection string</li> | ||
* </ul> | ||
* </p> | ||
* | ||
* @param props config properties | ||
*/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -67,72 +67,68 @@ | |
|
||
/** | ||
* This class handles the consumers interaction with zookeeper | ||
* <p/> | ||
* | ||
* Directories: | ||
* <p> | ||
* <b>1. Consumer id registry:</b> | ||
* <p/> | ||
* | ||
* <pre> | ||
* /consumers/[group_id]/ids[consumer_id] -> topic1,...topicN | ||
* /consumers/[group_id]/ids[consumer_id] -- topic1,...topicN | ||
* </pre> | ||
* <p/> | ||
* | ||
* A consumer has a unique consumer id within a consumer group. A consumer registers its id as | ||
* an ephemeral znode and puts all topics that it subscribes to as the value of the znode. The | ||
* znode is deleted when the client is gone. A consumer subscribes to event changes of the | ||
* consumer id registry within its group. | ||
* </p> | ||
* <p> | ||
* The consumer id is picked up from configuration, instead of the sequential id assigned by | ||
* ZK. Generated sequential ids are hard to recover during temporary connection loss to ZK, | ||
* since it's difficult for the client to figure out whether the creation of a sequential znode | ||
* has succeeded or not. More details can be found at | ||
* (http://wiki.apache.org/hadoop/ZooKeeper/ErrorHandling) | ||
* </p> | ||
* <p> | ||
* <b>2. Broker node registry:</b> | ||
* <p/> | ||
* <pre> | ||
* /brokers/[0...N] --> { "host" : "host:port", | ||
* /brokers/[0...N] -- { "host" : "host:port", | ||
* "topics" : {"topic1": ["partition1" ... "partitionN"], ..., | ||
* "topicN": ["partition1" ... "partitionN"] } } | ||
* </pre> | ||
* <p/> | ||
* This is a list of all present broker brokers. A unique logical node id is configured on each | ||
* broker node. A broker node registers itself on start-up and creates a znode with the logical | ||
* node id under /brokers. | ||
* <p/> | ||
* | ||
* The value of the znode is a JSON String that contains | ||
* <p/> | ||
* | ||
* <pre> | ||
* (1) the host name and the port the broker is listening to, | ||
* (2) a list of topics that the broker serves, | ||
* (3) a list of logical partitions assigned to each topic on the broker. | ||
* </pre> | ||
* <p/> | ||
* | ||
* A consumer subscribes to event changes of the broker node registry. | ||
* </p> | ||
* <p/> | ||
* | ||
* | ||
* <p> | ||
* <b>3. Partition owner registry:</b> | ||
* <p/> | ||
* | ||
* <pre> | ||
* /consumers/[group_id]/owner/[topic]/[broker_id-partition_id] --> consumer_node_id | ||
* /consumers/[group_id]/owner/[topic]/[broker_id-partition_id] -- consumer_node_id | ||
* </pre> | ||
* <p/> | ||
* | ||
* This stores the mapping before broker partitions and consumers. Each partition is owned by a | ||
* unique consumer within a consumer group. The mapping is reestablished after each | ||
* rebalancing. | ||
* </p> | ||
* <p/> | ||
* | ||
* | ||
* <p> | ||
* <b>4. Consumer offset tracking:</b> | ||
* <p/> | ||
* | ||
* <pre> | ||
* /consumers/[group_id]/offsets/[topic]/[broker_id-partition_id] --> offset_counter_value | ||
* /consumers/[group_id]/offsets/[topic]/[broker_id-partition_id] -- offset_counter_value | ||
* </pre> | ||
* <p/> | ||
* | ||
* Each consumer tracks the offset of the latest message consumed for each partition. | ||
* </p> | ||
* | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
|
@@ -816,9 +812,8 @@ public void handleStateChanged(KeeperState state) throws Exception { | |
/** | ||
* register consumer data in zookeeper | ||
* <p> | ||
* register path: /consumers/groupid/ids/groupid-consumerid <br/> | ||
* register path: /consumers/groupid/ids/groupid-consumerid <br> | ||
* data: {topic:count,topic:count} | ||
* </p> | ||
* | ||
* @param zkGroupDirs zookeeper group path | ||
* @param consumerIdString groupid-consumerid | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,8 +38,7 @@ | |
* 3. 4 byte CRC32 of the payload | ||
* 4. N - 6 byte payload | ||
* </pre> | ||
* | ||
* </p> | ||
* | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -23,7 +23,7 @@ | |
* For the last writeable file: | ||
* <ul> | ||
* <li>size</li> | ||
* </p> | ||
* </ul> | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -27,7 +27,7 @@ | |
* producer | ||
* | ||
* @author adyliu ([email protected]) | ||
* @see {@link AbstractCallbackHandler} | ||
* @see AbstractCallbackHandler | ||
* @since 1.0 | ||
*/ | ||
public interface CallbackHandler<T> { | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,10 +38,10 @@ | |
/** | ||
* Handles the server's interaction with zookeeper. The server needs to register the following | ||
* paths: | ||
* <p/> | ||
* <p> | ||
* <pre> | ||
* /topics/[topic]/[node_id-partition_num] | ||
* /brokers/[0...N] --> host:port | ||
* /brokers/[0...N] -- host:port | ||
* </pre> | ||
* | ||
* @author adyliu ([email protected]) | ||
|
@@ -113,9 +113,8 @@ private int getPartitions(String topic) { | |
/** | ||
* register broker in the zookeeper | ||
* <p> | ||
* path: /brokers/ids/<id> <br/> | ||
* path: /brokers/ids/<id> <br> | ||
* data: creator:host:port | ||
* </p> | ||
*/ | ||
public void registerBrokerInZk() { | ||
logger.info("Registering broker " + brokerIdPath); | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -31,7 +31,6 @@ | |
* <p> | ||
* see https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/ | ||
* Mx4jTool.java | ||
* </p> | ||
* | ||
* @author adyliu ([email protected]) | ||
* @since 1.0 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters