reshard test

[hampsterx]

kinesis.consumer DEBUG Shard shardId-000000000001 got 5 records
kinesis.checkpointers DEBUG Heartbeating shardId-000000000000@49615604846537535747729525809208528210816652204025315330
kinesis.checkpointers DEBUG Heartbeating shardId-000000000001@49615604846559836492928056432312587228681247061115404306
tests INFO Consumed 15 records
tests INFO Stream UPDATING
tests INFO Shard shardId-000000000000 start=49615604846537535747729525809166215807130139976752168962 end=
tests INFO Shard shardId-000000000001 start=49615604846559836492928056432307751525402788338258149394 end=
kinesis.checkpointers DEBUG Heartbeating shardId-000000000000@49615604846537535747729525809208528210816652204025315330
kinesis.checkpointers DEBUG Heartbeating shardId-000000000001@49615604846559836492928056432312587228681247061115404306
tests INFO Stream ACTIVE
tests INFO Shard shardId-000000000000 start=49615604846537535747729525809166215807130139976752168962 end=49615604846548686120328791120735774740446850284244697090
tests INFO Shard shardId-000000000001 start=49615604846559836492928056432307751525402788338258149394 end=49615604846570986865527321743877310458719498645750677522
tests INFO Shard shardId-000000000068 start=49615604849748843056317935541547359238391504652088639490 end=
tests INFO Shard shardId-000000000069 start=49615604849771143801516466164688894956664153013594619922 end=
tests INFO Shard shardId-000000000070 start=49615604849793444546714996787830430674936801375100600354 end=
tests INFO Shard shardId-000000000071 start=49615604849815745291913527410971966393209449736606580786 end=
kinesis.producer DEBUG flush queue=50 overflow=0
kinesis.producer DEBUG doing flush with 50 record (50 items) @ 50 kb
kinesis.producer INFO flush complete with 50 record (50 items) @ 50 kb
..
Heartbeating shardId-000000000001@49615604846559836492928056432336765745073539644609527826
kinesis.consumer ERROR 'NextShardIterator'
Traceback (most recent call last):
  File "/home/tim/Workspace/async-kinesis/kinesis/consumer.py", line 121, in _fetch
    await self.fetch()
  File "/home/tim/Workspace/async-kinesis/kinesis/consumer.py", line 266, in fetch
    if not result["NextShardIterator"]:
KeyError: 'NextShardIterator'
...
kinesis.consumer DEBUG Shard shardId-000000000000 caught up, sleeping 2s
kinesis.checkpointers DEBUG Heartbeating shardId-000000000000@49615604846537535747729525809225453172291257012471201794
kinesis.checkpointers DEBUG Heartbeating shardId-000000000001@49615604846559836492928056432336765745073539644609527826
tests INFO Expected 100 records. Consumed 44 records
kinesis.consumer DEBUG Closing Connection..
kinesis.checkpointers DEBUG Cancelling heartbeat task..
kinesis.checkpointers INFO test-5390a0c5/151976 stopping..
kinesis.checkpointers INFO test-5390a0c5/151976 deallocated on shardId-000000000000@49615604846537535747729525809225453172291257012471201794
kinesis.checkpointers INFO test-5390a0c5/151976 deallocated on shardId-000000000001@49615604846559836492928056432336765745073539644609527826
kinesis.producer DEBUG Closing Connection.. (stream status:ACTIVE)

#26 @jmcgrath207 might merge this and tests can fail forever until we get this behaviour in, no rush :)

[jmcgrath207]

@hampsterx sounds good, should be able to get to #26 this weekend #slow_committer lol

[jmcgrath207]

@hampsterx Thanks for the test it really help me out especially that note with the patch for kinesalite.

I went ahead and rebased #26 with this pull request so you can delete this now