logstash-input-dynamodb crashing logstash

[gcarreto]

Having a problem with plugin once an error ocurred with dynamodb. Dynamodb fails to retrieve a success response and after that logstash loses data. How can we avoid the lost of data? how can we implement a logic to help Kinesis to rollback to a previous sequence number range so it can go thru the data again and avoid loses.

Here is the error:

ERROR: com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator: LeasingException encountered in lease renewing th
read
com.amazonaws.services.kinesis.leases.exceptions.DependencyException: com.amazonaws.AmazonServiceException: Unable to parse HTTP response content (Ser
vice: AmazonDynamoDBv2; Status Code: 500; Error Code: null; Request ID: null)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.convertAndRethrowExceptions(LeaseManager.java:558)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.renewLease(LeaseManager.java:372) at com.amazonaws.services.kinesis.leases.impl.LeaseRenewer.renewLease(LeaseRenewer.java:106)
at com.amazonaws.services.kinesis.leases.impl.LeaseRenewer.renewLeases(LeaseRenewer.java:85) at com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator.runRenewer(LeaseCoordinator.java:199)
at com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator$RenewerRunnable.run(LeaseCoordinator.java:135)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.amazonaws.AmazonServiceException: Unable to parse HTTP response content (Service: AmazonDynamoDBv2; Status Code: 500; Error Code: null;
Request ID: null)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1219)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:803)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1799)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateItem(AmazonDynamoDBClient.java:1614)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.renewLease(LeaseManager.java:363)
... 11 more
log4j, [2016-02-16T02:13:11.953] ERROR: com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator: LeasingException encountered in lease renewing thread
com.amazonaws.services.kinesis.leases.exceptions.DependencyException: com.amazonaws.AmazonServiceException: Unable to parse HTTP response content (Service: AmazonDynamoDBv2; Status Code: 500; Error Code: null; Request ID: null)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.convertAndRethrowExceptions(LeaseManager.java:558)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.renewLease(LeaseManager.java:372)
at com.amazonaws.services.kinesis.leases.impl.LeaseRenewer.renewLease(LeaseRenewer.java:106)
at com.amazonaws.services.kinesis.leases.impl.LeaseRenewer.renewLeases(LeaseRenewer.java:85)
at com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator.runRenewer(LeaseCoordinator.java:199)
at com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator$RenewerRunnable.run(LeaseCoordinator.java:135)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.amazonaws.AmazonServiceException: Unable to parse HTTP response content (Service: AmazonDynamoDBv2; Status Code: 500; Error Code: null; Request ID: null)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1219)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:803)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1799)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateItem(AmazonDynamoDBClient.java:1614)
at com.amazonaws.services.kinesis.leases.impl.LeaseManager.renewLease(LeaseManager.java:363)
... 11 more