1

I've setup a Kinesis agent on EC2 instance having windows operating system. Its getting me the following error for multiple sinks I've specified in my appsetting file.

what could be wrong here and what can be done to avoid it?

Amazon.KinesisTap.Hosting.LogManager ERROR KinesisFirehoseSink client dev_dw_fh_s3_dl_inventtransorigin_snk exception: Amazon.KinesisFirehose.AmazonKinesisFirehoseException: Signature expired: 20190610T083014Z is now earlier than 20190610T092115Z (20190610T092615Z - 5 min.) ---> Amazon.Runtime.Internal.HttpErrorResponseException: The remote server returned an error: (400) Bad Request. ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.

This error occurs randomly. There doesn't seem to be any connection lost issue since its running on EC2 instance also, the clock is not slow.

2019-06-10 04:26:16.2103 Amazon.KinesisTap.Hosting.LogManager ERROR KinesisFirehoseSink client dev_dw_fh_s3_dl_inventtransorigin_snk exception: Amazon.KinesisFirehose.AmazonKinesisFirehoseException: Signature expired: 20190610T083014Z is now earlier than 20190610T092115Z (20190610T092615Z - 5 min.) ---> Amazon.Runtime.Internal.HttpErrorResponseException: The remote server returned an error: (400) Bad Request. ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.    at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)    at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.HttpRequest.<GetResponseAsync>d__16.MoveNext()
--- End of inner exception stack trace ---    at Amazon.Runtime.Internal.HttpRequest.<GetResponseAsync>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.HttpHandler`1.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.Unmarshaller.<InvokeAsync>d__3`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()   
--- End of inner exception stack trace ---    at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException(IExecutionContext executionContext, HttpErrorResponseException exception)    at Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext executionContext, Exception exception)    at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.CredentialsRetriever.<InvokeAsync>d__7`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__5`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__1`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Amazon.KinesisTap.AWS.KinesisFirehoseSink.<OnNextAsync>d__7.MoveNext()
James Z
  • 12,209
  • 10
  • 24
  • 44
SabaBaig
  • 11
  • 1
  • https://stackoverflow.com/questions/44017410/signature-expired-is-now-earlier-than-error-invalidsignatureexception – Spiff Jun 10 '19 at 11:40
  • Possible duplicate of [Signature expired: is now earlier than error : InvalidSignatureException](https://stackoverflow.com/questions/44017410/signature-expired-is-now-earlier-than-error-invalidsignatureexception) – rene Jun 10 '19 at 12:24
  • Yes I also visited these links, but as mentioned above already that the time on windows ec2 instance is correct. Clock is in sync with CST time zone. What else could be the reason then? also, the access key and secret key mentioned in configuration of agent are fine. it occurs occasionally. Do you think in this case the data loss is possible? – SabaBaig Jun 11 '19 at 06:53
  • I've Buffer Conditions as 1MB or 60 seconds at Kinesis firehose side. Could this be the reason that in case of larger files, it tries to make chunks of data larger than 60 MB and using expired signature with new request for some unknown reason? Note that I installed Kinesis agent on Dec, 2018. last release was in Nov, 2018. Meaning I have the latest release. Is there any data loss possible if it divides file into chunks and tries to put it again but signature is expired? what happens to data? – SabaBaig Jun 11 '19 at 06:58
  • @SabaBaig how did you solve this? – parlad Mar 08 '21 at 04:31

0 Answers0