IOfflineNumberCommitTo3E.Run

4/3/2024 12:50:01 PM (+26.567s) Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code InternalServerError

System.Net.Http.HttpRequestException: Request failed with status code InternalServerError
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+46ms Processing

Server:: dw0sdwk0004w4:4100
Worker:: 3f256b1e

+119msOfflineNumberCommitTo3ECommandHandler - Started
+333msNo of records retrieved : 38
+347ms----------------------------
+354msSubmission started for Number: 320830
+527msSubmission to 3E - Started at 4/3/2024 12:49:35 PM by Reid McLaughlin
+1.006sCurrent item:
+1.014sId - 320830
+1.038sTriggerType - FN, TaxYear - 2021, State/Fed - FED
+1.045sPreparing request for submission
+1.052sCommitting number to 3E
+26.530sAn error occured: Request failed with status code InternalServerError
+26.537sAn error occured: Request failed with status code InternalServerError

+1h 54m Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 10 of 10: Request failed with status code InternalServerErr…

Enqueue at:: 4/3/2024 12:49:31 PM

+26.403s Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code InternalServerError

System.Net.Http.HttpRequestException: Request failed with status code InternalServerError
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 167
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+30ms Processing

Server:: dw0sdwk000645:5168
Worker:: 856eb605

+66msOfflineNumberCommitTo3ECommandHandler - Started
+351msNo of records retrieved : 38
+358ms----------------------------
+365msSubmission started for Number: 320830
+578msSubmission to 3E - Started at 4/3/2024 10:55:00 AM by Reid McLaughlin
+734msCurrent item:
+741msId - 320830
+854msTriggerType - FN, TaxYear - 2021, State/Fed - FED
+861msPreparing request for submission
+871msCommitting number to 3E
+26.303sAn error occured: Request failed with status code InternalServerError
+26.329sAn error occured: Request failed with status code InternalServerError

+1h 12m Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+7ms Scheduled

Retry attempt 9 of 10: Request failed with status code InternalServerErr…

Enqueue at:: 4/3/2024 10:54:49 AM

+26.193s Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code InternalServerError

System.Net.Http.HttpRequestException: Request failed with status code InternalServerError
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 167
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+23ms Processing

Server:: dw0sdwk000645:5168
Worker:: 70032fe5

+49msOfflineNumberCommitTo3ECommandHandler - Started
+338msNo of records retrieved : 38
+347ms----------------------------
+354msSubmission started for Number: 320830
+499msSubmission to 3E - Started at 4/3/2024 9:41:40 AM by Reid McLaughlin
+628msCurrent item:
+636msId - 320830
+707msTriggerType - FN, TaxYear - 2021, State/Fed - FED
+713msPreparing request for submission
+720msCommitting number to 3E
+26.156sAn error occured: Request failed with status code InternalServerError
+26.163sAn error occured: Request failed with status code InternalServerError

+41m 29.734s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+180ms Scheduled

Retry attempt 8 of 10: Could not load file or assembly 'AG.A1.Scheduler.…

Enqueue at:: 4/3/2024 9:41:30 AM

+2.193s Failed

Can not change the state to 'Processing': target method was not found.

System.IO.FileNotFoundException

Could not load file or assembly 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.

System.IO.FileNotFoundException: Could not load file or assembly 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'AG.A1.Scheduler.Project.Contracts, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Hangfire.Common.TypeHelper.AssemblyResolver(String assemblyString)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Hangfire.Common.TypeHelper.CachedAssemblyResolver(AssemblyName assemblyName)
   at System.TypeNameParser.ResolveAssembly(String asmName, Func`2 assemblyResolver, Boolean throwOnError, StackCrawlMark& stackMark)
   at System.TypeNameParser.ConstructType(Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.TypeNameParser.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError, Boolean ignoreCase, StackCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Func`2 assemblyResolver, Func`4 typeResolver, Boolean throwOnError)
   at Hangfire.Common.TypeHelper.DefaultTypeResolver(String typeName)
   at Hangfire.Storage.InvocationData.DeserializeJob()

+24m 50.993s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 7 of 10: Request failed with status code InternalServerErr…

Enqueue at:: 4/3/2024 9:00:03 AM

+25.920s Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code InternalServerError

System.Net.Http.HttpRequestException: Request failed with status code InternalServerError
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+27ms Processing

Server:: dw0sdwk000645:5168
Worker:: 8d8c9a58

+36msOfflineNumberCommitTo3ECommandHandler - Started
+216msNo of records retrieved : 38
+234ms----------------------------
+252msSubmission started for Number: 320830
+342msSubmission to 3E - Started at 4/3/2024 8:34:51 AM by Reid McLaughlin
+430msCurrent item:
+436msId - 320830
+455msTriggerType - FN, TaxYear - 2021, State/Fed - FED
+462msPreparing request for submission
+470msCommitting number to 3E
+25.872sAn error occured: Request failed with status code InternalServerError
+25.879sAn error occured: Request failed with status code InternalServerError

+12m 48.197s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 6 of 10: Request failed with status code InternalServerErr…

Enqueue at:: 4/3/2024 8:34:43 AM

+27.413s Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code InternalServerError

System.Net.Http.HttpRequestException: Request failed with status code InternalServerError
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+33ms Processing

Server:: dw0sdwk000645:5168
Worker:: 856eb605

+56msOfflineNumberCommitTo3ECommandHandler - Started
+964msNo of records retrieved : 38
+981ms----------------------------
+991msSubmission started for Number: 320830
+1.119sSubmission to 3E - Started at 4/3/2024 8:21:36 AM by Reid McLaughlin
+1.637sCurrent item:
+1.647sId - 320830
+1.671sTriggerType - FN, TaxYear - 2021, State/Fed - FED
+1.687sPreparing request for submission
+1.697sCommitting number to 3E
+27.348sAn error occured: Request failed with status code InternalServerError
+27.365sAn error occured: Request failed with status code InternalServerError

+6m 34.587s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+7ms Scheduled

Retry attempt 5 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 4/3/2024 8:21:22 AM

+1m 40.763s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+27ms Processing

Server:: dw0sdwk000645:5168
Worker:: ce7a5530

+38msOfflineNumberCommitTo3ECommandHandler - Started
+231msNo of records retrieved : 38
+248ms----------------------------
+258msSubmission started for Number: 320830
+494msSubmission to 3E - Started at 4/3/2024 8:13:20 AM by Reid McLaughlin
+579msCurrent item:
+587msId - 320830
+631msTriggerType - FN, TaxYear - 2021, State/Fed - FED
+637msPreparing request for submission
+644msCommitting number to 3E
+1m 40.717sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 40.729sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+2m 49.236s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 4 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 4/3/2024 8:13:07 AM

+1m 40.924s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+26ms Processing

Server:: dw0sdwk000645:5168
Worker:: 70032fe5

+42msOfflineNumberCommitTo3ECommandHandler - Started
+267msNo of records retrieved : 38
+274ms----------------------------
+280msSubmission started for Number: 320830
+354msSubmission to 3E - Started at 4/3/2024 8:08:50 AM by Reid McLaughlin
+449msCurrent item:
+462msId - 320830
+498msTriggerType - FN, TaxYear - 2021, State/Fed - FED
+509msPreparing request for submission
+521msCommitting number to 3E
+1m 40.849sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 40.859sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+1m 4.304s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 3 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 4/3/2024 8:08:43 AM

+1m 40.790s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+40ms Processing

Server:: dw0sdwk000645:5168
Worker:: 8d8c9a58

+50msOfflineNumberCommitTo3ECommandHandler - Started
+310msNo of records retrieved : 38
+321ms----------------------------
+328msSubmission started for Number: 320830
+443msSubmission to 3E - Started at 4/3/2024 8:06:05 AM by Reid McLaughlin
+645msCurrent item:
+654msId - 320830
+674msTriggerType - FN, TaxYear - 2021, State/Fed - FED
+681msPreparing request for submission
+686msCommitting number to 3E
+1m 40.728sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+1m 40.736sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+27.843s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+10ms Scheduled

Retry attempt 2 of 10: The request was canceled due to the configured Ht…

Enqueue at:: 4/3/2024 8:06:01 AM

+2m 47.240s Failed

An exception occurred during performance of the job.

System.Threading.Tasks.TaskCanceledException

The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.CommitNumbersTo3E(NumberCommitInputWrapperDto numberCommitInputWrapper) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 165
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 130
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

+23ms Processing

Server:: dw0sdwk000645:5168
Worker:: 62925226

+46msOfflineNumberCommitTo3ECommandHandler - Started
+39.620sNo of records retrieved : 38
+39.629s----------------------------
+39.637sSubmission started for Number: 320830
+1m 442msSubmission to 3E - Started at 4/3/2024 8:03:50 AM by Reid McLaughlin
+1m 6.506sCurrent item:
+1m 6.515sId - 320830
+1m 7sTriggerType - FN, TaxYear - 2021, State/Fed - FED
+1m 7.010sPreparing request for submission
+1m 7.031sCommitting number to 3E
+2m 47.174sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
+2m 47.187sAn error occured: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

+27.677s Enqueued

Triggered by DelayedJobScheduler

Queue:: default

+20ms Scheduled

Retry attempt 1 of 10: Request failed with status code ServiceUnavailable

Enqueue at:: 4/3/2024 8:02:37 AM

+28.143s Failed

An exception occurred during performance of the job.

System.Net.Http.HttpRequestException

Request failed with status code ServiceUnavailable

System.Net.Http.HttpRequestException: Request failed with status code ServiceUnavailable
   at AG.A1.Framework.Rest.RESTClient.HandleResponse[T](RestResponse restResponse) in C:\agent\_work\1\s\A1Solution\src-framework\AG.A1.Framework\Rest\RESTClient.cs:line 357
   at AG.A1.Framework.Rest.RESTClient.GetInternalAsync[T](IRESTEndpoint apiEndpoint, IDictionary`2 headers, String expectedRootElement, IDictionary`2 parameters) in C:\agent\_work\1\s\A1Solution\src-framework\AG.A1.Framework\Rest\RESTClient.cs:line 186
   at AG.Integrations.RESTClients.AlliantOneApiREST.TransactionTemplateControllers.TransactionTemplateA1ApiClientController.GetTransactionTemplateByKey(String key) in C:\agent\_work\1\s\A1Solution\src-integrations\AG.Integrations.RESTClients\AlliantOneApiREST\TransactionTemplateController\TransactionTemplateA1ApiClientController.cs.cs:line 22
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Helpers.OfflineCommitNumbersTo3EHelper.HandleRequest(CommitNumbersTo3ERequest request, PerformContext performContext, IAlliantOneApiClientController alliantOneApiClientController, IE3EWrapperApiClientController wrapperApiClientController) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Helpers\OfflineCommitNumbersTo3EHelper.cs:line 66
   at AG.A1.Scheduler.Project.Application.Features.CommitNumberTo3E.Commands.OfflineNumberCommitTo3ECommandHandler.Run(PerformContext performContext, IJobCancellationToken cancellationToken) in C:\agent\_work\1\s\A1Solution\src-schedulers\AG.A1.Scheduler.Project.Application\Features\CommitNumberTo3E\Commands\OfflineNumberCommitTo3ECommandHandler.cs:line 73
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

+42.450s Processing

Server:: dw0sdwk000645:5168
Worker:: 856eb605

+638msOfflineNumberCommitTo3ECommandHandler - Started
+19.869sNo of records retrieved : 38
+19.878s----------------------------
+19.888sSubmission started for Number: 320830
+26.926sSubmission to 3E - Started at 4/3/2024 8:02:20 AM by Reid McLaughlin
+27.425sAn error occured: Request failed with status code ServiceUnavailable

+37ms Processing

Server:: dw0sdwk000645:2924
Worker:: ef9463ae

+179msOfflineNumberCommitTo3ECommandHandler - Started
+22.230sNo of records retrieved : 38
+22.247s----------------------------
+22.257sSubmission started for Number: 320830
+33.061sSubmission to 3E - Started at 4/3/2024 8:01:44 AM by Reid McLaughlin
+42.017sAn error occured: The operation was canceled.

+23ms Enqueued

Triggered by recurring job scheduler

Queue:: a1-numbers

CurrentCulture	`"en-US"`
CurrentUICulture	`"en-US"`
RecurringJobId	`"NUMBER-Offline-Commit-Number-To-3E-Task"`
RetryCount	`10`
Time	`1712131271`

Parameters

Requeue Delete State

4/3/2024 12:50:01 PM (+26.567s) Failed

System.Net.Http.HttpRequestException

+46ms Processing

+1h 54m Enqueued

+10ms Scheduled

+26.403s Failed

System.Net.Http.HttpRequestException

+30ms Processing

+1h 12m Enqueued

+7ms Scheduled

+26.193s Failed

System.Net.Http.HttpRequestException

+23ms Processing

+41m 29.734s Enqueued

+180ms Scheduled

+2.193s Failed

System.IO.FileNotFoundException

+24m 50.993s Enqueued

+10ms Scheduled

+25.920s Failed

System.Net.Http.HttpRequestException

+27ms Processing

+12m 48.197s Enqueued

+10ms Scheduled

+27.413s Failed

System.Net.Http.HttpRequestException

+33ms Processing

+6m 34.587s Enqueued

+7ms Scheduled

+1m 40.763s Failed

System.Threading.Tasks.TaskCanceledException

+27ms Processing

+2m 49.236s Enqueued

+10ms Scheduled

+1m 40.924s Failed

System.Threading.Tasks.TaskCanceledException

+26ms Processing

+1m 4.304s Enqueued

+10ms Scheduled

+1m 40.790s Failed

System.Threading.Tasks.TaskCanceledException

+40ms Processing

+27.843s Enqueued

+10ms Scheduled

+2m 47.240s Failed

System.Threading.Tasks.TaskCanceledException

+23ms Processing

+27.677s Enqueued

+20ms Scheduled

+28.143s Failed

System.Net.Http.HttpRequestException

+42.450s Processing

+37ms Processing

+23ms Enqueued

4/3/2024 8:01:11 AM Created

State